Skip to main content

Fostering implementation of knowledge into health practice: study protocol for the validation and redevelopment of the Knowledge Uptake and Utilization Tool

Abstract

Background

Measurement of what knowledge is taken-up and how that information is used to inform practice and policies can provide an understanding about the effectiveness of knowledge uptake and utilization processes. In 2007, the Knowledge Uptake and Utilization Tool (KUUT) was developed to evaluate the implementation of knowledge into practice. The KUUT has been used by numerous large health organizations despite limited validity evidence and a narrow understanding about how the tool is used in practice and interpreted by users. As such, the overall purpose of this protocol is to redevelop the KUUT and gather validity evidence to examine and support its use in various health-related organizations. This protocol paper outlines a validation and redevelopment procedure for the KUUT using the unitary view of validity.

Methods

The protocol outlined in this article proceeds through four phases, starting with redeveloping the tool, then evaluating validity evidence based on: test content, response processes and internal structure. The initial phase gathers information to redevelop the tool, and evaluates item content and response format. The second phase evaluates response process validity evidence by examining how a variety of users interact with the tool. In the third phase, the tool will be pilot tested with knowledge users and, in the final phase, psychometric properties of the tool will be examined and a final scoring structure will be determined. A knowledge translation plan described herein outlines where the final tool will be housed and how the information about the tool will be disseminated.

Discussion

This protocol outlines a procedure to gather different sources of validity evidence for the KUUT. By addressing limitations in the original KUUT, such as complexities with scoring, a redeveloped KUUT supporting validity evidence will enhance the ability of health-related organizations to effectively use this tool for its intended purpose.

Background

Research in health settings aims to both improve knowledge and translate this knowledge beyond the individual to group or systemic levels. Measuring what knowledge is taken-up and how that information is used to inform practice and policies can provide an understanding about the effectiveness of knowledge uptake and utilization (KUU) processes. However, large gaps exist between the knowledge that is revealed through research and the knowledge that is used in practice. To address this gap, Skinner [1] developed the Knowledge Uptake and Utilization Tool (KUUT). Over a decade since its initial development, the KUUT has been used and adapted by a variety of large national level health organizations such as the Public Health Agency of Canada, Health Canada and the Canadian Partnership Against Cancer. Although there are no other broad measures that evaluate KUU that are known to experts in the field [2, 3], the KUUT has weak validity evidence and limitations with its current use (e.g. complexities with scoring). Furthermore, while the process of knowledge exchange between producers of knowledge as well as users of knowledge is dominant in healthcare policy [4] and has flourished there, the KUUT has also become relevant to other health-related organizations. As such, a more robust, redeveloped KUUT with stronger validity evidence to support its use is needed to confidently measure KUU and improve the ease of use. The purpose of this protocol paper is to outline the processes that will be undertaken to obtain validity evidence for an updated and redeveloped KUUT.

As a concept, KUU refers to the implementation of knowledge into practice within the broad field of knowledge transfer and exchange (KTE). How KTE is defined and operationalized varies greatly depending on many factors such as the disciplinary field, commitment to community, industry, government, level of stakeholder engagement and funding priorities as well as the background and training of researchers [5,6,7]. As noted by Graham et al., it has been well recognized and accepted that, in the last decade, a key aspect of KTE includes the integration of knowledge users during the research process to improve the “relevance, applicability and impact of results” ([8], p. 2). The idea of integrated knowledge translation promotes interaction and collaboration between researchers (or other producers of knowledge) and knowledge users in the KTE process. Subsequently, the concept of KUU involves an integrated KTE approach to communicate and collaborate with potential knowledge users, as well as the exchange and use of the knowledge by, and between, stakeholders (e.g. researchers, organizations) and knowledge users (i.e. general public, policy-makers) in both health and health-related organizations [4, 9]. Operationalization of KUU involves using research- or organizational-generated knowledge, which are often originally communicated in written form and then applying this knowledge into practice, policies or programme development [10,11,12].

Although KUU includes the process of KTE, it focuses more heavily on how knowledge users integrate knowledge products into their practice. Knowledge products include the ways knowledge is customized before it is spread to target audiences [11, 13], and in this study includes documents (e.g. reports, briefing notes, journal articles), data, websites, workshops, training and other activities used to share information with knowledge users. Effective dissemination of knowledge products (e.g. organizational reports) improves the chances that they are being used by policy, practice and programme decision-makers to generate larger and faster societal impact [4, 8]. Although it can be difficult to measure improvements or increases in the KTE process, such as through integrated knowledge translation or the uptake and utilization of knowledge products, there are increasingly more requests to apply and evaluate KTE efforts — and these components are what the KUUT aims to measure.

The importance of both identifying and measuring KUU has grown over the years. In the last decade, anticipated KTE goals, activities and rationale have been increasingly requested by funding agencies. It is rare, however, for funders, organizations or researchers to publish how the KTE efforts were taken up, utilized and resulted in change [14]. For example, lead funders for health research in Canada are increasingly funding more projects that are ‘partnered’, meaning that researchers must enter into a project with identified partners that will contribute, benefit from and/or use knowledge generated from a project [15, 16]. Studies reviewing KTE-related strategies focus on the impact of specific methods and very few evaluate the impact of KTE products or processes [17,18,19,20]. Furthermore, in a literature review of KTE practices and outcomes, no tools were identified to evaluate the uptake of knowledge products and the KUUT aims to fill this gap [21].

Given increased priority for research that demonstrates how findings will improve outcomes for individuals, communities or target populations, there is a need and appetite for a KUU tool with sufficient validity evidence to support its use [2, 3]. More specifically, validity information is needed to ascertain how the tool evaluates the concept of KUU. Validity is defined as “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” ([22], p. 11). Investigating validity means evaluating how a tool is interpreted and the claims that are made about the tool [22, 23]. Validity information will also help to understand how the tool operates in different environments with different stakeholders. Currently, the existing validity evidence for the KUUT provides some preliminary information about test content by comparing the degree of association of knowledge use and KUUT outcomes, and uses qualitative interviews with a public health unit and the Level of Use scale [24, 25]. This evaluation found that there was 93% agreement between the KUUT and qualitative assessment [25]. As originally developed by Skinner [1], the KUUT is primarily informed by theory, including Knott and Wildavsky’s [26] Seven Standards of Utilization. In addition, two extra stages were included in the original 2007 KUUT [1] — awareness and effort. These stages were added by Skinner because 'awareness' of the knowledge product (or knowledge process) is necessary to take up and utilize the knowledge, and the knowledge user making an 'effort' to use that knowledge is a component of utilization. Altogether, the KUUT considers nine categories that are necessary for effective KUU (Table 1). However, the existing validity evidence does not establish how well the theoretical evidence is represented in the measure or how well the tool represents the concept of KUU. The protocol presented in this paper considers the gaps in validity evidence for the KUUT using current measurement perspectives from the unified validity theory [22, 27].

Table 1 Stages/Standard of Knowledge Utilizationa

The overall purpose of this study is to redevelop the KUUT and gather validity evidence for its use in health and health-related organizations. This paper outlines a validation protocol for the redevelopment and testing of the KUUT using the unitary view of validity. In particular, several sources of validity evidence will be explored, which include test content, response processes and internal structure. These sources of validity evidence will (1) provide information about the relevance and representativeness of items towards the KUU (test content), (2) evaluate how respondents both use and interpret the tool (response processes), and (3) provide preliminary psychometric evidence to confirm its factor structure and reliability (internal structure). These sources of evidence build an integrative judgment of how well the tool evaluates KUU, and helps to support the inferences that are made about the test score [28, 29]. In outlining the protocol for this investigation, this study uses integrated knowledge translation approaches and offers an example for how the unified validity theory can be applied, as application of this modern validity approach continues to be underutilized [22, 30, 31]. Altogether, the research protocol outlined here and subsequent evidence will contribute towards how well the tool evaluates the concept of KUU. Evaluating and strengthening validity evidence for the KUUT will aid implementation of this concept by knowledge users. It will also help organizations to determine the most effective ways of disseminating and using knowledge internally, and prompt knowledge users to consider how they might use new evidence that would be of interest to many organizations.

Methods

Scale development and validation practices that employ the unified validity theory [22, 27] will inform the redevelopment and validation processes of the KUUT at all stages.Within the unified view, validity is centred around the construct or the unobservable phenomenon that one aims to measure [22, 32, 29]. The research protocol outlined herein includes four phases that will redevelop the KUUT and gather different sources of validity information. The protocol starts by revising the original KUUT, then proceeds to gather information and evaluate the following sources of validity: test content, response processes and internal structure. Each phase is outlined in detail below, including a description of the process as well as participants and subsequent analysis (Fig. 1). A knowledge translation plan for disseminating the redeveloped KUUT is also included.

Fig. 1
figure1

Redeveloping the KUUT and gathering validity evidence

Phase 1: evaluate item content and response format on the KUUT

This initial phase will revise the KUUT using information from a literature review and interview feedback from users of the KUUT. Next, the tool content will be tested by asking experts to provide feedback about items, response format and instructions.

Part 1a – gather preliminary information to revise the tool

Since the original development of the KUUT by Skinner [1], both informal and formal feedback has been received about its content and response format. This initial phase consists of gathering preliminary information from a literature review, as well as interview feedback about the KUUT from KUUT users (ORE#31771). A scoping review by the research team has begun to explore the various tools that are used to evaluate KTE processes. Ongoing interviews with KUUT users are providing insight about the types of organizations, projects, knowledge products or processes that the KUUT is being used for as well as how the KUUT is being adapted to fit different contexts. Using the aforementioned information as well as guidelines in tool development and validation practices [22, 33], the original KUUT is undergoing revisions by two researchers (SS and KS). Items are being re-worded, added, or removed and the response scale refined. Revision of items includes consideration of their relevance to theory and how each item relates to the concept of KUU. Overall, the scale is being simplified and shortened to facilitate the ease of use, as well as a greater ability to investigate psychometric properties.

Part 1b – review content and response format

In this phase, information about the redeveloped KUUT will be received from experts and any further modifications will be made regarding its clarity, relevance, ease and representativeness. Specifically, information about the test content, response format and instructions will be evaluated.

Participants and analysis

Gathering validity evidence based on test content will begin by assembling subject matter experts who are knowledgeable in the fields of KTE, KUU and evaluation, as well as experiential experts who have practical experience with KTE and KUU. Subject matter and experiential experts will be invited to examine the content of the KUUT and judge the relevance and representativeness of items included in the redeveloped tool. We anticipate the group of subject matter and experiential experts to include a minimum of 25 people who will be recruited from several health-related organizations.

Both subject matter and experiential experts will judge the content of the KUUT using an online or in-person survey. Respondents will provide feedback on the shortened KUUT about the following: (1) clarity of administration, (2) clarity of instructions, (3) relevance of items to KUU and (4) ease of response format. In addition, respondents will provide narrative feedback for each of the items in the KUUT and judge the appropriateness of each item towards the overall scale. A content validity index [34] will be used to indicate the proportion of experts (both subject matter and experiential) who endorse each of the items. Any items that have a content validity index of less than 0.80 will be marked for review [35].

Qualitative information about the content will be gathered by suggestions from subject matter and experiential experts about items that need to be modified from their narrative feedback. To evaluate the narrative feedback, content analysis [36] will be employed to organize descriptions, followed by themes to summarize feedback at the item and overall test level. This information, along with information gathered in Phase 2, will be used to inform further modifications to the KUUT.

Phase 2: understand respondent interactions with the KUUT

Response processes provide validity evidence by explaining how respondents interact with a measure and relating this information to how a test is used and interpreted. Response processes are an aspect of validity that examine the process models involved in a measurement task and include aspects such as cognitive processes [32]. In order to understand how individuals respond the way they do on the KUUT, this aspect of the research study will investigate the accuracy and consistency of responses to the KUUT using a combination of think-aloud interviews and verbal probing. Using the process of a think-aloud interview, also termed concurrent verbalization, respondents will be asked to think out loud as they are completing the measure and the interviewer will also probe about specific information, such as why an item may reflect a particular theoretic stage or ways the item can be reworded [37]. During the think-aloud interview, respondents will be asked to describe the things that come to mind as they are responding to items on the measure. Interviewers will probe for more information to prompt respondents to explain what they are thinking or saying to themselves.

Participants and analysis

A small subsample of both subject matter and experiential experts from each of the knowledge user organizations in Phase 1 will be asked to respond to the KUUT while thinking aloud. Using think aloud interviews will enable researchers to examine what, if any, differences exist between various knowledge users and an opportunity to draw specific considerations for sex and gender of users, as well as ensuring diversity of the sample. Respondents will be asked to think out loud as they are completing the measure and the interviewer will also probe about specific information, such as why an item may reflect a particular theoretic stage or ways the item can be reworded [37]. The information gathered here will help to justify interpretations of this tool by providing evidence for how and why users respond the way they do [28]. Altogether, the information obtained from Phases 1 and 2 will inform final modifications before pilot testing.

Phase 3: pilot testing with knowledge users

The modified KUUT will be pilot tested with participants in various health-related organizations.

Participants and analysis

Pilot testing will involve real world application of the redeveloped KUUT with various knowledge user organizations. Each knowledge user organization has individuals who are recipients of disseminated knowledge products; these individuals are henceforth referred to as ‘secondary knowledge users’. During pilot testing, knowledge users will introduce a new knowledge product to their secondary knowledge users and, approximately 2–3 months after the new knowledge product is introduced, the secondary knowledge users will complete the redeveloped KUUT. As a period of time is required for secondary knowledge users to take up and utilize a disseminated knowledge product, this is why the KUUT will be administered 2–3 months after dissemination. Opportunities for knowledge user products include the following:

  • Secondary knowledge users at the 36 public health units will receive various knowledge products from Public Health Ontario;

  • Successful applicants of the Sunnybrook Health Sciences Centre seed grant programme, who are required to disseminate the knowledge products that they produce;

  • The Canadian Observatory on Homelessness would like to measure the impact of the evidence they are building through their Making the Shift and A Way Home Canada projects — more specifically, through 12 demonstration projects being conducted in 10 communities in the provinces of Alberta and Ontario, they are testing intervention models aiming to support young people as they exit homelessness or the risk for homelessness; and

  • Homewood Research Institute has a recovery monitoring system that synthesizes data from patients into a dashboard that is being developed to provide outcome reports for clinicians.

A month after the KUUT is first used, re-test information will be gathered by the same participants. The aim is to have a total sample size of at least 300 secondary knowledge users to pilot test the redeveloped tool and to ensure a strong sample size for subsequent psychometric testing in the next phase.

Phase 4: investigate dimensionality, item analyses and reliability

Using the data gathered in Phase 3, the redeveloped KUUT will be evaluated for internal structure to examine dimensionality. In addition, the quality of items and reliability will be evaluated.

Analysis

Factor analysis will be used to examine the dimensionality of the tool and confirm the scoring structure. Using an exploratory factor analysis, the scale will be evaluated to explore dimensionality. In addition, item analyses will be conducted to analyse the quality of items and determine the final selection of items (e.g. inter-item correlations, item-total and corrected item-total correlations, alpha-if-item-deleted, and factor loadings). Internal consistency and test–retest reliability estimates will also be obtained.

Knowledge translation plan

This study also includes its own knowledge translation plan strategy that will, through the process of gathering validity information for this tool (also acting as a form of integrated knowledge translation) and through dissemination of the final KUUT, engage KTE and evaluation hubs such as the Knowledge Translation Canada, the Knowledge Translation Program at St. Michael’s Hospital, Institute for Knowledge Mobilization and Canadian Evaluation Society. In addition, organizations such as the Public Health Agency of Canada, Health Canada, the National Collaborating Center for Public Policy, and Public Health Ontario will be involved in the knowledge translation plan. We will also network with other organizations that might have an interest in measuring KUU. The redeveloped KUUT will be housed in an open access format on Dr Skinner’s website [38] and will be disseminated through webinars open to any organization. During the end of the KTE phase when we disseminate the final KUUT through presentations and webinars, we will also evaluate the usefulness of the KUUT [39] through an online survey with new users and determine whether it is more useful for some users than others (e.g. for non-profit vs. government organizations; research vs. evaluation utilization; large vs. small organizations, etc.).

Discussion

The value of research, for community-based and government organizations alike, is based on the usefulness of the knowledge produced [40, 41]. Furthermore, health-based organizations rely heavily on knowledge products, such as evidence-based research, to develop or improve policies and programmes. Government-funded projects that aim to address priority issues such as poverty, homelessness or mental health frequently support intervention research or implementation science projects with a strong evaluation component that measures change (e.g. Homelessness Hub, Local Poverty Reduction Fund of the Ontario Trillium Foundation, etc.). The capacity for organizations to evaluate how knowledge products and processes have been taken up greatly varies [42] and the KUUT serves to fill this need.

While this protocol details how validity evidence will be gathered for the KUUT, it is important to understand subsequent conclusions that can be made about this tool once this validity evidence is obtained. Within the unified view, validity is no longer considered from a dichotomous perspective, where validity evidence determines whether a tool is ‘valid’ or ‘not valid’, or deems a tool as conclusively ‘validated’ [22]. The shift in conceptions of validity from a fixed or static property requires consideration that validity is an evolving concept that changes with time and use, in various settings [32]. As such, validity information is contextualized and dependent on the use of the tool. Gathering validity information and the processes involved in redevelopment of a tool will highlight the strength of evidence to support the use of the tool in different health-related organizations. Although application of the unified validity framework remains largely underutilized [28, 29], this validity approach encourages an understanding of how the tools are interpreted by users and is currently endorsed by the Standards for Educational and Psychological Testing [22]. Furthermore, as recommended in studies that investigate validation practices, articulating validity approaches are initial steps that support the foundation of a strong validity argument [22, 28]. Applying this framework also recognizes the consequences and possible negative implications of testing, which are important but often overlooked considerations [43]. Overall, the information gathered in this protocol will provide information about how the KUUT is used and understood in practice, thereby strengthening the validity evidence for this tool. Whether the validity evidence is sufficient to warrant use in settings that are different from those tested here is uncertain; however, we aim to test this tool with a wide audience to meet the needs of, and support its use with a number of interested health-related organizations.

Conclusion

This validation protocol will provide evidence for the KUUT to evaluate how well knowledge dissemination, uptake and utilization, or transfer of knowledge products are used by knowledge users and stakeholders, including diverse groups of women, men and gender-diverse individuals. The KUUT may also be used to hold researchers and other knowledge producers accountable, help organizations determine the most effective ways of disseminating and using knowledge internally, and prompt knowledge users to consider how else they might use new evidence that would be of interest to many organizations. A redeveloped tool with validity evidence to support it allows relevant organizations to confidently use the tool to measure KUU and make appropriate inferences from the measure.

Availability of data and materials

Not applicable.

Abbreviations

KTE:

knowledge transfer and exchange

KUU:

knowledge uptake and utilization

KUUT:

Knowledge Uptake and Utilization Tool

References

  1. 1.

    Skinner K. Developing a tool to measure knowledge exchange outcomes. Can J Program Eval. 2007;22:49–73.

    Google Scholar 

  2. 2.

    Center of Expertise for Waters. Evaluating Knowledge Exchange: A Review. Scotland: CREW; 2012.

    Google Scholar 

  3. 3.

    Gervais MJ, Marion C, Dagenais C, Chiocchio F, Houlfort N. Dealing with the complexity of evaluating knowledge transfer strategies: guiding principles for developing valid instruments. Res Eval. 2016;25:62–9.

    Article  Google Scholar 

  4. 4.

    Mitton C, Adair CE, Mckenzie E, Patten SB, Waye B, Mitton C, et al. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 2007;85:729–68.

    Article  Google Scholar 

  5. 5.

    Canadian Institutes of Health Research. Guide to Knowledge Translation Planning at CIHR: Integrated and End-Of-Grant Approaches. Ottawa: CIHR; 2012. http://www.cihr.ca/e/45321.html. Accessed 23 Nov 2019

  6. 6.

    Greenhalgh T, Wieringa S. Is it time to drop the “knowledge translation” metaphor? A critical literature review. J R Soc Med. 2011;104:501–9.

    Article  Google Scholar 

  7. 7.

    Tetroe JM, Graham ID, Foy R, Robinson N, Martin P, Wensing M, et al. Health research funding agencies’ support and promotion of knowledge translation : An international study. Milbank Q. 2008;86:125–55.

    Article  Google Scholar 

  8. 8.

    Graham ID, Kothari A, McCutcheon C, Alvarez G, Banner D, Botti M, et al. Moving knowledge into action for more effective practice, programmes and policy: Protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13:1–15.

    Article  Google Scholar 

  9. 9.

    Kiefer L, Frank J, Di Ruggiero E, Dobbins M, Manuel D, Gully PR, et al. Fostering evidence-based decision-making in Canada. Can J Public Health. 2005;96:I1–19.

    PubMed  Google Scholar 

  10. 10.

    Estabrooks CA. The conceptual structure of research utilization. Res Nurs Health. 1999;22:203–16.

    CAS  Article  Google Scholar 

  11. 11.

    Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: Time for a map? J Contin Educ Heal Prof. 2006;26:13–24.

    Article  Google Scholar 

  12. 12.

    Kothari A, Birch S, Charles C. “Interaction” and research utilisation in health policies and programs: Does it work? Health Policy. 2005;71:117–25.

    Article  Google Scholar 

  13. 13.

    Gagnon M. Knowledge dissemination and exchange of knowledge. Vol. Section 5., Knowledge Translation in Health Care: Moving from Evidence to Practice. Canadian Institutes of Health Research. 2015. http://www.cihr-irsc.gc.ca/e/documents/kt_in_health_care_chapter_5.1_e.pdf. Accessed 23 Nov 2019.

  14. 14.

    Scott SD, Albrecht L, O’Leary K, Ball GDC, Dryden DM, Hartling L, et al. A protocol for a systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2011;6:58.

    Article  Google Scholar 

  15. 15.

    Ellis KS. Knowledge Translation and the Governance of Health Research in Canada: A Critical Discourse Analysis: Electronic Thesis and Dissertation Repository; 2014. https://ir.lib.uwo.ca/etd/2198. Accessed 23 Nov 2019.

  16. 16.

    McLean R, Tucker J. Evaluation of CIHR’s Knowledge Translation Funding Program. Ottawa: Canadian Institutes of Health Research; 2014. p. 1–47.

    Google Scholar 

  17. 17.

    Bhattacharyya OK, Estey EA, Zwarenstein M. Methodologies to evaluate the effectiveness of knowledge translation interventions: a primer for researchers and health care managers. J Clin Epidemiol. 2011;64:32–40 https://doi.org/10.1016/j.jclinepi.2010.02.022.

    Article  Google Scholar 

  18. 18.

    Larocca R, Yost J, Dobbins M, Ciliska D, Butt M. The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012;12:751.

    Article  Google Scholar 

  19. 19.

    Salter K, Kothari A. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. 2014;9:115.

    Article  Google Scholar 

  20. 20.

    Yamada J, Shorkey A, Barwick M, Widger K, Stevens BJ. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:1–10.

    Article  Google Scholar 

  21. 21.

    Clark G. Promising Practices in Knowledge Translation for Research Users. 2008. https://static1.squarespace.com/static/56afc7218259b53bd8383cb8/t/56be58692fe13102bbfdf672/1455315053362/Promising_Practices_In_Knowledge_Translation.pdf. Accessed 23 Nov 2019

  22. 22.

    American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association; 2014.

  23. 23.

    Kane MT. Content-related validity evidence. In: Downing SM, Haladyna TM, editors. Handbook of Test Development. Mahwah: Lawrence Erlbaum Associates Publishers; 2006. p. 131–53.

    Google Scholar 

  24. 24.

    Hall GE, Loucks SF, Rutherford WL, Newlove BW. Levels of use of the innovation: a framework for analyzing innovation adoption. J Teach Educ. 1975;26:52–6.

    Article  Google Scholar 

  25. 25.

    Bonin EN. How Context Influences Knowledge Use in Public Health Units. Waterloo: University of Waterloo; 2007.

    Google Scholar 

  26. 26.

    Knott J, Wildavsky A. If dissemination is the solution, what is the problem? Sci Commun. 1980;1:537–78.

    Google Scholar 

  27. 27.

    Bandalos DL. Chapter 11: Validity. In: Measurement Theory and Applications for the Social Sciences. New York: Guilford Press; 2017. p. 254–97.

    Google Scholar 

  28. 28.

    Messick S. Validity. In: Linn RL, editor. Educational Measurement (3rd ed). New York: Macmillan; 1989. p. 13–103.

    Google Scholar 

  29. 29.

    Messick S. Validity of psychological assessment. Am Psychol. 1995;50:741–9.

    Article  Google Scholar 

  30. 30.

    Zumbo BD, Chan EKH, editors. Validity and Validation in Social, Behavioral and Health Sciences. Switzerland: Springer International Publishing; 2014. p. 1–329.

    Google Scholar 

  31. 31.

    Shankar S, Marshall SK, Zumbo BD. A systematic review of validation practices for the Goal Attainment Scaling measure. J Psychoeducational Assess. 2019. https://doi.org/10.1177/0734282919840948.

  32. 32.

    Loevinger J. Objective tests as instruments of psychological theory. Psychol Rep. 1957;3:635–94.

    Google Scholar 

  33. 33.

    DeVellis RF. Scale Development: Theory and Applications. 6th ed. SAGE: Thousand Oaks; 2012.

    Google Scholar 

  34. 34.

    Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28:563–75.

    Article  Google Scholar 

  35. 35.

    Beck CT, Gable RK. Ensuring content validity: an illustration of the process. J Nursing Meas. 2001;9:201–15.

    CAS  Article  Google Scholar 

  36. 36.

    Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.

    Article  Google Scholar 

  37. 37.

    Gadermann AM, Guhn M, Zumbo BD. Investigating the substantive aspect of construct validity for the satisfaction with life scale adapted for children: a focus on cognitive processes. Soc Indicators Res. 2010;100:37–60.

    Article  Google Scholar 

  38. 38.

    Skinner K. Knowledge Uptake and Utilization Tool. 2019. http://www.kellyskinner.ca/. Accessed 23 Nov 2019

  39. 39.

    Gilbride D, Vandergoot D, Golden K, Stensrud R. Development and validation of the employer openness survey. Rehab Counselling Bull. 2006;49:81–9.

    Article  Google Scholar 

  40. 40.

    Lavis JN, Permanand G, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 14: organising and using policy dialogues to support evidence-informed policymaking. Health res Policy Syst. 2009;7 Suppl 1:S14.

    Article  Google Scholar 

  41. 41.

    Lavis JN, Robertson D, Woodside JM, Mcleod CB, Lavis JN, Robertson D. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q. 2003;81:221–48.

    Article  Google Scholar 

  42. 42.

    Bourgeois I, Cousins JB. Understanding dimensions of organizational evaluation capacity. Am J Eval. 2013;34:299–319.

    Article  Google Scholar 

  43. 43.

    Cizek GJ, Rosenberg SL, Koons HH. Sources of validity evidence for educational and psychological tests. Educ Psychol Meas. 2008;68:397–412.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the support from our colleague Steve Montague as well as generous funding from the Performance and Planning Exchange (PPX) and a University of Waterloo CIHR Research Incentive Fund grant.

Funding

Funding was received from the Performance and Planning Exchange (PPX) and a University of Waterloo CIHR Research Incentive Fund grant.

Author information

Affiliations

Authors

Contributions

SS took the lead in designing, writing and obtaining input from all authors in this paper. SS and KS worked together to design the validation protocol and review the previous development and use of the KUUT. MEMN helped design the figure with SS, and JB provided input regarding the scoping review. All authors (SS, KS, MEMN & JB) reviewed and critically revised the final version of the manuscript before submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kelly Skinner.

Ethics declarations

Ethics approval and consent to participate

Ethics has been approved by the Office of Research Ethics, University of Waterloo (ORE - #31771).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Shankar, S., Skinner, K., Morton Ninomiya, M.E. et al. Fostering implementation of knowledge into health practice: study protocol for the validation and redevelopment of the Knowledge Uptake and Utilization Tool. Health Res Policy Sys 17, 105 (2019). https://doi.org/10.1186/s12961-019-0503-5

Download citation

Keywords

  • validation
  • knowledge uptake and utilization
  • integrated knowledge translation
  • knowledge translation
  • implementation
  • knowledge transfer and exchange
  • tools
  • unified validity theory