Co-creating physical activity interventions: a mixed methods evaluation approach
Health Research Policy and Systems volume 19, Article number: 37 (2021)
Co-creation strategies, such as cooperative planning, are promising as a means to ensure that physical activity interventions address real-world problems and are tailored to the target group. This has already been validated in diverse settings. However, questions targeting the transferability of cooperative planning to new settings and the key factors influencing its success or failure remain unclear. At the same time, co-creation processes are complex, and evaluation can be challenging. Following calls for detailed reporting, this paper describes the programme activities, the underlying logic, and methodological design of a study that aims to evaluate the transfer of cooperative planning to new settings and to explore the associated key determinants.
Cooperative planning was utilized as a strategy to target physical activity promotion in three real-world German settings in the nursing care and automotive mechatronics sectors. This involved researchers working alongside stakeholders from practice and policy to conjointly develop new interventions to promote physical activity in physically demanding jobs. A pragmatic approach is used to evaluate both the transferability and key determinants of this strategy. We developed a logic model for this co-creation process that describes the underlying assumptions and guides the evaluation. The evaluation outcomes of this study include planning meetings, newly developed interventions, and the determinants that are likely to affect cooperative planning. Quantitative and qualitative data will be collected using questionnaires, documents, and interviews. The quantitative data will be analysed descriptively, while the qualitative data will mainly be analysed using qualitative content analysis, split by settings. Subsequently, data triangulation will be used to integrate the quantitative and qualitative findings, which will then be compared across all three settings.
The study findings will contribute to a better understanding of co-creation strategies, their transferability, and key determinants. The practical implications can include a checklist for assessing key determinants and a guideline for transferring cooperative planning into new settings to benefit more people. Ultimately, this study will help to advance co-creation strategies and may be relevant for researchers, practitioners, and policy-makers targeting physical activity promotion in various contexts.
Trial registration: Open Science Framework: https://osf.io/r6xnt/ (retrospectively registered).
Since the early years of health-related physical activity (PA) research , there has been a growing body of literature providing strong evidence regarding the beneficial effects of PA on an individual’s health [2, 3]. In accordance with the position that PA may work as medicine, a special focus should be put on population groups who are exposed to increased health risks and who may particularly benefit from the positive molecular and physiological effects of PA . Even though work has entered the digital and automated era and a reduction in sedentary time has recently become a top priority in PA promotion [5,6,7,8], we must not lose sight of occupational groups who experience high physical demands during working life . In line with the “settings for health” defined by the World Health Organization (, p. 362), work sites and schools appear to be favourable settings to promote health and PA by reaching people who work and learn there.
However, two aspects need to be considered in this context. First, when we aim to promote PA in professions with a high physical workload, the special demands of the occupational group must be taken into account. For example, there are indications that a high level of occupational PA is associated with a number of unfavourable health outcomes [11, 12]. In light of this, PA promotion among people with physically demanding jobs should not focus solely on increasing PA levels; the emphasis should be placed on strengthening competencies that are essential to master PAs in a healthy manner . Second, the surrounding structures must be considered, as research has shown that supportive environments and policies are important for health behaviour changes [14, 15]. Taken together, it seems imperative to involve the target group and other nonacademic stakeholders as the experts in their settings [16,17,18] to identify appropriate PA interventions in physically demanding work settings.
Against this backdrop, co-creation, defined as “collaborative public health intervention development by academics working alongside other stakeholders” (, p. 2), is promising in terms of creating PA interventions. Involving different stakeholders in such strategies has the potential to systematically address real-world problems , to develop interventions that are tailored to the end user [18, 20], and to achieve sustainable outputs and impact [16, 21, 22]. The German research project PArC-AVE (Physical Activity-related Health Competence in Apprenticeship and Vocational Education) utilizes a co-creation strategy called cooperative planning (CP)  to develop interventions aimed to promote PA in the nursing care and automotive mechatronics sectors – two physically demanding professions. The goal of CP is to achieve changes at both the structural and individual levels that facilitate the adoption and/or maintenance of a physically active and healthy lifestyle; more specifically, by creating PA-friendly structures and strengthening Physical Activity-related Health Competence (PAHCO) [24, 25] among individuals. Previous research findings have shown promising results from CP in a private vocational education centre for health professions and in a vocational education centre of a German automotive manufacturer . In both settings, new PA interventions were developed and implemented, and several capacities for PA promotion were built up. Thus far, it remains unclear whether CP can successfully be transferred to other settings to reach more people in the nursing care and automotive mechatronics sectors (in the sense of “scaling up” [27, 28]). Beyond that, the aforementioned study  identified differences between the nursing and automotive settings, mainly concerning the active participation of the target group and facilitators as well as barriers to intervention development and implementation (e.g. financial resources, organizational support). Although these differences have not been investigated in detail, we nevertheless note the importance of influencing factors, such as the attitudes of the involved stakeholders, support from organizations or individuals, and financial resources (see also ). This raises the question of which determinants are key enablers of and barriers to CP and thus need to be considered in future PA-promotion programmes.
These unanswered questions targeting the transferability and relevant determinants of CP gain more weight in light of the increasing body of studies using CP [30,31,32,33] or similar strategies in PA promotion and health promotion [22, 34, 35]; not forgetting the critical voices discussing the limitations and challenges of co-creation strategies [36, 37]. However, co-creating new interventions to promote PA is a complex process due to the high number of involved interest groups, the variability of outcomes, and the flexibility of the process [16, 38, 39]. Moreover, practical challenges for evaluation may arise from the real-world settings in which research in the PArC-AVE Project is conducted [40, 41]. Hence, this study uses a pragmatic approach, which seems favourable for the evaluation of complex programmes in real-world settings [41, 42]. We will build on the most relevant and best available data sources to gain a better understanding of how CP can be transferred to new settings and the factors linked to success or failure.
The current paper describes in detail the evaluation design of a study that aims to investigate the transferability of CP and to explore the key determinants for developing and implementing PA interventions. Following calls for more precise reporting of such studies (e.g. [29, 38, 43, 44]), we provide a thorough description of the activities that were undertaken, the underlying programme logic, and the evaluation design. Ultimately, this study will help to increase the knowledge base of co-creation strategies in the field of PA promotion and to foster their scientific as well as practical advancement.
Study settings and co-creation strategy
Building on the first research findings from 2015 to 2018 , the PArC-AVE Project aims at transferring CP as a co-creation strategy into three new settings in the nursing care and automotive mechatronics sectors from 2018 to 2021. The new settings are a state vocational education centre for health professions in a medium-sized city (setting A: 200 nursing students enrolled in the nursing programme), a state vocational education centre for health professions in a large city (setting B: 180 nursing students enrolled in the nursing programme), and the assembly department of a German automotive manufacturer (setting C: 12 000 employees in the assembly department), all located in Bavaria, Germany. In all three settings, the overall goal is to conjointly develop new PA interventions.
Separate CP processes were undertaken in each setting (setting A, setting B, setting C), including the preparation, planning, and implementation phases [26, 45]. Figure 1 provides a summary overview of these processes. After the initial cooperation requests to settings A, B, and C (November 2016) and the subsequent positive funding decision (February 2018), the activities started in spring 2018. During the preparation phase, the initial meetings were used to inform the project partners about the project idea and to identify setting-specific structures, processes, and the relevant stakeholders (April/May 2018). A detailed explorative situation analysis was conducted in all settings to collect more information about the context and the needs of the target groups (April–July 2018). This was mainly done during several visits by the researchers to each setting and the utilization of a criteria catalogue, based on the ecological model of Bauman et al.  and on earlier findings , to collect the relevant information. In the planning phase, planning meetings were held to develop new PA interventions by a planning group of different stakeholders from research, practice, and policy (September 2018–December 2019). For further information on the phases and common rules of the planning meetings as well as the specific roles of the participants, see the description of earlier CP processes (, p. 1580–1581, Table 1). Finally, a set of new interventions was finalized in an action plan for each setting. In the implementation phase, the interventions were implemented under the guidance of the practitioners, starting after the last planning meetings (from July/November 2019).
When evaluating such processes, logic models are a valuable first step, describing the potential mechanisms of how a programme works [46, 47]. Moreover, they have recently been identified as a potential framework for the evaluation of PA interventions in a systematic review by Fynn et al. . In the present study, we developed a logic model to illustrate the planned activities and expected effects within the PArC-AVE Project and, in the next step, to guide the evaluation of CP and its determinants (Fig. 2). Usually, a logic model includes inputs, activities, outputs, and outcome components ; it can also include contextual factors that are expected to influence the success of the programme . In our case, inputs can be defined as the resources provided by researchers, practitioners, and policy-makers to realize cooperation and planning meetings in a particular setting, such as human, financial, and organizational resources. Activities are all project meetings and visits in settings A, B, and C – most importantly, the planning meetings involving stakeholders from research, practice, and policy with the overall aim of developing PA interventions. Outputs, as the direct products of programme activities, are the developed interventions, documented in one action plan per setting. In accordance with the goal of CP including structural and individual changes, we differentiate between outcomes at the structural and individual levels. An outcome at the structural level is the implementation of the developed PA interventions; at the individual level, it is a change in the target group’s PA behaviour, PAHCO, and/or health status. Contextual factors are defined as the determinants that are likely to influence a CP process and its success or failure. Based on earlier findings within the project  and a literature screening of factors relevant to co-creation processes and the implementation of co-created interventions in January/February 2020, we identified the following determinants to be considered in the present evaluation: champion, commitment, empowerment, engagement, group effectiveness, leadership, organizational culture, organizational readiness, ownership, and resources. An overview of these predefined determinants and their definitions is provided as an additional file (see Additional file 1).
Research questions and evaluation design
Two core research questions were identified to address the aim of this study:
Research question 1: How (un)successful is the transfer of cooperative planning to other settings?
Research question 2: Which key factors have an influence on the success or failure of cooperative planning?
The evaluation outcomes were subsequently derived from our logic model (see Table 1). To answer the first research question, planning meetings and the developed interventions will function as evaluation outcomes; measuring outcomes at the individual level (i.e. PA behaviour, PAHCO, health status) in a pre-post design was not possible in this study due to the early practitioner-initiated implementation of interventions. We will thus refer to the logic model components of activities, outputs, and structural outcomes to assess the success or failure of CP in settings A, B, and C. To answer the second research question, predefined and further determinants will function as evaluation outcomes, referring to the logic model component of contextual factors. Building on the differences between settings found in earlier research, we will contrast settings A, B, and C to examine potential differences and similarities. This will be followed by a final assessment of the success/failure of transferred CP and its relationships with the key determinants.
Inspired by the principles of a pragmatic evaluation , this study uses a mixed methods design to evaluate the transfer of CP and its determinants across three different settings . Combining different methods and triangulating the quantitative and qualitative data will, finally, enable us to adequately answer both research questions .
The data will be collected using the following sources (quan = quantitative; qual = qualitative): questionnaires (quan), structured minutes (qual), action plans (qual), and interviews (qual). During the writing and submission process of this paper, the data collection already started. The moments of measurement are indicated for each data source in Fig. 1 and in the following text. An overview of the evaluation outcomes, data sources, and study participants split by research questions is depicted in Table 1.
The predefined determinant of organizational readiness was assessed using the Organizational Readiness for Implementing Change (ORIC) questionnaire by Shea et al. , which has been shown to be a reliable and valid instrument of organizational readiness for change. This questionnaire contains 12 items using a five-point Likert scale to assess the domains of change commitment and change efficacy. As we could not find a German version during the preparation phase in 2018, we translated the questionnaire using forward and back translation with monolingual tests . The forward translation was done by a native German speaker and the back translation was done by a native English speaker. Afterwards, three researchers validated and discussed both versions to refine the ORIC questionnaire in the German language. We also added an introductory description to explain the meaning of the term “change” in the given context, which in our case, are the changes at the organizational level concerning PA promotion that result from the research project. Yet, for future studies of organizational readiness in the German language, it should be noted that a German version of the ORIC questionnaire has recently been tested and published by Lindig et al. .
According to Weiner et al. , the assessment of organizational readiness should take place before the process of change begins. Utilizing a maximum variation sampling scheme , we asked all stakeholders who attended the first planning meeting in each setting, excluding the researchers, to participate in this paper-based survey before the meeting started (September 2018/January 2019).
One CP questionnaire per setting was developed to investigate (a) the organization and realization of planning meetings, (b) the implementation status of PA interventions, and (c) the impact of predefined determinants on CP.
(a) We developed items assessing the contribution of different stakeholder groups to the planning meetings, the planning group’s structure and organization, the reasons for participation, and the satisfaction with CP. Most items were assessed using a five-point Likert scale, and some were assessed using a nominal scale or a dichotomous format.
(b) For each PA intervention documented in the setting-specific action plan, items were formulated concerning the status of implementation as well as the fit and sustainability of the intervention. Items were assessed using either a nominal scale, a five-point Likert scale, or a dichotomous format.
(c) We developed items for each predefined determinant (excluding organizational readiness, see the aforementioned ORIC questionnaire), based on their definitions and, if available, already existing instruments (see Additional file 1). Most items were assessed using a five-point Likert scale; only items concerning the determinant champion were assessed using a dichotomous format and open questions.
Given the restrictions due to the COVID-19 pandemic in 2020, we decided to conduct the CP questionnaires online using SoSci Survey (SoSci Survey GmbH, version 3.2.12, https://www.soscisurvey.de). In September 2020, all stakeholders who had attended at least one planning meeting, excluding the researchers, were contacted via email and asked to participate in the survey by following a setting-specific link (maximum variation sampling ).
Documents: structured minutes and action plans
During the preparation and planning phases (April 2018–November 2019), structured minutes were taken of all meetings in the three settings. These contain information about the date and duration of the meetings, the participating stakeholders, and the main points raised during the discussions. At the end of the planning phase (July/November 2019), the newly developed interventions were finalized and described in one action plan per setting.
Semi-structured interviews will be conceptualized to further assess key factors influencing CP, namely the development and implementation of interventions, and the way CP was used. In addition, the interviews will allow us to clarify potential ambiguities that may arise during the analysis of the questionnaires.
We will use setting-specific timelines to illustrate the entire CP processes from 2018 to 2021 and to support the temporal classification of influencing factors and moments. Timeline interviews are primarily used in life history research to analyse how life stories are related to the broader environmental, political, and social contexts , although they have already been employed to evaluate participatory research processes . In the present evaluation, we will initially prepare one timeline template for each setting, drawing on information from structured minutes. These will include all meetings and further project-related activities in chronological order. Second, we will develop setting-specific interview guidelines by building on data collected via structured minutes, action plans, and questionnaires. The leading questions will target further influencing key factors, an overall appraisal of CP, and the clarification of inconclusive results from the questionnaires. Moreover, these questions will provide guidance to refine the setting-specific timelines, for example, by adding relevant key factors and moments that emerge during the interviews.
From November 2020 to January 2021, we will invite the main stakeholders in each setting via email to participate in an interview (purposeful sampling of information-rich cases ). One interview per participant will be conducted using online conference software, for example, Zoom (Zoom Video Communications, Inc.). Before the interviews start, the interviewees will be provided with written information about the study and will be asked to give their consent to participate.
The collected data will be analysed for each data source separately, split by settings. Data triangulation will then be used to integrate the quantitative and qualitative findings in order to answer research questions 1 and 2 . Subsequently, the findings will be compared across settings A, B, and C to identify potential differences and similarities.
Data from the ORIC questionnaire will be analysed using SPSS Statistics (IBM). First, the items will be grouped according to the domains of change commitment and change efficacy. Second, the total ORIC, commitment, and efficacy scores will be calculated and analysed descriptively.
The setting-specific CP questionnaires will be analysed using SPSS Statistics (IBM). We will generate descriptive statistics to report characteristics of the planning meetings and the developed interventions as well as the impact of predefined determinants.
Documents: structured minutes and action plans
Data from the structured minutes and action plans will be analysed descriptively by reporting the number and dates of the meetings, the number and characteristics of the involved stakeholders, and the number and content of PA interventions.
All interviews will be audio-recorded and transcribed verbatim. For reasons of anonymity, we will substitute working positions for people’s names and pseudonyms for institutions and city names. The transcripts will be analysed using qualitative content analysis, involving a deductive and inductive definition of categories . The data analysis will comprise the following steps: (1) initial text work, (2) deductive development of the main categories based on the interview guidelines, (3) coding of the entire material using the main categories, (4) compilation of all coded text passages with the same main category, (5) inductive definition of subcategories based on the transcribed material, (6) coding of the entire material using the refined categories, and (7) evaluation and interpretation . MAXQDA (VERBI Gmbh) will be used for data coding and analysis. Two researchers will develop and double-check the main categories and subcategories and apply them to the interview transcripts. Inconsistencies will be discussed and resolved within the research team. The timelines created through the interviews will be pooled to generate one timeline per setting that will include the most important project activities and impactful factors and moments. Inconsistencies will be verified via data from structured minutes or general project documentation and, if necessary, clarified by interviewees or discussed in the team.
This methodological article describes the evaluation design of a study that aims to investigate a co-creation strategy – CP – in the field of PA promotion and, specifically, its transferability to new settings as well as the associated key determinants. CP was utilized in three real-world settings to develop and implement interventions targeting the promotion of PA and PAHCO among nursing students and employees working in an assembly department. Following recommendations to provide sufficient detail for such evaluation studies (see e.g. [29, 43]), this paper reports on the co-creation strategy, the underlying logic, and the evaluation design. Below, we outline the expected implications of this study.
First, the findings of this study will contribute to a better understanding of CP used as a strategy for promoting a physically active lifestyle. In particular, it will contribute to answering questions about the transferability or scaling up of co-creation strategies and the associated key determinants. Although co-creation in health promotion is neither a novel nor unknown topic in research, these are still relevant issues that are being discussed internationally (e.g. [22, 43, 59]). This pragmatic evaluation, emphasising relevance and practicability [41, 42], builds on a comprehensive data base and will, finally, enable us to increase the knowledge base of CP and similar strategies. In addition to our focus on school and occupational environments, the study findings might also be of interest for co-creation endeavours in other contexts, their further examination, and successful application.
From a practical point of view, recommendations for research, practice, and policy might be derived from the study results. For example, a checklist to systematically assess the key determinants of CP may help implementers decide whether more support (e.g. by leadership) or resources (e.g. time) are required even before collaboration starts. Another practical contribution could be the development of a guideline that supports the transfer of CP to further settings to reach more people (see scaling up [27, 28]). Taking into account recently published research findings, such as an initial guidance for research partnerships by Hoekstra et al. , a CP guideline might include all the necessary steps and helpful advices to successfully prepare, conduct, and evaluate a CP process. We think that both a checklist of key determinants and a guideline for CP could be useful tools for researchers in this field, but also for practitioners or policy-makers who are responsible for PA promotion (e.g. in schools, companies, communities).
In addition, we anticipate a substantial contribution to research examining strategies for promoting PA in physically demanding jobs. As described above, we hypothesise that this issue requires a special, target group- and setting-centred perspective. However, as the potential of CP as a suitable strategy has already been demonstrated elsewhere , our study will provide new scientific and practical insights and could even function as a starting point for further research in this field.
In summary, the present study will likely improve the understanding of how co-creation strategies can best be applied to address PA promotion in new settings. In doing so, it will add value to science and practice concerning the scaling up and advancement of such strategies. The results of this study may be relevant for co-creation and PA-promotion researchers, but also for people from practice and policy who deliver or make decisions about PA interventions.
Availability of data and materials
The data sets generated and/or analysed during the current study, as well as the relevant research materials (e.g. questionnaires, interview guidelines), will be available from the corresponding author on reasonable request.
Organizational Readiness for Implementing Change
Physical Activity-related Health Competence
Physical Activity-related Health Competence in Apprenticeship and Vocational Education
Morris JN, Heady JA, Raffle PAB, Roberts CG, Parks JW. Coronary heart-disease and physical activity of work. Lancet. 1953;262:1111–20. https://doi.org/10.1016/S0140-6736(53)91495-0.
Lee I-M, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet. 2012;380:219–29. https://doi.org/10.1016/S0140-6736(12)61031-9.
Warburton DER, Bredin SSD. Health benefits of physical activity: a systematic review of current systematic reviews. Curr Opin Cardiol. 2017;32:541–56. https://doi.org/10.1097/HCO.0000000000000437.
Pedersen BK. The physiology of optimizing health with a focus on exercise as medicine. Annu Rev Physiol. 2019;81:607–27. https://doi.org/10.1146/annurev-physiol-020518-114339.
Pronk NP, Kottke TE. Physical activity promotion as a strategic corporate priority to improve worker health and business performance. Prev Med. 2009;49:316–21. https://doi.org/10.1016/j.ypmed.2009.06.025.
Gardner B, Smith L, Lorencatto F, Hamer M, Biddle SJH. How to reduce sitting time? A review of behaviour change strategies used in sedentary behaviour reduction interventions among adults. Health Psychol Rev. 2016;10:89–112. https://doi.org/10.1080/17437199.2015.1082146.
Stamatakis E, Ekelund U, Ding D, Hamer M, Bauman AE, Lee I-M. Is the time right for quantitative public health guidelines on sitting? A narrative review of sedentary behaviour research paradigms and findings. Br J Sports Med. 2019;53:377–82. https://doi.org/10.1136/bjsports-2018-099131.
Ross R, Chaput J-P, Giangregorio LM, Janssen I, Saunders TJ, Kho ME, et al. Canadian 24-hour movement guidelines for adults aged 18–64 years and adults aged 65 years or older: an integration of physical activity, sedentary behaviour, and sleep. Appl Physiol Nutr Metab. 2020;45:S57–102. https://doi.org/10.1139/apnm-2020-0467.
Pate RR. The report of the US Physical Activity Guidelines Advisory Committee: important findings for employers. Am J Health Promot. 2019;33:313–4. https://doi.org/10.1177/0890117118816750b.
Nutbeam D. Health promotion glossary. Health Promot Int. 1998;1:349–64.
Holtermann A, Krause N, van der Beek AJ, Straker L. The physical activity paradox: six reasons why occupational physical activity (OPA) does not confer the cardiovascular health benefits that leisure time physical activity does. Br J Sports Med. 2018;52:149–50. https://doi.org/10.1136/bjsports-2017-097965.
Cillekens B, Lang M, van Mechelen W, Verhagen E, Huysmans MA, Holtermann A, et al. How does occupational physical activity influence health? An umbrella review of 23 health outcomes across 158 observational studies. Br J Sports Med. 2020;54:1474–81. https://doi.org/10.1136/bjsports-2020-102587.
Carl J, Grüne E, Popp J, Pfeifer K. Physical activity promotion for apprentices in nursing care and automotive mechatronics—competence counts more than volume. Int J Environ Res Public Health. 2020;17:793. https://doi.org/10.3390/ijerph17030793.
Bauman AE, Reis RS, Sallis JF, Wells JC, Loos RJF, Martin BW. Correlates of physical activity: why are some people physically active and others not? Lancet. 2012;380:258–71. https://doi.org/10.1016/S0140-6736(12)60735-1.
Kahn-Marshall JL, Gallant MP. Making healthy behaviors the easy choice for employees: a review of the literature on environmental and policy changes in worksite health promotion. Health Educ Behav. 2012. https://doi.org/10.1177/1090198111434153.
Beckett K, Farr M, Kothari A, Wye L, Le May A. Embracing complexity and uncertainty to create impact: exploring the processes and transformative potential of co-produced research through development of a social impact model. Health Res Policy Syst. 2018;16:118. https://doi.org/10.1186/s12961-018-0375-0.
Graham ID, McCutcheon C, Kothari A. Exploring the frontiers of research co-production: the Integrated Knowledge Translation Research Network concept papers. Health Res Policy Syst. 2019;17:88. https://doi.org/10.1186/s12961-019-0501-7.
Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5:2. https://doi.org/10.1186/s40900-018-0136-9.
McConnell T, Best P, Davidson G, McEneaney T, Cantrell C, Tully M. Coproduction for feasibility and pilot randomised controlled trials: learning outcomes for community partners, service users and the research team. Res Involv Engagem. 2018;4:32. https://doi.org/10.1186/s40900-018-0116-0.
Leask CF, Sandlund M, Skelton DA, Chastin SFM. Co-creating a tailored public health intervention to reduce older adults’ sedentary behaviour. Health Educ J. 2017;76:595–608. https://doi.org/10.1177/0017896917707785.
Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012;90:311–46. https://doi.org/10.1111/j.1468-0009.2012.00665.x.
Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94:392–429. https://doi.org/10.1111/1468-0009.12197.
Rütten A. Kooperative Planung und Gesundheitsförderung: Ein Implementationsansatz [Cooperative planning and health promotion: an implementation approach]. J Public Health. 1997;5:257–72. https://doi.org/10.1007/BF02956335.
Sudeck G, Pfeifer K. Physical activity-related health competence as an integrative objective in exercise therapy and health sports—conception and validation of a short questionnaire. German J Exerc Sport Res. 2016;46:74–87. https://doi.org/10.1007/s12662-016-0405-4.
Carl J, Sudeck G, Pfeifer K. Competencies for a healthy physically active lifestyle—reflections on the model of physical activity-related health competence. J Phys Activity Health. 2020;17:688–97. https://doi.org/10.1123/jpah.2019-0442.
Popp J, Carl J, Grüne E, Semrau J, Gelius P, Pfeifer K. Physical activity promotion in German vocational education: does capacity building work? Health Promot Int. 2020;35:1577–89. https://doi.org/10.1093/heapro/daaa014.
World Health Organization and ExpandNet. Nine steps for developing a scaling-up strategy. Geneva: World Health Organization; 2010.
Reis RS, Salvo D, Ogilvie D, Lambert EV, Goenka S, Brownson RC. Scaling up physical activity interventions worldwide: stepping up to larger and smarter approaches to get people moving. Lancet. 2016;388:1337–48. https://doi.org/10.1016/S0140-6736(16)30728-0.
Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci. 2016;11:38. https://doi.org/10.1186/s13012-016-0399-1.
Frahsa A, Rütten A, Roeger U, Abu-Omar K, Schow D. Enabling the powerful? Participatory action research with local policymakers and professionals for physical activity promotion with women in difficult life situations. Health Promot Int. 2012;29:171–84. https://doi.org/10.1093/heapro/das050.
Rütten A, Wolff A, Streber A. Interaktiver Wissenstransfer in der Gesundheitsförderung: das GESTALT-Projekt. Erste Ergebnisse der Erprobung eines Ansatzes zur nachhaltigen Implementierung evidenzbasierter Bewegungsprogramme [Interactive knowledge to action in health promotion: the GESTALT project. Initial results of a pilot study on sustainable implementation of an evidence-based programme]. Gesundheitswesen. 2016;78:359–66. https://doi.org/10.1055/s-0035-1548882.
Loss J, Brew-Sam N, Metz B, Strobl H, Sauter A, Tittlbach S. Capacity building in community stakeholder groups for increasing physical activity: results of a qualitative study in two German communities. Int J Environ Res Public Health. 2020;17:2306. https://doi.org/10.3390/ijerph17072306.
Strobl H, Ptack K, Töpfer C, Sygusch R, Tittlbach S. Effects of a participatory school-based intervention on students’ health-related knowledge and understanding. Front Public Health. 2020;8:122. https://doi.org/10.3389/fpubh.2020.00122.
Hoekstra F, Mrklas KJ, Khan M, McKay RC, Vis-Dunbar M, Sibley KM, et al. A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature. Health Res Policy Syst. 2020;18:51. https://doi.org/10.1186/s12961-020-0544-9.
Verloigne M, Altenburg TM, Chinapaw MJM, Chastin S, Cardon G, de Bourdeaudhuij I. Using a co-creational approach to develop, implement and evaluate an intervention to promote physical activity in adolescent girls from vocational and technical schools: a case control study. Int J Environ Res Public Health. 2017;14:862. https://doi.org/10.3390/ijerph14080862.
Flinders M, Wood M, Cunningham M. The politics of co-production: risks, limits and pollution. Evid Policy J Res Debate Pract. 2016;12:261–79. https://doi.org/10.1332/174426415X14412037949967.
Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Syst. 2019;17:33. https://doi.org/10.1186/s12961-019-0432-3.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. https://doi.org/10.1136/bmj.a1655.
Campbell H, Vanderhoven D. Knowledge that matters: realising the potential of co-production. Manchester: N8 Research Partnership; 2016. http://www.n8research.org.uk/research-focus/urban-and-community-transformation/co-production/. Accessed 9 Oct 2017. Manchester; 2016.
Bamberger M, Rugh J, Mabry L. RealWorld evaluation: working under budget, time, data, and political constraints. 2nd ed. Thousand Oaks: Sage; 2012.
Crane M, Bauman A, Lloyd B, McGill B, Rissel C, Grunseit A. Applying pragmatic approaches to complex program evaluation: a case study of implementation of the New South Wales Get Healthy at Work program. Health Promot J Austr. 2019;30:422–32. https://doi.org/10.1002/hpja.239.
Malterud K, Aamland A, Iden KR. Small-scale implementation with pragmatic process evaluation: a model developed in primary health care. BMC Fam Pract. 2018;19:93. https://doi.org/10.1186/s12875-018-0778-6.
Gagliardi AR, Kothari A, Graham ID. Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know. J Epidemiol Community Health. 2017;71:105–6. https://doi.org/10.1136/jech-2016-207743.
Fynn JF, Hardeman W, Milton K, Murphy J, Jones A. A systematic review of the use and reporting of evaluation frameworks within evaluations of physical activity interventions. Int J Behav Nutr Phys Act. 2020;17:107. https://doi.org/10.1186/s12966-020-01013-7.
Rütten A, Gelius P. Building policy capacities: an interactive approach for linking knowledge to action in health promotion. Health Promot Int. 2013;29:569–82. https://doi.org/10.1093/heapro/dat006.
Cooksy LJ, Gill P, Kelly PA. The program logic model as an integrative framework for a multimethod evaluation. Eval Program Plan. 2001;24:119–28. https://doi.org/10.1016/S0149-7189(01)00003-9.
W.K. Kellogg Foundation. Using logic models to bring together planning, evaluation, and action: logic model development guide. Battle Creek, Michigan: W.K. Kellogg Foundation; 2004.
McLaughlin JA, Jordan GB. Using logic models. In: Newcomer KE, Hatry HP, Wholey JS, editors. Handbook of practical program evaluation. Hoboken: Jossey-Bass, A Wiley imprint; 2015. p. 62–87. https://doi.org/10.1002/9781119171386.ch3.
Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd ed. Los Angeles, London, New Delhi, Singapore, Washington DC: Sage; 2011.
O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341:c4587. https://doi.org/10.1136/bmj.c4587.
Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9:7. https://doi.org/10.1186/1748-5908-9-7.
Maneesriwongul W, Dixon JK. Instrument translation process: a methods review. J Adv Nurs. 2004;48:175–86. https://doi.org/10.1111/j.1365-2648.2004.03185.x.
Lindig A, Hahlweg P, Christalle E, Scholl I. Translation and psychometric evaluation of the German version of the Organisational Readiness for Implementing Change measure (ORIC): a cross-sectional study. BMJ open. 2020;10:e034380. https://doi.org/10.1136/bmjopen-2019-034380.
Weiner BJ, Amick H, Lee SYD. Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65:379–436. https://doi.org/10.1177/1077558708317802.
Patton MQ. Qualitative research and evaluation methods. 4th ed. Los Angeles: Sage; 2014.
Adriansen HK. Timeline interviews: a tool for conducting life history research. QS. 2012;3:40–55. https://doi.org/10.7146/qs.v3i1.6272.
Hoekstra F, Martin Ginis KA, Allan V, Kothari A, Gainforth HL. Evaluating the impact of a network of research partnerships: a longitudinal multiple case study protocol. Health Res Policy Syst. 2018;16:107. https://doi.org/10.1186/s12961-018-0377-y.
Kuckartz U. Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung [Qualitative content analysis: methods, practice, computer support]. 3rd ed. Weinheim: Beltz Juventa; 2016.
Heaton J, Day J, Britten N. Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implement Sci. 2016;11:20. https://doi.org/10.1186/s13012-016-0383-9.
Open Access funding enabled and organized by Projekt DEAL. The research network Capital4Health (http://www.capital4health.de/en) and the associated PArC-AVE Project, in which the study is conducted, are funded by the German Federal Ministry of Education and Research [grant number 01EL1821A].
Ethics approval and consent to participate
Ethical approval for research within the PArC-AVE Project was granted by the Ethical Committee of the Friedrich-Alexander University Erlangen-Nürnberg [January 15, 2019; sign 467_18 B]. Participants in this study will be asked to provide written or verbal consent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Popp, J., Grüne, E., Carl, J. et al. Co-creating physical activity interventions: a mixed methods evaluation approach. Health Res Policy Sys 19, 37 (2021). https://doi.org/10.1186/s12961-021-00699-w