Skip to main content

Co-creating physical activity interventions: a mixed methods evaluation approach

Abstract

Background

Co-creation strategies, such as cooperative planning, are promising as a means to ensure that physical activity interventions address real-world problems and are tailored to the target group. This has already been validated in diverse settings. However, questions targeting the transferability of cooperative planning to new settings and the key factors influencing its success or failure remain unclear. At the same time, co-creation processes are complex, and evaluation can be challenging. Following calls for detailed reporting, this paper describes the programme activities, the underlying logic, and methodological design of a study that aims to evaluate the transfer of cooperative planning to new settings and to explore the associated key determinants.

Methods

Cooperative planning was utilized as a strategy to target physical activity promotion in three real-world German settings in the nursing care and automotive mechatronics sectors. This involved researchers working alongside stakeholders from practice and policy to conjointly develop new interventions to promote physical activity in physically demanding jobs. A pragmatic approach is used to evaluate both the transferability and key determinants of this strategy. We developed a logic model for this co-creation process that describes the underlying assumptions and guides the evaluation. The evaluation outcomes of this study include planning meetings, newly developed interventions, and the determinants that are likely to affect cooperative planning. Quantitative and qualitative data will be collected using questionnaires, documents, and interviews. The quantitative data will be analysed descriptively, while the qualitative data will mainly be analysed using qualitative content analysis, split by settings. Subsequently, data triangulation will be used to integrate the quantitative and qualitative findings, which will then be compared across all three settings.

Discussion

The study findings will contribute to a better understanding of co-creation strategies, their transferability, and key determinants. The practical implications can include a checklist for assessing key determinants and a guideline for transferring cooperative planning into new settings to benefit more people. Ultimately, this study will help to advance co-creation strategies and may be relevant for researchers, practitioners, and policy-makers targeting physical activity promotion in various contexts.

Trial registration: Open Science Framework: https://osf.io/r6xnt/ (retrospectively registered).

Peer Review reports

Background

Since the early years of health-related physical activity (PA) research [1], there has been a growing body of literature providing strong evidence regarding the beneficial effects of PA on an individual’s health [2, 3]. In accordance with the position that PA may work as medicine, a special focus should be put on population groups who are exposed to increased health risks and who may particularly benefit from the positive molecular and physiological effects of PA [4]. Even though work has entered the digital and automated era and a reduction in sedentary time has recently become a top priority in PA promotion [5,6,7,8], we must not lose sight of occupational groups who experience high physical demands during working life [9]. In line with the “settings for health” defined by the World Health Organization ([10], p. 362), work sites and schools appear to be favourable settings to promote health and PA by reaching people who work and learn there.

However, two aspects need to be considered in this context. First, when we aim to promote PA in professions with a high physical workload, the special demands of the occupational group must be taken into account. For example, there are indications that a high level of occupational PA is associated with a number of unfavourable health outcomes [11, 12]. In light of this, PA promotion among people with physically demanding jobs should not focus solely on increasing PA levels; the emphasis should be placed on strengthening competencies that are essential to master PAs in a healthy manner [13]. Second, the surrounding structures must be considered, as research has shown that supportive environments and policies are important for health behaviour changes [14, 15]. Taken together, it seems imperative to involve the target group and other nonacademic stakeholders as the experts in their settings [16,17,18] to identify appropriate PA interventions in physically demanding work settings.

Against this backdrop, co-creation, defined as “collaborative public health intervention development by academics working alongside other stakeholders” ([18], p. 2), is promising in terms of creating PA interventions. Involving different stakeholders in such strategies has the potential to systematically address real-world problems [19], to develop interventions that are tailored to the end user [18, 20], and to achieve sustainable outputs and impact [16, 21, 22]. The German research project PArC-AVE (Physical Activity-related Health Competence in Apprenticeship and Vocational Education) utilizes a co-creation strategy called cooperative planning (CP) [23] to develop interventions aimed to promote PA in the nursing care and automotive mechatronics sectors – two physically demanding professions. The goal of CP is to achieve changes at both the structural and individual levels that facilitate the adoption and/or maintenance of a physically active and healthy lifestyle; more specifically, by creating PA-friendly structures and strengthening Physical Activity-related Health Competence (PAHCO) [24, 25] among individuals. Previous research findings have shown promising results from CP in a private vocational education centre for health professions and in a vocational education centre of a German automotive manufacturer [26]. In both settings, new PA interventions were developed and implemented, and several capacities for PA promotion were built up. Thus far, it remains unclear whether CP can successfully be transferred to other settings to reach more people in the nursing care and automotive mechatronics sectors (in the sense of “scaling up” [27, 28]). Beyond that, the aforementioned study [26] identified differences between the nursing and automotive settings, mainly concerning the active participation of the target group and facilitators as well as barriers to intervention development and implementation (e.g. financial resources, organizational support). Although these differences have not been investigated in detail, we nevertheless note the importance of influencing factors, such as the attitudes of the involved stakeholders, support from organizations or individuals, and financial resources (see also [29]). This raises the question of which determinants are key enablers of and barriers to CP and thus need to be considered in future PA-promotion programmes.

These unanswered questions targeting the transferability and relevant determinants of CP gain more weight in light of the increasing body of studies using CP [30,31,32,33] or similar strategies in PA promotion and health promotion [22, 34, 35]; not forgetting the critical voices discussing the limitations and challenges of co-creation strategies [36, 37]. However, co-creating new interventions to promote PA is a complex process due to the high number of involved interest groups, the variability of outcomes, and the flexibility of the process [16, 38, 39]. Moreover, practical challenges for evaluation may arise from the real-world settings in which research in the PArC-AVE Project is conducted [40, 41]. Hence, this study uses a pragmatic approach, which seems favourable for the evaluation of complex programmes in real-world settings [41, 42]. We will build on the most relevant and best available data sources to gain a better understanding of how CP can be transferred to new settings and the factors linked to success or failure.

The current paper describes in detail the evaluation design of a study that aims to investigate the transferability of CP and to explore the key determinants for developing and implementing PA interventions. Following calls for more precise reporting of such studies (e.g. [29, 38, 43, 44]), we provide a thorough description of the activities that were undertaken, the underlying programme logic, and the evaluation design. Ultimately, this study will help to increase the knowledge base of co-creation strategies in the field of PA promotion and to foster their scientific as well as practical advancement.

Methods

Study settings and co-creation strategy

Building on the first research findings from 2015 to 2018 [26], the PArC-AVE Project aims at transferring CP as a co-creation strategy into three new settings in the nursing care and automotive mechatronics sectors from 2018 to 2021. The new settings are a state vocational education centre for health professions in a medium-sized city (setting A: 200 nursing students enrolled in the nursing programme), a state vocational education centre for health professions in a large city (setting B: 180 nursing students enrolled in the nursing programme), and the assembly department of a German automotive manufacturer (setting C: 12 000 employees in the assembly department), all located in Bavaria, Germany. In all three settings, the overall goal is to conjointly develop new PA interventions.

Separate CP processes were undertaken in each setting (setting A, setting B, setting C), including the preparation, planning, and implementation phases [26, 45]. Figure 1 provides a summary overview of these processes. After the initial cooperation requests to settings A, B, and C (November 2016) and the subsequent positive funding decision (February 2018), the activities started in spring 2018. During the preparation phase, the initial meetings were used to inform the project partners about the project idea and to identify setting-specific structures, processes, and the relevant stakeholders (April/May 2018). A detailed explorative situation analysis was conducted in all settings to collect more information about the context and the needs of the target groups (April–July 2018). This was mainly done during several visits by the researchers to each setting and the utilization of a criteria catalogue, based on the ecological model of Bauman et al. [14] and on earlier findings [26], to collect the relevant information. In the planning phase, planning meetings were held to develop new PA interventions by a planning group of different stakeholders from research, practice, and policy (September 2018–December 2019). For further information on the phases and common rules of the planning meetings as well as the specific roles of the participants, see the description of earlier CP processes ([26], p. 1580–1581, Table 1). Finally, a set of new interventions was finalized in an action plan for each setting. In the implementation phase, the interventions were implemented under the guidance of the practitioners, starting after the last planning meetings (from July/November 2019).

Fig. 1
figure1

Overview of cooperative planning within the PArC-AVE Project (summary of settings A, B, and C). CP cooperative planning, ORIC Organizational Readiness for Implementing Change

Logic model

When evaluating such processes, logic models are a valuable first step, describing the potential mechanisms of how a programme works [46, 47]. Moreover, they have recently been identified as a potential framework for the evaluation of PA interventions in a systematic review by Fynn et al. [44]. In the present study, we developed a logic model to illustrate the planned activities and expected effects within the PArC-AVE Project and, in the next step, to guide the evaluation of CP and its determinants (Fig. 2). Usually, a logic model includes inputs, activities, outputs, and outcome components [47]; it can also include contextual factors that are expected to influence the success of the programme [48]. In our case, inputs can be defined as the resources provided by researchers, practitioners, and policy-makers to realize cooperation and planning meetings in a particular setting, such as human, financial, and organizational resources. Activities are all project meetings and visits in settings A, B, and C – most importantly, the planning meetings involving stakeholders from research, practice, and policy with the overall aim of developing PA interventions. Outputs, as the direct products of programme activities, are the developed interventions, documented in one action plan per setting. In accordance with the goal of CP including structural and individual changes, we differentiate between outcomes at the structural and individual levels. An outcome at the structural level is the implementation of the developed PA interventions; at the individual level, it is a change in the target group’s PA behaviour, PAHCO, and/or health status. Contextual factors are defined as the determinants that are likely to influence a CP process and its success or failure. Based on earlier findings within the project [26] and a literature screening of factors relevant to co-creation processes and the implementation of co-created interventions in January/February 2020, we identified the following determinants to be considered in the present evaluation: champion, commitment, empowerment, engagement, group effectiveness, leadership, organizational culture, organizational readiness, ownership, and resources. An overview of these predefined determinants and their definitions is provided as an additional file (see Additional file 1).

Fig. 2
figure2

Logic model of the PArC-AVE Project. PA physical activity, PAHCO Physical Activity-related Health Competence

Research questions and evaluation design

Two core research questions were identified to address the aim of this study:

Research question 1: How (un)successful is the transfer of cooperative planning to other settings?

Research question 2: Which key factors have an influence on the success or failure of cooperative planning?

The evaluation outcomes were subsequently derived from our logic model (see Table 1). To answer the first research question, planning meetings and the developed interventions will function as evaluation outcomes; measuring outcomes at the individual level (i.e. PA behaviour, PAHCO, health status) in a pre-post design was not possible in this study due to the early practitioner-initiated implementation of interventions. We will thus refer to the logic model components of activities, outputs, and structural outcomes to assess the success or failure of CP in settings A, B, and C. To answer the second research question, predefined and further determinants will function as evaluation outcomes, referring to the logic model component of contextual factors. Building on the differences between settings found in earlier research, we will contrast settings A, B, and C to examine potential differences and similarities. This will be followed by a final assessment of the success/failure of transferred CP and its relationships with the key determinants.

Table 1 Evaluation outcomes and data sources

Inspired by the principles of a pragmatic evaluation [41], this study uses a mixed methods design to evaluate the transfer of CP and its determinants across three different settings [49]. Combining different methods and triangulating the quantitative and qualitative data will, finally, enable us to adequately answer both research questions [50].

Data collection

The data will be collected using the following sources (quan = quantitative; qual = qualitative): questionnaires (quan), structured minutes (qual), action plans (qual), and interviews (qual). During the writing and submission process of this paper, the data collection already started. The moments of measurement are indicated for each data source in Fig. 1 and in the following text. An overview of the evaluation outcomes, data sources, and study participants split by research questions is depicted in Table 1.

Quantitative data

ORIC questionnaire

The predefined determinant of organizational readiness was assessed using the Organizational Readiness for Implementing Change (ORIC) questionnaire by Shea et al. [51], which has been shown to be a reliable and valid instrument of organizational readiness for change. This questionnaire contains 12 items using a five-point Likert scale to assess the domains of change commitment and change efficacy. As we could not find a German version during the preparation phase in 2018, we translated the questionnaire using forward and back translation with monolingual tests [52]. The forward translation was done by a native German speaker and the back translation was done by a native English speaker. Afterwards, three researchers validated and discussed both versions to refine the ORIC questionnaire in the German language. We also added an introductory description to explain the meaning of the term “change” in the given context, which in our case, are the changes at the organizational level concerning PA promotion that result from the research project. Yet, for future studies of organizational readiness in the German language, it should be noted that a German version of the ORIC questionnaire has recently been tested and published by Lindig et al. [53].

According to Weiner et al. [54], the assessment of organizational readiness should take place before the process of change begins. Utilizing a maximum variation sampling scheme [55], we asked all stakeholders who attended the first planning meeting in each setting, excluding the researchers, to participate in this paper-based survey before the meeting started (September 2018/January 2019).

CP questionnaires

One CP questionnaire per setting was developed to investigate (a) the organization and realization of planning meetings, (b) the implementation status of PA interventions, and (c) the impact of predefined determinants on CP.

  • (a) We developed items assessing the contribution of different stakeholder groups to the planning meetings, the planning group’s structure and organization, the reasons for participation, and the satisfaction with CP. Most items were assessed using a five-point Likert scale, and some were assessed using a nominal scale or a dichotomous format.

  • (b) For each PA intervention documented in the setting-specific action plan, items were formulated concerning the status of implementation as well as the fit and sustainability of the intervention. Items were assessed using either a nominal scale, a five-point Likert scale, or a dichotomous format.

  • (c) We developed items for each predefined determinant (excluding organizational readiness, see the aforementioned ORIC questionnaire), based on their definitions and, if available, already existing instruments (see Additional file 1). Most items were assessed using a five-point Likert scale; only items concerning the determinant champion were assessed using a dichotomous format and open questions.

Given the restrictions due to the COVID-19 pandemic in 2020, we decided to conduct the CP questionnaires online using SoSci Survey (SoSci Survey GmbH, version 3.2.12, https://www.soscisurvey.de). In September 2020, all stakeholders who had attended at least one planning meeting, excluding the researchers, were contacted via email and asked to participate in the survey by following a setting-specific link (maximum variation sampling [55]).

Qualitative data

Documents: structured minutes and action plans

During the preparation and planning phases (April 2018–November 2019), structured minutes were taken of all meetings in the three settings. These contain information about the date and duration of the meetings, the participating stakeholders, and the main points raised during the discussions. At the end of the planning phase (July/November 2019), the newly developed interventions were finalized and described in one action plan per setting.

Interviews

Semi-structured interviews will be conceptualized to further assess key factors influencing CP, namely the development and implementation of interventions, and the way CP was used. In addition, the interviews will allow us to clarify potential ambiguities that may arise during the analysis of the questionnaires.

We will use setting-specific timelines to illustrate the entire CP processes from 2018 to 2021 and to support the temporal classification of influencing factors and moments. Timeline interviews are primarily used in life history research to analyse how life stories are related to the broader environmental, political, and social contexts [56], although they have already been employed to evaluate participatory research processes [57]. In the present evaluation, we will initially prepare one timeline template for each setting, drawing on information from structured minutes. These will include all meetings and further project-related activities in chronological order. Second, we will develop setting-specific interview guidelines by building on data collected via structured minutes, action plans, and questionnaires. The leading questions will target further influencing key factors, an overall appraisal of CP, and the clarification of inconclusive results from the questionnaires. Moreover, these questions will provide guidance to refine the setting-specific timelines, for example, by adding relevant key factors and moments that emerge during the interviews.

From November 2020 to January 2021, we will invite the main stakeholders in each setting via email to participate in an interview (purposeful sampling of information-rich cases [55]). One interview per participant will be conducted using online conference software, for example, Zoom (Zoom Video Communications, Inc.). Before the interviews start, the interviewees will be provided with written information about the study and will be asked to give their consent to participate.

Data analysis

The collected data will be analysed for each data source separately, split by settings. Data triangulation will then be used to integrate the quantitative and qualitative findings in order to answer research questions 1 and 2 [50]. Subsequently, the findings will be compared across settings A, B, and C to identify potential differences and similarities.

Quantitative data

ORIC questionnaire

Data from the ORIC questionnaire will be analysed using SPSS Statistics (IBM). First, the items will be grouped according to the domains of change commitment and change efficacy. Second, the total ORIC, commitment, and efficacy scores will be calculated and analysed descriptively.

CP questionnaires

The setting-specific CP questionnaires will be analysed using SPSS Statistics (IBM). We will generate descriptive statistics to report characteristics of the planning meetings and the developed interventions as well as the impact of predefined determinants.

Qualitative data

Documents: structured minutes and action plans

Data from the structured minutes and action plans will be analysed descriptively by reporting the number and dates of the meetings, the number and characteristics of the involved stakeholders, and the number and content of PA interventions.

Interviews

All interviews will be audio-recorded and transcribed verbatim. For reasons of anonymity, we will substitute working positions for people’s names and pseudonyms for institutions and city names. The transcripts will be analysed using qualitative content analysis, involving a deductive and inductive definition of categories [58]. The data analysis will comprise the following steps: (1) initial text work, (2) deductive development of the main categories based on the interview guidelines, (3) coding of the entire material using the main categories, (4) compilation of all coded text passages with the same main category, (5) inductive definition of subcategories based on the transcribed material, (6) coding of the entire material using the refined categories, and (7) evaluation and interpretation [58]. MAXQDA (VERBI Gmbh) will be used for data coding and analysis. Two researchers will develop and double-check the main categories and subcategories and apply them to the interview transcripts. Inconsistencies will be discussed and resolved within the research team. The timelines created through the interviews will be pooled to generate one timeline per setting that will include the most important project activities and impactful factors and moments. Inconsistencies will be verified via data from structured minutes or general project documentation and, if necessary, clarified by interviewees or discussed in the team.

Discussion

This methodological article describes the evaluation design of a study that aims to investigate a co-creation strategy – CP – in the field of PA promotion and, specifically, its transferability to new settings as well as the associated key determinants. CP was utilized in three real-world settings to develop and implement interventions targeting the promotion of PA and PAHCO among nursing students and employees working in an assembly department. Following recommendations to provide sufficient detail for such evaluation studies (see e.g. [29, 43]), this paper reports on the co-creation strategy, the underlying logic, and the evaluation design. Below, we outline the expected implications of this study.

First, the findings of this study will contribute to a better understanding of CP used as a strategy for promoting a physically active lifestyle. In particular, it will contribute to answering questions about the transferability or scaling up of co-creation strategies and the associated key determinants. Although co-creation in health promotion is neither a novel nor unknown topic in research, these are still relevant issues that are being discussed internationally (e.g. [22, 43, 59]). This pragmatic evaluation, emphasising relevance and practicability [41, 42], builds on a comprehensive data base and will, finally, enable us to increase the knowledge base of CP and similar strategies. In addition to our focus on school and occupational environments, the study findings might also be of interest for co-creation endeavours in other contexts, their further examination, and successful application.

From a practical point of view, recommendations for research, practice, and policy might be derived from the study results. For example, a checklist to systematically assess the key determinants of CP may help implementers decide whether more support (e.g. by leadership) or resources (e.g. time) are required even before collaboration starts. Another practical contribution could be the development of a guideline that supports the transfer of CP to further settings to reach more people (see scaling up [27, 28]). Taking into account recently published research findings, such as an initial guidance for research partnerships by Hoekstra et al. [34], a CP guideline might include all the necessary steps and helpful advices to successfully prepare, conduct, and evaluate a CP process. We think that both a checklist of key determinants and a guideline for CP could be useful tools for researchers in this field, but also for practitioners or policy-makers who are responsible for PA promotion (e.g. in schools, companies, communities).

In addition, we anticipate a substantial contribution to research examining strategies for promoting PA in physically demanding jobs. As described above, we hypothesise that this issue requires a special, target group- and setting-centred perspective. However, as the potential of CP as a suitable strategy has already been demonstrated elsewhere [26], our study will provide new scientific and practical insights and could even function as a starting point for further research in this field.

In summary, the present study will likely improve the understanding of how co-creation strategies can best be applied to address PA promotion in new settings. In doing so, it will add value to science and practice concerning the scaling up and advancement of such strategies. The results of this study may be relevant for co-creation and PA-promotion researchers, but also for people from practice and policy who deliver or make decisions about PA interventions.

Availability of data and materials

The data sets generated and/or analysed during the current study, as well as the relevant research materials (e.g. questionnaires, interview guidelines), will be available from the corresponding author on reasonable request.

Abbreviations

CP:

cooperative planning

ORIC:

Organizational Readiness for Implementing Change

PA:

physical activity

PAHCO:

Physical Activity-related Health Competence

PArC-AVE:

Physical Activity-related Health Competence in Apprenticeship and Vocational Education

qual:

qualitative

quan:

quantitative

References

  1. 1.

    Morris JN, Heady JA, Raffle PAB, Roberts CG, Parks JW. Coronary heart-disease and physical activity of work. Lancet. 1953;262:1111–20. https://doi.org/10.1016/S0140-6736(53)91495-0.

    CAS  Article  PubMed  Google Scholar 

  2. 2.

    Lee I-M, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet. 2012;380:219–29. https://doi.org/10.1016/S0140-6736(12)61031-9.

    Article  PubMed  PubMed Central  Google Scholar 

  3. 3.

    Warburton DER, Bredin SSD. Health benefits of physical activity: a systematic review of current systematic reviews. Curr Opin Cardiol. 2017;32:541–56. https://doi.org/10.1097/HCO.0000000000000437.

    Article  PubMed  Google Scholar 

  4. 4.

    Pedersen BK. The physiology of optimizing health with a focus on exercise as medicine. Annu Rev Physiol. 2019;81:607–27. https://doi.org/10.1146/annurev-physiol-020518-114339.

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Pronk NP, Kottke TE. Physical activity promotion as a strategic corporate priority to improve worker health and business performance. Prev Med. 2009;49:316–21. https://doi.org/10.1016/j.ypmed.2009.06.025.

    Article  PubMed  Google Scholar 

  6. 6.

    Gardner B, Smith L, Lorencatto F, Hamer M, Biddle SJH. How to reduce sitting time? A review of behaviour change strategies used in sedentary behaviour reduction interventions among adults. Health Psychol Rev. 2016;10:89–112. https://doi.org/10.1080/17437199.2015.1082146.

    Article  PubMed  Google Scholar 

  7. 7.

    Stamatakis E, Ekelund U, Ding D, Hamer M, Bauman AE, Lee I-M. Is the time right for quantitative public health guidelines on sitting? A narrative review of sedentary behaviour research paradigms and findings. Br J Sports Med. 2019;53:377–82. https://doi.org/10.1136/bjsports-2018-099131.

    Article  PubMed  Google Scholar 

  8. 8.

    Ross R, Chaput J-P, Giangregorio LM, Janssen I, Saunders TJ, Kho ME, et al. Canadian 24-hour movement guidelines for adults aged 18–64 years and adults aged 65 years or older: an integration of physical activity, sedentary behaviour, and sleep. Appl Physiol Nutr Metab. 2020;45:S57–102. https://doi.org/10.1139/apnm-2020-0467.

    Article  PubMed  Google Scholar 

  9. 9.

    Pate RR. The report of the US Physical Activity Guidelines Advisory Committee: important findings for employers. Am J Health Promot. 2019;33:313–4. https://doi.org/10.1177/0890117118816750b.

    Article  PubMed  Google Scholar 

  10. 10.

    Nutbeam D. Health promotion glossary. Health Promot Int. 1998;1:349–64.

    Article  Google Scholar 

  11. 11.

    Holtermann A, Krause N, van der Beek AJ, Straker L. The physical activity paradox: six reasons why occupational physical activity (OPA) does not confer the cardiovascular health benefits that leisure time physical activity does. Br J Sports Med. 2018;52:149–50. https://doi.org/10.1136/bjsports-2017-097965.

    Article  PubMed  Google Scholar 

  12. 12.

    Cillekens B, Lang M, van Mechelen W, Verhagen E, Huysmans MA, Holtermann A, et al. How does occupational physical activity influence health? An umbrella review of 23 health outcomes across 158 observational studies. Br J Sports Med. 2020;54:1474–81. https://doi.org/10.1136/bjsports-2020-102587.

    Article  PubMed  Google Scholar 

  13. 13.

    Carl J, Grüne E, Popp J, Pfeifer K. Physical activity promotion for apprentices in nursing care and automotive mechatronics—competence counts more than volume. Int J Environ Res Public Health. 2020;17:793. https://doi.org/10.3390/ijerph17030793.

    Article  PubMed Central  Google Scholar 

  14. 14.

    Bauman AE, Reis RS, Sallis JF, Wells JC, Loos RJF, Martin BW. Correlates of physical activity: why are some people physically active and others not? Lancet. 2012;380:258–71. https://doi.org/10.1016/S0140-6736(12)60735-1.

    Article  PubMed  Google Scholar 

  15. 15.

    Kahn-Marshall JL, Gallant MP. Making healthy behaviors the easy choice for employees: a review of the literature on environmental and policy changes in worksite health promotion. Health Educ Behav. 2012. https://doi.org/10.1177/1090198111434153.

    Article  PubMed  Google Scholar 

  16. 16.

    Beckett K, Farr M, Kothari A, Wye L, Le May A. Embracing complexity and uncertainty to create impact: exploring the processes and transformative potential of co-produced research through development of a social impact model. Health Res Policy Syst. 2018;16:118. https://doi.org/10.1186/s12961-018-0375-0.

    Article  PubMed  PubMed Central  Google Scholar 

  17. 17.

    Graham ID, McCutcheon C, Kothari A. Exploring the frontiers of research co-production: the Integrated Knowledge Translation Research Network concept papers. Health Res Policy Syst. 2019;17:88. https://doi.org/10.1186/s12961-019-0501-7.

    Article  PubMed  PubMed Central  Google Scholar 

  18. 18.

    Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5:2. https://doi.org/10.1186/s40900-018-0136-9.

    Article  PubMed  PubMed Central  Google Scholar 

  19. 19.

    McConnell T, Best P, Davidson G, McEneaney T, Cantrell C, Tully M. Coproduction for feasibility and pilot randomised controlled trials: learning outcomes for community partners, service users and the research team. Res Involv Engagem. 2018;4:32. https://doi.org/10.1186/s40900-018-0116-0.

    Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Leask CF, Sandlund M, Skelton DA, Chastin SFM. Co-creating a tailored public health intervention to reduce older adults’ sedentary behaviour. Health Educ J. 2017;76:595–608. https://doi.org/10.1177/0017896917707785.

    Article  Google Scholar 

  21. 21.

    Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012;90:311–46. https://doi.org/10.1111/j.1468-0009.2012.00665.x.

    Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94:392–429. https://doi.org/10.1111/1468-0009.12197.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Rütten A. Kooperative Planung und Gesundheitsförderung: Ein Implementationsansatz [Cooperative planning and health promotion: an implementation approach]. J Public Health. 1997;5:257–72. https://doi.org/10.1007/BF02956335.

    Article  Google Scholar 

  24. 24.

    Sudeck G, Pfeifer K. Physical activity-related health competence as an integrative objective in exercise therapy and health sports—conception and validation of a short questionnaire. German J Exerc Sport Res. 2016;46:74–87. https://doi.org/10.1007/s12662-016-0405-4.

    Article  Google Scholar 

  25. 25.

    Carl J, Sudeck G, Pfeifer K. Competencies for a healthy physically active lifestyle—reflections on the model of physical activity-related health competence. J Phys Activity Health. 2020;17:688–97. https://doi.org/10.1123/jpah.2019-0442.

    Article  Google Scholar 

  26. 26.

    Popp J, Carl J, Grüne E, Semrau J, Gelius P, Pfeifer K. Physical activity promotion in German vocational education: does capacity building work? Health Promot Int. 2020;35:1577–89. https://doi.org/10.1093/heapro/daaa014.

    Article  PubMed  PubMed Central  Google Scholar 

  27. 27.

    World Health Organization and ExpandNet. Nine steps for developing a scaling-up strategy. Geneva: World Health Organization; 2010.

    Google Scholar 

  28. 28.

    Reis RS, Salvo D, Ogilvie D, Lambert EV, Goenka S, Brownson RC. Scaling up physical activity interventions worldwide: stepping up to larger and smarter approaches to get people moving. Lancet. 2016;388:1337–48. https://doi.org/10.1016/S0140-6736(16)30728-0.

    Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci. 2016;11:38. https://doi.org/10.1186/s13012-016-0399-1.

    Article  PubMed  PubMed Central  Google Scholar 

  30. 30.

    Frahsa A, Rütten A, Roeger U, Abu-Omar K, Schow D. Enabling the powerful? Participatory action research with local policymakers and professionals for physical activity promotion with women in difficult life situations. Health Promot Int. 2012;29:171–84. https://doi.org/10.1093/heapro/das050.

    Article  PubMed  Google Scholar 

  31. 31.

    Rütten A, Wolff A, Streber A. Interaktiver Wissenstransfer in der Gesundheitsförderung: das GESTALT-Projekt. Erste Ergebnisse der Erprobung eines Ansatzes zur nachhaltigen Implementierung evidenzbasierter Bewegungsprogramme [Interactive knowledge to action in health promotion: the GESTALT project. Initial results of a pilot study on sustainable implementation of an evidence-based programme]. Gesundheitswesen. 2016;78:359–66. https://doi.org/10.1055/s-0035-1548882.

    Article  PubMed  Google Scholar 

  32. 32.

    Loss J, Brew-Sam N, Metz B, Strobl H, Sauter A, Tittlbach S. Capacity building in community stakeholder groups for increasing physical activity: results of a qualitative study in two German communities. Int J Environ Res Public Health. 2020;17:2306. https://doi.org/10.3390/ijerph17072306.

    Article  PubMed Central  Google Scholar 

  33. 33.

    Strobl H, Ptack K, Töpfer C, Sygusch R, Tittlbach S. Effects of a participatory school-based intervention on students’ health-related knowledge and understanding. Front Public Health. 2020;8:122. https://doi.org/10.3389/fpubh.2020.00122.

    Article  PubMed  PubMed Central  Google Scholar 

  34. 34.

    Hoekstra F, Mrklas KJ, Khan M, McKay RC, Vis-Dunbar M, Sibley KM, et al. A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature. Health Res Policy Syst. 2020;18:51. https://doi.org/10.1186/s12961-020-0544-9.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Verloigne M, Altenburg TM, Chinapaw MJM, Chastin S, Cardon G, de Bourdeaudhuij I. Using a co-creational approach to develop, implement and evaluate an intervention to promote physical activity in adolescent girls from vocational and technical schools: a case control study. Int J Environ Res Public Health. 2017;14:862. https://doi.org/10.3390/ijerph14080862.

    Article  PubMed Central  Google Scholar 

  36. 36.

    Flinders M, Wood M, Cunningham M. The politics of co-production: risks, limits and pollution. Evid Policy J Res Debate Pract. 2016;12:261–79. https://doi.org/10.1332/174426415X14412037949967.

    Article  Google Scholar 

  37. 37.

    Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Syst. 2019;17:33. https://doi.org/10.1186/s12961-019-0432-3.

    Article  PubMed  PubMed Central  Google Scholar 

  38. 38.

    Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. https://doi.org/10.1136/bmj.a1655.

    Article  PubMed  PubMed Central  Google Scholar 

  39. 39.

    Campbell H, Vanderhoven D. Knowledge that matters: realising the potential of co-production. Manchester: N8 Research Partnership; 2016. http://www.n8research.org.uk/research-focus/urban-and-community-transformation/co-production/. Accessed 9 Oct 2017. Manchester; 2016.

  40. 40.

    Bamberger M, Rugh J, Mabry L. RealWorld evaluation: working under budget, time, data, and political constraints. 2nd ed. Thousand Oaks: Sage; 2012.

    Google Scholar 

  41. 41.

    Crane M, Bauman A, Lloyd B, McGill B, Rissel C, Grunseit A. Applying pragmatic approaches to complex program evaluation: a case study of implementation of the New South Wales Get Healthy at Work program. Health Promot J Austr. 2019;30:422–32. https://doi.org/10.1002/hpja.239.

    Article  PubMed  Google Scholar 

  42. 42.

    Malterud K, Aamland A, Iden KR. Small-scale implementation with pragmatic process evaluation: a model developed in primary health care. BMC Fam Pract. 2018;19:93. https://doi.org/10.1186/s12875-018-0778-6.

    Article  PubMed  PubMed Central  Google Scholar 

  43. 43.

    Gagliardi AR, Kothari A, Graham ID. Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know. J Epidemiol Community Health. 2017;71:105–6. https://doi.org/10.1136/jech-2016-207743.

    Article  PubMed  Google Scholar 

  44. 44.

    Fynn JF, Hardeman W, Milton K, Murphy J, Jones A. A systematic review of the use and reporting of evaluation frameworks within evaluations of physical activity interventions. Int J Behav Nutr Phys Act. 2020;17:107. https://doi.org/10.1186/s12966-020-01013-7.

    Article  PubMed  PubMed Central  Google Scholar 

  45. 45.

    Rütten A, Gelius P. Building policy capacities: an interactive approach for linking knowledge to action in health promotion. Health Promot Int. 2013;29:569–82. https://doi.org/10.1093/heapro/dat006.

    Article  PubMed  Google Scholar 

  46. 46.

    Cooksy LJ, Gill P, Kelly PA. The program logic model as an integrative framework for a multimethod evaluation. Eval Program Plan. 2001;24:119–28. https://doi.org/10.1016/S0149-7189(01)00003-9.

    Article  Google Scholar 

  47. 47.

    W.K. Kellogg Foundation. Using logic models to bring together planning, evaluation, and action: logic model development guide. Battle Creek, Michigan: W.K. Kellogg Foundation; 2004.

  48. 48.

    McLaughlin JA, Jordan GB. Using logic models. In: Newcomer KE, Hatry HP, Wholey JS, editors. Handbook of practical program evaluation. Hoboken: Jossey-Bass, A Wiley imprint; 2015. p. 62–87. https://doi.org/10.1002/9781119171386.ch3.

    Chapter  Google Scholar 

  49. 49.

    Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd ed. Los Angeles, London, New Delhi, Singapore, Washington DC: Sage; 2011.

    Google Scholar 

  50. 50.

    O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341:c4587. https://doi.org/10.1136/bmj.c4587.

    Article  PubMed  Google Scholar 

  51. 51.

    Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9:7. https://doi.org/10.1186/1748-5908-9-7.

    Article  PubMed  PubMed Central  Google Scholar 

  52. 52.

    Maneesriwongul W, Dixon JK. Instrument translation process: a methods review. J Adv Nurs. 2004;48:175–86. https://doi.org/10.1111/j.1365-2648.2004.03185.x.

    Article  PubMed  Google Scholar 

  53. 53.

    Lindig A, Hahlweg P, Christalle E, Scholl I. Translation and psychometric evaluation of the German version of the Organisational Readiness for Implementing Change measure (ORIC): a cross-sectional study. BMJ open. 2020;10:e034380. https://doi.org/10.1136/bmjopen-2019-034380.

    Article  PubMed  PubMed Central  Google Scholar 

  54. 54.

    Weiner BJ, Amick H, Lee SYD. Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65:379–436. https://doi.org/10.1177/1077558708317802.

    Article  PubMed  Google Scholar 

  55. 55.

    Patton MQ. Qualitative research and evaluation methods. 4th ed. Los Angeles: Sage; 2014.

    Google Scholar 

  56. 56.

    Adriansen HK. Timeline interviews: a tool for conducting life history research. QS. 2012;3:40–55. https://doi.org/10.7146/qs.v3i1.6272.

    Article  Google Scholar 

  57. 57.

    Hoekstra F, Martin Ginis KA, Allan V, Kothari A, Gainforth HL. Evaluating the impact of a network of research partnerships: a longitudinal multiple case study protocol. Health Res Policy Syst. 2018;16:107. https://doi.org/10.1186/s12961-018-0377-y.

    Article  PubMed  PubMed Central  Google Scholar 

  58. 58.

    Kuckartz U. Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung [Qualitative content analysis: methods, practice, computer support]. 3rd ed. Weinheim: Beltz Juventa; 2016.

    Google Scholar 

  59. 59.

    Heaton J, Day J, Britten N. Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implement Sci. 2016;11:20. https://doi.org/10.1186/s13012-016-0383-9.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Open Access funding enabled and organized by Projekt DEAL. The research network Capital4Health (http://www.capital4health.de/en) and the associated PArC-AVE Project, in which the study is conducted, are funded by the German Federal Ministry of Education and Research [grant number 01EL1821A].

Author information

Affiliations

Authors

Contributions

JS and KP contributed to obtaining funding for the study. All authors made substantial contributions to the design of the study. JP drafted the manuscript with input from EG, JC, JS, and KP. EG, JC, JS, and KP reviewed and provided feedback on the manuscript drafts. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Johanna Popp.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for research within the PArC-AVE Project was granted by the Ethical Committee of the Friedrich-Alexander University Erlangen-Nürnberg [January 15, 2019; sign 467_18 B]. Participants in this study will be asked to provide written or verbal consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Definitions and instruments underlying the operationalization of predefined determinants.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Popp, J., Grüne, E., Carl, J. et al. Co-creating physical activity interventions: a mixed methods evaluation approach. Health Res Policy Sys 19, 37 (2021). https://doi.org/10.1186/s12961-021-00699-w

Download citation

Keywords

  • Coproduction
  • Cooperative planning
  • Participation
  • Pragmatic evaluation
  • Logic model
  • Health promotion
  • School
  • Workplace
  • Nursing care
  • Automotive mechatronics