Skip to main content

Table 2 Criteria for evaluating cross-disciplinarity in health policy and systems research

From: A critical realist synthesis of cross-disciplinary health policy and systems research: defining characteristic features, developing an evaluation framework and identifying challenges

Domains

Appraisal criteria

Specific indicators and references

Illustrative evaluation questions

Context

Scope of the problem and solution space1.

Articulation of the problem [13, 22, 24, 53,54,55, 71, 73, 92]

Has the problem been adequately described, in terms of its nature, scope and relevance?

Justification of a cross-disciplinary approach to the problem [23, 54, 55, 69, 70, 76]

Is there a clear description of how cross-disciplinarity provides a more useful approach to addressing the problem than the alternatives?

Nature of research question and purpose1.1.1.

Fit of the research question with cross-disciplinary perspective [53, 54, 67, 73, 85]

Does the frame of the research question reflect a cross-disciplinary perspective?

Statement of expected cross-disciplinary outcomes [26, 52, 57, 62, 93, 97]

Are the expected cross-disciplinary outcomes of the research explicitly stated?

Appropriateness of the study purpose for addressing the research question [53,54,55,56, 67, 73, 85]

To what extent does the research purpose reflect the cross-disciplinary question under investigation?

Conceptualisation of research question and purpose [28, 53,54,55,56,57, 63]

Does the analysis and/or frame of the research question and purpose demonstrate critical consideration of theory and evidence from multiple disciplines relevant to the context?

Study setting1.

Description of study setting [29, 53, 57, 68, 70, 71]

Does the study setting as described reflect the research problem and question specified?

Justification of setting for cross-disciplinary cross-disciplinary study [53, 55, 60, 67, 70]

To what extent does the context of the setting necessitate a cross-disciplinary approach?

Mechanism

Theoretical, methodological and institutional balance 1.1.1.1.1.1.

Representation of multiple disciplinary perspectives [13, 19, 20, 23, 52, 53, 55, 56, 63, 66, 68, 69, 71,72,73]

Have the diverse disciplinary orientations been clearly defined to allow for evaluation?

Balance of institutional representation [52, 54, 56, 66, 69, 70, 76]

If the research crosses institutional boundaries, are the relevant institutions and disciplines clearly stated in the study?

Means of integrating diverse perspectives, theories and methods [28, 53, 56, 59, 67, 73]

Are the methods for disciplinary integration clearly defined?

Fit of methods for purpose [26, 29, 58, 77, 92]

Do the methods for generating and analysing data fit a cross-disciplinary strategy?

Criteria for selection and composition of the scientific team [19, 20, 54, 57, 60, 69, 73]

Are the criteria used for selecting the team suitable for the research problem?

Fit of team members’ backgrounds to the research problem and solution space [13, 20, 21, 23, 52, 57, 63, 67, 68, 72]

Do the team members’ background and expertise fit the problem space?

Appropriate distribution of responsibility [19, 24, 54, 56, 69, 70]

Does the definition of participating stakeholders’ roles reflect their disciplinary representation in the project?

Scale of disciplinary crossing1.

Extent of cross-disciplinary knowledge integration [13, 19, 22, 52, 67, 68, 70, 71, 76]

To what extent (high/medium/low) does the study cross diverse disciplines or fields?

Stages of disciplinary crossing in the research process [52, 53, 56, 67, 72]

At what stage of the study are disciplinary boundaries crossed?

Shared (cross-disciplinary) communication 1.1.

Commitment to a cross-disciplinary communication framework [13, 21, 56, 57, 63, 68, 70, 72, 79]

Are there clear efforts to ensure effective cross-disciplinary communication throughout the project?

Indication of collegial decision-making among team members [19, 30, 53, 56, 62]

Does the design have mechanisms in place to allow for collective decision-making?

Interactive and iterative process [52, 53, 59, 69, 71, 76, 79]

Does the communication structure reveal clear potential for team members to regularly interact and jointly evaluate their assumptions and processes?

Conflict resolution1.

Identification and response to researchers’ own conflicts [19, 23, 24, 58, 59, 67, 68, 70]

Are the possible/practical disciplinary and personal conflicts among researchers stated and addressed?

Explicit indication of cross-disciplinary/institutional ethics [24, 54, 58, 59, 62, 70]

Have the different disciplinary and institutional ethical positions and assumptions been adequately explained and addressed?

Outcome

Validity 1.1.

Achievement of cross-disciplinary purpose [52, 55, 69, 76, 98]

Has the cross-disciplinary purpose of the research been fulfilled?

Cross-disciplinary balance was maintained as planned [27, 58, 77, 80, 91, 95]

Has the intended disciplinary and institutional balance been adequately adhered to?

Consistency/coherence of common vocabulary [19,20,21,22, 24, 52, 54, 57, 67, 71, 72]

Do the research results convincingly represent an interplay of the different disciplinary voices?

Value/Yield

Additional contribution (value addition) [13, 21, 26, 52,53,54, 56, 67, 69, 92, 98]

Has extra scientific and social impact been demonstrated beyond that attainable through monodisciplinary approaches?

Problem-solving capacity [21, 30, 54, 57, 63, 64, 66, 73]

Do the study results contribute to a fundamental understanding of the problem or actionable solutions or a combination of both?

Novelty or innovation [55, 64, 67, 68, 71, 74, 79]

Have the research results demonstrated novelty by virtue of the use of a cross-disciplinary approach?