Domains | Appraisal criteria | Specific indicators and references | Illustrative evaluation questions |
---|---|---|---|
Context | Scope of the problem and solution space1. | Articulation of the problem [13, 22, 24, 53,54,55, 71, 73, 92] | Has the problem been adequately described, in terms of its nature, scope and relevance? |
Justification of a cross-disciplinary approach to the problem [23, 54, 55, 69, 70, 76] | Is there a clear description of how cross-disciplinarity provides a more useful approach to addressing the problem than the alternatives? | ||
Nature of research question and purpose1.1.1. | Fit of the research question with cross-disciplinary perspective [53, 54, 67, 73, 85] | Does the frame of the research question reflect a cross-disciplinary perspective? | |
Statement of expected cross-disciplinary outcomes [26, 52, 57, 62, 93, 97] | Are the expected cross-disciplinary outcomes of the research explicitly stated? | ||
Appropriateness of the study purpose for addressing the research question [53,54,55,56, 67, 73, 85] | To what extent does the research purpose reflect the cross-disciplinary question under investigation? | ||
Conceptualisation of research question and purpose [28, 53,54,55,56,57, 63] | Does the analysis and/or frame of the research question and purpose demonstrate critical consideration of theory and evidence from multiple disciplines relevant to the context? | ||
Study setting1. | Does the study setting as described reflect the research problem and question specified? | ||
Justification of setting for cross-disciplinary cross-disciplinary study [53, 55, 60, 67, 70] | To what extent does the context of the setting necessitate a cross-disciplinary approach? | ||
Mechanism | Theoretical, methodological and institutional balance 1.1.1.1.1.1. | Representation of multiple disciplinary perspectives [13, 19, 20, 23, 52, 53, 55, 56, 63, 66, 68, 69, 71,72,73] | Have the diverse disciplinary orientations been clearly defined to allow for evaluation? |
Balance of institutional representation [52, 54, 56, 66, 69, 70, 76] | If the research crosses institutional boundaries, are the relevant institutions and disciplines clearly stated in the study? | ||
Means of integrating diverse perspectives, theories and methods [28, 53, 56, 59, 67, 73] | Are the methods for disciplinary integration clearly defined? | ||
Do the methods for generating and analysing data fit a cross-disciplinary strategy? | |||
Criteria for selection and composition of the scientific team [19, 20, 54, 57, 60, 69, 73] | Are the criteria used for selecting the team suitable for the research problem? | ||
Fit of team members’ backgrounds to the research problem and solution space [13, 20, 21, 23, 52, 57, 63, 67, 68, 72] | Do the team members’ background and expertise fit the problem space? | ||
Appropriate distribution of responsibility [19, 24, 54, 56, 69, 70] | Does the definition of participating stakeholders’ roles reflect their disciplinary representation in the project? | ||
Scale of disciplinary crossing1. | Extent of cross-disciplinary knowledge integration [13, 19, 22, 52, 67, 68, 70, 71, 76] | To what extent (high/medium/low) does the study cross diverse disciplines or fields? | |
Stages of disciplinary crossing in the research process [52, 53, 56, 67, 72] | At what stage of the study are disciplinary boundaries crossed? | ||
Shared (cross-disciplinary) communication 1.1. | Commitment to a cross-disciplinary communication framework [13, 21, 56, 57, 63, 68, 70, 72, 79] | Are there clear efforts to ensure effective cross-disciplinary communication throughout the project? | |
Indication of collegial decision-making among team members [19, 30, 53, 56, 62] | Does the design have mechanisms in place to allow for collective decision-making? | ||
Interactive and iterative process [52, 53, 59, 69, 71, 76, 79] | Does the communication structure reveal clear potential for team members to regularly interact and jointly evaluate their assumptions and processes? | ||
Conflict resolution1. | Identification and response to researchers’ own conflicts [19, 23, 24, 58, 59, 67, 68, 70] | Are the possible/practical disciplinary and personal conflicts among researchers stated and addressed? | |
Explicit indication of cross-disciplinary/institutional ethics [24, 54, 58, 59, 62, 70] | Have the different disciplinary and institutional ethical positions and assumptions been adequately explained and addressed? | ||
Outcome | Validity 1.1. | Achievement of cross-disciplinary purpose [52, 55, 69, 76, 98] | Has the cross-disciplinary purpose of the research been fulfilled? |
Cross-disciplinary balance was maintained as planned [27, 58, 77, 80, 91, 95] | Has the intended disciplinary and institutional balance been adequately adhered to? | ||
Consistency/coherence of common vocabulary [19,20,21,22, 24, 52, 54, 57, 67, 71, 72] | Do the research results convincingly represent an interplay of the different disciplinary voices? | ||
Value/Yield | Additional contribution (value addition) [13, 21, 26, 52,53,54, 56, 67, 69, 92, 98] | Has extra scientific and social impact been demonstrated beyond that attainable through monodisciplinary approaches? | |
Do the study results contribute to a fundamental understanding of the problem or actionable solutions or a combination of both? | |||
Have the research results demonstrated novelty by virtue of the use of a cross-disciplinary approach? |