Skip to main content

Table 3 Examples of evaluation feedback, team decisions and adaptations

From: Using developmental evaluation to enhance continuous reflection, learning and adaptation of an innovation platform in Australian Indigenous primary healthcare

Evaluation findingsDecisions and adaptations
Increase the number, input and attendance of health service providers input and attendance at CRE-IQI bi-annual meetingsCRE-IQI bi-annual meeting agendas were amended to include ‘health service showcases’, in which health services staff could present their quality improvement work and discuss opportunities for research collaboration and knowledge translation. Presentation topics were determined through consultation processes at previous bi-annual meetings and with the management committee. The CRE-IQI funded health services staff members to attend these meetings, which were held in different locations to encourage participation by a range of groups
Increase attendance by Indigenous stakeholders at bi-annual meetingsPersonalised invitations to bi-annual meetings were extended to Indigenous stakeholders via telephone rather than emails, with funds specifically allocated for Indigenous stakeholders to attend. Increased steps were taken to ensure a culturally safe environment at the meetings, including their formal opening and closing by Indigenous people
Explicitly promote the shared values and working principles of the CRE-IQIThe principles of practice of the innovation platform were highlighted and discussed at the start of all bi-annual meetings, and purposely applied when developing criteria for the allocation of funds for CRE-IQI activities, such as ‘seed grants’ to develop research
Focus on capacity-strengthening, particularly in relation to Indigenous direction of, and participation in, researchDedicated funding was allocated for a 12-month ‘research capacity-building’ position and a lead group to oversee and provide guidance on capacity-building strategies and activities. The terminology was changed to ‘capacity-strengthening’ in recognition of the existing strengths and knowledge among stakeholders, and monthly online research capacity-strengthening meetings held using ‘Zoom’ software
Grow Indigenous leadership in CRE-IQI management and researchAn additional Indigenous researcher was appointed to the CRE-IQI leadership. Purposeful encouragement of co-leadership arrangements was established, whereby all new research grants and projects were to have an Indigenous leader or a co-leadership arrangement with a non-Indigenous member of the team
Enable Indigenous members to engage in CRE-IQI direction and governance despite their high workloads and multiple leadership commitmentsThe decision was made to embed and disperse Indigenous leadership and participation across all levels of the innovation platform, rather than having one overall reference group. This included the appointment of an Indigenous researcher to the leadership team; purposeful engagement and funding to increase attendance by Indigenous people and organisations at all meetings; greater focus and attention on embedding the ‘principles of practice’ established at the start of the innovation platform; and co-leadership arrangements with Indigenous and non-Indigenous collaborators on all projects
Provide information to CRE-IQI stakeholders through mechanisms such as meetings, new publications and news from the networkBased on feedback, a monthly email to all CRE-IQI stakeholders was established that would later become a regular and official online CRE-IQI newsletter
Boost engagement with, and readership of, the CRE-IQI newsletterTo improve readability and engagement, the monthly newsletter was adjusted to include more illustrative material and articles from stakeholders. Following research into the most effective dissemination times, the monthly newsletter was disseminated on a Tuesday or Thursday at either 10 am or 2 pm
Ensure that administrative data collected by the CRE-IQI (e.g. attendance at bi-annual meetings, publications, grants awarded) is capable of the required data analysisData collection procedures were reviewed, specifically, what was being collected and how, and, importantly, what aspects would need to be reported and the aggregations required. Changes were made and standard nomenclature adopted
Increase the focus on and engagement in research translationResearch translation strategies were prioritised and developed over a series of workshops at bi-annual meetings and discussions at management committee meetings. CRE-IQI stakeholders were provided with training opportunities in knowledge translation skills, including the use of social media, influencing policy and other relevant topics. CRE-IQI social media accounts were established and reviewed, resulting in increased use of Twitter to communicate research activities and findings. A dedicated research translation working group was convened and a position established to support projects and translation across the CRE-IQI. In the final year, knowledge synthesis workshops were held in which members collaboratively identified and prioritised the overall findings and key messages from CRE-IQI research. Research translation products were produced in a range of formats targeting different audiences
Respond to CRE-IQI stakeholders’ identified need for training in a range of relevant topicsTraining needs were addressed through the establishment of webinar research seminars and face-to-face masterclasses. At each bi-annual meeting, participants were invited to suggest further topics to meet their professional development needs, such as social media training to extend research translation, engaging policy-makers in dissemination of research findings and using Indigenous methodologies
Strengthen CRE-IQI engagement with policy-making processesResources were directed into writing targeted policy and parliamentary submissions that drew on CRE-IQI research. Policy masterclasses were offered to members early in the CRE-IQI’s establishment and again in its final year. Relevant policy-makers were invited to bi-annual meetings with the aim of having their input into the development of research products, such as key messages, and of building relationships with them over an extended period. Final products of research projects included policy briefs, and the publication of a summary of overall policy messages from the CRE-IQI’s research. At the end of its funding period, the CRE-IQI targeted key policy-makers for briefings about the research findings
Prioritise further collaborative research in Indigenous primary healthcare quality improvementCollaborative processes were undertaken to identify and refine the research priorities. These processes included presenting and working up ideas at bi-annual meetings, discussing research needs in management committee meetings and holding a series of smaller more focused workshops. A decision was made to develop a submission for funding beyond the innovation platform, with revised leadership arrangements to reflect the DE outcomes. This resulted in a proposal for a collaborative research network led by an Indigenous chief investigator, with 50% of the leadership team identifying as Indigenous
  1. CRE-IQI Centre for Research Excellence in Integrated Quality Improvement, DE developmental evaluation