Setting and target populations: building on the POSD evaluation study
A multi-country evaluation of the impact of the 2007 POSD is currently underway (POSD evaluation study: http://www.onecaribbeanhealth.org/), led by the University of the West Indies, to assess successes and gaps in the development and implementation of policy on NCDs in the Caribbean region. The POSD evaluation study has developed a network of stakeholders and collected a range of qualitative and quantitative data that can serve directly as resources for the development of the model described here. As part of this study, data are being collated on mortality, morbidity and risk factor trends across the 20 CARICOM member states. A core component of the POSD evaluation study were case studies in seven Caribbean countries of the policy responses to NCDs, including qualitative in-depth interviews with stakeholders from government and other sectors. In addition, government and regional policy initiatives around the prevention and control of NCDs since 2007 are being appraised. The model development will build on findings and stakeholder contacts from the POSD evaluation study (Fig. 1).
Middle-income countries were chosen to evaluate system dynamics methodologies in health in resource-limited settings. One country is Jamaica, as the largest English-speaking middle-income country in the Caribbean. The second is Belize, which is politically and culturally part of the Caribbean, but has a unique environment being a Central American mainland, English-speaking middle-income country with access to data on diabetes prevalence, risk factors and its effects from continuing monitoring in the country. The third is St Vincent and the Grenadines, a small, middle-income island nation.
Development of an initial conceptual model
Milstein et al. [23] developed a comprehensive diabetes model for the CDC in the USA to reflect, as much as possible, a generic model for diabetes. The core of this model will be used as a basis in this project. The CDC model will be adapted to a Caribbean context, which differs from the USA in demographics, ethnic admixture, health system structure, food policy, built environment and resources. Furthermore, policymakers participating in the POSD evaluation study have expressed the need to include upstream determinants of diabetes and other NCDs in any policymaking endeavour. These include the broader determinants of diet and physical activity, such as the price and availability of different foods and the structure of the built environment. The CDC model explicitly excluded these determinants as outside the model boundaries set by stakeholders partly due to a lack of sufficient evidence to satisfy stakeholders and partly because the model, as it was constructed, satisfied the purposes of that study [16]. The policies of interest for evaluation in this study also differ from those of the CDC and thus the model must follow the needs of the context for which it is being designed. Despite these differences, a core epidemiological structure exists in the CDC model, which is generic and will be used as the starting point for model adaptation and development (Fig. 2).
This initial concept model will be used to engage stakeholders and promote systems thinking during in-depth interviews. The model focuses explicitly on incorporating shared risk factors for NCDs and their upstream determinants, which are potential targets of policy interventions. Total diabetes prevalence will be used as the primary outcome measure of interest (in system dynamics referred to as the ‘reference mode’) for policy interventions.
The interviews will be used to gather qualitative data from stakeholders on their perspectives regarding diabetes, its determinants, and effects in their own countries and the region. The development will be an iterative process with each phase informing the other, thus refining the model. As part of this iterative process, the model structure will be compared to the CDC diabetes model [23]. This will provide an additional way of promoting critical consideration of the model structure through understanding the basis and the consequences of any differences.
Review of existing qualitative data and stakeholder selection
The POSD evaluation study conducted in-depth interviews with NCD experts in countries on their views regarding progress on policies laid out by the POSD. Using data collected from the evaluation the investigators will undertake a formal thematic qualitative analysis of existing interview data as it relates to drivers and determinants of the diabetes system in the region, particularly to the upstream determinants of diet and physical activity through obesity. The review will also be used to identify stakeholders to be invited to participate in and for follow-up in-depth interviews, and insights from the prior interviews will be revisited and discussed in these planned follow-ups. Further stakeholders will be identified using snowball sampling and referral from contacts within and using the existing POSD network in the three project countries. For practical reasons, and in keeping with recommendations for model building [24], no more than 10 stakeholders will be selected for each country.
Review of quantitative evidence
A systematic literature review will provide evidence on the quantitative historical relationships between variables in the model. The empirical data will be used to calibrate the model. A systematic review is being conducted of studies performed within the last 10 years in the Caribbean describing the prevalence of type 2 diabetes, its risk factors and their social determinants, and healthcare coverage and outcomes in people with diabetes. This evidence review will build upon recent scoping and systematic reviews into the social determinants of diabetes in the Caribbean [25]. In addition, any available, unpublished, datasets in the three project countries will be identified. This will be facilitated by the POSD evaluation and ongoing work collating mortality and risk factor datasets within the CARICOM countries. In addition, stakeholders will be asked during the qualitative interviews, if they know of any local datasets.
In the absence of sufficient quantitative data from the Caribbean for a relationship within the conceptual model, it will be necessary to use data for calibration from other parts of the world. Care will be taken to use data matching a context and ethnic mix as close as possible to the Caribbean populations and such data will be assumed to come with greater uncertainty in the calibration exercise.
Stakeholder interviews and new data collection
Stakeholders for the in-depth qualitative interviews will involve members of the Ministries of Health, including the Chief Medical Officer or NCD focal point, other relevant government ministries, such as education, transport and agriculture; leaders from civil society organisations, including diabetes organisations and church groups; and prominent private sector leaders.
A semi-structured interview guide will be used, eliciting stakeholders’ estimation and gaining feedback on the draft preliminary model, and contextual information of trends in diabetes, obesity and physical inactivity, and the underlying determinants in their settings; similarly, stakeholder views on trends in access to diabetes care and determinants of blood glucose, pressure, and lipid control and their determinants will be explored. All interviews will be recorded, transcribed verbatim and analyzed thematically [26], and used to generate causal maps [27].
Country-level model building
A conceptual model will be developed for each of the three study countries using stakeholder input and data gathered from the evidence review. Stakeholders will be given a report of the conceptual model and given an opportunity to submit changes for one round of iteration. The models will be finalised and a quantitative model produced for each country using country-level data where possible. The models will be compared and simulations will be produced using a single set of policies and scenarios for testing.
Group model building and an inter-country regional model
The investigators will use a revised conceptual model to include input from country-level work as the basis for the development of an inter-country regional model. The model development will include stakeholders with regional expertise who will meet for a group model building workshop. Over 2 days, the workshop will provide an introduction to system dynamics modelling and systems thinking to stakeholders to develop a regional diabetes model. Following the model building workshop, stakeholders will be given two opportunities for revision and feedback. These comments will be included in the revision and development of the model. Once there is general agreement from stakeholders on the conceptual model, evidence from quantitative data collection will be applied to develop a quantitative model.
Quantitative inter-country regional model, validation and policy simulations
Once the conceptual model has been finalised, estimates for each of the parameters and variables will be incorporated using data obtained from the quantitative data collection. The model will be used to try and replicate trends in risk factors and outcomes available from studies in the region as much as possible from 2000 to the present. One of the time frames that will be used for simulations is to 2025 in order to help guide potential policies for achieving relevant targets of the WHO Global Monitoring Framework [22]. The model will also be extended to 2050 to give enough time to assess the presence and influence of slow feedback mechanisms. Testing the model will involve undertaking sensitivity analyses for the parameters included and also ‘extreme case’ modelling [28] will help to assess the plausibility of the simulations being produced.
The primary targets for the option appraisal and simulation will be centred on policy recommendations generated from the POSD evaluation study, relevant targets in the Global Monitoring Framework, and policies identified by stakeholders, including no increase in obesity or diabetes prevalence by 2025, and a reduction in premature diabetes mortality by 25%. Various alternative scenarios, including adoption and implementation of policies related to risk factors and treatment, exogenous scenarios and testing one against the other, will be evaluated using the simulations. Potential policies will be evaluated alone and in plausible combinations, as guided by stakeholders, against a timeline of achievement by 2025, without a time horizon, and estimating a time horizon for achievement.
Special attention will be put into stratifying the model by gender and socioeconomic status to explore how the potential interventions for the prevention and control of diabetes impact on health equity.
Stakeholder model evaluation
Once the quantitative model has been tested, a simulation test environment (management flight simulator) will be developed for stakeholders to interact with the model. Stakeholders will be interviewed and surveyed to gather feedback on the model building process, utilisation of the simulation tool, and how process and outputs could be made more useful to them.
Ethical review
The study has been submitted for ethical review and approved by the University of the West Indies, Cave Hill/Barbados Ministry of Health Research Ethics Committee/Institutional Review board.