- Research
- Open access
- Published:
Health system description and assessment: a scoping review of templates for systematic analyses
Health Research Policy and Systems volume 22, Article number: 82 (2024)
Abstract
Background
Understanding and comparing health systems is key for cross-country learning and health system strengthening. Templates help to develop standardised and coherent descriptions and assessments of health systems, which then allow meaningful analyses and comparisons. Our scoping review aims to provide an overview of existing templates, their content and the way data is presented.
Main body
Based on the WHO building blocks framework, we defined templates as having (1) an overall framework, (2) a list of indicators or topics, and (3) instructions for authors, while covering (4) the design of the health system, (5) an assessment of health system performance, and (6) should cover the entire health system. We conducted a scoping review of grey literature published between 2000 and 2023 to identify templates. The content of the identified templates was screened, analyzed and compared. We found 12 documents that met our inclusion criteria. The building block `health financing´ is covered in all 12 templates; and many templates cover ´service delivery´ and ´health workforce’. Health system performance is frequently assessed with regard to ‘access and coverage’, ‘quality and safety’, and ‘financial protection’. Most templates do not cover ‘responsiveness’ and ‘efficiency’. Seven templates combine quantitative and qualitative data, three are mostly quantitative, and two are primarily qualitative. Templates cover data and information that is mostly relevant for specific groups of countries, e.g. a particular geographical region, or for high or for low and middle-income countries (LMICs). Templates for LMICs rely more on survey-based indicators than administrative data.
Conclusions
This is the first scoping review of templates for standardized descriptions of health systems and assessments of their performance. The implications are that (1) templates can help analyze health systems across countries while accounting for context; (2) template-guided analyses of health systems could underpin national health policies, strategies, and plans; (3) organizations developing templates could learn from approaches of other templates; and (4) more research is needed on how to improve templates to better achieve their goals. Our findings provide an overview and help identify the most important aspects and topics to look at when comparing and analyzing health systems, and how data are commonly presented. The templates were created by organizations with different agendas and target audiences, and with different end products in mind. Comprehensive health systems analyses and comparisons require production of quantitative indicators and complementing them with qualitative information to build a holistic picture.
Clinical Trial Registration: Not applicable.
Background
Health systems strengthening is key to achieving Sustainable Development Goal (SDG) 3 “to ensure healthy lives and promote well-being for all at all ages” [1], and is high up on the global research and policy agendas [2, 3]. Strengthening health systems requires a comprehensive approach targeting different health system building blocks (see Box 1) and a thorough understanding of the multiple relationships and interactions between them [4].
Describing and analysing the building blocks of health systems’ and their functioning is a precondition for assessing them, and an effective assessment supports health systems strengthening [5,6,7,8]. Utilizing a standardized format or template when describing and assessing health systems can support cross-country comparisons. This is because the reports that are produced based on templates follow the same structure, thus simplifying the extraction of comparable information and benchmarking [9,10,11,12,13,14,15], as well as the identification of high-performing areas of health systems and areas for improvement [16, 17]. Therefore, templates provide a basis for informed, evidence-based decision-making, promoting transparency, and facilitating cross-country learning and collaboration [18, 19]. Finally, standardized documents describing health systems support performance assessment by providing a common way of understanding health systems and their functioning, along with a set of indicators that allow for the comparison of health system performance across different countries. A number of international agencies have developed templates to describe and assess health systems and to facilitate international comparisons and ultimately highlight areas and policies for improvement [20, 21].
However, there are challenges associated with developing and maintaining standardized health system documents for cross-country comparison, as this exercise requires a common definition and understanding of the elements and functioning of health systems to be described, and a common way of describing, analyzing and interpreting these elements. It requires an inventory of consensual indicators that accurately capture and describe the elements and functions of health systems, that are reliable and available across countries [16, 17]. Effective templates that guide authors on how to write these standardized health system documents mitigate these challenges. Not only do templates guide authors on which elements and functions to describe, they also provide definitions and instructions on how to describe them, how to collect, present and interpret data and indicators, and how to compare one country to others. Yet, to our knowledge, an inventory of such templates does not exist, and a comparison of templates’ contents, strategies of guidance and list of indicators proposed is missing.
This paper fills these gaps by presenting a scoping review of templates that guide authors on how to analyze and compare health systems and to assess their performance. We assess templates against the WHO’s building blocks framework [22], which we take as foundational. Specifically, our objectives were (1) to identify existing templates that serve as guides for describing and analyzing health systems; and (2) to explore how the templates describe the design and performance of health systems, i.e. the methods of data collection and presentation and the topics covered.
Materials and methods
A scoping review was conducted to identify existing templates. We conducted exploratory searches of templates between February and December 2020, which helped us conceptualize “what a template is” and what its defining elements are. This definition then informed the search strategy and the inclusion and exclusion criteria. We used Arksey and O’Malley [23] as guidance for our review stages (identify research question, identify relevant studies, study selection, charting data, reporting results). Because the tools and documents sought were grey literature, a systematic grey literature search approach was adopted following Godin et al. [24]. The protocol of this scoping review was registered with Open Science Framework (https://osf.io) and is accessible under the registration number https://doi.org/10.17605/OSF.IO/CVHJD. The review is reported according to the PRISMA-ScR extension for Scoping review guidelines [25].
Conceptualization of “templates”
We identified six criteria that define a ‘template’ for the purpose of our analysis. These included: (1) the existence of an overall framework, (2) the provision of a list of indicators or clear descriptions of the topics and elements to be included, (3) instructions for authors on how to write an informative document, (4) guidance for the description of the design of the health system, and (5) an approach for the assessment of health system performance. In addition, (6) a “template” should cover the entire health system, which we operationalized as covering at least four of the six functions of WHO’s building blocks (see Box 1) [22].
Data collection
A database search was conducted to identify templates from July to December 2023. Four information sources were identified as suggested by Godin [24]. First, four grey literature databases were identified (OpenGrey, ELDIS, WHOLIS, Google Scholar); second, a systematic search of the internet was conducted using Google, complemented by Open AI (BING AI, ELICIT). Third, websites from organizations that operate on an international level and whose agenda includes overall health or health systems were searched (e.g., WHO, OECD, World Bank). For a list of all targeted websites see supplement 1. Fourth, tacit knowledge of all authors, experts in health systems analysis, were integrated in the search.
A list of search terms (strings) was created initially based on the expertise of the co-authors and refined through a trial run. The search terms are presented in Table 1. Table 2 presents one example (out of many) of the search strategy. Due to the different characteristics of each database´s search engine mechanisms, different search strategies were applied.Footnote 1 Only the first 100 hits were screened. All targeted websites were hand-searched. Full electronic search strategies are provided in Supplement 2.
Inclusion criteria
We searched primarily for templates, but also looked for reports that described health systems so we could search for the template that guided the report. Eligibility criteria for inclusion in the review were:
-
1.
Documents that are used for health system description, comparison or performance assessment at the international level
-
2.
Published between 2000 and 2023
-
3.
Most current version of the document
-
4.
Template must include six defining criteria
-
a.
Include an overall framework
-
b.
Include a list of indicators or clear descriptions that depict the topics and elements included
-
c.
Include instructions for authors on how to write an informative document
-
d.
Guide on how to describe the design of the health system
-
e.
Include an approach for the assessment of the health system performance
-
f.
Cover the entire health system, which we operationalized as covering at least four of the six functions of WHO’s building blocks
-
a.
Exclusion criteria
-
1.
National reports or templates—because these are not designed to fit multiple health systems.
-
2.
Documents that focus only on specific aspects of health systems such as health for older adults, health inequalities.
Once a report met all inclusion criteria, except criterion 3 (Include instructions for authors), attempts were made to obtain the relevant template from the authors or publishing organizations. There were no language restrictions in the scoping review.
Screening and extraction process
The screening process included three stages. First, the hits of each search were screened by title. Duplicates were removed and different issues of a report were condensed into one. Second, the introductions, abstracts and tables of contents were screened. Third, the full documents were assessed regarding the fulfilment of the inclusion criteria. Those that did, were defined as “templates” (Table 3). The third step was performed by two authors (IP and RW), and in case of divergent screening decisions, the reports in question were discussed with a third author (WQ).
Data charting process and analysis
A data charting form was developed containing the templates name, their methods of data collection, building blocks and intermediate and final goals of the WHO’s framework. Two reviewers (RW, IP) charted data independently and discussed the results. Inconsistencies were discussed with a third author (WQ). For all identified “templates”, we matched the chapters of the documents to WHO’s building blocks framework to assess whether the tools covered at least four of the six building blocks. We assessed data collection approaches suggested by the templates, for example through interviews, surveys, literature review, administrative data and records (Table 4). In addition, we mapped chapters to the six intermediate and overall goals of the building blocks framework (Table 5). This was followed by an in-depth analysis of the templates.
Results
Identifying templates
The scoping review identified 14 documents that met the inclusion criteria. Eight of these were publicly available templates. Six documents were reports, of which two could not be obtained from the authors (Country Health Profiles by EMRO; and Western Pacific Country Health Information Profiles). Ultimately, twelve templates could be included in the analysis. The screening process is illustrated in Fig. 2.
Table 3 presents the 12 templates identified. Another 26 documents met most of our criteria, but not all, or focused on specific topics (and not most building blocks), such as primary care; therefore, they were not considered “templates” (see Appendix 3 in the e-supplement). All tools and templates, with one exception (CEF-AHI), were developed by major organizations such as the WHO, the OECD, the USAID, the Commonwealth Fund, or collaborations between them.
Content of the templates
Table 4 provides an overview of data collection approaches suggested by the different templates. The OECD HSCS [32, 33] and the CMWF [27] are mostly qualitative, asking authors to answer closed and open-ended questions or suggesting in-depth interviews with stakeholders for data collection. Three templates, the CEF-AHI, WHO-MBB, and the USAID UHC-MF, are exclusively quantitative [26, 37, 39]. They focus only on quantitative indicators based on administrative data, censuses, and surveys. Seven templates, the HiT template [29], the SoHEU [36], the USAID HSAA [38], the PAHO HSCP [34], the CHSSP [28], the HSiAI [30], and the MFUHA [31] combine qualitative and quantitative approaches, describing and analyzing health systems with (qualitative) descriptions and (quantitative) indicators (see the first column of Table 4).
Table 4 also presents the health system functions covered by each template, indicating the names of the chapters where the topics are covered. Table 4 also shows that some functions, such as ‘service delivery’ and ‘financing’, are often covered in a separate, dedicated chapter. Other functions do not always have their own dedicated chapters, such as ‘health workforce’. Only the `Health Financing´ building block is covered in all the 12 templates, followed by ‘service delivery’ and ‘health workforce’, which are covered by 11 of the 12 templates. ‘Leadership and governance’ (10/12) and ‘medical products’ (7/12) follow in descending order. The building block least covered is Health information (5/12). The building block ‘medical products’ is almost exclusively covered by templates targeting middle- and low-income countries.
Table 5 presents the health system goals covered by each template. Two templates, i.e. the HiT, and the SoHEU, assess all health system goals. Eight templates consider most health system goals, while two templates, i.e. the WHO’s MBB and the USAID’s UHC-MF, focus only on a relatively narrow range of health system goals. The most frequently covered goals are ‘access and coverage’ (11/12) ‘quality and safety’ (10/12), and ‘financial protection’ (9/12). The other goals were covered at maximum by half of the templates (‘improved health’, 6/12). ‘Responsiveness’ and ‘efficiency’ are not covered by seven of the twelve templates.
Most templates (eight of twelve) also cover topics beyond WHO’s building blocks framework. Some cover “extra topics” to contextualize the health system (e.g. socio-demographic and political context, and determinants of health), while others assess changes and reforms. Further topics covered in the templates are resilience; ageing and long-term care; provider payment mechanisms; equity, competition and resource allocation (see the last column of Table 5).
Discussion
This is the first systematic scoping review of templates for standardized descriptions of health systems and for systematic assessments of performance. The review identified 12 templates, most of them written by international organizations. While most health system building blocks are addressed by all templates, some building blocks are less frequently covered, such as ‘medical products’ and ‘health information systems’. Similarly, some health system goals are not addressed by several templates, such as ‘responsiveness’ and ‘efficiency’. Seven templates provide guidance for collecting and presenting both quantitative and qualitative data, while three are mostly quantitative, and two are primarily qualitative. Our findings have implications for policy makers, for researchers and for organizations developing templates.
For policy makers, it is important to realize that health systems strengthening requires a systematic understanding of health systems. Many countries are developing strategic health system plans [41] and ‘health systems performance assessment’ strategies [5, 8, 42,43,44,45,46]. To assess health systems, policymakers and researchers require several tools that complement and build on each other. First, an analytical framework that defines the topics and aspects of health systems to be planned or assessed; second, descriptions of the structure and performance of the health system, based on the framework chosen; and third evaluations on the extent to which the system is performing and meeting its objectives [43]. While there are several analytical frameworks that define ‘a health system’ [5, 43, 46,47,48,49], and guides for health system performance assessments [5, 38, 42, 44, 50, 51], our work fills the knowledge gap in the second type of tools that describe the structures and functioning of the health system. First, by identifying the existing templates, second by analyzing their content, scope and methods, and third by highlighting how to improve these templates for a more systematic understanding of health systems structures and functioning.
Second, our work invites researchers to fill knowledge gaps. It calls for future research on a range of topics including: (1) identification of indicators ‘frequently used’ by templates, and their availability in different regions; (2) development of concepts and indicators for certain building blocks and health system goals (e.g. governance, health information, and responsiveness); and (3) the definition of “core” topics for templates. In addition, future work should explore whether templates achieve their goals of leading to standardized descriptions and assessments of health systems that are useful at the national level by informing policy-makers; and at the international level by facilitating health systems comparisons.
Finally, for organizations developing templates, our work might contribute to developing consensus towards elements that should be included in systematic analyses and assessments. In addition, results provide insights on three important aspects: (1) the relevance of context, (2) the potential to use more qualitative data for health systems analyses (3) and gaps of existing templates.
Templates support standardized descriptions of health systems but need to reflect context
The templates identified in this study were created by organizations with different agendas and target audiences, and with different end products in mind, i.e. ranging from brief descriptions or assessments to full studies with detailed descriptions and assessments. Templates aimed at different groups of countries, either clustering them by region, such as the African Region [12, 26], the Americas [52], Europe [30, 36], or by income. About half of the templates correspond either to health systems of high-income countries (HICs) (SoHEU [36]) or low- and middle-income countries (LMICs) (CEF-AHI [26], MFUHA [31], AHOP [28], UHC-MF [37], HSAA [38], WHO-MBB [39]). Five templates have been used to guide descriptions of HIC and LMICs (CMWF [27], HiT [29], HSCS, HSCP [32, 33], HSiAI [30]).
The first reason for this is that organizations developing templates usually target specific regions, with unique needs and characteristics, or different income-level countries. Some organizations target either HICs or LMICs, but not both. For example, the OECD focuses primarily on HICs, whereas the WHO prioritized projects that promote LMICs. Certain organizations are region-specific and target a certain region. For example, PAHO targets Latin-American countries, AHOP and USAID target mainly African countries, and the European Observatory on Health Systems and Policies focusses on the European region.
Second, templates apparently cover data and information that is mostly relevant to these specific regions or either for HICs or LMICs. For example, while templates focusing on HICs cover diseases such as cancer and dementia, templates for LMICs focus on maternal and infant mortality, tropical diseases, and undernutrition [53,54,55,56]. While non-communicable diseases (NCDs) represent the main burden and mortality reasons also in regions such as Africa, NCDs are neglected in international development assistance and health system strengthening efforts, as is reflected in the templates. Templates focusing on Latin American countries address tropical diseases, while templates focusing on Europe focus on non-communicable and cardiovascular diseases. This finding raises the question of whether it is appropriate to cover different regions, LMIC and HIC systems in one comparison, given their idiosyncrasies. For example, LMICs rely more on traditional practitioners and healers [57,58,59] or receive more foreign donor funds than HICs [60, 61]. Organizations comparing and assessing health systems should take these differences into consideration, while attempting to compare the same health system functions and goals [62]. Third, templates that target HICs rely more on health service utilization and other routinely collected data that is less available in LMICs, while templates dedicated to LMICs generally rely on fewer indicators, and on survey-based indicators that are more commonly available for these countries [63]. These findings suggest that quantitative data and indicators are generally less available for LMICs [64,65,66], which calls for national and international agencies to improve and ensure the collection of robust data and health statistics as a precondition to describe, compare and assess the performance of health systems.
Nonetheless, templates should be generic enough to be suitable for all types of countries, allowing for adaptations to reflect context and particularities [43]. Resolving the apparent tension between standardising templates while reflecting context can be achieved by having multiple organizations with different target audiences compile a template jointly and agreeing on a common framework that suits different types of countries, and covers the topics to be covered within the health systems building blocks and goals, the methods of data collection, and the set of indicators proposed. Templates can achieve this dual objective by proposing a list of core elements and functions that must be described within each building block of the health system, and another list of elective topics that could be chosen based on the context of the country being described. While all countries should describe the essential elements of each building block and to what extent they achieve the ultimate goals, there can be additional aspects that are related to each building block, which are relevant only for specific contexts. For example, all countries should cover the building block ‘workforce’, and describe trends in density of nurses and physicians and geographical distribution as core topics. Yet, the focus on skill mix and the challenges faced by each country may differ, and can be described as elective topics that reflect context. As an elective topic, skill mix lends itself to context-specific elaboration: while HICs could focus on the balance between primary and specialist care, the role of physician assistants and specialist nurses, and explore concerns related to quality of workforce training, LMICs could highlight a more diverse range of professionals such as community workers, include traditional medicine in the analysis, and explore concerns regarding the shift of specialized workers to private practice. A third and related strategy consists of compiling a list of core indicators that support the description and analysis of core topics, and a list of elective indicators from which authors can choose how to describe the contextual factors. There has been extensive work on defining a list of ‘core’ indicators [16, 17], which can be further developed to define the set of indicators for the elective topics [18].
There is an opportunity to use more qualitative data to analyze health systems
Existing macro-level health systems research has often focused on quantitative indicators, e.g. for classification of health systems into typologies [6, 62, 67, 68], for cross-country analyses of health reforms [69], or comparisons of performance [68, 70]. However, this data has several drawbacks: in high-income countries, there are frequent breaks in series and changing indicator definitions. In low-income countries, data on human and capital resources, service provision, and health system performance is often unavailable or outdated [63, 67, 81].
In several templates, qualitative descriptions are suggested to provide additional information on institutional or organizational health system characteristics, or explanations and interpretations of reported quantitative indicators, e.g. to explain breaks in series or discrepancies between national and international data. Qualitative information is more appropriate to capture processes, changes, and outcomes, and (over-)reliance on quantitative indicators may result in comparisons that are limited to quantifiable parameters. Therefore, comprehensive health systems analyses and comparisons require complementing quantitative indicators with qualitative information to build a more holistic picture [62, 64, 68]. If reporting of this information is sufficiently standardized, it can be useful for cross-country comparisons of health systems characteristics.
Qualitative data can be systematically collected from different sources including grey and academic literature, interviews with policymakers, scholars, health workers and managers, as well as interviews and focus groups among providers, patients and the general population. By using a standard interview guide, standard questionnaire, or table to be completed, data can be collected in a standardized manner that can be easily compared across countries. Qualitative data is particularly important when quantitative data or indicators are not collected, are not reliable or outdated. In addition, these sources can provide valuable information not captured by quantitative indicators, such as governance features, sources and content of information systems, or supply chains of pharmaceuticals. Interviews with stakeholders further shed light on processes that may complement or explain quantitative data, such as the considerations applied in health technology assessment beyond cost-effectiveness; (un)intended outcomes of payment mechanisms, such as transparency of financial flows, and additional parameters of quality of care that are not measured by quantitative indicators. Providers’, patients’ and the general population’s perspectives may add further value to quantitative indicators in understanding a host of issues, including quality of care, access barriers, dropout rates of health professions, or patient admission and treatment decisions.
Health information systems, responsiveness and efficiency are missing in some templates
The building block ‘Health information’ is not always covered by existing templates, and the health system goals of ‘responsiveness’ and ‘efficiency’ are sometimes missing. The reasons for the relative lack of attention to these topics remain unclear [7, 11, 44, 71,72,73,74] but could be related to the lower availability (or absence) of quantitative indicators to describe and assess these topics in some of the countries. Another reason could be the complexity in defining and collecting data for indicators that precisely capture these concepts. For example, measuring efficiency requires controlling for variations in quality of care, which is often not linked to cost data [75]. Moreover, measuring the achievement of health system goals, such as responsiveness, improved health, and efficiency, requires isolating the effects of determinants that fall outside the health system, which remains a challenge, and is still a work in progress [76].
The lack of attention to ‘Health information systems’ is surprising, given their central role for generating data that can be used to describe and steer health systems [71, 77]. We believe that this topic deserves more attention in the revision of existing or the development of new templates. Likewise, ‘governance’ is not always covered in existing templates, despite its importance for health systems' performance and resilience [78]. Potentially, the systematic description and assessment of governance could be improved through the incorporation of governance indicators that have been developed by various organizations [79]. Similarly, templates could be improved with regard to their approach to describing and measuring ‘efficiency’ and ‘responsiveness’ based on several analytical frameworks and indicators that have been developed [7, 9,10,11, 73, 75, 80,81,82,83,84,85].
Limitations
This study has several limitations. First, our overview of templates’ contents is omitting details and nuances, given that it was impossible to describe the actual questions and indicators of each template. Second, the choice to define and analyse templates based on the WHO’s health systems framework (2007) [22] may have biased our results, as we excluded from further analyses all documents that did not cover most building blocks. There are many different frameworks for the analysis of health systems, in part because a society’s cultural values influence how policy-makers, researchers or other stakeholders conceptualize and measure the performance of health system goals [48]. The choice of a certain framework certainly influences the outcomes measured—i.e., the ‘goals’ of the health system. Other frameworks or approaches would have yielded different results. Despite the disadvantages of the WHO framework, it is commonly used by policy-makers and researchers in HICs and LMICs. Therefore, we believe that it was the most suitable framework for our purpose. Finally, three of the authors (AM, BR and EvG) are authors of two of the templates, which may have influenced the conceptualization of a “template” and the choice of defining criteria. However, they tried to be as impartial as possible in this work, and the other authors balanced this potential bias.
Conclusions
Standardized reports of health systems’ characteristics and systematic assessments of their performance can support health systems strengthening. Templates should guide authors to describe the structure and functioning of any health system, while allowing for flexibility to account for context. Our review of templates shows that some health system building blocks (i.e. ‘governance’ and ‘health information systems’) and certain health system goals (i.e. ‘responsiveness’ and ‘efficiency’) are missing in several templates. A comprehensive health systems analysis and comparison requires a combination of quantitative indicators complemented with qualitative information to build a more holistic picture. The implications of our findings are that (1) policy-makers should demand systematic template-guided analyses of their health systems to underpin national health policies, strategies, and plans; (2) organisations developing templates should be inspired to learn from approaches of other templates at describing and assessing health systems and consider the incorporation of more qualitative information; and (3) researchers should strive to fill important knowledge gaps on indicators for health systems descriptions and assessments, as well as exploring how templates can be further improved to better achieve the goal of supporting policy-makers in strengthening health systems worldwide.
Data availability and materials
Data is publicly available.
Notes
For example, in some databases, the Boolean operators did not work (ELDIS) or the search field was limited to a certain number of characters (Google scholar).
Abbreviations
- CEF-AHI:
-
Common evaluation framework for the African Health Initiative
- CHSSP:
-
Country Health Systems and Service Profile
- CMWF:
-
Commonwealth Fund
- EMRO:
-
WHO’s Eastern Mediterranean Region
- EU:
-
European Union
- HIC:
-
High Income Countries
- HiT:
-
Health Systems in Transition
- HSAA:
-
Health System Assessment Approach
- HSCP:
-
Health Systems Country Profiles
- HSCS:
-
Health systems Characteristics survey
- HSiAI:
-
Health Systems in Action insights
- LMIC:
-
Low and Middle-income countries
- MFUHA:
-
Monitoring Framework for Universal Health in the Americas
- NCD:
-
Non-communicable disease
- OECD:
-
The Organisation for Economic Co-operation and Development
- PAHO:
-
Pan American Health Organization
- QUAL:
-
Qualitative
- QUAN:
-
Quantitative
- SoHEU:
-
State of Health in the EU
- UHC:
-
Universal Health Coverage
- UHC-MF:
-
Universal Health Coverage Monitoring Framework
- USAID:
-
United States Agency for International Development
- WHO:
-
World Health Organization
- WHOLIS:
-
WHO Library and Digital Information Networks
- WHO-MBB:
-
WHO Monitoring the Building Blocks
References
United Nations. Sustainable Development Goals|SDG Monitor. Available: https://www.sdgmonitor.co/sdgs. Accessed 22 Jul 2021.
Naimoli JF, Saxena S, Hatt LE, Yarrow KM, White TM, Ifafore-Calfee T. Health system strengthening: prospects and threats for its sustainability on the global health policy agenda. Health Policy Plann. 2018;33:85. https://doi.org/10.1093/heapol/czx147.
Kieny MP, Bekedam H, Dovlo D, Fitzgerald J, Habicht J. Strengthening health systems for universal health coverage and sustainable development. Bull World Health Organ. 2017;95(7):537–9. https://doi.org/10.2471/BLT.16.187476.
de Savigny D, Adam T, editors. Systems thinking for health systems strengthening. Geneva: World Health Organization; 2009.
Murray C, Frenk J. A WHO Framework for Health System Performance Assessment. 2018. Available: https://www.who.int/healthinfo/paper06.pdf.
Burau V, Blank RH, Pavolini E. Typologies of healthcare systems and policies. In: The Palgrave International Handbook of Healthcare Policy and Governance. London: Palgrave Macmillan; 2015. p. 101–15. https://doi.org/10.1057/9781137384935_7.
Cylus J, Papanicolas I, Smith P. Comparative measures of health system efficiency. In: Cylus J, Papanicolas I, Smith P, editors. Health systems performance comparison. Berkshire: Open University Press; 2013.
Papanicolas I, Kringos D, Klazinga NS, Smith PC. Health system performance comparison: new directions in research and policy. Health Policy (New York). 2013;112(1–2):1–3. https://doi.org/10.1016/J.HEALTHPOL.2013.07.018.
Cylus J, Papanicolas I, Smith P. A framework for thinking about health system efficiency. In: Cylus J, Papanicolas I, Smith P, editors. Health Policy Series No. 46 Health System Efficiency How to make measurement matter for policy and management. WHO Regional Office for Europe: Denmark/London; 2016.
Cylus J, Papanicolas I, Smith P. Health system efficiency: how to make measurement matter for policy and management. Copenhagen: European Observatory on Health Systems and Policies. 2016. Available: https://www.ncbi.nlm.nih.gov/books/NBK436888. Accessed 25 May 2021.
Cylus J, Papanicolas I, Smith PC. How to make sense of health system efficiency comparisons? 2017. Available: https://www.euro.who.int/__data/assets/pdf_file/0005/362912/policy-brief-27-eng.pdf.
AHOP African Health Observatory Platform. Available: https://ahop.aho.afro.who.int/. Accessed 24 Mar 2022.
NAO: North American Observatory on Health Systems and Policies | Institute of Health Policy, Management and Evaluation. Available: https://ihpme.utoronto.ca/research/research-centres-initiatives/nao/. Accessed 24 Mar 2022.
Asia Pacific Observatory on Health Systems and Policies. Available: https://apo.who.int/. Accessed 24 Mar 2022.
European Observatory on Health Systems and Policies. Available: https://eurohealthobservatory.who.int/. Accessed 24 Mar 2022.
Perić N, Hofmarcher MM, Simon J. Headline indicators for monitoring the performance of health systems: findings from the European Health Systems_Indicator (euHS_I) survey. Arch Public Health. 2018;76(1):1–17. https://doi.org/10.1186/S13690-018-0278-0/TABLES/7.
Forde I, Morgan D, Klazinga NS. Resolving the challenges in the international comparison of health systems: the must do’s and the trade-offs. Health Policy (New York). 2013;112(1–2):4–8. https://doi.org/10.1016/J.HEALTHPOL.2013.01.018.
Noto G, Corazza I, Kļaviņa K, Lepiksone J, Nuti S. Health system performance assessment in small countries: the case study of Latvia. Int J Health Plann Manage. 2019;34(4):1408–22. https://doi.org/10.1002/HPM.2803.
Sevim F, Aldogan EU. Evaluation of health systems performance of OECD countries using MOORA method. J Health Manag. 2024;26(1):172–83. https://doi.org/10.1177/09720634231215131.
OECD Health Statistics 2021. Available: https://www.oecd.org/els/health-systems/health-data.htm. Accessed 24 Mar 2022.
USAID. Health Systems Strengthening|U.S. Agency for International Development. Available: https://www.usaid.gov/global-health/health-systems-innovation/health-systems-strengthening. Accessed 24 Mar 2022.
World Health Organization. Every body’s business: strengthening health systems to improve health outcomes who’s framework for action. Geneva, 2007. Available: https://www.who.int/healthsystems/strategy/everybodys_business.pdf.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. https://doi.org/10.1080/1364557032000119616.
Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. 2015;4(1):1–10. https://doi.org/10.1186/S13643-015-0125-0/FIGURES/2.
Tricco AC, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. https://doi.org/10.7326/M18-0850.
Bryce J, Requejo JH, Moulton LH, Ram M, Black RE. A common evaluation framework for the African Health Initiative. BMC Health Serv Res. 2013;13(S2):1–9. https://doi.org/10.1186/1472-6963-13-S2-S10.
Tikkanen R, Osborn R, Mossialos E, Djordjevic A, Wharton G. International Health Care System Profiles|Commonwealth Fund. 2020. Available: https://www.commonwealthfund.org/international-health-policy-center/system-profiles. Accessed 15 Jul 2021.
World Health Organization. Regional Office for Africa. AHOP Country Health Systems and Services Profile (CHSSP): an overview. 2023. Available: https://ahop.aho.afro.who.int. Accessed 03 Nov 2023.
Rechel B, Maresso A, Van Ginneken E. Health Systems in Transition—Template for Authors. 2019 Brussels, 2019. Available: https://www.euro.who.int/__data/assets/pdf_file/0009/393498/HiT-template-for-web-for-authors-2019.pdf. Accessed 05 Jan 2022.
European Observatory on Health Systems and Policies. European Observatory’s Health Systems in Action insights. Available: https://eurohealthobservatory.who.int/publications/health-systems-in-action-insights. Accessed 27 Oct 2023.
Pan American Health Organization. Monitoring Framework for Universal Health in the Americas. Washington, D.C., 2021. Available: https://iris.paho.org/handle/10665.2/53918. Accessed 03 Nov 2023.
OECD. Health systems characteristics survey. 2019. Available: https://qdd.oecd.org/subject.aspx?Subject=hsc. Accessed 03 Nov 2023.
Paris V, Devaux M, Wei L. Health Systems Institutional Characteristics: a Survey of 29 OECD Countries. Paris, 2010. https://doi.org/10.1787/5kmfxfq9qbnr-en.
PAHO and WHO. Methodological guidelines, health system profiles, monitoring and analyzing health systems change. Washington, DC, 2006. Available: https://www3.paho.org/hq/index.php?option=com_docman&task=doc_download&gid=10244&Itemid=270&lang=en. Accessed 15 Jul 2021.
OECD and European Observatory on Health Systems and Policies, Bulgaria: Country Health Profile 2021, State of Health in the EU. Paris: OECD Publishing, 2021. https://doi.org/10.1787/c1a721b0-en.
European Observatory on Health Systems and Policies and OECD, “State of Health in the EU Country Health Profiles. 2019. Available: https://ec.europa.eu/health/state/country_profiles_en. Accessed 15 Jul 2021.
Universal health coverage monitoring framework indicators and proxy indicators. 2017.
USAID. Health Systems Assessment Approach. 2017. Available: https://hsaamanual.org/download-the-pdf/. Accessed 15 Jul 2021.
WHO. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva, 2010. Available: https://www.who.int/healthinfo/systems/WHO_MBHSS_2010_full_web.pdf. Accessed 09 Jul 2021.
Lorenzoni L, et al. Health spending projections to 2030: new results based on a revised OECD methodology. In: OECD Health Working Papers, No. 110. Paris: OECD Publishing; 2019. https://doi.org/10.1787/5667f23d-en.
WHO. Country Planning Cycle Database. Available https://extranet.who.int/countryplanningcycles/. Accessed 19 Dec 2022.
Health system performance assessment-European Public Health. Available: http://www.europeanpublichealth.com/health-systems/health-system-performance-assessment/. Accessed 24 May 2021.
Irene Papanicolas JF, Rajan D, Karanikolos M, Soucat A. Health system performance assessment: a framework for policy analysis. Health Policy Ser. 2022;57:246.
Murray CJL, Evans DB. Health systems performance assessment: debates, methods and empiricism. Geneva: World Health Organization; 2003.
Papanicolas I, Smith PC. Health System Performance Comparison: An Agenda for Policy, Information and Research. Maidenhead: Open University Press, 2013. Available: http://eprints.lse.ac.uk/54802/1/__libfile_REPOSITORY_Content_Papanicolas%2C%20I_Health%20system%20performance_Papanicolas_Health%20system%20performance_2013.pdf, Accessed 29 Jul 2021.
Braithwaite J, et al. Health system frameworks and performance indicators in eight countries: a comparative international analysis. SAGE Open Med. 2017;5:2050312116686516. https://doi.org/10.1177/2050312116686516.
Sacks E, et al. Beyond the building blocks: integrating community roles into health systems frameworks to achieve health for all. BMJ Glob Health. 2019;3(Suppl 3): e001384. https://doi.org/10.1136/BMJGH-2018-001384.
Collins C, Green A. Valuing health systems : a framework for low and middle income countries. London: Sage. 2022. Available: https://books.google.com/books/about/Valuing_Health_Systems.html?hl=iw&id=qPanAwAAQBAJ. Accessed 08 Nov 2023.
Rechel B, et al. A framework for health system comparisons: the health systems in transition (HiT) Series of the European Observatory on Health Systems and Policies. 2019; pp. 279–296. https://doi.org/10.1007/978-1-4939-8715-3_15.
European Commission. Health systems performance assessment. Available: https://health.ec.europa.eu/health-systems-performance-assessment_en. Accessed 30 Oct 2023.
OECD. Health System Performance Assessment-OECD. Available: https://www.oecd.org/health/health-system-performance-assessment.htm. Accessed 30 Oct 2023.
PAHO and WHO. Monitoring framework for universal health in the Americas. 2021. Available: https://iris.paho.org/bitstream/handle/10665.2/53918/9789275122709_eng.pdf?sequence=1&isAllowed=y. Accessed 02 Nov 2023.
Murray CJL, et al. Global burden of 87 risk factors in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. 2020;396(10258):1223–49. https://doi.org/10.1016/S0140-6736(20)30752-2.
WHO. Global Health Risks: Mortality and Burden of Disease Attributable to Selected Major Risks. Geneva, 2009.
WHO. Investing to Overcome the Global Impact of Neglected Tropical Diseases. World Health Organization. Geneva, 2015.
Stevens P. Diseases of poverty and the 10/90 Gap. Islington, Nov. 2004.
Kim JK, Kim KH, Shin YC, Jang BH, Ko SG. Utilization of traditional medicine in primary health care in low- and middle-income countries: a systematic review. Health Policy Plan. 2020;35(8):1070–83. https://doi.org/10.1093/HEAPOL/CZAA022.
James PB, Wardle J, Steel A, Adams J. Traditional, complementary and alternative medicine use in Sub-Saharan Africa: a systematic review. BMJ Glob Health. 2018;3:895. https://doi.org/10.1136/bmjgh-2018-000895.
Oyebode O, Kandala N-B, Chilton PJ, Lilford RJ. Use of traditional medicine in middle-income countries: a WHO-SAGE study. Health Policy Plan. 2016;31:984–91. https://doi.org/10.1093/heapol/czw022.
Huffstetler HE, et al. The impacts of donor transitions on health systems in middle-income countries: a scoping review. Health Policy Plan. 2022;37:1188–202. https://doi.org/10.1093/heapol/czac063.
Ann Grépin K, Beth Pinkstaff C, Cyrus Shroff Z, Ghaffar A. Donor funding health policy and systems research in low-and middle-income countries: how much, from where and to whom. https://doi.org/10.1186/s12961-017-0224-6.
de Carvalho G, Schmid A, Fischer J. Classifications of health care systems: do existing typologies reflect the particularities of the Global South? Global Social Policy. 2020. https://doi.org/10.1177/1468018120969315.
Polin K, Shuftan N, Webb E, Opoku D, Droti B, Quentin W. Data for health system comparison and assessment in the African Region: a review of 63 indicators available in international databases. J Global Health. 2024. https://doi.org/10.7189/jogh.14.04118.
Stevens GA et al. Statement guidelines for accurate and transparent health estimates reporting: the GATHER statement. www.thelancet.com. p. 19, https://doi.org/10.1016/S0140-6736(16)30388-9.
Abouzahr C, Boerma T, Hogan D. Global estimates of country health indicators: useful, unnecessary, inevitable? Glob Health Action. 2017;10:1. https://doi.org/10.1080/16549716.2017.1290370.
Missoni E, Pacileo G, Tediosi F. Global health governance and policy : an introduction. Oxon and New York: Routledge, 2019. Available: https://www.routledge.com/Global-Health-Governance-and-Policy-An-Introduction/Missoni-Pacileo-Tediosi/p/book/9780815393290. Accessed 08 Nov 2023.
Reibling N, Ariaans M, Wendt C. Worlds of healthcare: a healthcare system typology of OECD countries. Health Policy (New York). 2019;123(7):611–20. https://doi.org/10.1016/J.HEALTHPOL.2019.05.001.
Frisina Doetter L, Schmid A, de Carvalho G, Rothgang H. Comparing apples to oranges? Minimizing typological biases to better classify healthcare systems globally. Health Policy Open. 2021;2:100035. https://doi.org/10.1016/J.HPOPEN.2021.100035.
Moreno-Serra R, Wagstaff A. System-wide impacts of hospital payment reforms: evidence from Central and Eastern Europe and Central Asia. J Health Econ. 2010;29(4):585–602. https://doi.org/10.1016/j.jhealeco.2010.05.007.
Wendt C. Changing healthcare system types. Soc Policy Adm. 2014;48(7):864–82. https://doi.org/10.1111/SPOL.12061.
Colombo F, Oderkirk J, Slawomirski L. Health information systems, electronic medical records, and big data in global healthcare: progress and challenges in OECD countries. In: Handbook of global health. Cham: Springer International Publishing; 2020. p. 1–31. https://doi.org/10.1007/978-3-030-05325-3_71-1.
Papanicolas I, Smith PC. International Comparisons of Health Systems. In: Pananicolas I, Smith PC, Eds. Health system performance comparison. An agenda for policy, information and research. European Observatory on Health Systems and Policies Series, 2013. Available: https://books.google.de/books?hl=iw&lr=&id=-n1FBgAAQBAJ&oi=fnd&pg=PA75&dq=International+Comparisons+of+Health+Systems%E2%80%9D.+in+Papanicolas+and+Smith+&ots=DerM2PDUFA&sig=Nv0BWlOfh7xgoaoFyNyguNtfPSo&redir_esc=y#v=onepage&q=International%20Comparisons%20of%20Health%20Systems%E2%80%9D.%20in%20Papanicolas%20and%20Smith&f=false. Accessed 08 Feb 2021.
Mirzoev T, Kane S. What is health systems responsiveness? Review of existing knowledge and proposed conceptual framework. BMJ Glob Health. 2017;2(4): e000486. https://doi.org/10.1136/BMJGH-2017-000486.
Health system performance comparison. An agenda for policy, information and research. 2017. Available: https://www.euro.who.int/en/about-us/partners/observatory/publications/studies/health-system-performance-comparison.-an-agenda-for-policy,-information-and-research-2013. Accessed 22 Dec 2020.
Hussey PS, et al. A systematic review of health care efficiency measures. Health Serv Res. 2009;44(3):784. https://doi.org/10.1111/J.1475-6773.2008.00942.X.
Murray CJL, Salomon JA, Mathers C. Policy and practice a critical examination of summary measures of population health. Bull World Health Organ. 2000; 78(8).
Panteli D, et al. Health and Care Data. Health Syst Transit. 2023; 25(2). Available: www.healthobservatory.eu. Accessed 01 Nov 2023.
Greer SL, Wismar M, Figueras J. Strengthening Health System Governance European Observatory on Health Systems and Policies Series. Open University Press, 2016. Available: https://www.euro.who.int/__data/assets/pdf_file/0004/307939/Strengthening-health-system-governance-better-policies-stronger-performance.pdf. Accessed 22 Dec 2022.
Worldwide Governance Indicators. Available: https://info.worldbank.org/governance/wgi/. Accessed 22 Dec 2022.
OECD and EC. Scoping paper on health system efficiency measurement (cooperation between the OECD and EC in promoting efficiency in health care). 2016. Available: https://www.oecd.org/els/health-systems/Scoping-Paper-Measuring-efficiency-in-health-system.pdf. Accessed 01 Nov 2023.
European Commission. Tools and methodologies to assess the efficiency of health care services in Europe. An overview of current approaches and opportunities for improvement Report by the Expert Group on Health System Performance Assessment. 2019. https://doi.org/10.2875/77000.
Charlesworth A, Or Z, Spencelayh E. Efficiency measurement for policy formation and evaluation. In: Cylus J, Papanicolas I, Smith PC, editors. Health system efficiency: how to make measurement matter for policy and management, vol. 1. Copenhagen: WHO Regional Office for Europe; 2016. p. 167–202.
Cylus J, Papanicolas I, Smith PC. Using data envelopment analysis to address the challenges of comparing health system efficiency. Glob Policy. 2017;8:60–8. https://doi.org/10.1111/1758-5899.12212.
De Silva A et al. A framework for measuring responsiveness. Geneva, 2000. Available: https://www.who.int/responsiveness/papers/paper32.pdf. Accessed 09 Jul 2021.
Mbau R, et al. Analysing the efficiency of health systems: a systematic review of the literature. Appl Health Econ Health Policy. 2023;21(2):205–24. https://doi.org/10.1007/S40258-022-00785-2/TABLES/7.
Acknowledgements
We thank Nathan Shuftan and Erin Webb for their valuable comments and ideas that helped to improve this paper. We acknowledge Alida Tiekoetter for validating the results, and Nure Jannat Ara Riya for helping compile the tables.
Funding
Open Access funding enabled and organized by Projekt DEAL. This research was produced as part of a programme of work conducted by the African Health Observatory Platform on Health Systems and Policies (AHOP) and supported by the Bill & Melinda Gates Foundation.
Author information
Authors and Affiliations
Contributions
WQ and RW designed, IP and RW executed the study. All authors wrote and critically reviewed the paper.
Corresponding author
Ethics declarations
Ethical approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Creative commons license
All co-authors confirm the agreement to publish the article under the terms of a Creative Commons Attribution 3.0 IGO license.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Waitzberg, R., Pfundstein, I., Maresso, A. et al. Health system description and assessment: a scoping review of templates for systematic analyses. Health Res Policy Sys 22, 82 (2024). https://doi.org/10.1186/s12961-024-01166-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12961-024-01166-y