Skip to main content

Improving the validity, relevance and feasibility of the continuum of care framework for maternal health in South Africa: a thematic analysis of experts’ perspectives

Abstract

Background

The continuum of care is a key strategy for ensuring comprehensive service delivery for maternal health, while acknowledging the role of the social determinants of health. However, there is little research on the operationalisation of the framework by decision-makers and implementers to address maternal health challenges. The framework should be measurable and feasible for implementation in low- and middle-income country contexts. In this study, we explore experts’ perspective on monitoring indicators for continuum of care and key issues related to their use in the South African context.

Methods

We conducted key informant interviews with a range of experts in decision-making and programme implementation roles in the health system and relevant sectors. Key informants provided their perspectives on systematically selected, nationally representative monitoring indicators in terms of validity, relevance and feasibility. We interviewed 13 key informants and conducted a thematic analysis of their responses using multi-stage coding techniques in Atlas.ti 8.4.

Results

Experts believed that the continuum of care framework and monitoring indicators offer a multisectoral perspective for maternal health intervention missing in current programmes. To improve validity of monitoring indicators, experts suggested reflection on the use of proxy indicators and improvement of data to allow for equity analysis. In terms of relevance and feasibility, experts believe there was potential to foster co-accountability using continuum of care indicators. However, as experts stated, new indicators should be integrated that directly measure intersectoral collaboration for maternal health. In addition, experts recommended that the framework and indicators should evolve over time to reflect evolving policy priorities and public health challenges.

Conclusion

Experts, as decision-makers and implementers, helped identify key issues in the application of the continuum of care framework and its indicators. The use of local indicators can bring the continuum of care framework from an under-utilised strategy to a useful tool for action and decision-making in maternal health. Our findings point to measurement issues and systematic changes needed to improve comprehensive monitoring of maternal health interventions in South Africa. Our methods can be applied to other low- and middle-income countries using the continuum of care framework and locally available indicators.

Peer Review reports

Background

A review of national and global sources estimated that maternal mortality for South Africa was between 138 and 157.9 deaths per 100,000 live births in 2015 [1]. This is an improvement from the range of 140 to 174.1 deaths per 100,000 live births reported in 2013 [1]. The latest South African Demographic and Health Survey shows an increase in pregnancy-related mortality ratio (number of pregnancy-related deaths per 100,000 live births) from 150 in 1998 to 536 in 2016 [2]. Pregnancy related-deaths are classified as any deaths from any cause occurring during the ‘maternal risk period’ from pregnancy to 6 weeks postpartum [3]. The measure is sometimes used in South Africa due to challenges in estimation of the maternal mortality ratio [2]. Based on the resources the country invests in health, these estimates for maternal and pregnancy-related mortality should be lower [4]; they should reflect better progress towards the Sustainable Development Goals’ target for maternal mortality (a national goal of below 100 deaths per 100,000 live births) [5].

A continuum of care approach has been recommended for the country to improve maternal health outcomes [6,7,8]. Continuum of care for maternal health is the delivery of a myriad health promotion, preventive and curative interventions from pre-conception to postnatal care [7,8,9]. It is defined as “access to care provided by families and communities, by outpatient and outreach services, and by clinical services throughout the lifecycle, including adolescence, pregnancy, childbirth, the postnatal period, and childhood. Saving lives depends on high coverage and quality of integrated service-delivery packages throughout the continuum, with functional linkages between levels of care in the health system and between service-delivery packages, so that the care provided at each time and place contributes to the effectiveness of all the linked packages” ([9] p. 1359). Delivery of a continuum of care for maternal health is integrated with child health interventions, though in this study we focus only on the former. The aim of the continuum of care approach is to improve the efficiency and effectiveness of health service organisation; it increases the integration of services and reduces the duplication of effort, increases linkages between different stages and levels of care and, ultimately, improves health outcomes and outcomes of care [9, 10].

National stakeholders in the South African health system (the Department of Health (DOH) and partner organisations) outlined a continuum of care framework as presented in Fig. 1 [7]. The framework serves to frame health policy and planning by promoting comprehensive service delivery involving all levels of the system across the lifecycle. It also highlights the importance of the social determinants of health acting at the community level of care to influence health outcomes. The social determinants of maternal health include structural determinants related to governance and policies as well as cultural and social values [11, 12]. In addition, the community context of living and other material conditions, social capital and social cohesion, health system, behavioural and psychosocial, and biological factors also influence maternal health outcomes as intermediary determinants [11, 12]. In the South African framework, there is a focus on ‘intersectoral factors’ that cross-cut a few social determinants of health. The South African health system aims to improve maternal health through multisectoral action to address the social determinants of health [6, 13, 14]. The current framework differs from other models that typically define the continuum of care for maternal health only from pregnancy to the postnatal period, with the exclusion social determinants [15].

Fig. 1
figure 1

Continuum of care framework for maternal and child health in South Africa. The figure shows the packages of interventions necessary for provision to mothers over time and place. The vertical dimension is place, which ranges from family and community level to regional hospital. The horizontal dimension is time, which ranges from pre-pregnancy to childhood. Each place dimension is colour coded differently to highlight interventions needed at that level over time. Also crucial to the continuum of care are connecting the vertical levels of caregiving (place) and connecting caregiving over the lifecycle (time). The interventions outlined for each level come from evidence-based guidelines and were decided consultatively for the South African government by the relevant health, scientific and development bodies stated on the report. Fig. 1 is reproduced from a publicly available government report [7]. There are no permissions required to reproduce figures and tables produced by the National Department of Health and made available in public reports

The current strategic framework for maternal health in South Africa envisions future monitoring and evaluation of services from a continuum of care perspective [16]. Quality of care initiatives also recommend action on the continuum of care to improve maternal health in South Africa [6]. Although the strategic goal exists, as yet there is no defined approach for the planning of services and monitoring of performance along the continuum of care. There is a research gap in defining and validating a comprehensive set of indicators that can be used to assess and monitor the continuum of care for maternal health in the country. Implementing and monitoring the comprehensive set of services on the continuum will complement current quality of care efforts in the health system to improve maternal health outcomes.

In a preliminary study [17], we used the continuum of care framework (Fig. 1) to guide the identification of relevant indicators for all services, using local data sources. These data sources included the South African District Health Information System and household surveys. The aim was to propose a set of indicators for monitoring the continuum of care for maternal health in South Africa. The available indicators are currently used in programmes to monitor various maternal health and community interventions. In this study, we ask experts to reflect on their potential use within the continuum of care framework. Using existing indicators relieves the data collection burden from an already overwhelmed health system workforce.

For existing indicators to support policy and programme decision-making, they have to be valid, relevant and feasible in the new context for which they are intended [18,19,20]. To expand on the preliminary study that identified the indicators, in this study we aim to explore experts’ perspectives on key issues that arise in re-orienting indicators to monitor the continuum of care. The purpose of the study is to strengthen the continuum of care framework and improve its potential use in maternal health policy and planning. The study is part of a larger project to identify, evaluate and use available indicators to track continuum of care performance for maternal health in South Africa.

Methods

Aim

The aim of this study was to assess experts’ perspectives of the validity, relevance and feasibility of selected indicators and the indicator framework for the continuum of care for maternal health in South Africa. The study complements others in a larger project that identified the relevant indicators [15, 17] and will be used in the future to assess performance.

Design

We conducted semi-structured key informant interviews with policy and programme decision-makers from the health system and other sectors. We focused on the framework and the domains of ‘empowerment’, ‘nutrition’ and ‘quality of care’. Our preliminary research had shown that ‘quality of care’ was a weak area of continuum of care measurement [15]; ‘empowerment’ and ‘nutrition’ are broad and multidimensional domains that needed further contextualisation for South Africa. The use of specific indicator domains also helps to delve into the thematic issues affecting the validity of the indicators in general. The individual indicators used present an opportunity for participants to elaborate by making concrete examples from the indicator framework.

Preliminary research

Indicators were selected from local data sources using criteria developed by the authors. The criteria were (1) frequent data collection, (2) reliability, (3) already in use in policy- and decision-making in South Africa, (4) alignment with the continuum of care framework, and (5) nationally and sub-nationally representative data [17]. The National Indicator Data Set of the District Health Information System of South Africa was used to select health service indicators and the annual General Household Survey was used to select data on services related to social determinants of health [17]. These indicators are presented in Table 1.

Table 1 List of indicators across the continuum of care for maternal health in South Africa

Participants

We sought a purposeful sample of experts, with variation in experiences across maternal health policy and programmes, social determinants of health, monitoring and evaluation, and multisectoral collaboration. Our sample included five experts from the health system (national and provincial DOH) working in maternal health policy and programme design, monitoring and evaluation (M&E), and quality assurance; four academics with experience in maternal health and social determinants of health (such as public health nutrition and empowerment); and four experts with experience in multisectoral collaboration to address social determinants of health (from Department of Agriculture, Forestry and Fisheries (nutrition and food security division), bilateral and development organisations, and Department of Planning, Monitoring and Evaluation). All experts have experience working in South Africa or similar low- and middle-income country (LMIC) contexts.

Data collection

We searched online sources, used the authors’ networks and snowballing techniques to identify experts (potential participants), and kept a list in Microsoft Excel 2010. Potential participants were contacted via email and those available were sent informational material ahead of the scheduled interview. The informational material introduced the study and provided the table of indicators on which experts based their responses (Table 1). The interview guide and consent form were also sent ahead of the interview. Before data collection, the interview guide was tested on four suitable experts that were not part of the study sample, and refined for clarity of questions. After refining the guide, the interviews were conducted among 13 experts between January and May 2019. We contacted 19 experts, 16 confirmed participation and only 13 were interviewed due to time constraints on experts’ part. One author (MM) interviewed the participants in face-to-face meetings and via telephone. The open-ended nature of interview questions as well as probing by the interviewer reduced telephone participants’ likelihood to put less effort in responses (satisficing). Interview questions were not of a personal or sensitive nature, thus reducing the social desirability bias and lack of trust that may occur in telephone interviews. Similarly, the nature of questions did not impede response quality due to the lack of anonymity in face-to-face interviews. The interviews lasted between 40 min and 1 hour. The researcher took notes during and immediately after interview using Microsoft Word 2010; data was entered into Atlas.ti 8.4 for coding and analysis.

Data analysis

The three themes of validity, feasibility and relevance guided the design of the open ended interview guide and a priori coding of responses: these themes are defined in Box 1. Validity referred to experts’ opinion of the suitability of indicators to measure and monitor specific domains. Relevance and feasibility refer to experts’ perspective on whether the framework and its indicators can support action and decision-making in the health system as well as multisectoral collaboration for maternal health. These concepts were adapted from Blas et al.’s [18] study of the measurement and monitoring of social determinants of health in LMICs. In Box 1, we merge both technical (ease of data acquisition, analysis and interpretation) and policy and programmatic feasibility (messages from indicators are ‘communicable’ and ‘comprehensible’) as described by Blas et al. [18]. Other interpretations of validity may intertwine with relevance, and feasibility is an important consideration in the use of indicators [21].

We conducted in vivo coding of responses under the a priori themes, followed by closer analysis and open coding of emerging concepts. A third stage of coding involved the synthesis of emerging concepts into sub-themes (list coding), including any contrasting views. Multistage coding is an essential component of thematic analysis and helps with the layering of issues and development of themes in a transparent, rigorous manner [22]. One author was involved in the coding, and two authors were involved in the analysis.

Trustworthiness

Although using experts’ perspectives to assess validity is considered the least rigorous validation method [23], it is the most justified in our study because the goal was to explore the framework and indicators by those likely to use them. In addition, all the indicators are already in use for decision-making in their relevant sectors. We used a transparent and systematic approach for selecting indicators. The approach used a systematic review to define the measurement model for the continuum of care for maternal health, local data sources for availability of indicators, and individual indicators based on evidence from published literature and policy documents [15, 17]. This study serves as a qualitative evaluative exercise meant to support the interpretation of the continuum of care framework and its indicators in the South African context.

Reflexivity

The framing of questions for the interviews reflects authors’ knowledge and perspectives of what is important to explore given current policy directions for maternal health in South Africa and other LMICs. There are similarities between South Africa and other LMICs in terms of systemic priorities for addressing maternal health challenges such as action on social determinants of health as well as improved quality of care and monitoring and evaluation. The main areas of interest were defined in advance and thus influenced the emergence of sub-themes. We defined validity, reliability and feasibility in advance because indicators are quantitative measures whose quality can be assessed from specific perspectives. Without guiding themes, participants could have probably had broader interpretations of the indicators and indicator framework.

Ethics

Institutional ethics clearance was obtained from the University of the Western Cape Biomedical Research Ethics Committee as well as the departments where experts’ worked (where prior permission was needed). All transcripts were confidentially kept and are accessible only to authors, and names of experts are not included in the analysis and reporting.

Results

In this section, we discuss key cross-cutting issues raised by participants regarding the framework and indicators of the continuum of care for maternal health. These issues emerged as sub-themes under the overarching themes of validity, relevance and feasibility.

Validity of the framework and indicators

According to participants, the framework and indicators provided a holistic, multisectoral perspective of interventions needed to improve maternal health in the country. Participants saw the framework as a good basis for thinking about future integrated monitoring and evaluation of maternal health. As a participant stated:

These frameworks have a role because multisectoral action plans need to be monitored … The frameworks are useful to groups that care about the holistic perspective.” (Public health nutrition and M&E researcher, academic institution)

In multisectoral platforms, participants believe frameworks such as the continuum of care can help monitor interventions by different sectors towards improved maternal health outcomes. A participant with experience in multisectoral public health programmes elaborated:

Frameworks are useful for program design and to coalesce stakeholders around an idea; if there is buy-in they can get everyone aligned around the same goals and strategies … It has to be the foundational basis if it is to be useful for monitoring and evaluation.” (Multisectoral programme specialist, development organisation)

Some of the indicators in the continuum of care framework already form part of national multisectoral strategic frameworks. From a multisectoral monitoring and evaluation perspective, a participant reflected:

Continuum of care would fall under the outcomes monitoring, especially related to the Medium Term Expenditure Frameworks (MTEF), which is what all departments use to set their strategic plans, including the Department of Health (DOH). So from the MTEF the DOH will get the target of reducing maternal mortality in order to contribute to the national development plan. But there are also indicators in outcomes monitoring that are monitored for the improvement of maternal health – and this is about ARV [anti-retroviral drugs] access, antenatal care, attendance of early postnatal care etc …” (M&E specialist, Department of Planning, Monitoring and Evaluation)

The Medium Term Expenditure Framework is a multisectoral strategic budgeting framework used by the government to plan the country’s development through all its departments. Continuum of care, as a phenomenon, is seen as an outcome that can be monitored to track the collective action of sectors. This implies the need for formulation of a composite outcome indicator for the continuum of care from the existing set.

The validity discussion also led to the emergence of two sub-themes reflecting gaps and challenges perceived by participants, which were (1) the use of proxy indicators and (2) measurement of indicators for sub-groups within maternal health.

Use of proxy indicators

Proxy indicators provide an indirect measurement of interventions and domains when direct data is not available. Participants felt that some of the proxy indicators used could not adequately measure interventions targeted for maternal health. An example was multidimensional quality of care or the ideal clinic status indicator (Indicator 20, Table 1). This indicator measures quality of care from different dimensions at the facility level. Although it captures some maternal health components, it more broadly indicates the quality environment in which women and other population groups receive care. Thus, the participants felt that the indicator should be supplemented by others that more specifically measured quality of care for maternal health. Similarly, the fortification of foods indicator (Indicator 3, Table 1) measures intervention at the community level, but not utilisation by women of reproductive age.

Participants felt that the use of proxy indicators represented a compromise when data is unavailable. In the context of the continuum of care, participants felt there needed to be a balance between collection of data on new, more direct indicators and their usefulness for decision-making. As a respondent stated:

These are good indicators and they are comprehensive enough because we don’t even want to come up with too many indicators. Dietary diversity score [for example] is based on food groups and although we may need to go deep into having macro- and micro-nutrient information, and measure the micro-nutrient adequacy score, these are used by specialists. What we have is adequate for programs and to inform us about the situation in South Africa.” (Maternal health and nutrition researcher, academic institution).

Thus, there are indicators that may be useful for researchers but not policy and programme implementers. For example, the dietary diversity score (Indicator 35, Table 1) is used in research and interventions that improve food security for individuals and populations. In contrast, other indicators related to nutrition may be outside of the scope of monitoring inputs/interventions along the continuum of care for maternal health.

Measurement for sub-groups

Participants reflected on measurement and monitoring of the indicators along the continuum of care, and the extent to which they represent services to sub-groups of women of reproductive age. As a non-homogenous population group for maternal health, women of reproductive age may have different barriers in accessing services along the continuum. For example, a participant states that the framework requires,

“… tweaking of some indicators to reflect the context of measurement. For example, the accessibility of the facility to people with disabilities. In the context of maternal health language disability is the most critical than physical disability” (Policy-maker, DOH).

Tweaking then implies modification or integration of indicators that reflect local public health needs and promotes equitable service provision. Another example stated by participants was services to younger mothers, whose indicators can measure the quality elements of youth friendliness and interpersonal communication between providers and patients (i.e. patient experience of care). When there is no sub-grouping, it may be difficult to design and monitor targeted services along the continuum of care that also address inequitable access. The rural and urban divide was another equity dimension mentioned by participants as influential on the social determinants of health and access to care, and thus important to assess from the continuum of care perspective. Some indicators, such as obstetric emergency transport in rural vs. urban areas (Indicators 22–26, Table 1), were a good demonstration of this stratification, according to participants. To the extent possible, other indicators should explore these sub-divisions.

In terms of equity and the social determinants of maternal health, a participant stated:

There is a balance between acknowledging social determinants of health are important and focusing on the interventions that the health system needs to do. For the most part, for example in maternal health, the health system works on their own evidence of what preventable causes of death are and addresses those. Social determinants of health are acknowledged. But we need an equity lens to the maternal mortality rates that can contribute to a reporting of the attribution of other determinants on outcomes.” (Technical Advisor, DOH)

Thus, the health system can also integrate social determinants of health data to help attribute maternal health outcomes to interventions and exposures outside of the health sector.

In summary, the participants felt that the framework and its indicators were valid in terms of providing a broad multisectoral perspective of maternal health intervention necessary to achieve and attribute outcomes. Validity could be improved by disaggregation of indicators to reflect maternal health subgroups for equity analysis and collection of more direct data. However, the collection of more data should be an exercise in balancing indicator utility and their precision.

Policy and programmatic relevance and feasibility

According to participants, the multisectoral nature of the indicators means that they can complement multisectoral collaboration for maternal health at strategic levels. Participants stated that there was various multisectoral work on-going between health and other sectors in South Africa such as agriculture (nutrition and food security) and social development (women’s empowerment). However, these were restricted to the operational level without shared monitoring and evaluation frameworks. According to participants, this leads to a fragmentation in goals and perpetuates the current ‘silo’ functioning of separate government departments. According to a participant:

From the health perspective it has been about getting other sectors to do what health wants them to do. Which might not be best approach and a stumbling block” (Technical Advisor, DOH)

In contrast, a co-accountability mechanism for maternal health outcomes will use shared frameworks that integrate indicators from different sectors. According to participants, this solves the essential problem, whereby it is “… difficult to get sectors to care about the work that other sectors do, especially their monitoring indicators” (Public health nutrition researcher, academic institution). Using the example of the role of nutrition in maternal health outcomes, another participant stated that:

There are more than 60 programs in different government departments dealing with food security and nutrition, including health. Nothing much happens in multisectoral action – a coordinating department should be having an information management system as a resource for anyone who wants to do research on food security and health” (Technical Advisor and Programme Co-ordinator, Department of Agriculture, Forestry and Fisheries)

According to the participant, this points to the lack of relevant shared platforms and tools for cross-sectoral learning and research for informed decision-making.

Two sub-themes emerged that relate to the main gaps and challenges related to relevance and feasibility of the framework and its indicators, which were (1) lack of indicators for intersectoral action and (2) the need for the framework to reflect the evolving policy and public health context in the country.

Lack of indicators for intersectoral action

Participants reflected on whether the indicators adequately captured the interface between health and other sectors in multisectoral collaboration to improve maternal health outcomes. At the interface of the health system and other sectors, participants stated that the continuum of care framework should include indicators that measure intersectoral action. According to a participant:

This [the framework] doesn’t get at the intersectorality; thus, the multisectoral nature is clearly a second thought. There is no extra information on intersectoral expected interventions or domains …” (M&E specialist, development organisation)

The inclusion of specific indicators for intersectorality, according to participants, may improve the relevance and use of the continuum of care framework in decision-making in the country. Intersectoral indicators can include indicators of co-coverage, which measure efforts of multiple sectors simultaneously. As a participant elaborates: “A co-coverage indicator will, for instance, look at impact of agriculture, water and education, gender all together at the household level” (Public health nutrition and M&E researcher, academic institution). This implies that a co-coverage indicator can be a composite metric of interventions by different sectors.

Besides intersectorality, participants also stated the need for harmonisation of definitions and metadata to improve cross-sectoral use and interpretation. As an example, a participant compared how different sectors may define ‘child’ – in civil registration as an individual below 16 years of age, in social development sector as below 18 years of age, and in the health sector the differentiation between children and adolescents. This logic can be applied to maternal health indicators within the continuum of care framework to ensure key concepts or sub-groups are described uniformly. Harmonisation and intersectorality can thus enhance the ability to use indicators within multisectoral collaboration for maternal health.

The importance of local policy and public health context

According to participants, the relevance of the indicators can be enhanced by their alignment with current priorities for maternal health in the country. These priorities reflect subnational action and targets in maternal health services as well as the prevailing social determinants of health. In terms of services, participants noted gaps in measurement of maternal nutrition and mental health interventions at the health facility level. In terms of social determinants of health, participants reflected on the need for more empowerment factors and their impact on maternal health. In particular, issues of gender-based violence and coercion were relevant as they were prevalent in the country and affect women’s reproductive choices, pregnancy experiences, child health outcomes and maternal mental health. According to participants, the current framework should reflect these missing and crucial services and indicators. As one participant stated: “It is important to have a framework, but the framework should not be set in stone and need to be updated every five years with latest data” (Empowerment and maternal health researcher, academic institution). These data will be reflective of emerging policy issues and public health concerns, and thus keep the framework and its indicators relevant to context.

Another aspect related to new policy priorities is respectful maternal care, which emphasises positive birth experiences for women delivering in facilities. According to a participant, indicators are needed that monitor the continuum of care from the patient perspective – current indicators are heavily oriented towards provision of services, and not enough on how patients experience care. Participants raised more contextual issues such as health literacy challenges and women’s traditional roles in South African households, which is tied to their empowerment and may affect their health and nutrition outcomes. The reflection on relevance thus reveals entangled contextual factors and highlights their necessity if the framework is to continually reflect the environment it is intended to monitor. Some of these factors are already reflected in the framework (e.g. empowerment) and others need to be collected in future iterations as data becomes available (e.g. maternal mental health, respectful maternal care).

In summary, the relevance and feasibility of the framework is centred on the potential to flatten out the current vertical or ‘silo’ mentality of sectors, persistent even in multisectoral platforms. However, improvements need to be made to include indicators of concomitant impact of multiple sectors/intersectorality. Additionally, the framework should continue to evolve and reflect emerging policies, public health challenges and health services that impact maternal health outcomes in South Africa.

Discussion

One of the important ways to improve maternal health outcomes in South Africa is to strengthen health service monitoring to support planning and accountability in the health system [24]. From a public health perspective, this also means acknowledging social determinants of health and their role in maternal health [25]. Our study assessed experts’ perspectives on potential monitoring indicators for the maternal health continuum of care in South Africa. Participants in our study believed that the continuum of care framework and the selected indicators provided a needed multisectoral perspective of maternal health interventions in South Africa. This multisectoral perspective, according to participants, can serve as a foundation for integrated monitoring of health and non-health sector interventions, and shared accountability for maternal health outcomes in the country.

The indicators appraised by participants in this study show that gaps remain in reporting services along the continuum of care. For the health sector, there are gaps in indicators because not all services are reported in the health system [26] and some are monitored through parallel information systems [27, 28]. An example is quality of care, whose detailed audits are reported elsewhere, and only composite indicators are included in the routine data set [29]. This affects accessibility of data [29] and the lack of indicators for the framework. Other gaps, as identified by participants, were a lack of indicators for experience of care, mental health and services to vulnerable groups (such as disabled women or young mothers). These gaps may be reflective of a lack of health service delivery in the country, not just reporting challenges. This is particularly true for maternal mental health services, which are lagging behind in primary healthcare in South Africa [30, 31].

The framework of indicators needs to be continually improved to include the missing indicators when services and data become available. However, participants also highlighted the importance of balancing the need for more data with the utility of the indicators from a decision-making perspective. The use of already available indicators for new monitoring and evaluation goals reduces the burden of data collection on health workers [32], and fulfils the goal of getting more intelligence out of the health data already available in South Africa [33]. Thus, we recommend the use of available data to measure and monitor the continuum of care, with a consultative process for additions/modifications to better align with evolving priorities.

The validity of indicators is affected by how well they measure the intended constructs. Sometimes, proxy indicators are used in the absence of direct measures. Participants believed that proxy indicators that reflect community coverage instead of the maternal health population should be removed. In the absence of alternatives, however, such proxy indicators can remain until future improvements in measurement are made. Validating maternal health indicators should be an on-going process of re-evaluation to ensure that indicators are truly reflective of constructs and context [21].

Besides validity and gaps in measurement, participants also reflected on the relevance and feasibility of available indicators. For relevance, participants stated that indicators should appropriately reflect current policy priorities, social issues and public health challenges. South Africa has unique problems in maternal health, including teenage pregnancy [34], gender inequality and gender-based violence [35], inequitable access and quality of maternal care as well as risky social exposures [36, 37]. There are age, race/ethnicity, residential, socioeconomic, disability and other differences in maternal health experiences among women in South Africa [36, 38,39,40]; indicators need to reflect these disparities in order to create buy-in from users and effectively support decision-making and action. To improve the relevance of available indicators and the framework as a whole, we recommend strengthening of the health information system to collect data on disaggregation variables and support equity analysis. The inadequacy of disaggregation data is a weakness of the current health information system in South Africa and other LMICs [1, 28, 41], and needs to be improved to support effective use of health service data in the future.

The feasibility of indicators was mainly tied to the perceived ease of use in intersectoral platforms, according to participants in this study. Additionally, there is a need to go beyond tracking sector performance individually and to integrate measurement of intersectoral collaboration for maternal health. Intersectoral collaboration refers to the mutually beneficial, integrated efforts of sectors who ideally share a common M&E framework [42]. According to our participants, intersectoral collaboration indicators would capture the true interface of the health system with other sectors. Intersectoral indicators are crucial because they have been used to address the social determinants of health and improve equity, accountability and planning for services in the era of the Sustainable Development Goals [18, 43,44,45]. Currently, intersectoral indicators for reproductive, maternal, and newborn, child and adolescent health are insufficiently measured and tracked [42]. Based on our findings, we recommend future research to document and develop indicators of intersectoral collaboration for inclusion in the continuum of care framework. In addition, we recommend future integration of data from other sectors and intersectoral collaboration into the health information system for accessibility and more comprehensive monitoring of interventions. The health system already acknowledges the need to integrate relevant non-health sector data and this needs to be implemented [13, 27].

The experts who participated in our study are stakeholders who have insight on the possible implementation challenges and measurement gaps in the indicator framework for the continuum of care. In this way, the study identifies key issues for improvement of the indicator framework to better support the goal for comprehensive monitoring and evaluation of maternal health in South Africa. We thus believe that this assessment of validity, relevance and feasibility of the indicator framework can be the basis for its future refinement and use in decision-making and action for maternal health in South Africa. Similar studies can be conducted in other LMICs to enhance monitoring and decision-making on multisectoral and comprehensive maternal service delivery. Studies should involve local stakeholders to help shape indicators that consider within-country priorities and support targeted decision-making.

Limitations

Our study does not reflect all of the possible perspectives of stakeholders or experts in maternal health and determinants in South Africa. It is also not a consensus-building process on how to improve the continuum of care framework or on which final list of indicators to use. Rather, it is a reflection by diverse stakeholders to identify key issues that can make the indicator framework implementable. Our study is also not a final validation of the indicators, as it is part of on-going research that will also use quantitative methods to measure the available indicators and derive composite metrics to track progress in health system performance in South Africa.

Conclusion

This study highlighted experts’ perspectives on the framework and indicators for the continuum of care for maternal health in South Africa. The study provided insights into the potential utility of the continuum of care framework to maternal health planning and programming in South Africa. It provides recommendations from the perspective of decision-makers and implementers to improve the use of valid, relevant and feasible indicators. The framework can be applied to assessment of comprehensive health services and monitoring and evaluation. Validity considerations focused on the adjustment of indicators to measure maternal health more accurately. Indicators were relevant from a multisectoral perspective but did not sufficiently reflect current policy directions and contextual factors. To improve feasibility, indicators should also be amenable to future monitoring of intersectoral collaboration to address the social determinants of maternal health. These findings help improve future measurement of indicators on the continuum of care for maternal health. They also point to systematic changes needed to improve monitoring of comprehensive maternal health service delivery in South Africa. These include future research in developing missing indicators, improved health service delivery, and strengthening health information systems to integrate data and indicators from other sectors. We recommend similar studies in other LMICs that involve stakeholders in repurposing and interpreting existing indicators, to reduce reporting burden and ultimately improve continuum of care monitoring in maternal health. These studies can support within-country improvements in service delivery across the continuum, which complement global, cross-national goals for maternal health.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. No participant confidential information will be released.

Abbreviations

DOH:

Department of Health

LMICs:

Low- and middle-income countries

M&E:

Monitoring and evaluation

NIDS:

National Indicator Data Set

References

  1. Day C, Gray A, Ndlovu N. Health and related indicators. South African Health Rev. 2018;(Nov 2018):139–250. http://hst.org.za/uploads/files/sahr2006.pdf.

  2. Department of Health, Statistics South Africa, South African Medical Research Council. South Africa Demographic and Health Survey 2016 .Pretoria; 2019. https://dhsprogram.com/pubs/pdf/FR337/FR337.pdf.

  3. Garenne M, Kahn K, Collinson MA, Gómez-olivé FX. Maternal mortality in rural South Africa: the impact of case definition on levels and trends. Int J Women's Health. 2013;5:457–63.

    Article  Google Scholar 

  4. Rispel L. Analysing the progress and fault lines of health sector transformation in South Africa. South Afr Health Rev. 2016;(May 2016):17–24.

  5. Department of Planning Monitoring and Evaluation. Medium-Term Strategic Framework MTSF 2014-2019. Pretoria: Department of Planning Monitoring and Evaluation South Africa; 2019.

  6. Department of Health. Saving Mothers 2014–2016: Seventh triennial report on confidential enquiries into maternal deaths in South Africa: Executive Summary. Pretoria: Department of Health South Africa; 2016.

  7. Bradshaw D, Chopra M, Kerber K, Lawn J, Moodley J, Pattinson R, et al. Every Death Counts: Saving the Lives of Mothers, Babies and Children in South Africa. Every Death Counts: Saving the Lives of Mothers, Babies and Children in South Africa. Cape Town; 2008. http://www.mrc.ac.za/sites/default/files/attachments/2016-06-30/edcrptfinal.pdf. Accessed 9 Jun 2018.

  8. The Partnership For Maternal Newborn & Child Health. A Global Review of Key Interventions Related to Reproductive, Maternal, Newborn and Child Health. Geneva: The Partnership for Maternal Newborn & Child Health (PMNCH); 2011.

  9. Kerber KJ, de Graft-Johnson JE, Bhutta ZA, Okong P, Starrs A, Lawn JE. Continuum of care for maternal, newborn, and child health: from slogan to service delivery. Lancet. 2007;370(9595):1358–69.

    Article  Google Scholar 

  10. de Graft-Johnson J, Kerber K, Tinker A, Otchere S, Narayanan I, Shoo R, et al. The maternal, newborn, and child health continuum of care. In: Lawn J, Kerber K, editors. Opportunities for Africa’s Newborns. Geneva: World Health Organization; 2006. p. 24–36.

    Google Scholar 

  11. Khanna R, Sri BS. Social determinants approach to maternal deaths. Maternal, newborn, child and adolescent health. https://www.who.int/maternal_child_adolescent/epidemiology/maternal-death-surveillance/case-studies/india-social-determinants/en/. Accessed 7 Jan 2020.

  12. World Health Organization. Closing the gap: policy into practice for social determinants of health. Geneva: WHO; 2011.

    Google Scholar 

  13. Pillay Y, Barron P. On the path to reach the SDG targets: Decreasing maternal and child mortality in South Africa. South Africa Med J. 2018;3(Suppl 1):S2–3.

    Article  Google Scholar 

  14. Bhardwaj S, Pattinson R, Sa F, Kauchali S, Sa F, Dlamini N, et al. Implementation of strategies to improve programme effectiveness lead to an improvement in maternal and child health outcomes in South Africa. South Africa Med J. 2018;108(March):44–52.

    Article  Google Scholar 

  15. Mothupi M, Knight L, Tabana H. Measurement approaches in continuum of care for maternal health: a critical interpretive synthesis of evidence from LMICs and its implications for the South African context. BMC Health Serv Res. 2018;18(539):1–9.

    Google Scholar 

  16. Department of Health. Strategic Plan for Maternal, Newborn, Child and Women’s Health (MNCWH) and Nutrition in South Africa: 2012–2016. Pretoria: Department of Health South Africa; 2012.

  17. Mothupi MC, Knight L, Tabana H. Indicators and gaps for monitoring service provision along the continuum of care for maternal health in South Africa. 2020. Preprint (Version #1) available at Research Square. https://doi.org/10.21203/rs.2.22005/v1.

  18. Blas E, Ataguba JE, Huda TM, Bao GK, Rasella D, Gerecke MR. The feasibility of measuring and monitoring social determinants of health and the relevance for policy and programme - a qualitative assessment of four countries. Glob Health Action. 2016;9(1):29002.

    Article  Google Scholar 

  19. Sherr K, Fernandes Q, Kanté AM, Bawah A, Condo J, Mutale W. Measuring health systems strength and its impact : experiences from the African Health Initiative. BMC Health Serv Res. 2017;17:827.

  20. Ross JA, Campbell OMR, Bulatao R. The Maternal and Neonatal Programme Effort Index (MNPI). Trop Med Int Health. 2001;6(10):787–98.

    Article  CAS  Google Scholar 

  21. Benova L, Moller A, Moran AC. “What gets measured better gets done better ”: The landscape of validation of global maternal and newborn health indicators through key informant interviews. PLoS One. 2019;14(11):1–16.

    Article  Google Scholar 

  22. Fereday J, Muir-Cochrane E. Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development Jennifer. Int J Qual Methods. 2006;5(1) http://www.ualberta.ca/~iiqm/backissues/5_1/pdf/fereday.pdf.

    Article  Google Scholar 

  23. Bolarinwa O. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195 http://www.npmj.org/text.asp?2015/22/4/195/173959.

    Article  Google Scholar 

  24. Bamford L. Maternal, Newborn and Child Health. In: Padarath A, English R, editors. South African Health Review 2012/2013. 2012/2013. Durban, South Africa: Health Systems Trust; 2012. p. 49–66.

    Google Scholar 

  25. Rispel L, Nieuwoudt S. Mainstreaming the social determinants of health in South Africa: rhetoric or reality? South African Health Rev. 2012:89–114.

  26. English R, Masilela T, Barron P, Schonfeldt A. Health information systems in South Africa. South African Health Rev. 2011:81–9.

  27. Moodley J, Fawcus F, Pattinson R. Improvements in maternal mortality in South Africa. South African Med J. 2018;108(3a):s4–8.

    Google Scholar 

  28. Khumalo F. Health management information systems. South African Health Rev. 2006;2006:65–76.

    Google Scholar 

  29. Moodley J, Pattinson RC, Fawcus S, Schoon MG, Moran N, Shweni PM. The confidential enquiry into maternal deaths in South Africa: a case study. BJOG. 2014;121(Suppl):53–60.

    Article  Google Scholar 

  30. Honikman S, van Heyningen T, Field S, Baron E, Tomlinson M. Stepped care for maternal mental health: a case study of the perinatal mental health project in South Africa. PLoS Med. 2012;9(5):e1001222.

    Article  Google Scholar 

  31. Petersen I, Lund C. Mental health service delivery in South Africa from 2000 to 2010: one step forward, one step back. South African Med J. 2011;101(10):751–7.

    Google Scholar 

  32. Klazinga N, Stronks K, Delnoij D, Verhoeff A. Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective. Int J Qual Heal Care. 2001;13(6):433–8.

    Article  CAS  Google Scholar 

  33. Department of Health. District Health Management Information System (DHMIS) Policy. Pretoria: Department of Health South Africa; 2011.

  34. Jonas K, Crutzen R, Van Den Borne B, Sewpaul R, Reddy P. Teenage pregnancy rates and associations with other health risk behaviours: a three-wave cross-sectional study among South African school-going adolescents. Reprod Health. 2016;13(1):50.

  35. Cooper D, Morroni C, Orner P, Moodley J, Harries J, Cullingworth L, et al. Ten years of democracy in South Africa: Documenting transformation in reproductive health policy and status. Reprod Health Matters. 2004;12(24):70–85.

    Article  Google Scholar 

  36. Wabiri N, Chersich M, Zuma K, Blaauw D, Goudge J, Dwane N. Equity in maternal health in South Africa: analysis of health service access and health status in a national household survey. PLoS One. 2013;8(9):e73864.

    Article  CAS  Google Scholar 

  37. Wabiri N, Chersich M, Shisana O, Blaauw D, Rees H, Dwane N. Growing inequities in maternal health in South Africa: a comparison of serial national household surveys. BMC Pregnancy Childbirth. 2016;16:256. https://doi.org/10.1186/s12884-016-1048-z.

    Article  PubMed  PubMed Central  Google Scholar 

  38. World Health Organization, International Center for Equity in Health. State of inequality: reproductive, maternal and child health. Geneva: WHO; 2015.

    Google Scholar 

  39. Boerma JT, Bryce J, Kinfu Y, Axelson H, Victora CG. Mind the gap: equity and trends in coverage of maternal, newborn, and child health services in 54 countdown countries. Lancet. 2008;371:1259–67.

    Article  Google Scholar 

  40. Fried J, Sunderji A, Birch S, Eyles J. The reason that I did not go – determinants of the use of antenatal care services in South Africa, two decades after the end of apartheid. Can J Afr Stud. 2013;47(1):27–50.

    Google Scholar 

  41. Agyepong IA, Sewankambo N, Binagwaho A, Coll-Seck AM, Corrah T, Ezeh A, et al. The path to longer and healthier lives for all Africans by 2030: the Lancet Commission on the future of health in sub-Saharan Africa. Lancet. 2017;390(10114):2803–59.

    Article  Google Scholar 

  42. Rasanathan K, Damji N, Atsbeha T, Brune Drisse MN, Davis A, Dora C, et al. Ensuring multisectoral action on the determinants of reproductive, maternal, newborn, child, and adolescent health in the post-2015 era. Br Med J (Clin Res Ed). 2015;351:h4213.

    Google Scholar 

  43. Blas E, Roebbel N, Rajan D, Valentine N. Intersectoral planning for health and health equity. In: Schemts G, Rajan D, Kadandale S, editors. Strategizing national health in the 21st century: A Handbook. Geneva: World Health Organization; 2016.

    Google Scholar 

  44. Valentine NB, Bonsel GJ. Exploring models for the roles of health systems’ responsiveness and social determinants in explaining universal health coverage and health outcomes. Glob Health Action. 2016;9(1):29329.

    Article  Google Scholar 

  45. Pega F, Valentine NB, Rasanathan K, Hosseinpoor R, Torgersen TP, Ramanathan V, et al. The need to monitor actions on the social determinants of health. Bull World Health Organ. 2017;95:784–7.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the participants of the study for their time and generous insights.

Funding

This work is based on the research supported by the South African Research Chairs Initiative of the Department of Science and Technology and National Research Foundation of South Africa (grant no 82769). The authors would also like to acknowledge funding from the South African Medical Research Council and the Belgian Development Cooperation, through the Institute of Tropical Medicine Antwerp. Any opinion, finding and conclusion or recommendation expressed in this material is that of the authors and not the funders.

Author information

Authors and Affiliations

Authors

Contributions

MM and LM conceptualised the study. MM conducted data collection and analysis. LM guided additional analysis of data. MM, LM and HT were involved in writing and revision of manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mamothena Carol Mothupi.

Ethics declarations

Ethics approval and consent to participate

Participants signed an informed consent form prior to commencement of interviews. Institutional ethics clearance was obtained from the University of the Western Cape Biomedical Research Ethics Committee as well as the departments where experts’ worked (where prior permission was needed).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Interview guide.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mothupi, M.C., Knight, L. & Tabana, H. Improving the validity, relevance and feasibility of the continuum of care framework for maternal health in South Africa: a thematic analysis of experts’ perspectives. Health Res Policy Sys 18, 28 (2020). https://doi.org/10.1186/s12961-020-0537-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12961-020-0537-8

Keywords