- Open Access
- Open Peer Review
Incremental cost and health gains of the 2016 WHO antenatal care recommendations for Rwanda: results from expert elicitation
Health Research Policy and Systemsvolume 17, Article number: 36 (2019)
High-quality evidence of effectiveness and cost-effectiveness is rarely available and relevant for health policy decisions in low-resource settings. In such situations, innovative approaches are needed to generate locally relevant evidence. This study aims to inform decision-making on antenatal care (ANC) recommendations in Rwanda by estimating the incremental cost-effectiveness of the recent (2016) WHO antenatal care recommendations compared to current practice in Rwanda.
Two health outcome scenarios (optimistic, pessimistic) in terms of expected maternal and perinatal mortality reduction were constructed using expert elicitation with gynaecologists/obstetricians currently practicing in Rwanda. Three costing scenarios were constructed from the societal perspective over a 1-year period. The two main inputs to the cost analyses were a Monte Carlo simulation of the distribution of ANC attendance for a hypothetical cohort of 373,679 women and unit cost estimation of the new recommendations using data from a recent primary costing study of current ANC practice in Rwanda. Results were reported in 2015 USD and compared with the 2015 Rwandan per-capita gross domestic product (US$ 697).
Incremental health gains were estimated as 162,509 life-years saved (LYS) in the optimistic scenario and 65,366 LYS in the pessimistic scenario. Incremental cost ranged between $5.8 and $11 million (an increase of 42% and 79%, respectively, compared to current practice) across the costing scenarios. In the optimistic outcome scenario, incremental cost per LYS ranged between $36 (for low ANC attendance) and $67 (high ANC attendance), while in the pessimistic outcome scenario, it ranged between $90 (low ANC attendance) and $168 (high ANC attendance) per LYS. Incremental cost effectiveness was below the GDP-based thresholds in all six scenarios.
Implementing the new WHO ANC recommendations in Rwanda would likely be very cost-effective; however, the additional resource requirements are substantial. This study demonstrates how expert elicitation combined with other data can provide an affordable source of locally relevant evidence for health policy decisions in low-resource settings.
In evidence-informed health policy, meta-analysis of randomised controlled trials is considered to be at the top of the hierarchy of good evidence, whereas evidence from specialists and clinical practice is considered to be at the bottom . Internationally published health research is of limited local relevance for low-income countries, given that most of the evidence is generated from high-income countries . Many systematic reviews do not reflect the priorities and problems of low-resource settings [2, 3], and even when the reviews are relevant, the interventions and treatments suggested may be unavailable, unaffordable or inappropriate for the setting .
One example in practice are the new recommendations on antenatal care (ANC) for improved pregnancy outcomes issued by WHO in November 2016 , which replace the 2002 Focused ANC model . The major new features of the 2016 WHO recommendations are eight ANC contacts during pregnancy and one obstetric ultrasound examination before gestational week 24 for all pregnant women. The ultrasound recommendation is based on a Cochrane systematic review from 2015 that includes 11 randomised controlled trials on populations from six countries (Australia, Norway, South Africa, Sweden, the United Kingdom and the United States of America) [4, 6]; no low- or lower- to middle-income countries were included in this review.
Rwanda’s current ANC policy was developed based on the 2002 WHO recommendations, which suggest four focused ANC visits for normal pregnancy . In 2014/2015 only 45% of pregnant women attended the full four ANC visits, and the first visit was only in the fifth month of gestation on average . Maternal mortality in Rwanda has been significantly reduced over the last 15 years from 1071 per 100,000 live-births in 2000 to 210 per 100,000 live births in 2014/2015 [8, 9]. This is still three times higher than the global target for maternal mortality in the Sustainable Development Goals, namely to reduce the global maternal mortality ratio to less than 70 per 100,000 live births . In its strategy 2018–2024, Rwanda aims to reduce the maternal mortality ratio to 126 per 100,000 live births .
Decision-makers constrained with insufficient budgets need evidence from health economic evaluations to make informed decisions about the efficient use of resources for meeting population health needs . Economic evaluations help to identify, measure and compare the cost and health impacts of competing health interventions as well as their scalability and sustainability . However, the use of economic evaluation is still limited in low- and middle-income countries . One reason is the unavailability and limited local relevance of published research, which calls for alternative sources of evidence to be used. One method increasingly used in such situations is expert elicitation, which can be used to collect quantitative and qualitative information such as proportions, percentages and probabilities of events .
The aim of this study was to inform decision-making on ANC recommendations in Rwanda by estimating the incremental cost and health gains of implementing the 2016 WHO recommendations compared to current practice.
The intervention of interest was ANC for normal (uncomplicated) pregnancies in the public healthcare system in Rwanda. The cost and health outcomes of current ANC practice were compared with a hypothetical situation in which the 2016 WHO ANC recommendations are adopted and implemented over a 1-year time period. The study was a (prospective) simulation-based cost effectiveness study for a hypothetical cohort of 373,679 women, equal to the number of women who attended ANC at least once in Rwanda in the year 2014 .
The unit cost of current ANC practice, and current patterns of ANC attendance, were obtained from two sub-studies of Maternal Health Research in Rwanda (MatHeR), a research programme comprising cross-sectional health facility and household surveys conducted in Rwanda in 2014 [16, 17]. Incremental resource use was identified and quantified by comparing the key activities during each of the eight visits detailed in the 2016 WHO ANC recommendations , with resource use under the current four-visit model . Primary data was collected using expert elicitation to estimate health gains in terms of mortality decreases as detailed below. Health outcomes were expressed as life-years saved (LYS) using current mortality from the 2015 Annual Health Statistical Booklet for the Rwandan Ministry of Health  and life expectancy from the report of the Rwanda Fourth General Population and Housing Census 2012 . Incremental cost-effectiveness ratios were estimated from the societal perspective comprising health sector and household costs.
A list of all data sources, the kind of information obtained, and the use of that information are presented in Additional file 1.
Incremental cost was estimated in three steps. First, assumptions were made about the unit cost of each of the eight ANC visits under the 2016 WHO recommendations. Second, the distribution of ANC attendance was simulated in the hypothetical situation that Rwanda implements the new recommendations, and the cost of ANC was obtained for each pregnant woman in the hypothetical cohort. Third, incremental total cost and incremental cost per woman were computed as the difference between average cost from the simulation and the average cost of current ANC practice. Each of these steps is described in detail below.
Assumptions about unit cost
Assumptions about the unit cost of the eight visits were based on our previous study of the unit cost of the four ANC visits that women with normal (uncomplicated) pregnancies are expected to complete under the current ANC policy in Rwanda . That study used a micro-costing approach to collect and analyse resource use by healthcare providers and households from eight public and faith-based health facilities in the Northern province and in Kigali city in Rwanda.
The main changes implied by the 2016 WHO ANC recommendations compared to current practice in Rwanda that have considerable resource use implications are (1) an increase in the number of visits from four to eight contacts during pregnancy; (2) one early obstetric ultrasound examination before week 24 of gestation, in place of selective ultrasound only; and (3) repetition of some laboratory tests in late pregnancy. Based on the above changes, the following assumptions related to incremental changes in the unit cost were made:
Firstly, the cost of an ultrasound examination, estimated at $3 , was added to the cost of the first ANC visit. Neither the WHO recommendation nor the systematic review of trials that underpinned it  specified whether the ultrasound should be conducted during the first trimester (Visit 1) or early in the second trimester, as long as it is before week 24 of gestation. Because women in Rwanda typically register late for ANC, and to simplify the calculations, we added the cost of ultrasound to Visit 1. Otherwise, the cost of the first visit was assumed to be equal to current cost ($21), thus including a number of laboratory tests and a mosquito net. Secondly, Visits 2, 4, 5 and 7 in the new model were taken to be ordinary routine ANC visits, and therefore equal in cost to Visits 2 and 3 in the current model ($6). Third, Visits 3 and 6 in the new model were assumed to be routine visits but with the addition of two repeat laboratory tests, namely tests for bacteriuria and for anaemia, for which costs were estimated at $3 and $0.75, respectively [4, 21]. The last visits in the two models were assumed to be equal in cost ($11). The summary of the unit cost in the currently implemented four-visit model and unit cost estimates in the new eight-visit model are presented in Table 1.
Simulation of ANC attendance
Three costing scenarios, corresponding to different assumptions about ANC attendance, were simulated using R software (version 3.3.2, The R Foundation for Statistical Computing). The assumptions were based on current attendance patterns in Rwanda, the schedule suggested in the 2016 WHO ANC recommendations and previous literature on ANC attendance in other countries.
ANC attendance under the current ANC policy in Rwanda has been previously reported using a household survey conducted in 2014 . The analysis of the dataset, to which we had access, showed that the mean number of ANC visits is 3.3 per woman, with a median of 3 and a standard deviation of 0.8. A histogram of the distribution, which approximates a normal distribution, is provided in Additional file 2.
Given the current distribution, it is unlikely that if Rwanda implements the 2016 WHO ANC recommendations, the average number of visits becomes eight per woman as specified in the recommendations, at least in the short to medium term. This assumption is further corroborated by findings in the Cochrane review  and the WHO ANC trial  that, while women in high-income countries generally complete at least the minimum number of ANC visits recommended by the policy in place, in low- and middle-income countries, women generally do not reach the recommended number of ANC visits. One reason is late registration in ANC , as is the case in Rwanda and the region [24,25,26,27,28,29]. A summary of ANC attendance in the studies included in the Cochrane systematic review is presented in Additional file 3.
The scenarios were generated using Monte Carlo simulation, a numerical technique that assists in generating random numbers and forces them to follow a pre-determined probability distribution . The hypothetical cohort was 373,679 women, equal to the number of women who attended ANC at least once in Rwanda in 2014 . Since the true distribution is unknown, the mean and standard deviation were used to simulate a normal distribution . The simulations resulted in the following three scenarios of ANC attendance in the new eight-visit model. In the ‘conservative’ Scenario 1, the mean number of ANC visits per woman is five; a pregnant woman who attends the minimum number of visits, attends only once; and the maximum number of visits is eight. In Scenario 2, the mean number of visits is six, the minimum is two, and the maximum number is 10. In the ‘ambitious’ Scenario 3, the mean number of visits is seven, the minimum number is three, and the maximum number is 11. Histograms of the distributions of the three attendance scenarios are presented in Additional file 2.
For the purpose of presenting alternative policy options, a scenario of full compliance to the current ANC guidelines was also constructed, in which all women attend the full four ANC visits.
Incremental cost estimation
Incremental cost was estimated in four main steps. First, the cost of ANC was calculated for each member of the 373,679 cohort by multiplying the simulated number of ANC visits for each individual by their respective unit costs. For example, the cost of ANC for a woman who attends five visits was calculated as the sum of Visits 1–5 in Table 1. Second, the total cost of ANC in the simulation was calculated by summing up the cost of ANC for all individuals. This procedure was repeated 100 times and differences between the minimum and maximum were compared to check for reliability of the simulation. Third, the average total cost was used to compute average cost of ANC per woman. Finally, incremental cost was computed by comparing each of the four costing scenarios with the cost of current ANC practice.
Health outcome measurement
Two health outcomes scenarios (optimistic, pessimistic) were constructed to estimate the incremental health gains from the 2016 WHO recommendations compared to current practice. First, changes expected in maternal and perinatal mortality were estimated using expert elicitation. Second, the resulting mortality reductions were transformed into deaths averted and life-years saved using secondary data.
Expert elicitation using aspects of the Delphi process was used to estimate how implementing the 2016 WHO ANC recommendations in Rwanda would potentially affect perinatal and maternal mortality. The respondents were selected using the inclusion criteria of having specialist training in obstetrics and to have practiced in Rwanda for at least 5 years after training. A total of 70 gynaecologists/obstetricians registered in Rwanda were identified, of which 19 fulfilled the inclusion criteria. All 19 were invited for interview by e-mail (with the questionnaire as an attachment; Additional file 4). One week later, an appointment was proposed via telephone. Three specialists declined the invitation either due to time constraints or travel. Two other specialists agreed to participate but were not available for interview within the specified 3-week timeframe. Six specialists did not respond to the invitation. Finally, eight specialists were interviewed in April 2017.
The interviewer-administered questionnaire was composed of three main parts. The first part described trends in maternal and newborn health with key indicators in Rwanda and changes to ANC practice proposed by the new 2016 WHO ANC recommendations. The second part included questions about the potential implications of the new recommendations on cause-specific and overall perinatal and maternal mortality. The third part invited the respondents to suggest improvements to ANC practice other than implementing the 2016 WHO ANC recommendations in full, and to estimate the possible implications on perinatal and maternal mortality.
According to WHO, a maternal death is “the death of a woman while pregnant or within 42 days of termination of pregnancy, irrespective of the duration and site of the pregnancy, from any cause related to or aggravated by the pregnancy or its management but not from accidental or incidental causes” . While perinatal mortality has different definitions , according to WHO, the perinatal period commences at 22 completed weeks of gestation and ends 7 completed days after birth. Perinatal mortality refers, therefore, to deaths of the fetus from 22 completed gestational weeks (154 days) and deaths in the first week of life (7 days) .
The first draft of the questionnaire was developed by the first author and revised based on comments from the multi-disciplinary co-authors’ team, composed of health economists, obstetricians and public health researchers. All interviews were conducted by the first author. The questionnaire is presented in Additional file 4.
After an initial analysis of the responses, it was decided to give respondents an opportunity to re-evaluate their responses after examination of the group response, following the principles of the Delphi process . A second round of opinion collection was therefore conducted in July–August 2017. The eight respondents were re-contacted by email and phone and presented with the anonymised group responses. Five specialists revised their estimates, two kept their initial estimates, and one did not reply. In the latter case, non-response was taken as confirmation of the respondent’s initial estimates. Each respondent’s estimates of the maternal and perinatal mortality changes from the two rounds are presented in Additional file 5.
Responses from the second round were retained as inputs in the analysis. Two outcome scenarios were constructed by ordering responses to each of the two indicators from low to high and dividing into two equal sub-groups labelled either the ‘optimistic’ health outcome scenario, composed of relatively higher reductions in mortality, or the ‘pessimistic’ health outcome scenario, composed of relatively lower reductions in maternal and perinatal mortality. The mean response in each sub-group was retained as the parameter estimate for each scenario.
Estimation of life-years saved (LYS)
The estimated reductions in mortality were converted into deaths averted over a hypothetical 1-year implementation period in order to summarise the two mortality figures in one unit of measurement. The number of deaths averted was calculated using death counts from the 2015 Annual Health Statistical Booklet for the Rwandan Ministry of Health . In 2015, maternal and perinatal mortality deaths were estimated at 371 and 10,076, respectively . Then, maternal LYS were calculated using proportions of maternal mortality by age groups from the Demographic and Health Survey 2014/2015 report , life expectancy at birth for women (66 years) for life-years of mothers, and life expectancy at birth for both sexes (64 years) for perinatal life-years, from the 2012 Rwanda Census . LYS were discounted a 3% rate (Additional file 6).
Incremental cost effectiveness of the 2016 WHO ANC recommendations compared to current ANC practice in Rwanda was expressed as cost per LYS for each of the three costing scenarios (reflecting different attendance assumptions) and the two health outcome scenarios.
No official or country-specific estimate of the cost effectiveness threshold is currently available for Rwanda. Thresholds based on Gross Domestic Product (GDP), previously recommended by WHO  in cases where there is no official threshold, were therefore used. Any intervention that averts a disability-adjusted life-year for the cost of a given country’s annual GDP per capita or less is considered to be ‘very cost effective’, and an intervention is labelled as not cost effective if the cost is more than three times per capita GDP . The Rwandan GDP per capita was $697 in (2015 USD) .
Cost of the 2016 WHO ANC recommendations in Rwanda
The total and incremental cost of the 2016 WHO ANC recommendations from the 100 simulations for each of the three attendance scenarios is presented in Table 2. The results suggest that, on average, the cost of ANC per woman ranges between $53 and $69 (standard deviation $7). The results further suggest that, on average, the total national cost of ANC ranges between $19.8 million and $24.9 million. The difference between minimum and maximum values for each scenario across the simulations are 0.001% and below, which indicates that the simulation model is reliable.
The average cost of current ANC practice has previously been reported to be $44 per woman, and the total national cost to be $13.9 million . The incremental total cost of the 2016 WHO ANC recommendations ranges therefore between $5.9 million and $11 million across the attendance scenarios (Table 2). The costs increase by 42%, 64% and 79% compared to current practice in Scenarios 1–3, respectively. Full utilisation under the current policy would increase costs by 18%.
Life-years saved (LYS)
Implementation of the 2016 WHO ANC guidelines was estimated to reduce perinatal mortality by 22.5% and 55% in the pessimistic and optimistic health outcome scenarios, respectively, equivalent to 2267 and 5542 perinatal deaths avoided per year (Table 3). The estimated reduction in maternal mortality was 7% and 52% in the pessimistic and optimistic scenarios, respectively, representing 26 and 195 maternal deaths avoided. In total, the incremental health gains were 65,366 LYS in the pessimistic scenario and 162,509 LYS in the optimistic scenario, of which 99% and 98%, respectively, were due to perinatal deaths avoided.
Incremental costs per LYS from implementation of the 2016 WHO ANC recommendations in different cost and outcome scenarios are presented in Table 4. Results indicate that incremental cost per LYS ranges between $36 and $67 in the optimistic outcome scenario and between $90 and $168 in the pessimistic scenario. Thus, implementing the new recommendations in Rwanda would be very cost-effective in all the scenarios presented in this study.
This study analyses the incremental cost-effectiveness of the 2016 WHO ANC recommendations for Rwanda using evidence from expert elicitation, simulation of attendance patterns, and estimation of their unit cost implications. The results indicate that implementing the new 2016 WHO ANC recommendations in Rwanda would likely be very cost-effective. However, the additional resource requirements are significant for both the health sector and for households. The $5.8 million, $8.8 million and $11 million in incremental costs for the three costing scenarios represent 3%, 5% and 6% of the 2017/2018 health sector budget ($183.1 million) , respectively. The 2016 WHO recommendations include doubling the number of ANC visits per woman with an uncomplicated pregnancy, adding some new activities, in particular one routine obstetric ultrasound, and repeating some laboratory tests. Our results suggest that the new recommendations do not imply doubling the cost of the currently implemented four-visit model. The main explanation is that the first visit, which represents nearly half of the cost of the current four-visit schedule, remains by far the most costly visit also in the eight-visit schedule. For households, the cost of ANC is likely to increase nearly in proportion to the number of visits. Although this study did not separately report the relative importance of households and health sector, the household contribution was representing 9% of the total societal cost of current ANC practice in Rwanda .
The findings of this study suggest large health gains, ranging between 65,366 and 162,509 LYS per year of implementation, mostly due to a reduction in perinatal mortality. These results from expert elicitation are consistent with previous literature that suggests the critical changes suggested by the new recommendations (such as doubling the number of ANC visits and a routine ultrasound examination before 24 weeks) are associated with saving babies’ rather than mothers’ lives [40,41,42]. The maternal health package is composed of a set of interdependent interventions, for which the relative effectiveness of each element of the package is hard to measure . Maternal deaths occur mainly during labour, delivery and in the immediate postpartum, and postpartum haemorrhage, sepsis and other direct causes together represent close to the half of the causes of maternal mortality . Therefore, while it has been estimated that ANC can save 26% of maternal deaths in low-resource settings, saving women’s lives depends to a larger extent on the quality of obstetric care rather than ANC [45, 46]. Other explanations for the relative emphasis on perinatal rather than maternal LYS in the results are differences in the current level of mortality and the larger relative weight associated with a perinatal death in the LYS computation. Together, these imply that a given percentage reduction in perinatal mortality is equivalent to a relatively larger number of LYS than the same reduction in maternal mortality.
Current data on population health and the cost of care is increasingly becoming available and locally relevant even in low-resource settings. However, scientifically rigorous methods are still rarely applied to use that evidence to inform health policy decision-making. In this study, we combined primary costing and attendance data with results from simulation and expert elicitation to estimate the implications on cost and health outcomes of a change in current ANC policy suggested by new WHO recommendations. Expert elicitation is increasingly used in healthcare research, specifically in model-based health economic evaluations [47, 48] and in developing evidence-based clinical guidelines . It is used to complement information from other type of evidence especially when data are unavailable, expensive or difficult to collect  or when the available data are not relevant. Current limitations to using expert elicitation include limited guidance on how to successfully conduct expert elicitation , and how to deal with cognitive heuristics and biases . In this study, we provided respondents with references to the available literature on common causes of perinatal and maternal mortality in Rwanda and in low-income countries, in order to limit bias by overlooking pertinent evidence. However, the small number of experts who participated in the expert elicitation (n = 8) is one of the limitations of this study. It was mainly due to limited number of active practitioners who fulfil the inclusion criteria. We believe that expert elicitation has contributed by adding scientific rigor to the ongoing policy debate and demonstrated its potential as an alternative solution for evidence-informed decision-making.
Further research is needed on the mortality effects of the new recommendations but also on the relationship between ANC attendance and health benefits, especially in low-income countries where mortality is still high. At present, the available evidence is insufficient to create combined scenarios of attendance and health outcomes, which we dealt with in this study by allowing combinations that may be considered unlikely, such as the combination of the high cost (attendance) with the pessimistic outcome scenario. Another limitation is that this study limited the consideration of health gains to maternal and perinatal mortality. In a previous study, we found evidence that health-related quality of life 1 year postpartum was positively associated with adequate ANC utilisation . Other studies have similarly shown that the benefits of ANC extend beyond mortality to a reduction in maternal morbidity, women’s life satisfaction , children’s health status , and control of other diseases such as malaria and HIV. Increased use of ANC can affect maternal morbidity in two ways – first, regular exposure to some ANC interventions can reduce the risk of certain health conditions during pregnancy; for example, there is evidence suggesting that daily iron supplementation reduces the risk of maternal puerperal sepsis . Secondly, screening for existing maternal health conditions such as anaemia, HIV, syphilis, malaria, etc., reduces their burden on mothers who regularly attend ANC services, through increased chances for early detection and management, especially in high prevalence settings . Health promotion activities included in the WHO ANC recommendations like hygiene, nutrition, and physical activity, can also be translated into better nutrition and hygiene for other children.
The method used in this study to compute total cost of ANC per woman by adding up the unit cost of ANC visits according to the schedule might not reflect reality, in particular when the number of visits falls well below the recommended number. For example, a pregnant woman who comes for her second ANC visit close to delivery, in effect receives Visit 8, instead of Visit 2, as has been assumed here. Furthermore, the 2016 WHO recommendations have other cost implications for Rwanda that have not been explored in this analysis. For example, the large increase in the number of ANC visits is likely to replace other health facility services. On the other hand, there may be some economies of scale, i.e. the same infrastructures are used by more people and consequently at lower cost per woman. Economies of scope could also be expected if, for example, ultrasound equipment could be used for other purposes such as gynaecological examinations. There will likely be increased need for supervision and training of staff; however, this analysis did not consider any one-off costs, such as ultrasound training, or the procurement of new ultrasound machines. Currently in Rwanda, obstetric ultrasound is only available in hospitals and performed by physicians [33, 54, 55]. There are disparities between rural and urban health facilities in terms of the availability and quality of ultrasound equipment [33, 55]. Lastly, the projection of cost and health gains from an actual future policy change would require minor adjustments to the results presented here because prices, attendance, mortality and life expectancy data refer to figures collected in 2014–2015.
Implementation of the new 2016 WHO ANC recommendations has the potential to cost-effectively reduce perinatal and maternal mortality in Rwanda. Based on these findings, it can be concluded that the Government should consider implementation of these recommendations. However, given the significant investment required for implementation, and a narrow fiscal space, an alternative is that the health sector adopts a phased approach to implementation, starting with the most feasible activities within the 2016 WHO ANC recommendation. This could imply, for example, increasing the recommended number of visits to some level that can be managed with current staffing levels. Nevertheless, there is also a need to investigate the reasons behind the low attendance to ANC within the current ANC policy and to address them, and hence, preparing the ground for an expanded ANC package. Our study also demonstrates that expert elicitation can be successfully combined with other sources of data to support health policy decision-making, especially when high-quality evidence of effectiveness and cost effectiveness is not available or locally relevant.
gross domestic product
Masic I, Miokovic M, Muhamedagic B. Evidence based medicine – new approaches and challenges. Acta Inform Med. 2008;16:219–25.
Richards T. Poor countries lack relevant health information, says Cochrane editor. BMJ. 2004;328:1140669.
Mcmichael C, Waters E, Volmink J. Evidence-based public health: what does it offer developing countries ? J Public Health. 2005;27(2):215–21.
World Health Organization. WHO Recommendation on Antenatal Care for Positive Pregnancy Experience. Geneva: WHO; 2016.
World Health Organization Antenatal Care Trial Research Group. WHO Antenatal Care Randomized Trial: Manual for the Implementation of the New Model. Geneva: WHO; 2002.
Whitworth M, Bricker L, Mullan C. Ultrasound for fetal assessment in early pregnancy. Cochrane Database Syst Rev. 2015;7:CD007058.
WHO Antenatal care Randomized Trial: Manual for the Implementation of the New Model. 2002. http://apps.who.int/iris/bitstream/handle/10665/42513/WHO_RHR_01.30.pdf. Accessed 26 Mar 2019.
National Institute of Statistics of Rwanda (NISR), Ministry of Health (MOH). Rwanda Demographic and Health Survey 2014–15. 2015.
Ministere de la Sante du Rwanda. Rwanda Santé pour tous: Enquete Demographque et de Sante. 2000. https://dhsprogram.com/publications/publication-FR125-DHS-Final-Reports.cfm. Accessed 26 Mar 2019.
World Health Organization. SDG 3: Ensure Healthy Lives and Promote Wellbeing for All At All Ages. Geneva: WHO; 2017. p. 81–126. http://www.who.int/sdg/targets/en/. Accessed 26 Mar 2019.
The Ministry of Health [Rwanda]. Fourth Health Sector Strategic Plan July 2018 – June 2024. 2018. moh.gov.rw/fileadmin/templates/Docs/FINALH_2-1.pdf. Accesed 14 Aug 2018.
Rabarison KM, Bish CL, Massoudi MS, Giles WH. Economic evaluation enhances public health decision making. Front Public Health. 2015;3:164.
Mills A. Reflections on the development of health economics in low- and middle-income countries. Proc R Soc B Biol Sci. 2014;281:20140451.
Butler AJ, Thomas MK, Pintar KDM. Systematic review of expert elicitation methods as a tool for source attribution of enteric illness. Foodborne Pathog Dis. 2015;12:367–82.
Rwanda Ministry of Health. https://hmis.moh.gov.rw/hmis/dhis-web-commons/security/login.action. Accessed 26 Mar 2019.
Hitimana R, Lindholm L, Krantz G, Nzayirambaho M, Pulkki-Brännström A-M. Cost of antenatal care for the health sector and for households in Rwanda. BMC Health Serv Res. 2018;18:262. https://doi.org/10.1186/s12913-018-3013-1s.
Rurangirwa AA, Mogren I, Nyirazinyoye L, Ntaganira J, Krantz G. Determinants of poor utilization of antenatal care services among recently delivered women in Rwanda; a population based study. BMC Pregnancy Childbirth. 2017;17:142.
The Ministry of Health [Rwanda]. 2015 Annual Health Statistical Booklet. 2016. http://www.moh.gov.rw/fileadmin/user_upload/HMIS/2016_Annual_Statistical_booklets_V9_08_03_2018.pdf. Accessed 26 Mar 2019.
National Institute of Statistics of Rwanda (NISR). Fourth Population and Housing Census, Rwanda, 2012. Rwanda; 2014. http://www.statistics.gov.rw/datasource/42. Accessed 14 Aug 2018.
Whitworth M, Bricker L, Neilson JP, Dowswell T. Ultrasound for fetal assessment in early pregnancy. Cochrane Database Syst Rev. 2010;(4):CD007058. https://doi.org/10.1002/14651858.CD007058.pub2.
Ullah A, Barman A, Ahmed I, Salam A. Asymptomatic bacteriuria in pregnant mothers: A valid and cost-effective screening test in Bangladesh. J Obstet Gynaecol. 2012;32:37–41.
Munjanja SP, Lindmark GNL. Randomised control trial of a reduced-visits programme of antenatal care in Harare, Zimbabwe. Lancet. 1996;348(9024):364–9.
Hofmeyr GJ, Hodnett ED. Antenatal care packages with reduced visits and perinatal mortality: a secondary analysis of the WHO antenatal care trial – Comentary: routine antenatal visits for healthy pregnant women do make a difference. Reprod Health. 2013;10:20.
Manzi A, Munyaneza F, Mujawase F, Banamwana L, Sayinzoga F, Thomson DR, et al. Assessing predictors of delayed antenatal care visits in Rwanda. BMC Pregnancy Childbirth. 2014;14:299.
Gross K, Alba S, Glass TR, Schellenberg JA, Obrist B. Timing of antenatal care for adolescent and adult pregnant women in south-eastern Tanzania. BMC Pregnancy Childbirth. 2012;12:16.
Brown CA, Sohani SB, Khan K, Lilford R, Mukhwana W. Antenatal care and perinatal outcomes in Kwale district, Kenya. BMC Pregnancy Childbirth. 2008;8:2.
Gebremeskel F, Dibaba Y, Admassu B. Timing of first antenatal care attendance and associated factors among pregnant women in Arba Minch town and Arba Minch District, Gamo Gofa zone, south Ethiopia. J Environ Public Health. 2015;2015:971506.
Turyasiima M, Tugume R, Openy A, Ahairwomugisha E, Opio R, Ntunguka M, Mahulo N, Akera P, Odongo-Aginya E. Determinants of first antenatal care visit by pregnant women at community based education, research and service sites in Northern Uganda. East Afr Med J. 2014;91(9):317–22.
Kisuule I, Kaye DK, Najjuka F, Ssematimba SK, Arinda A, Nakitende G, et al. Timing and reasons for coming late for the first antenatal care visit by pregnant women at Mulago hospital, Kampala Uganda. BMC Pregnancy Childbirth. 2013;13:121.
Farrance I, Frenkel R. Uncertainty in measurement: a review of Monte Carlo simulation using microsoft excel for the calculation of uncertainties through functional relationships, including uncertainties in empirically derived constants. Clin Biochem Rev. 2014;35(1):37–61.
Harrison RL, Carlo M. Introduction to Monte Carlo Simulation. AIP Conf Proc. 2010;1204:17–21.
World Health Organization. The WHO application of ICD-10 to deaths during pregnancy, childbirth and the puerperium: ICD-MM. 2012. https://www.who.int/reproductivehealth/publications/monitoring/9789241548458/en/. Accessed 14 Aug 2018.
Holmlund S, Ntaganira J, Edvardsson K, Lan PT, Sengoma JPS, Åhman A, et al. Improved maternity care if midwives learn to perform ultrasound: a qualitative study of Rwandan midwives’ experiences and views of obstetric ultrasound. Glob Health Action. 2017;10(1):1350451.
World Health Organization. Maternal and Perinatal Health. WHO. 2013. http://www.who.int/maternal_child_adolescent/topics/maternal/maternal_perinatal/en/#.WsKRt9XsgQk.mendeley. Accessed 2 Apr 2018.
Linstone HA, Turoff M. The Delphi Method – Techniques and Applications. 2002. https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf. Accessed 26 Mar 2019.
World Health Organization. Commission on Macroeconomics and Health. 2001. Macroeconomics and Health: Investing in Health for Economic Development. Geneva: WHO; 2001.
Hutubessy R, Chisholm D, Edejer TT. Generalized cost-effectiveness analysis for national-level priority-setting in the health sector. Cost Eff Resour Alloc. 2003;1:8.
World Bank. World Bank Database. 2015. http://data.worldbank.org/. Accessed 26 Mar 2019.
The Government of Rwanda. Law Determining the State Finances for the 2017/2018 Fiscal Year, Official Gazette no Special of 30/06/2017. 2017. https://www.cabri-sbo.org/uploads/bia/rwanda_2017_approval_external_enacted_budget_ministry_of_finance_comesa_eac_english_.pdf.
Dowswell T, Carroli G, Duley L, Gates S, Gülmezoglu AM, Khan-Neelofur D, et al. Alternative versus standard packages of antenatal care for low-risk pregnancy. Cochrane Database Syst Rev. 2010;10:CD000934.
Bricker L, Henderson J, Martin M. Ultrasound screening in pregnancy: a systematic review of the clinical effectiveness, cost-effectiveness and women’s views. Health Technol Assess. 2000;4(16):i–vi 1–193.
Vogel JP, Habib NA, Souza JP, Gülmezoglu AM, Dowswell T, Carroli G, et al. Antenatal care packages with reduced visits and perinatal mortality: A secondary analysis of the WHO Antenatal Care Trial. Reprod Health. 2013;10:19.
Jowett M. Safe motherhood interventions in low-income countries: An economic justification and evidence of cost effectiveness. Health Policy. 2000;53:201–28.
Black RE, Laxminarayan R, Temmerman M, Walker N. Reproductive, Maternal, Newborn, and Child Health. 3rd ed. Washington DC: The International Bank for Reconstruction and Development/The World Bank; 2016.
Sharma SK. Does antenatal care matter reducing maternal mortality in Nepal ? J Popul. 2006;14:59–76.
Oyerinde K. Can antenatal care result in significant maternal mortality reduction in developing countries? J Community Med Health Educ. 2013;3(2):2–3.
Leal J, Wordsworth S, Legood R, Blair E. Eliciting expert opinion for economic models: an applied example. Value Health. 2007;10:195–203. https://doi.org/10.1111/j.1524-4733.2007.00169.x.
Sullivan W, Payne K. The appropriate elicitation of expert opinion in economic models: making expert data fit for purpose. PharmacoEconomics. 2011;29:455–9.
Eibling D, Fried M, Blitzer A, Postma G. Commentary on the role of expert opinion in developing evidence-based guidelines. Laryngoscope. 2014;124(2):355–7.
Grigore B, Peters J, Hyde C, Stein K. A comparison of two methods for expert elicitation in health technology assessments. BMC Med Res Methodol. 2016;16:85. https://doi.org/10.1186/s12874-016-0186-3.
Morgan MG. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc Natl Acad Sci U S A. 2014;111:7176–84.
Hitimana R, Lindholm L, Krantz G, Nzayirambaho M, Condo J, Paul J, et al. Health-related quality of life determinants among Rwandan women after delivery: does antenatal care utilization matter? A cross-sectional study. J Health Popul Nutr. 2018;37(1):12.
Peña-Rosas JP, De-Regil LM, Garcia-Casal MN, Dowswell T. Daily oral iron supplementation during pregnancy. Cochrane Database Syst Rev. 2015;7:CD004736.
Shah SP, Epino H, Bukhman G, Umulisa I, Dushimiyimana JMV, Reichman A, et al. Impact of the introduction of ultrasound services in a limited resource setting: rural Rwanda 2008. BMC Int Health Hum Rights. 2009;9:4.
Edvardsson K, Ntaganira J, Åhman A, Sengoma JPS, Small R, Mogren I. Physicians’ experiences and views on the role of obstetric ultrasound in rural and urban Rwanda: a qualitative study. Trop Med Int Health. 2016;21:895–906.
We would like to show our gratitude to Sewe Macquins Odhiambo, post-doctoral researcher at the unit of Sustainable Health, Department of Public Health and Clinical Medicine, Umeå University, for assistance with the simulation of ANC attendance. We also thank Dr. Valens Nkubito Gatera and Mrs. Clarisse Uwimana for their help in identifying gynecologists and obstetricians practicing in Rwanda, and for organising appointments. Our gratitude goes also to the respondents, who, despite their busy schedules, agreed to take part in this study.
This study is part of the Maternal Health in Rwanda Research Program (MaTHeR) undertaken by the University of Rwanda in collaboration with Gothenburg University and Umeå University with funding from the Swedish International Development Agency. The funder played no role in the design of the study; the collection, analysis, or interpretation of the data, or the writing of the manuscript.
Availability of data and materials
The full dataset from the expert elicitation and simulation results are provided in Additional Files. The data on cross-sectional survey referred to in this study, was collected in the framework of the ongoing maternal health research in Rwanda (MatHeR) project. The dataset will be available to the public after the completion of the project. However, before that publication, data will be available from the corresponding author upon reasonable request.
Regis Hitimana, PhD is an Assistant Lecturer at the University of Rwanda, School of Public Health.
Lars Lindholm, PhD, is a Professor in Health Economics at Umeå University, Department of Epidemiology and Global Health.
Gunilla Krantz, MD, PhD is a Professor in Public Health Sciences at Gothenburg University, Department of Public Health and Community Medicine.
Ingrid Mogren, MD, PhD is a Professor in Obstetrics and Gynecology at Umeå University, Department of Clinical Science, Obstetrics and Gynecology.
Manasse Nzayirambaho, PhD is an Associate Professor of Public Health at the University of Rwanda, School of Public Health.
Jean Paul Semasaka Sengoma, MD, PhD is an Assistant Lecturer at the University of Rwanda.
Anni-Maria Pulkki-Brännström, PhD is a Senior Research Assistant at Umeå University, Department of Epidemiology and Global Health.
Ethics approval and consent to participate
This study mainly uses previously published data from peer-reviewed articles and official documents. Expert elicitation was the only data source that involved primary data collection. We sought prior opinion from the ethical committee of the College of Medicine and Health Sciences of the University of Rwanda. A formal ethical application was not submitted because no personally sensitive data was collected during expert elicitation, apart from the names and employers of the respondents, which were kept with confidentiality.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Summary of data sources used. (DOCX 21 kb)
Distribution of women according to the number of antenatal care visits attended, in the current four-visit model. (DOCX 42 kb)
Summary of antenatal care utilisation in different countries from the Cochrane systematic review of antenatal care and WHO trial. (DOCX 14 kb)
Questionnaire for collection of opinions on potential health outcomes that can be expected from the expansion of antenatal care programme in Rwanda. (DOCX 38 kb)
Calculation of incremental cost. (XLSX 34 kb)
Calculation of discounted Life Years Saved. (XLSX 33 kb)