Skip to main content
Download PDF
Download ePub

Improving interpretation of publically reported statistics on health and healthcare: the Figure Interpretation Assessment Tool (FIAT-Health)

Health Research Policy and Systems volume 16, Article number: 20 (2018) Cite this article

Abstract

Background

Policy-makers, managers, scientists, patients and the general public are confronted daily with figures on health and healthcare through public reporting in newspapers, webpages and press releases. However, information on the key characteristics of these figures necessary for their correct interpretation is often not adequately communicated, which can lead to misinterpretation and misinformed decision-making. The objective of this research was to map the key characteristics relevant to the interpretation of figures on health and healthcare, and to develop a Figure Interpretation Assessment Tool-Health (FIAT-Health) through which figures on health and healthcare can be systematically assessed, allowing for a better interpretation of these figures.

Methods

The abovementioned key characteristics of figures on health and healthcare were identified through systematic expert consultations in the Netherlands on four topic categories of figures, namely morbidity, healthcare expenditure, healthcare outcomes and lifestyle. The identified characteristics were used as a frame for the development of the FIAT-Health. Development of the tool and its content was supported and validated through regular review by a sounding board of potential users.

Results

Identified characteristics relevant for the interpretation of figures in the four categories relate to the figures’ origin, credibility, expression, subject matter, population and geographical focus, time period, and underlying data collection methods. The characteristics were translated into a set of 13 dichotomous and 4-point Likert scale questions constituting the FIAT-Health, and two final assessment statements. Users of the FIAT-Health were provided with a summary overview of their answers to support a final assessment of the correctness of a figure and the appropriateness of its reporting.

Conclusions

FIAT-Health can support policy-makers, managers, scientists, patients and the general public to systematically assess the quality of publicly reported figures on health and healthcare. It also has the potential to support the producers of health and healthcare data in clearly communicating their data to different audiences. Future research should focus on the further validation of the tool in practice.

Peer Review reports

Background

Every day, numerous figures related to health and healthcare are reported in all kinds of sources. Policy-makers, managers, scientists, patients and the general public use these figures to guide their thinking on topics of health and healthcare [1,2,3,4]. Based on these figures, inferences are made on the severity, magnitude or impact of a health issue in society [5], influencing the decision-making process of patients [6, 7] and public opinion, which is central to priority-setting in health policy [8] and science [9].

Ideally, people base their decisions on the best available evidence, retrieving the figures which support their thinking directly from the source in which the figures are initially published; however, this is often not the case [10,11,12]. When looking for information on health and healthcare, people will obtain information from secondary sources and organisations they deem reliable [13], such as sources found through internet searches, newspapers, information leaflets of consumer organisations, television programmes and scientific information provided by research institutes [14,15,16,17].

During the construction process of figures on health and healthcare, choices are made on definitions of what is counted and measured, which inclusion and exclusion criteria are used, and which methodology is applied; moreover, interests of the involved parties may influence the results [18]. Figures on health and healthcare are often reproduced and cited in reports, summaries, fact sheets, press releases and news messages. Through this process, mistakes and misunderstandings may easily occur, or figures may be deliberately manipulated [19,20,21]. Inadequate communication of the construction of these figures may result in a misreporting of estimates such as prevalence, disease severity and outcomes of research [3, 22, 23], eventually leading to wrongful interpretation of figures by readers of such publications [24]. Furthermore, multiple sources of information may report contradicting figures on the same topic [25]. Contradictory or unclear reporting may cause uncertainty regarding a health(care) topic [26, 27], creating a barrier [28, 29] or leading to avoidance of decision-making, or even lead to misinformed decisions [30, 31]. In addition, figures on similar topics leave room for political and anecdotal use, such as applying the figure that fits best with the agenda of its users [31, 32] (e.g. politicians [33], patient advocacy of patient organisations [34], or media generating attention by publishing negative figures [35]).

Increasingly, attention is being paid to the translation of evidence into policy and practice [36], as illustrated, for instance, by the development of the AGREE instrument, which assesses the quality of the process and reporting of clinical practice guidelines [37], the AIRE instrument, which assesses the methodological quality of healthcare indicators and the connected reports [38], and the GATHER statement, which assesses the reporting practice of global health estimates [39]. Such instruments, which seem to be actively used in practice, are aimed to give an in-depth assessment of the quality of research outcomes and the detailed reporting of health estimates.

Methods aimed at the in-depth assessment of figures reported in scientific or extensive research publications are widely available. However, there is a lack of methods through which a practical assessment can be made of publically reported statistics. Journalists, policy-makers or interested citizens may question whether they can trust certain figures, but often do not have the time or inclination to dive into the world of statistics and research methodology. A need exists for an easy to use tool that supports users to gain insight into the key characteristics that contribute to the interpretation of a figure on health or healthcare [40,41,42]. The objectives of this study are to (1) map the key characteristics relevant to the interpretation of figures on health and healthcare, and (2) to develop a Figure Interpretation Assessment Tool (FIAT-Health) enabling systematic assessment of publically reported figures on health and healthcare to improve the proper use of these figures by policy-makers, managers, scientists, patients and the general public.

Methods

The design of the FIAT-Health was guided by a qualitative approach, relying on data derived from various forms of expert consultation in the Netherlands.

Four topic categories of figures on health and healthcare

A broad range of data is available on health and healthcare, resulting in numerous figures of varying origin. This study focusses on four categories of figures, selected by the research group (the authors) because they differ strongly in their nature and type of data required, but are all commonly used in health policy, monitoring and healthcare planning. Moreover, these types of figures are found daily in the media by the general public. The wide range of figures covered by these categories is likely to generate insight with regards to the most essential characteristics of figures that are important for their interpretation, and are also generalisable to other figures on health and healthcare.

Morbidity figures

Morbidity figures are key in determining the incidence and prevalence of diseases in a population, forming the first ideal step to estimate the need for healthcare [43]. Figures on morbidity are typically collected through various sources, such as clinical registry data and health surveys.

Healthcare expenditure figures

Planning of healthcare services relies on the affordability of care, which is estimated through figures on healthcare expenditure [44, 45]. Depending on the type of system, expenditure data may be based on tax data, insurance claims, providers’ balance sheets, etc. Cost may, for example, be presented in relation to the GDP, per person, or in relation to certain diseases or types of care. In this category, figures are often the result of modelling.

Healthcare outcome figures

The quality of healthcare services is often perceived through figures on healthcare outcomes. Through these figures, the need for healthcare quality improvement and action on health is determined [46, 47]. This includes clinical outcomes, such as readmissions or complications and patient-reported outcomes. Data on healthcare outcomes is mostly derived through various clinical registries, administrative databases and patient surveys.

Lifestyle figures

Lifestyle is an important determinant of health, which is needed to estimate changes in the health of a population [48]. Often used examples are figures regarding physical activity, diet, and the use of alcohol, tobacco and drugs. Quantitative information on lifestyle is mostly collected using surveys and increasingly by wearable devices [49]. For macro-level figures, turnover of certain industries (e.g. tobacco) can also be used.

Data collection

The research took place in the Netherlands, involving experts from four academic institutions and one national public health institute. Through purposeful sampling [50], researchers and knowledge integration specialists who specialise in communicating research findings to policy-makers and the public, were selected, based on their extended expertise in the respective topic categories. Furthermore, science journalists and communication officers were consulted for their experience with reporting figures in the media. Expert consultations were carried out in various formats, constructed according to the emerging knowledge need and participation of experts, as outlined below. The data collection process is described in Fig. 1. Each consultation meeting lasted between 60 and 90 minutes. By taking notes during the meetings and adding written comments made by the participants, the first author compiled a report for each session.

Fig. 1
figure 1

Data collection process

Full size image

Stage 1

The first stage concerned the gathering of a broad list of characteristics on the practice of reporting figures. Data on the characteristics was obtained through the expert consultation for the morbidity category involving 33 specialists in the integration of knowledge on health and healthcare from the National Institute for Public Health and the Environment (RIVM). Fourteen groups of three experts each were asked to review the figures on a disease on four national websites from the RIVM [51,52,53,54]. To make groups of three, nine experts participated in two different groups. Experts were asked to note the type of prevalence or incidence, the population, year, sex, age, source of the figure and the source of the data on which the figure was based. They were asked to reflect on the clarity of the figures, give their opinion on the current publication practices, identify inconsistencies that they found between the collected figures across the four websites, and identify what information was needed to improve interpretation of the figures. Each group gathered face-to-face and gave a joint written response in a pre-structured Excel file. Their notes were compiled by the first author in a separate Excel file and summarised.

Stage 2

As in the first stage, a broad list of characteristics was derived, the second stage concerned the addition of characteristics particular to the category healthcare expenditure. A group process including a discussion was considered to provide deeper insight into these characteristics. Four senior health economists from two health economics research departments and the RIVM were invited. All four experts accepted the invitation, although only two were finally able to attend the group session. During the session, the participants were asked to note their thoughts on the interpretation of presented figures on healthcare expenditure supported by an example of a report; consequently, the notes were structured during the session and recorded by the two present researchers. Separate appointments were made with the participants who did not attend the first session. During these individual consultations, the findings of the first meeting were discussed in-depth, complemented and validated.

Stage 3

Based on the previous stages, an extended characteristics list was developed. At this stage, more in-depth questioning on the construction of figures was sought. These questions were asked during the expert consultation for the healthcare outcomes category. The consultation consisted of two in-person interviews and one phone interview with three senior researchers from two academic institutions and one private company. Open questions with a semi-structured format were used to gain insight into the considerations made during both the construction and use of figures on healthcare outcomes.

Stage 4

At the fourth stage, an early version of the tool was developed, and gaps in the list of the items needed to be identified and phrasing needed to be clarified. Making use of the available expertise at the research institute, the expert consultation for the lifestyle category was addressed through a meeting with 28 junior and senior researchers of a Public Health research department. Participants received a publication on a lifestyle figure. Three groups assessed the figure through an early version of the FIAT-Health, while three other groups assessed the figure without the structured support of the draft FIAT-Health. Their findings and experiences with assessing the figure were compared and discussed in a plenary session.

Furthermore, two science journalists writing for two online journalist platforms, and two communication officers from the RIVM were individually consulted on their experience with the communication of figures on health and healthcare towards the public. Both journalists had specific knowledge on the communication of figures on health and healthcare. The consultations were aimed at the use of figures in the media, and they shared their experience on how they make figures on health and healthcare understandable to the public and policymakers.

Data analysis

Based on the expert consultations [55], insight was gained through an inductive approach in the key characteristics of each of the four topic categories of figures on health and healthcare that are relevant to their interpretation. First, to gain an overview of characteristics relevant to the interpretation of figures, characteristics related to the use of figures and those related to the construction of figures were coded in the session reports by the first author, resulting in an overview of characteristics for each of the four categories, and were successively compiled into a single list. Finally, characteristics in this list were synthesised resulting in the key characteristics that are essential to the interpretation of most figures on health and healthcare.

The format of the expert consultations differed among the categories in terms of approach and number of participants. Instead of first collecting all data in a single stage, an iterative process was applied during which multiple draft versions of the FIAT-Health were developed over time based on the emerging characteristics and shared with the involved experts. Hence, in the fourth stage, during the final interview with a communication officer and the final interview with a science journalist, a preliminary draft to which participants could respond was presented, which allowed the testing of insights gained from prior sessions in subsequent consultations.

Development of the FIAT-Health

The key characteristics were structured in eight overarching themes that formed the basis of tool development, guiding the development of the content, form and outcome of the FIAT-Health. The themes comprised the structure on which the main components of the tool were based. Starting from this structure, the components suitable to address the themes were created.

The development of the tool was guided by constant dialogue amongst the research group, consisting of the authors of this paper. Furthermore, to gain insight into the expectations and needs of potential users of the tool, nine project leaders of the RIVM, who are highly experienced in the publication of figures on health and healthcare, were gathered in a sounding board. This sounding board supported the development of the tool by advising on the form and regularly reviewing its content. The panel gathered five times, during which draft versions of the tool were reviewed in detail. Feedback resulting from these meetings was used to adapt and refine the FIAT-Health. During one additional meeting, an early version of the instrument was tested by the sounding board and several knowledge integration specialists at the RIVM. In small groups they assessed an example of a published figure and reflected on the results.

Face validity was established through examination by the research group of the relevance, reasonability, unambiguousness and clarity of the content [55]. Content validity was assured through the review of the tool on both content as well as form by the sounding board, thereby assuring a balanced inclusion of the aspects relevant for the interpretation of figures on health and healthcare [55].

Results

Characteristics relevant to the interpretation of figures on health and healthcare were mapped for the four categories. Interpretation was considered to consist of characteristics relevant to the construction and use of a figure. The construction of a figure relates to the methodological considerations that impact the quality of a figure, and characteristics relevant to the use of a figure relates to the information which is needed to apply the figure in practice. As the objective was to identify the characteristics relevant to the interpretation of figures, only the characteristics related to the construction of the figure that could be understood by those without specialised knowledge were included. Furthermore, characteristics that were relevant to the use of a figure on health and healthcare and which related to the information the user of a figure would need to assess the usability of a figure in a certain context were also included. The resulting key characteristics that were relevant for the interpretation of each of the four categories of figures were grouped into eight themes.

Themes

Origin of the figure

This refers to the primary publication in which the figure was published for the first time. Any (secondary) publication that cites or refers to the figure should properly refer to this primary publication. The primary publication needs to be accessible to confirm the construction of the figure.

Credibility of the figure

Credibility is, to a large extent, a subjective judgement of the reader or user of a figure. This judgement is based on the expected expertise of the author and on possible conflicts of interest. Regarding expertise, a peer-reviewed scientific article from a well-known research group leads to higher expectations of credibility than an unknown blogger. Any financial or political interests may bias results or may appear to do so.

Expression of the figure

Two categories are distinguished, namely singular figures, such as an average or an absolute number, and composite figures such as a percentage or a fraction. The characterisation of the figure will impact the way the figure is perceived by its reader. When a figure is communicated in a singular form, the context of that figure might be lost to its user, while a composite form could obscure the actual figure. Either way, the form will impact the way the reader will value a figure.

Subject to which the figure applies

The definition of the subject is a decision made by the authors of the publication. Often, several definitions of the seemingly same subject are available. A more broad or narrow definition can impact figures considerably. Moreover, a misconception of the definition of the subject will result in a wrongful interpretation.

Population to which the figure applies

The population forms the basis for the figure. The inclusion and exclusion criteria have an impact to whom or to what the figure applies. The interpretation of the figure needs to be supported by the knowledge of the exact population the figure applies to.

The geographical area to which the figure applies

The geographical area concerns the location to which the figure is generalisable. While the geographical area has a large impact on the figure, this characteristic is easily miscommunicated. A figure which is valid for one geographical area may not be applied to another.

The time period to which the figure applies

The time period to which the figure applies relates to the question on whether a figure is relevant to the time in which it is reported. Often, a time period to which the figure applies is not mentioned in a secondary publication, implicitly assuming that the figure applies to the present. Whether a figure that is, for example, 2 years old is still relevant at the present time differs strongly between subjects and the aim for which it is used. Furthermore, experience reveals that figures which are counted at one point in time are often confused with longitudinal figures, i.e. point-prevalence and year-prevalence.

The process of counting and measuring

The process of counting and measuring is a broad theme covering different methodologies used to construct a figure, and the practical considerations of these methodologies. For each methodology, the main strengths and weaknesses need to be understood in order to interpret the figure.

Repetition of data collection

Furthermore, the repetition of data collection is considered in this theme. A figure derived from repeated or continuous data collection may be updated over time, or can be compared with earlier figures derived from the same data collection source.

Sampling

Often, a sample of the population is used. A sample should be large and varied enough to represent the entire population. Statistical reliability increases with sample size and if the sample is relatively small, the reader should be careful with attaching value to it.

Registries

Registries are used as the basis for many figures on health and healthcare. Here, the way the data is registered, and who registers the data, is of importance. The quality of a registry, and thus the figure derived from it, depends on the completeness of data and the care with which the data is entered.

Surveys

Questions asked and answers given to a survey determine the eventual figure. Furthermore, these questions should be carefully deduced to conclusions on the subject.

Direct observations

Data can be obtained through measurements made by researchers or field workers, or in other words, through direct observations. In some cases, this method has an advantage over a survey, but this may depend on the measurement instrument and the care with which the researcher measures. Obviously, many factors, for instance, patient experiences, cannot be easily observed.

Modelling

If a figure cannot be constructed through empirical methods, the possibility of modelling exists. In modelling, many figures are used for which assumptions are made by the modeller. These assumptions should be well grounded and transparent in order to interpret the reliability of the figure. The plausibility of certain assumptions may often be difficult to judge for many readers.

The FIAT-Health

The FIAT-Health was constructed out of the eight themes relevant to the interpretation of figures on health and healthcare. The inclusion of all features was decided upon during an iterative process through which the research group thoroughly examined each item included in the tool. Five draft versions of the tool were reviewed by the sounding board. The sounding board gathered five times in order to review the emerging tool, and four draft versions were presented and fine-tuned based on the received feedback. Consensus was reached on the fifth version. In the FIAT-Health, a distinction is made between the primary publication, which is the source in which the figure is reported for the first time, and the publication in which the figure is assessed, which can be both a publication referring to a figure from a primary publication or the primary publication itself.

The characteristics relevant to the interpretation of figures on health and healthcare are addressed through a closed question format with directed routing. As multiple definitions can be applied to the terms used in the FIAT-Health, all terms and phrasing of the questions were discussed extensively and approved by both the research group and the sounding board.

The FIAT-Health consists of 13 main questions with sub-questions, numbered 1–13, and two statements, numbered 14 and 15. Questions 1–13 guide the interpretation and assessment of the figure, and consist of two types of questions, namely characterisation questions and assessment questions; the former are neutral, whereas the latter express a value judgment. The characterisation questions will guide the user into understanding the figure without directing towards an assessment, while assessment questions do provide a distinction between a positive and negative answer. Three types of assessment questions are used, (1) factual questions, which relate to the accessibility of information on a particular theme; (2) assessment questions, for which the user is asked to give a value judgment on a particular theme; and (3) correspondence questions, which relate to the correspondence between the primary publication and the publication in which the user is assessing the figure, which is not relevant if the figure is assessed in a primary publication.

Completing all questions results in a structured, 1-page overview of answers given to questions 1–13. This structured overview supports users to order their thoughts about the strengths and weaknesses of the figure. Through statements 14 and 15, users respectively assess the correctness of the figure and assess the appropriateness of the report of the figure.

Answer format

Characterisation questions are answered using a dichotomous yes/no scale. These answers do not give a positive or negative value to the answer. Factual and correspondence questions are also answered using a dichotomous yes/no scale, on which the answer ‘yes’ is continuously positive and ‘no’ is negative. Answers to assessment questions are given through a numerical 4-point Likert scale (1 = negative, 4 = positive). To support the user of the FIAT-Health in assessing the figure, a 4-point scale was chosen, avoiding a middle option to stimulate users in forming an assessment [56]. Concluding statements 14 and 15 are accompanied by a 5-star scale, which has been found to be easy to understand in previous studies [57]. Furthermore, to guide the user of the FIAT-Health, the questionnaire is accompanied by a user guide including explanations of all questions.

The glossary used in the FIAT-Health is described in Table 1. The FIAT-Health 1.0 questionnaire, as provided in Table 2, was developed in Dutch. The translation to English was conducted through a forward-back translation by two bilingual translators. Discrepancies in translation were discussed within the research group and the translators until agreement was reached [55].

Table 1 Glossary of the FIAT-Health
Full size table
Table 2 The FIAT-Health 1.0 questionnaire
Full size table

Discussion

The purpose of this study was to develop a tool enabling a systematic assessment of publically reported figures on health and healthcare to improve assessment of these figures by policy-makers, managers, journalists, researchers, patients and the general public. The FIAT-Health can be used to assess publically reported figures on health and healthcare, and is recommended to be applied by (1) policy-makers to support their interpretation of figures to guide their decision-making process; (2) knowledge institutes and policy advisers to provide grounded advise on the use of figures on health and healthcare to policy-makers; (3) journalists, bloggers and information officers during the writing of a public report, summary or commentary; and (4) researchers for the translation of their own research to the public.

The tool should enable its user to (1) detect possible causes of bias associated with the construction of a figure; (2) distinguish the weaknesses and strengths associated to the methodology of a figure; (3) recognise in which context a figure may be used; and (4) detect inconsistencies in the communication of the figure reported to the public and in the primary publication. The FIAT-Health is designed to support its user to systematically interpret a figure on health or healthcare, facilitating the assessment of the quality of the figure, and the assessment of the appropriateness of the public report of this figure. The FIAT-Health is a practical tool avoiding extreme details while assuring coverage of all relevant aspects of the construction of the figure. Moreover, the tool should not generate a definite conclusion on the value of a figure, but should serve as a systematic guide for the thinking process of its user.

The importance of the characteristics found in this study is widely recognised to be influencing interpretation of figures, and are key to the reporting of figures on health and healthcare. Particular to this tool is the attention paid towards the clarity of definitions [58,59,60,61] and the credibility [21, 29] of the primary publication. Novel to this tool are the questions aimed to characterise the figure, which is often not addressed in any reporting guideline, but has a large influence on how the figure is interpreted [62, 63]. Furthermore, other themes are in accordance with the content of other tools supporting the reporting of quantitative evidence, namely the GATHER statement [39], the Drummond checklist [64] and the STROBE checklist [65]. Like the GATHER statement, the FIAT-Health is meant to improve reporting as well as to serve the information needs of decision-makers. The FIAT-Health is applicable to figures of varying origin. While knowledge usually relates to scientific evidence [66], the FIAT-Health is not limited to scientific evidence, and is thus usable in the many cases where figures are constructed through other means. Therefore, this tool does not dismiss the value of figures found through alternate methods, but allows the balanced interpretation of these figures.

Strengths and limitations

The main strength of the FIAT-Health is the development through collaboration with its potential users, ensuring the relevance of the content of the tool and its suitability for application in professional settings. As the tool is primarily based on the perspectives of both the research and funding institution, it is likely that the language used as well as its characteristics may be biased towards their research paradigm. The involvement of other experts with a different affinity with the problem might have resulted in fewer items. In subsequent research, a larger number of institutes should be involved to broaden the perspective of the tool. Experts involved in the consultation rounds were sensitive to the problem of inadequate reporting. To avoid bias in the selection of experts, different groups of experts were involved, including junior and senior researchers and knowledge integration specialists. Additionally, journalists were consulted. Researcher bias in the identification of items was avoided through regular interaction with the research group and the sounding board. Furthermore, focus groups may incite ‘group think’, where participants may adapt their opinion to fit the group. This bias was avoided by organising different forms of consultation, including individual interviews during the developmental process.

The development of the FIAT-Health was supported by a sounding board, a method comparable with a nominal group technique involving the review of material received in advance and discussion amongst experts forming consensus [67]. This study deviates from a nominal technique as the sounding board in this study involved several rounds and focussed on a qualitative assessment. Although the FIAT-Health is based upon the assumption that better information on evidence leads towards better decisions, it is not a decision support tool such as the SUPPORT tools [68], which have the goal to guide policy-makers directly.

Fully understanding all methodological characteristics of figures, and overseeing its consequences, is often extremely challenging and time consuming, even for those who work in research. Further, figures communicated in the media are often biased and misunderstood. The FIAT-Health is by no means a panacea that will fully solve this problem. While developing the tool, one of the main challenges was to find the right balance between thoroughness, on the one hand, and practicality and compactness on the other. It may be unavoidable that the tool is too complex and time consuming for some, and too superficial for others. Consequently, the expectation that an average newspaper-reader will invest time in assessing figures using the FIAT-Health is unrealistic; neither will an experienced statistician obtain great revelations using the FIAT-Health; however, the purpose of this research is not to aid these groups of people. The FIAT-Health fills the critical gap between the expert and the news-consumer, consisting of policy-makers and advisors, journalists, managers, patients and those of the general public whose decisions depend on the correctness of a figure and its reporting.

Steps ahead

Although face and content validity are established, the FIAT-Health 1.0 is not yet tested for user experience and construct validity. Therefore, a validation study will be performed during the next stage of this study. The current paper format of the tool does not facilitate easy usage, creating a possible barrier to the uptake of the instrument. To encourage uptake of the FIAT-Health into practice, an online version is under construction, facilitating the easy and efficient routing of a user through the questions, after which an overview of the answers can be provided. Nevertheless, despite its current practical limitations, the questions provided in the FIAT-Health are highly relevant, and can thus be applied in every-day practice immediately. As such, a sample of the content of the RIVM website ‘de Staat van Volksgezondheid en Zorg’ was systematically assessed using the FIAT-Health 1.0. The implementation of the FIAT-Health in the improvement process of the website is currently being explored.

Conclusion

The FIAT-Health is a tool enabling systematic assessment of publically reported figures on health and healthcare, to support a better understanding and interpretation of these figures by policy-makers, managers, researchers, patients, and the general public. The use of the tool results in a 1-page overview, representing the main strengths and weaknesses of a figure, filling the gap of scientifically developed methods which support the public reporting of figures on health and healthcare. As few systematic methods are available through which figures on health and healthcare can be interpreted, the FIAT-Health adds a practical approach through which users are better informed and supported in their decision-making processes.

Abbreviations

FIAT-Health:

Figure Interpretation Assessment Tool–Health

RIVM:

Dutch National Institute for Public Health and the Environment

References

  1. Vogel JP, Oxman AD, Glenton C, Rosenbaum S, Lewin S, Gulmezoglu AM, Souza JP. Policymakers' and other stakeholders' perceptions of key considerations for health system decisions and the presentation of evidence to inform those considerations: an international survey. Health Res Policy Syst. 2013;11:19.

  2. Schwitzer G, Mudur G, Henry D, Wilson A, Goozner M, Simbra M, Sweet M, Baverstock KA. What are the roles and responsibilities of the media in disseminating health information? PLoS Med. 2005;2(7):e215.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Young ME, Norman GR, Humphreys KR. Medicine in the popular press: the influence of the media on perceptions of disease. Plos One. 2008;3(10):e3552.

    Article  PubMed  PubMed Central  Google Scholar 

  4. de Goede J, van Bon-Martens MJH, Mathijssen JJP, Putters K, van Oers HAM. Looking for interaction: quantitative measurement of research utilization by Dutch local health officials. Health Res Policy Syst. 2012;10:9.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Williamson JML, Skinner CI, Hocken DB. Death and illness as depicted in the media. Int J Clin Pract. 2011;65(5):547–51.

    Article  CAS  PubMed  Google Scholar 

  6. Holmes-Rovner M, Charles S. The mammography screening controversy: who and what is heard in the press? Patient Educ Couns. 2003;51(1):75–81.

    Article  PubMed  Google Scholar 

  7. Passalacqua R, Caminiti C, Salvagni S, Barni S, Beretta GD, Carlini P, Contu A, Di Costanzo F, Toscano L, Campione F. Effects of media information on cancer patients' opinions, feelings, decision-making process and physician-patient communication. Cancer. 2004;100(5):1077–84.

    Article  PubMed  Google Scholar 

  8. Burstein P. The impact of public opinion on public policy: a review and an agenda. Polit Res Quart. 2003;56(1):29–40.

    Article  Google Scholar 

  9. Weingart P. Science and the media. Res Policy. 1998;27(8):869–79.

    Article  Google Scholar 

  10. Russell J, Greenhalgh T, Byrne E, McDonnell J. Recognizing rhetoric in health care policy analysis. J Health Serv Res Policy. 2008;13(1):40–6.

    Article  PubMed  Google Scholar 

  11. Greenhalgh T, Russell J. Evidence-based policymaking: a critique. Perspect Biol Med. 2009;52(2):304–18.

    Article  PubMed  Google Scholar 

  12. Black N. Evidence based policy: proceed with care. BMJ. 2001;323(7307):275–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Turney J. Public understanding of science. Lancet. 1996;347(9008):1087–90.

    Article  CAS  PubMed  Google Scholar 

  14. Dobbins M, Jack S, Thomas H, Kothari A. Public health decision-makers' informational needs and preferences for receiving research evidence. Worldviews Evid Based Nurs. 2007;4(3):156–63.

    Article  PubMed  Google Scholar 

  15. Ritter A. How do drug policy makers access research evidence? Int J Drug Policy. 2009;20(1):70–5.

    Article  PubMed  Google Scholar 

  16. Zardo P, Collie A. Type, frequency and purpose of information used to inform public health policy and program decision-making. BMC Public Health. 2015;15:381.

    Article  PubMed  PubMed Central  Google Scholar 

  17. McVay AB, Stamatakis KA, Jacobs JA, Tabak RG, Brownson RC. The role of researchers in disseminating evidence to public health practice settings: a cross-sectional study. Health Res Policy Syst. 2016;14:42.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–65.

    Article  PubMed  Google Scholar 

  19. Woloshin S, Schwartz LM, Casella SL, Kennedy AT, Larson RJ. Press releases by academic medical centers: not so academic? Ann Intern Med. 2009;150(9):613–8.

    Article  PubMed  Google Scholar 

  20. Bodemer N, Muller SM, Okan Y, Garcia-Retamero R, Neumeyer-Gromen A. Do the media provide transparent health information? A cross-cultural comparison of public information about the HPV vaccine. Vaccine. 2012;30(25):3747–56.

    Article  PubMed  Google Scholar 

  21. Caulfield T. The commercialisation of medical and scientific reporting. PLoS Med. 2005;1(3), e38.

    Article  Google Scholar 

  22. Frost K, Frank E, Maibach E. Relative risk in the news media: a quantification of misrepresentation. Am J Public Health. 1997;87(5):842–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, Mantz J, Ravaud P. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med. 2012;9(9):e1001308.

  24. Simmerling A, Janich N. Rhetorical functions of a 'language of uncertainty' in the mass media. Public Underst Sci. 2015;25(8):961–75. doi:10.1177/0963662515606681

  25. Goldman SR, Scardamalia M. Managing, understanding, applying, and creating knowledge in the information age: next-generation challenges and opportunities. Cognition Instruct. 2013;31(2):255–69.

    Article  Google Scholar 

  26. Schwitzer G. Addressing tensions when popular media and evidence-based care collide. BMC Med Inform Decis Mak. 2013;13 Suppl 3:S3. doi:10.1186/1472-6947-13-S3-S3.

  27. Andermann A, Pang T, Newton JN, Davis A, Panisset U. Evidence for health II: overcoming barriers to using evidence in policy and practice. Health Res Policy Syst. 2016;14:17.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.

    Article  PubMed  PubMed Central  Google Scholar 

  29. de Goede J, Putters K, van der Grinten T, van Oers HA. Knowledge in process? Exploring barriers between epidemiological research and local health policy development. Health Res Policy Syst. 2010;8:26.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Fischhoff B, Davis AL. Communicating scientific uncertainty. Proc Natl Acad Sci U S A. 2014;111:13664–71.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Furedi A. The public health implications of the 1995 'pill scare'. Hum Reprod Update. 1999;5(6):621–6.

    Article  CAS  PubMed  Google Scholar 

  32. The Editors. When evidence collides with anecdote, politics, and emotion: breast cancer screening. Ann Intern Med. 2010;152(8):531–2.

    Article  Google Scholar 

  33. Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients make sense of health statistics. Psychol Sci Public Interest. 2007;8(2):53–96.

    Article  PubMed  Google Scholar 

  34. McKeganey N, Russell C. Tobacco plain packaging: evidence based policy or public health advocacy? Int J Drug Policy. 2015;26(6):560–8.

    Article  PubMed  Google Scholar 

  35. Sandell T, Sebar B, Harris N. Framing risk: communication messages in the Australian and Swedish print media surrounding the 2009 H1N1 pandemic. Scand J Public Health. 2013;41(8):860–5.

    Article  PubMed  Google Scholar 

  36. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.

    Article  PubMed  PubMed Central  Google Scholar 

  37. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1):18–23.

    Article  Google Scholar 

  38. de Koning JS, Kallewaard M, Klazinga NS. Prestatie-indicatoren langs de meetlat – het AIRE instrument. TVGW. 2007;85:261.

    Article  Google Scholar 

  39. Stevens GA, Alkema L, Black RE, Boerma JT, Collins GS, Ezzati M, Grove JT, Hogan DR, Hogan MC, Horton R, et al. Guidelines for accurate and transparent health estimates reporting: the GATHER statement. PLoS Med. 2016;13(6), e1002056.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Walker N, Bryce J, Black RE. Interpreting health statistics for policymaking: the story behind the headlines. Lancet. 2007;369(9565):956–63.

    Article  PubMed  Google Scholar 

  41. Peters E. Beyond comprehension: the role of numeracy in judgments and decisions. Curr Dir Psychol Sci. 2012;21(1):31–5.

    Article  Google Scholar 

  42. Han PK, Moser RP, Klein WM, Beckjord EB, Dunlavy AC, Hesse BW. Predictors of perceived ambiguity about cancer prevention recommendations: sociodemographic factors and mass media exposures. Health Commun. 2009;24(8):764–72.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Jones IR. Health needs and their assessment: demography and epidemiology. In: Bowling A, editor. Research Methods in Health: Investigating Health and Health Services. 2nd ed. Buckingham: Open University Press; 2002. p. 73.

  44. Morris S, Devlin N, Parkin D, Spencer A. Economic Analysis in Healthcare. Chichester: Wiley; 2012.

    Google Scholar 

  45. OECD, Eurostat, WHO. A System of Health Accounts. Paris: OECD Publishing; 2011.

    Google Scholar 

  46. Geddis PW. Health service performance indicators. Ulster Med J. 1988;57(2):121–8.

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–81.

    Article  CAS  PubMed  Google Scholar 

  48. Mackenbach JP, Stronks K. Inleiding. In: Mackenbach JP, Stronks K, editors. Volksgezondheid en gezondheidszorg. 8th ed. Houten: Bohn Stafleu van Loghum; 2016. p. 36–7.

    Google Scholar 

  49. Loveday A, Sherar LB, Sanders JP, Sanderson PW, Esliger DW. Technologies that assess the location of physical activity and sedentary behavior: a systematic review. J Med Internet Res. 2015;17(8), e192.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42(5):533–44.

    Article  PubMed  PubMed Central  Google Scholar 

  51. De Staat van Volksgezondheid en Zorg. 2016. https://www.staatvenz.nl/. Accessed 4 November 2016.

  52. Volksgezondheidenzorg.info. 2016. https://www.volksgezondheidenzorg.info/. Accessed 4 November 2016.

  53. Nationaal Kompas Volksgezondheid. 2011. https://www.rivm.nl/Documenten_en_publicaties/Wetenschappelijk/Databases/Preventie_Ziekte_Zorg/Nationaal_Kompas_Volksgezondheid. Accessed 28 Feb 2018.

  54. De VTV-2014. 2016. http://www.eengezondernederland.nl/. Accessed 4 November 2016.

  55. Bowling A. Research Methods in Health, Investigating Health and Health Services. 2nd ed. Buckingham: Open University Press; 2002.

    Google Scholar 

  56. Nadler JT, Weston R, Voyles EC. Stuck in the middle. The use and interpretation of mid-points in items on questionnaires. J Gen Psychol. 2015;142(2):71–89.

    Article  PubMed  Google Scholar 

  57. Shwartz M, Restuccia JD, Rosen AK. Composite measures of health care provider performance: a description of approaches. Milbank Q. 2015;93(4):788–825.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Anema HA, van der Veer SN, Kievit J, Krol-Warmerdam E, Fischer C, Steyerberg E, Dongelmans DA, Reidinga AC, Klazinga NS, de Keizer NF. Influences of definition ambiguity on hospital performance indicator scores: examples from The Netherlands. Eur J Public Health. 2014;24(1):73–8.

    Article  PubMed  Google Scholar 

  59. Dorn HF. Methods of measuring incidence and prevalence of disease. Am J Public Health Nations Health. 1951;41(3):271–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Rosen T, Olsen J. Invited commentary: the art of making questionnaires better. Am J Epidemiol. 2006;164(12):1145–9.

    Article  PubMed  Google Scholar 

  61. Jaime Caro J, Eddy DM, Kan H, Kaltz C, Patel B, Eldessouki R, Briggs AH, ISPOR-AMCP-NPC Modeling CER Task Forces. Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):174–82.

    Article  CAS  PubMed  Google Scholar 

  62. Ancker JS, Kaufman D. Rethinking health numeracy: a multidisciplinary literature review. J Am Med Inform Assoc. 2007;14(6):713–21.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Yamagishi K. When a 12.86% mortality is more dangerous than 24.14%: implications for risk communication. Applied Cognitive Psychol. 1997;11(6):495–506.

    Article  Google Scholar 

  64. Drummond MF, Sculpter MJ, Torrance GW, O'Brien BJ, Stoddart GL. Methods for the Economic Evaluation of Health Care Programmes. 3rd ed. Oxford: Oxford University Press; 2005.

    Google Scholar 

  65. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, Initiative S. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Int J Surg. 2014;12(12):1495–9.

    Article  Google Scholar 

  66. Contandriopoulos D, Lemire M, Denis J-L, Tremblay É. knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q. 2010;88(4):444–83.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311:376–80.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Lavis JN, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP). Introduction. Health Res Policy Syst. 2009;7 Suppl 1:I1.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank all experts and the members of the sounding board for their valuable input and advice. We thank Antonia Verweij for her work on the collection of information on morbidity figures.

Funding

This research was funded by the Dutch National Institute for Public Health and the Environment (RIVM). URL: http://rivm.nl/. To optimise the relevance and applicability of the tool, the RIVM (represented by MJ van den Berg) was involved in data collection and analysis, decision to publish, and preparation of the manuscript.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Author information

Authors and Affiliations

  1. Department of Public Health, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

    Reinie G. Gerrits, Dionne S. Kringos, Michael J. van den Berg & Niek S. Klazinga

  2. National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands

    Michael J. van den Berg

Authors
  1. Reinie G. Gerrits
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dionne S. Kringos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael J. van den Berg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Niek S. Klazinga
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RG, DS, MB and NS designed the study and interpreted the outcomes. RG, DS and MB collected the data, and RG analysed the data and drafted the article. DS, MB and NS were major contributors to the writing of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Reinie G. Gerrits.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gerrits, R.G., Kringos, D.S., van den Berg, M.J. et al. Improving interpretation of publically reported statistics on health and healthcare: the Figure Interpretation Assessment Tool (FIAT-Health). Health Res Policy Sys 16, 20 (2018). https://doi.org/10.1186/s12961-018-0279-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12961-018-0279-z

Keywords

Health Research Policy and Systems

ISSN: 1478-4505

Contact us