Skip to main content

The reporting of funding in health policy and systems research: a cross-sectional study

Abstract

Background

Major research-reporting statements, such as PRISMA and CONSORT, require authors to provide information about funding. The objectives of this study were (1) to assess the reporting of funding in health policy and systems research (HPSR) papers and (2) to assess the funding reporting policies of journals publishing on HPSR.

Methods

We conducted two cross-sectional surveys for papers published in 2016 addressing HPSR (both primary studies and systematic reviews) and for journals publishing on HPSR (both journals under the ‘Health Policy and Services’ (HPS) category in the Web of Science, and non-HPS journals that published on HPSR). Teams of two reviewers selected studies and abstracted data in duplicate and independently. We conducted descriptive analyses and a regression analysis to investigate the association between reporting of funding by papers and the journal’s characteristics.

Results

We included 400 studies (200 systematic reviews and 200 primary studies) that were published in 198 journals. Approximately one-third (31%) of HPSR papers did not report on funding. Of those that did, only 11% reported on the role of funders (15% of systematic reviews and 7% of primary studies). Of the 198 journals publishing on HPSR, 89% required reporting of the source of funding. Of those that did, about one-third (34%) required reporting of the role of funders. Journals classified under the HPS category (n = 72) were less likely than non-HPS journals that published HPSR studies (n = 142) to require information on the role of funders (15% vs. 32%). We did not find any of the journals’ characteristics to be associated with the reporting of funding by papers.

Conclusions

Despite the majority of journals publishing on HPSR requiring the reporting of funding, approximately one-third of HPSR papers did not report on the funding source. Moreover, few journals publishing on HPSR required the reporting of the role of funders, and few HPSR papers reported on that role.

Peer Review reports

Background

There is growing evidence that source of funding is associated with the reporting of research results. A recent Cochrane methods systematic review found that industry-sponsored studies were more likely than non-industry-sponsored studies to report favourable efficacy results and favourable conclusions (relative incidence increased by 27% and 34%, respectively) [1]. Further, a literature survey of randomised controlled studies published in 2011 found that the majority of studies sponsored by industry reported favourable results [2]. Therefore, the reporting of funding in published evidence might help in interpreting results and highlight possible bias [3]. One study found that physicians were less confident in the results of trials that disclosed industry funding compared to those that did not [4]. In addition, physicians’ willingness to prescribe was higher for drugs assessed in government-funded trials compared to those assessed in industry-funded trials [4]. Another study evaluating 106 review articles found that affiliation of the review author with the tobacco industry was the only factor associated with concluding that passive smoking is not harmful [5].

Guidelines for reporting health research require the reporting of the source of funding and the role of funders given their influence on the design, conduct, analysis and reporting of research [6,7,8]. International Committee of Medical Journals Editors highlighted the importance of the reporting of funding sources as it could bias the viewpoint and the choice of topics [6]. The latest version of the CONSORT statement (short for Consolidated Standards of Reporting Trials) requires the reporting of the sources of funding and the level of involvement of funders [7]. Similarly, the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement underlined the importance of transparent funding reporting as systematic reviews have a critical role in decision-making [8].

While studies have investigated the reporting of funding by papers published in clinical journals [9,10,11,12] and the policies of public health journals regarding the reporting of funding, we are not aware of similar studies in health policy and systems research (HPSR). Therefore, the objectives of this study were (1) to assess the reporting of funding in reporting HPSR papers and (2) to assess the funding reporting policies of journals publishing on HPSR.

Methods

Overall design

We conducted two cross-sectional surveys, one for papers addressing HPSR and one for journals publishing on HPSR. Below, we describe the details of the two surveys. For both surveys, we considered the following definitions:

  • Funding: any sort of support to the planning, conduct or reporting of a study, in either monetary or non-monetary form (e.g. logistical support or writing assistance)

  • Funding statement: any text in the paper that provides information about the funding of the study, including a report of ‘no funding’

  • Funding policy: a policy that requires, at a minimum, the authors to disclose any source funding of the study

Survey of HPSR papers

Eligibility criteria and search

We included both primary studies and systematic reviews and excluded policy briefs, overviews of systematic reviews, economic evaluation and costing studies, technical reports, conference reports, proceedings, abstracts, editorials and opinion pieces.

To identify systematic reviews, we searched the Health Systems Evidence database of McMaster Health Forum for documents published in 2015 [13, 14]. The Health Systems Evidence database is a regularly updated database of systematic reviews in health policy and systems that relies on different sources of systematic reviews, has its own eligibility criteria, and has an independent and duplicate selection process.

To identify primary studies, we considered 72 journals listed under the ‘Health Policy and Services’ (HPS) category in Web of Science as of June 2016 [15]. We searched for primary studies (e.g. randomised controlled studies, cohort studies, qualitative studies) published in those journals in English in 2016. We considered those studies as eligible if they met the criteria of health systems topics developed by the McMaster Health Forum [13, 14], including governance, financial and delivery arrangements, and implementation strategies.

Study selection and data extraction

We developed a data extraction form with detailed instructions using the Research Electronic Data Capture (REDCap) tool hosted at the American University of Beirut. Teams of two reviewers selected studies and abstracted data in duplicate and independently, and resolved disagreements through discussion or with the help of a third reviewer, if needed.

A funding statement could refer to more than one funding contribution. For each study, we collected data on the number of authors, affiliation(s) of the first authors (private or public academic institution, government, not-for-profit organisation, private for-profit organisation, intergovernmental organisation), country of affiliation of the first author and its classification according to the World Bank list of economies issued in July 2015, reporting of study funding (not reported, reported as funded, reported as not funded), reporting of the source of funding (internal fund, governmental, private for-profit and private not-for-profit), and reporting of the role of funders (not reported, reported as ‘no involvement’, reported as involvement in specific stage(s) of research).

Survey of journals publishing HPSR papers

Journal selection

We considered two groups of journals. The first group consisted of all journals listed under the category of HPS by Web of Science as of June 2016. We considered only journals that had an online submission system to review their policies on websites and during the submission process [15]. That resulted in the exclusion of one inactive journal and one active journal that published by invitation only and had no information on reporting of funding on their website. The second group consisted of journals that published the systematic reviews on HPSR included in the first survey.

Data extraction

We collected the needed information from the instructions and forms accessible on the journal or publisher’s websites. Teams of two reviewers abstracted data in duplicate and independently using the REDCap tool. They resolved disagreements through discussion or with the help of a third reviewer if needed.

For each journal, we extracted information about whether the journal is categorised by the Web of Science as HPSR, impact factor, International Committee of Medical Journals Editors membership, COPE (Committee on Publication Ethics) membership, affiliation with a professional organisation, requirement of disclosure of conflicts of interest, requirement of disclosure of source of funding, and requirement of disclosure of role of funders.

Statistical analysis

We conducted descriptive analyses for all variables collected for the included papers and journals. We provided frequencies and percentages for categorical variables, and median and interquartile range for continuous variables. We compared the general characteristics of primary studies with those of the systematic reviews. In addition, we compared journals in which HPSR papers were published with journals listed under the category HPS according to the Web of Science 2016. We used the Mann–Whitney test to compare non-parametric continuous data and χ2 test to compare categorical data. We conducted simple and multiple regression analyses to investigate the association between reporting of funding by papers and the journal’s characteristics (except for the variable ‘requirement of disclosure of conflicts of interest’ due its high correlation with the variable ‘requirement of disclosure of source of funding’).

Results

Survey of HPSR papers

Table 1 shows the general characteristics of the 400 included HPSR papers. The median number of authors per paper was 4 (interquartile range 3–6). Systematic reviews had a significantly higher median number of authors compared with primary studies (p = 0.048). The majority of papers’ first authors was affiliated with public academic institutions (n = 288, 72%), with that percentage being lower in primary studies compared with systematic reviews (68% vs. 77%). A minority of papers’ first authors was affiliated with private for-profit (n = 5, 1%) and intergovernmental organisations (n = 1, 0.3%). The majority of first authors were from high-income countries (n = 368, 92%).

Table 1 General characteristics of the 400 included studies

Table 2 presents the reporting of funding in the 400 included HPSR papers. A substantive percentage of papers (n = 126, 31%) did not report any detail on the study funding. As for papers that reported being funded (n = 240, 60%), their percentage was higher among primary studies compared to systematic reviews (65% vs. 54%). Among the 240 funded papers, the majority were funded by government (n = 186, 78%). Most of the studies did not report on the role of funders (n = 214, 89%) (Fig. 1).

Table 2 Reporting of funding in the 400 included studies
Fig. 1
figure 1

Reporting of funding source and role of funders in the 400 health policy and systems research (HPSR) papers

Table 3 presents the status of reporting of funder involvement in each of the 15 stages of the research in studies that reported on involvement (whether positive or negative) in at least 1 of the stages. There was no reporting of the status of involvement (positive or negative) for 6 of the 15 stages of research.

Table 3 Status of the reporting of funder involvement in each of the 15 stages of the research, among studies that reported on any such involvement (whether positive or negative) (n = 22)

Survey of journals publishing HPSR papers

Table 4 shows the characteristics of journals that published the 400 included HPSR studies (n = 198). It also includes results for the 2 overlapping subgroups of journals that published the 200 primary studies (n = 55) and the 200 systematic reviews (n = 153), respectively. Out of 198 journals publishing on HPSR, the majority required the reporting of source of funding (n = 176, 89%). Of those that did, 34% (56 out of 176) required the reporting of the role of funders (Fig. 2). That percentage was 19% (10 out of 53) for journals that published the primary studies and 37% (49 out of 133) for journals that published the systematic reviews.

Table 4 The characteristics of journals that published articles on health policy and systems research (HPSR) studies
Fig. 2
figure 2

Requirements for the reporting of funding source and role of funders by the 198 journals publishing on health policy and systems research (HPSR)

Table 5 compares the characteristics of HPS journals (according to the Web of Science) (n = 72) and non-HPS journals that published the HPSR studies included in our first survey (n = 142). Only 15% of the HPS journals required reporting of the role of funders, compared to 32% of the non-HPS journals that published the HPSR studies.

Table 5 Comparison between the characteristics of journals listed as Health Policy and Services (HPS) and non-HPS journals that published health policy and systems research (HPSR) studies

Regression analysis

Table 6 presents the simple and multiple logistic regression of ‘reporting of funding’ in included studies with the journals’ funding policies and other covariates. ‘Reporting of funding’ was significantly associated with publication in journals with higher impact in the simple logistic regression (odds ratio (OR) 1.15, 95% confidence interval (CI) 1.00–1.33) but not in the multiple logistic regression (OR 1.13, 95% CI 0.97–1.31).

Table 6 Simple and multiple logistic regression of ‘reporting of funding’ in included studies with journals’ funding policies and other covariates

Discussion

Summary of findings

Approximately one-third of HPSR papers did not report any detail on study funding. Of those that did, only 11% reported on the role of funders. Although the vast majority of journals publishing on HPSR required reporting of the source of funding, only approximately one-third of these journals required reporting of the role of funders. Journals classified under the HPS category were less likely than non-HPS journals that published the HPSR studies to require the reporting of the role of funders. We did not find any of the journal’s characteristics to be associated with the reporting of funding by papers.

Strengths and limitations

This is the first study to address the reporting of funding in HPSR papers. Additionally, it is the first to assess the policies on reporting of funding of journals publishing on HPSR. We used duplicate and independent selection and data abstraction processes to ensure the validity of findings. In addition, conducting the two surveys in parallel enabled us to assess the association between the reporting of funding and the funding policies of journals. One possible limitation of this study is that journals might inform the authors about certain requirements on reporting of funding at the time of submission or acceptance that might not be reflected in the policies published on their website.

Interpretation of findings

While a third of HPSR papers did not provide any information about funding, most journals in which they were published had adequate policies requiring the reporting of the funding source. This reflects both a suboptimal compliance by authors with the funding policies and a deficient implementation by the journals. On the other hand, the low reporting of the role of funders (11%) seems to be related to the inadequacy of the journals’ policies since approximately one-third of the journals required the reporting of the role of funders. Journals classified under the HPS category were less likely than the non-HPS journals that published the HPSR studies to require reporting of the role of funders. This highlights the need for health policy journals to strengthen their policies for the reporting of research funding.

Comparison to other studies

HPSR papers do not compare favourably with clinical papers in terms of the reporting of funding. Recently, we found that 89% of clinical trial reports published in 2015 included funding statements [16], a percentage substantively higher than the 69% found for HPSR papers herein. In addition, we found that 50% of the clinical trials described as funded reported on the role of funders, a much higher percentage than the 11% found for HPSR papers herein [16]. Another study found that 35% of funded surgical trials published in 10 surgery journals reported on the role of study sponsors [9].

Most HPS journals have policies that require the reporting of source of funding (90%). While we did not identify any similar publication for clinical journals, one study found that 90% of public health journals have policies for reporting study funding [17]. On other hand, the percentage of journals requiring the reporting of the role of funders was low for both HPS (15%) and public health (23%) [17].

Reporting of funding’ was significantly associated with publication in journals with higher impact factor in the simple but not in the multiple logistic regression. Similarly, we previously found that the core clinical journals had better reporting of funding associated with the impact factor of the journal (OR 1.41, 95% CI 1.09–1.9) [16]. One potential explanation of this finding is that journals with high impact factors are more likely to enforce the implementation of policies related to the declaration of funding.

Conclusions

Implication for practice

Our study identified some gaps in the funding policies of HPS journals that are associated with low reporting of funding information in HPSR papers. HPS journals need to implement more detailed policies requiring authors to report relevant details such as the role of funders, type of support (e.g. monetary, provision of supply) and amount of fund [16]. Moreover, journals need to better enforce their funding policies. We have previously proposed the type of funding information that should be reported, along with a fillable PDF document as a standardised instrument for the reporting of funding information. Additionally, reporting statements (such as PRISMA and CONSORT) could require more details, e.g. in terms of the specific roles that the funder plays.

Implications for future research

Future research could assess the reasons for low reporting of funding information in HPSR, as well as the accuracy and completeness of the reported funding information. Moreover, it would be interesting to assess the effectiveness of interventions to improve reporting of funding such as the use of standardised reporting instruments.

Abbreviations

CI:

confidence interval

HPS:

health policy and services

HPSR:

health policy and systems research

OR:

odds ratio

References

  1. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry Sponsorship and Research Outcome. Cochrane. 2017. https://www.cochrane.org/MR000033/METHOD_industry-sponsorship-and-research-outcome. Accessed 6 Aug 2018.

  2. Flacco ME, Manzoli L, Boccia S, Capasso L, Aleksovska K, Rosso A, Scaioli G, De Vito C, Siliquini R, Villari P. Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor. J Clin Epidemiol. 2015;68(7):811–20.

    Article  PubMed  Google Scholar 

  3. Bero LA. Why the Cochrane Risk of Bias Tool Should Include Funding Source as a Standard Item. Cochrane Library. 2013. http://www.cochranelibrary.com/editorial/10.1002/14651858.ED000075. ​Accessed 6 Aug 2018.

  4. Kesselheim AS, Robertson CT, Myers JA, Rose SL, Gillet V, Ross KM, Glynn RJ, Joffe S, Avorn J. A randomized study of how physicians interpret research funding disclosures. N Engl J Med. 2012;367(12):1119–27.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  5. Barnes DE, Bero LA. Why review articles on the health effects of passive smoking reach different conclusions. JAMA. 1998;279(19):1566–70.

    Article  PubMed  CAS  Google Scholar 

  6. International Committee of Medical Journals Editors. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. http://www.icmje.org/icmje-recommendations.pdf. Accessed 6 Aug 2018.

  7. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8(1):18.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Bridoux V, Moutel G, Schwarz L, Michot F, Herve C, Tuech J-J. Disclosure of funding sources and conflicts of interest in phase III surgical trials: survey of ten general surgery journals. World J Surg. 2014;38(10):2487–93.

    Article  PubMed  Google Scholar 

  10. Hodgson R, Allen R, Broderick E, Bland JM, Dumville JC, Ashby R, Bell-Syer S, Foxlee R, Hall J, Lamb K. Funding source and the quality of reports of chronic wounds trials: 2004 to 2011. Trials. 2014;15(1):19.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Mugambi MN, Musekiwa A, Lombard M, Young T, Blaauw R. Association between funding source, methodological quality and research outcomes in randomized controlled trials of synbiotics, probiotics and prebiotics added to infant formula: a systematic review. BMC Med Res Methodol. 2013;13(1):137.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Voineskos SH, Coroneos CJ, Ziolkowski NI, Kaur MN, Banfield L, Meade MO, Chung KC, Thoma A, Bhandari M. A systematic review of surgical randomized controlled trials: Part 2. Funding source, conflict of interest, and sample size in plastic surgery. Plastic Reconstruct Surg. 2016;137(2):453e–61e.

    Article  CAS  Google Scholar 

  13. About Health Systems Evidence. https://www.healthsystemsevidence.org/about. Accessed 6 Aug 2018.

  14. Lavis JN, Wilson MG, Moat KA, Hammill AC, Boyko JA, Grimshaw JM, Flottorp S. Developing and refining the methods for a ‘one-stop shop’ for research evidence about health systems. Health Res Policy Syst. 2015;13(1):10.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Khamis AM, Hakoum MB, Bou-Karroum L, Habib JR, Ali A, Guyatt G, El-Jardali F, Akl EA. Requirements of health policy and services journals for authors to disclose financial and non-financial conflicts of interest: a cross-sectional study. Health Res Policy Syst. 2017;15(1):80.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Hakoum MB, Jouni N, Abou-Jaoude EA, Hasbani DJ, Abou-Jaoude EA, Lopes LC, Khaldieh M, Hammoud MZ, Al-Gibbawi M, Anouti S. Characteristics of funding of clinical trials: cross-sectional survey and proposed guidance. BMJ Open. 2017;7(10):e015997.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Daou KN, Hakoum MB, Khamis AM, Bou-Karroum L, Ali A, Habib JR, Semaan AT, Guyatt G, Akl EA. Public health journals’ requirements for authors to disclose funding and conflicts of interest: a cross-sectional study. BMC Public Health. 2018;18(1):533.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This project was funded by the American University of Beirut Faculty of Medicine’s Medical Practice Plan funds. The funder had no role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; or decision to submit the manuscript for publication. The authors and their contributions to the manuscript are independent from the funder.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on request.

Author information

Authors and Affiliations

Authors

Contributions

AMK, LBK, MBH, FEJ and EAA conceived and designed the study. AMK, LBK and MBH respectively coordinated different parts of the study. EAA had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. AMK, LBK and MBH ran the search and study selection processes. AMK, LBK, MBH, MAG and JRH extracted the data. AMK and EAA analysed and interpreted the data. AMK wrote the first draft of the manuscript with EAA. All authors critically revised the manuscript and approved the final manuscript. The lead author EAA affirms that this manuscript is an honest, accurate and transparent account of the study being reported, that no important aspects of the study have been omitted, and that any discrepancies from the study as planned have been explained.

Corresponding author

Correspondence to Elie A. Akl.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

EAA and MBH are authors of a number of published papers on funding. All other authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khamis, A.M., Bou-Karroum, L., Hakoum, M.B. et al. The reporting of funding in health policy and systems research: a cross-sectional study. Health Res Policy Sys 16, 83 (2018). https://doi.org/10.1186/s12961-018-0356-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12961-018-0356-3

Keywords