- Study Protocol
- Open Access
COVID-19: protocol for observational studies utilizing near real-time electronic Australian general practice data to promote effective care and best-practice policy—a design thinking approach
Health Research Policy and Systems volume 19, Article number: 122 (2021)
Health systems around the world have been forced to make choices about how to prioritize care, manage infection control and maintain reserve capacity for future disease outbreaks. Primary healthcare has moved into the front line as COVID-19 testing transitions from hospitals to multiple providers, where tracking testing behaviours can be fragmented and delayed. Pooled general practice data are a valuable resource which can be used to inform population and individual care decision-making. This project aims to examine the feasibility of using near real-time electronic general practice data to promote effective care and best-practice policy.
The project will utilize a design thinking approach involving all collaborators (primary health networks [PHNs], general practices, consumer groups, researchers, and digital health developers, pathology professionals) to enhance the development of meaningful and translational project outcomes. The project will be based on a series of observational studies utilizing near real-time electronic general practice data from a secure and comprehensive digital health platform [POpulation Level Analysis and Reporting (POLAR) general practice data warehouse]. The study will be carried out over 1.5 years (July 2020–December 2021) using data from over 450 general practices within three Victorian PHNs and Gippsland PHN, Eastern Melbourne PHN and South Eastern Melbourne PHN, supplemented by data from consenting general practices from two PHNs in New South Wales, Central and Eastern Sydney PHN and South Western Sydney PHN.
The project will be developed using a design thinking approach, leading to the building of a meaningful near real-time COVID-19 geospatial reporting framework and dashboard for decision-makers at community, state and nationwide levels, to identify and monitor emerging trends and the impact of interventions/policy decisions. This will integrate timely evidence about the impact of the COVID-19 pandemic related to its diagnosis and treatment, and its impact across clinical, population and general practice levels.
Since its identification in December 2019, SARS-CoV-2 and its associated coronavirus disease (COVID-19) has had a devastating effect on communities around the world, leading to a rapidly escalating mortality toll of over a million people . Health systems around the world have been forced to make rapid choices about how to prioritize care, manage infection control and maintain reserve capacity for future disease outbreaks . The interruption of normal patterns of healthcare and the suspension of services has meant that the pandemic has also had a major impact on the detection and treatment of many non-COVID-19 conditions .
The healthcare consequences associated with the COVID-19 pandemic range from people’s avoidance of contact with healthcare settings, either for fear of contracting COVID-19 or as a means of reducing pressure on the health system , or because of the increased financial stress caused by the pandemic . Early evidence from an Australian Bureau of Statistics Household Impacts of COVID-19 Survey showed that one in 14 Australians (1.4 million adults) were unable to see their general practitioner (GP) or other health professional in person during the early April 2020 to early May 2020 period . Australian Medicare figures revealed that pathology testing fell by almost 30%; consultations with specialists also fell by 8%, and medical operations by 27% . The pandemic has had a disproportionate impact on disadvantaged communities, helping to entrench inequalities, as a consequence of the economic measures used to contain the virus . The slump in general practice visits may have detrimental and long-term effects on patient care and outcomes, especially if it impacts on the diagnosis of new conditions, recommended disease and cancer screening programmes, or ongoing monitoring of patients with chronic disease [7, 8].
Whilst the initial focus has been on COVID-19 and its mortality, the effects of SARS-CoV-2 are evolving into (at this stage) a three-pronged disease:
Acute COVID-19, the disease responsible for most of the initial morbidity and in particular mortality 
Paediatric disease, variously called paediatric inflammatory multisystem syndrome temporally associated with SARS-CoV-2 (PIMS-TS) in Europe and also known as multisystem inflammatory syndrome in children (MIS-C) in the United States) .
Whilst general practices were initially severely affected by the drop in visits in the immediate aftermath of the COVID-19 outbreak, the utilization of Medicare funding for telehealth helped to reinforce community access to general practices and non-GP specialists. This change represented a potentially significant shift in how patient care can be delivered in the future. Australian Bureau of Statistics figures over a 4-week period across early April to early May 2020 showed that one in six Australians (17%) used telehealth services (mostly in the form of telephone calls) . Despite the importance of telehealth initiatives, many patients still require a face-to-face option . In recognition of this, on 30 March 2020 the Commonwealth Government announced temporary increases to incentive payments (Practice Incentives Program Quality Improvement) for general practice to increase support for bulk-billed services with the aim of supporting essential face-to-face care access for patients .
Identifying the impact of COVID-19 on the healthcare system in near real time is an urgent priority for critical decision-making about resource allocation, control measures, decision-making, policy development and community communication. Primary healthcare has moved into the front line as COVID-19 testing transitions from hospitals to multiple general practice providers, where tracking testing behaviours can be fragmented and delayed.
This paper outlines a project funded by the Digital Health Cooperative Research Centre (DHCRC) which will examine the feasibility of using near real-time electronic general practice data to promote effective care and best-practice policy through the following mechanisms: (1) the building of a near real-time COVID-19 geospatial reporting framework; (2) the generation of timely and critical evidence about the impact of COVID-19; (3) the building of a predictive geospatial analytics dashboard for timely, evidence-based decision-making; and (4) the establishment of an evidence-based suite of general practice outcome measures required to monitor the quality and effectiveness of care related to incidence and prevalence, recovery and mortality.
The project was formally identified initially as a project concept proposal to the DHCRC involving a collaboration between Macquarie University, Outcome Health, Gippsland, Eastern Melbourne, South Eastern Melbourne Primary Health Networks (PHNs) and the Royal College of Pathologists of Australasia Quality Assurance Programs. The proposal was assessed by the DHCRC using its eligibility threshold criteria to meet its research and investment framework. Following this, we developed a project plan which was reviewed by the DHCRC according to its merit and value criteria, which led to the establishment of a project agreement (17 June 2020).
The codesign aspects of the project will involve all the project collaborators (PHNs, general practices, consumer groups, researchers, and digital health developers, pathology professionals) as part of an integrated collaboration to enhance the development of meaningful and translational project outcomes. The project will utilize a design thinking approach to incorporate user needs and feedback throughout the development of the research process. This approach involves rounds of ideation, prototyping and testing  to enhance understanding of underlying problems or unpredictable situations .
The project builds upon an established collaboration between Macquarie University, the Royal College of Pathologists Quality Assurance Programs, Outcome Health and Gippsland, Eastern and South Eastern PHNs, which evaluated the appropriateness and quality use of pathology in general practice  using pooled electronic general practice data . This project provided an overview of pathology testing within Australian general practice, along with reporting on variation across key indicators including vitamin D testing, glycated haemoglobin (HbA1c) and kidney function tests for monitoring type 2 diabetes, ferritin testing for iron deficiency and prostate-specific antigen (PSA) testing .
A series of population studies will be carried out over the period July 2020–December 2021 utilizing near real-time electronic general practice data to promote effective care and best-practice policy. The type of studies examining the impact of the COVID-19 pandemic will include (1) the uptake of general practice telehealth services; (2) medication prescribing; (3) the impact on pathology testing; and (4) general practice consultations to long-term aged care facilities.
Data examination and analysis will be performed using Stata/MP 16 (StataCorp) , R v4.0.2 (R Core Team)  and SAS 9.4 (SAS Institute) statistical software . Statistical methods will include descriptive and inferential statistics, depending on the components of the project. Where spatial or temporal evaluations are required, stochastic (e.g., mixed models) and deterministic models (e.g., Bayesian structural time series) will be developed as per the study aims, and if applicable, incorporated into machine learning for building prediction models. The methods will be structured according to the reporting of studies conducted using observational routinely collected health data or pharmacoepidemiology data (RECORD-PE) . The RECORD-PE checklist deals specifically with real-world research and evaluation using data from electronic health records such as that provided by electronic general practice sources .
The studies will be centred on 450 general practices within three Victorian PHNs, Gippsland PHN, Eastern Melbourne PHN and South Eastern Melbourne PHN. These PHNs cover metropolitan and rural regions across a combined area totalling 48,903 km2, delivering healthcare to 3,132,382 Australians . Data will be supplemented by participation of two PHNs from New South Wales, Central and Eastern Sydney PHN and South Western Sydney PHN, with Central and Eastern Sydney PHN incorporating a further 350 general practices.
Outcome Health, as a data custodian, uses its POpulation Level Analysis and Reporting (POLAR) Data Space to provide a secure and comprehensive digital health platform which collects data from consenting general practices across participating PHNs [17, 18, 24,25,26]. Data variables include de-identified demographic information about patients and general practices, as well as visit records (diagnosis, past history, medications, immunization, radiology) and pathology test records (test name and result).
Sample size considerations
The research partners in this project have been amongst the first in Australia to successfully use and analyse electronic patient data from general practice and evaluate patient outcomes in both cross-sectional and longitudinal studies [18, 19]. This body of evidence (initially based on > 350 practices) includes a comprehensive assessment of outcomes for patients with type 2 diabetes , and an investigation of iron deficiency diagnosis in general practice . The research outlined in this paper is centred on data from > 800 general practices, representing a total of > 4 million patients (~ 56% female). This sample will provide sufficient scope to detect significant variation in practices across patient and general practice demographic domains.
The study will involve the building of a meaningful near real-time COVID-19 geospatial reporting framework and dashboard for decision-makers at community, state and nationwide levels, to identify and monitor emerging trends and the impact of interventions/policy decisions. This will include:
The generation of timely and critical evidence (e.g., publicly available reports, peer-reviewed publications and media presentations) about the impact of COVID-19 across the following care level dimensions:
Clinical: Patient-level impacts based on:
Diagnostics (most common COVID-19 symptoms; alarm flags for COVID-19, e.g. white blood cells, lymphocytes, platelet counts; risk factors, e.g. smoking, respiratory failure, etc.);
Medications (the dangers of prescribing antiviral drugs, e.g. drug–drug, drug–disease interactions; risks associated with chloroquine/hydroxychloroquine, angiotensin-converting enzyme [ACE] inhibitors, aggravation of symptoms, etc.); and
Patient care (Considerations during pregnancy, care of infants, use of nebulisers, recommendations for paediatric patients etc.).
Population: What populations are being impacted, not only from a direct COVID-19 perspective, but from a regular care perspective (e.g., impact on chronic disease, preventative care and mental health).
Business: What impact has COVID-19 had on general practice (e.g., number and types of services, tests, medications, etc.) from a business and financial perspective? Types of interaction with patients, telehealth vs face-to-face, how has this worked, what are the immediate and future implications? How has this impacted patient care in relation to population and clinical care?
Development of a predictive geospatial analytics dashboard for timely, evidence-based decision-making at community, state and nationwide levels to include:
Visual representations of patient status and clinical environments that change dynamically in near real-time.
Risk prediction tools based on statistical modelling incorporating demographics, comorbidities, patient symptoms and risk factors.
Incorporation of displays and decision support features tailored to the needs and preferences of key recipients including GPs, PHNs and government health agencies.
Patient and public involvement in research
Each of the project partners have strong records of collaborative research which includes engagement with patients and the public. The Macquarie University research team has an established Consumer Reference Group made up of consumer representatives who have contributed to the design, development and promulgation of research aims [28, 29]. The Consumer Reference Group was initiated following a national stakeholder forum (which brought together representatives from 14 stakeholder groups including patient organizations, clinicians and healthcare professionals) that outlined key patient safety challenges related to the utilization of digital health and the diagnostic process . The project will draw on established PHN stakeholder (patient and public) involvement using design thinking approaches to inform research questions, the design and conduct of studies and choice of outcome measures.
The electronic general practice data available for this project represent a rich source of information about general practice activity and patient care processes. However, there are limitations in the data, including variables that may not be recorded in a standardized way, or inconsistencies in how information is documented and identified. In some cases this may be due to the lack of comparability between software packages used by general practices, specialists, pathology and medical imaging providers. Furthermore, free-text data of comorbidities and diagnoses increase the possibility of underreporting. Nevertheless, the collection of general practice data directly from general practices at the time of patient care provides a valuable source of pooled data, which can help to power major research investigations.
The project will establish an operational and governance structure consisting of the following:
Project management team (PMT) which will consist of representatives of the collaborating partners (Macquarie University, Outcome Health, Gippsland, Eastern Melbourne, and South Eastern Melbourne PHNs). The PMT will be responsible for the day-to-day functioning of the Project. It will draw on a team of clinical, technical and digital health analytics experts drawn from each of the collaborating partners, who will provide regular timely advice and guidance to generate meaningful robust, validated outcomes. This will include liaison with all stakeholder groups (general practice, nurses, managers and consumers).
Project Control Group that will consist of key representatives from the collaborating partners and the DHCRC with responsibility to oversee the Project’s progress, priorities and direction. The Project Control Group will meet quarterly and be presented with interim reports for consideration during the course of the Project.
Ethics and dissemination
Outcome Health has received ethical approval from the Royal Australian College of General Practitioners National Research and Evaluation Ethics Committee (NREEC) 17-008 POLAR GP data warehouse. The Macquarie University Human Research Ethics Committee (HREC) Medical Sciences Committee has granted approval to the project (Reference no. 5202067517176).
The operational objectives of the project have been designed to contribute significantly to the enhancement of quality general practice care in response to COVID-19. Findings will be regularly communicated to GPs, PHNs, the community and nationally through near real-time snapshot reports. Findings will be disseminated in peer-reviewed academic journals and presentations (national and international conferences and industry forums).
General practice services have a critical role to play in Australia’s response to COVID-19, not least because they are often the place of initial health system contact for most Australians . They are essential as suppliers of information to the community in terms of healthcare advice to mitigate the spread of disease and protect the health of the community .
General practice activity is also key to identifying and monitoring the health of the community, providing an early warning system of the spread of the pandemic and targeting areas where healthcare may be avoided or delayed, leading to the possibility of missed diagnoses, medications and treatment, which may have serious future consequences for patients and the healthcare system .
Until recently, research into Australian general practice has been hampered by the lack of reliable data and expertise in data management, information systems and quality improvement infrastructures . Pooled general practice data are a valuable resource, which can be used to inform decision-making including individual care, population health and business and government strategy. The research partners in this project have been amongst the first in Australia to successfully use and analyse electronic patient data from general practice and evaluate patient outcomes in both cross-sectional and longitudinal studies [17, 19].
The operational objectives of this project have been designed to contribute to the enhancement of quality general practice in response to the COVID-19 pandemic, providing a foundation for improved care monitoring and planning and more timely decision-making and evidence-based policy development.
Availability of data and materials
Primary health networks
Paediatric inflammatory multisystem syndrome
Multisystem inflammatory syndrome in children
Digital Health Cooperative Research Centre
POpulation Level Analysis and Reporting
Reporting of studies conducted using observational routinely collected data
Project management team
National Research and Evaluation Ethics Committee
Human Research Ethics Committee
World Health Organization. Coronavirus disease (COVID-19) Situation Report. https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200811-covid-19-sitrep-204.pdf?sfvrsn=1f4383dd_2 Accessed 17 Aug 2020.
Reed S. Nuffied Trust Briefing July 2020 - Resuming health services during the Covid-19 pandemic. What can the NHS learn from other countries? https://www.nuffieldtrust.org.uk/research/resuming-health-services-during-the-covid-19-pandemic-what-can-the-nhs-learn-from-other-countries 2020; Accessed 3 Aug 2020
Carr S. Missed and delayed diagnoses of non-COVID conditions - collateral harm from a pandemic. ImproveDx Newslett. 2020;7(4):1–7.
Mareiniss DP. The impending storm: COVID-19, pandemics and our overwhelmed emergency departments. Am J Emerg Med. 2020;38:1293.
Zhang Y, Liu J, Scott A. Who is avoiding necessary healthcare during the COVID-19 pandemic? Research Insights. https://melbourneinstitute.unimelb.edu.au/__data/assets/pdf_file/0019/3401821/ri2020n13.pdf Accessed: 23 July 2020: Melbourne Institute: Applied Economic & Social Research, The University of Melbourne; 2020.
Australian Bureau of Statistics. Household Impacts of COVID-19 Survey, 29 Apr - 4 May 2020. https://www.abs.gov.au/AUSSTATS/abs@.nsf/allprimarymainfeatures/50884FE62C2C85C0CA258576008061F0?opendocument Accessed: 5 August 2020. 2020.
Know Pathology. Why Australians are being urged not to delay pathology tests during COVID-19 restrictions 2020 https://knowpathology.com.au/2020/04/23/do-not-delay-pathology-tests-during-covid-19-restrictions/?utm_source=ActiveCampaign&utm_medium=email&utm_content=Spread+the+word+-+Don+t+Skip+Tests&utm_campaign=KPKH+April+2020+newsletter.
Victoria State Government Health and Human Services. Coronavirus COVID-19 daily update 2020 https://www.dhhs.vic.gov.au/coronavirus-covid-19-daily-update.
Lam RPK, Hung KKC, Lau EHY, et al. Clinical, laboratory, and radiological features indicative of novel coronavirus disease (COVID-19) in Emergency Departments–a Multicentre Case-Control Study in Hong Kong. J Am Coll Emerg Physicians Open. 2020;1:597–608.
Mahase E. Covid-19: What do we know about “long covid”? Br Med J. 2020;370:1–8.
Greenhalgh T, Knight M, Buxton M, et al. Management of post-acute covid-19 in primary care. Br Med J. 2020;370:m3026.
Feldstein LR, Rose EB, Horwitz SM, et al. Multisystem inflammatory syndrome in US children and adolescents. N Engl J Med. 2020;383(4):334–46.
Knapp J. The importance of face-to-face consultations during the coronavirus pandemic: Why and how for general practice Australia. 2020 https://nwmphn.org.au/news/importance-face-face-consultations-coronavirus-pandemic-general-practice/.
Australian Government Department of Health. Coronavirus (COVID-19) Primary Care Package - Practice Incentive Payments. https://www.health.gov.au/sites/default/files/documents/2020/03/coronavirus-covid-19-primary-care-package-practice-incentive-payments.pdf Accessed 17 August 2020.
Altman M, Huang TT, Breland JY. Peer reviewed: Design thinking in health care. Prev Chronic Dis. 2018;15:117.
Roberts JP, Fisher TR, Trowbridge MJ, et al. A design thinking framework for healthcare management and innovation. Healthcare. 2016;4(1):11–4.
Sezgin G, Georgiou A, Hardie RA, et al. Compliance with pathology testing guidelines in Australian general practice: protocol for a secondary analysis of electronic health record data. BMJ Open. 2018;8(11):e024223.
Imai C, Hardie R-A, Franco GS, et al. Harnessing the potential of electronic general practice pathology data in Australia: an examination of the quality use of pathology for type 2 diabetes patients. Int J Med Inform. 2020;141:104189.
Hardie R-A, Sezgin G, Imai C, et al. Enhancing patient outcomes through evaluation of the appropriateness and quality use of pathology in general practice: a report to the Department of Health Quality Use of Pathology Program Sydney: Australian Institute of Health Innovation; 2020.
Stata/MP. https://www.stata.com/statamp/ Accessed 11 Sept 2020.
The R Project for Statistical Computing. www.r-project.org Accessed 11 Sept 2020.
SAS Institute. https://www.sas.com/en_au/home.html Accessed 11 Sept 2020.
Langan SM, Schmidt SA, Wing K, et al. The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE). Br Med J. 2018;363:3532.
Pearce C, Shearer M, Gardner K, et al. A division’s worth of data. Aust Fam Phys. 2011;40(3):167.
Pearce C, McLeod A, Patrick J, et al. Coding and classifying GP data: the POLAR project. BMJ Health Care Inform. 2019;26:e100009.
Pearce C, McLeod A, Rinehart N, et al. What a Comprehensive, Integrated Data Strategy Looks Like: The Population Level Analysis and Reporting (POLAR) Program. In: Ohno-Machado L, Seroussi B, editors., et al., MedInfo 2019: Health and Wellbeing e-Networks for All. Amsterdam: IOS Press; 2019. p. 303–7.
Sezgin G, Monagle P, Loh TP, et al. Clinical thresholds for diagnosing iron deficiency: comparison of functional assessment of serum ferritin to population based centiles. Sci Rep. 2020;10(1):1–12.
Dahm M, Brown A, Martin D, et al. Interaction and innovation – practical strategies for inclusive consumer-driven research in health services. BMJ Open. 2019;2019:e031555.
Dahm M, Georgiou A, Herkes R, et al. Patient groups, clinicians and healthcare professionals agree - all test results need to be seen, understood and followed up. Diagnosis. 2018;5:215–22.
Kidd M. Australia’s primary care COVID-19 response. Aust J Gen Pract. 2020;49(Suppl2):1.
Desborough J, Hall SL, de Toca L, et al. Australia’s national COVID-19 primary care response. Med J Aust. 2020;213(3):104–6.
Bhatia RS, Levinson W, Shortt S, et al. Measuring the effect of Choosing Wisely: an integrated framework to assess campaign impact on low-value care. BMJ Qual Saf. 2015;24(8):523–31.
The project is funded by a Digital Health Cooperative Research Centre Partnership Project Grant (DHCRC-0118). The funder had no role in the design of the protocol and the writing of the manuscript.
Ethics approval and consent to participate
Outcome Health has received ethical approval from the Royal Australian College of General Practitioners NREEC 17-008 POLAR GP data warehouse. The Macquarie University Human Research Ethics Committee (HREC) Medical Sciences Committee has granted approval to the project (Reference no. 5202067517176).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Georgiou, A., Li, J., Pearce, C. et al. COVID-19: protocol for observational studies utilizing near real-time electronic Australian general practice data to promote effective care and best-practice policy—a design thinking approach. Health Res Policy Sys 19, 122 (2021). https://doi.org/10.1186/s12961-021-00772-4
- General practice
- Design thinking
- Quality of care
- Observational studies