- Research
- Open access
- Published:
Integrating qualitative research methods into care improvement efforts within a learning health system: addressing antibiotic overuse
Health Research Policy and Systems volume 14, Article number: 63 (2016)
Abstract
Background
Despite reports advocating for integration of research into healthcare delivery, scant literature exists describing how this can be accomplished. Examples highlighting application of qualitative research methods embedded into a healthcare system are particularly needed. This article describes the process and value of embedding qualitative research as the second phase of an explanatory, sequential, mixed methods study to improve antibiotic stewardship for acute sinusitis.
Methods
Purposive sampling of providers for in-depth interviews improved understanding of unwarranted antibiotic prescribing and elicited stakeholder recommendations for improvement. Qualitative data collection, transcription and constant comparative analyses occurred iteratively.
Results
Emerging themes and sub-themes identified primary drivers of unwarranted antibiotic prescribing patterns and recommendations for improving practice. These findings informed the design of a health system intervention to improve antibiotic stewardship for acute sinusitis. Core components of the intervention are also described.
Conclusion
Qualitative research can be effectively applied in learning healthcare systems to elucidate quantitative results and inform improvement efforts.
Background
In two recent reports, the National Academy of Medicine defined the Learning Healthcare System as an iterative, innovative process to improve healthcare delivery and outcomes [1, 2]. Ideally, learning healthcare systems leverage advances in information technology to identify variation in services and patient outcomes, then establish a feedback loop between researchers, clinicians and leadership to rapidly evaluate and improve quality and efficiency [3–5]. Despite evidence that interventions adapted to local contexts and cultures can achieve success, there is limited practical literature describing this process [6, 7].
We report our experience within Kaiser Permanente Southern California, an integrated healthcare delivery system with 14 medical centres and over 4 million members, as an example of how qualitative and quantitative methods may be combined to support the rapid cycle improvement process [8]. This study specifically sought to understand processes and drivers associated with overuse of antibiotics for treatment of acute sinusitis, as well as to garner physician stakeholders’ recommended strategies for improving prescribing behaviours at the point of care.
Overuse of non-recommended antibiotics to treat acute sinusitis is a global healthcare problem associated with unwarranted costs and burden to health systems worldwide. For example, in the United Kingdom, more than 90% of patients presenting with acute sinusitis are prescribed antibiotics [9, 10]. Another European study of six different countries found that between 56% and 87% of acute sinusitis encounters resulted in antibiotics contrary to guidelines, thereby exposing patients to unwarranted harm and costs [11]. Reports in the United States are similar, with approximately 30 million individuals affected by acute sinusitis every year [12–14] and 85–98% receiving a prescription for antibiotics [15]. Noting limited research in the United States on the rates of inappropriate antibiotic prescribing in the outpatient setting, Fleming-Dutra et al. [16] found an estimated annual antibiotic prescription rate of 506 per 1000 population from 2010 to 2011. The authors determined that, of these, an estimated 353 prescriptions were likely appropriate and called for improvements in antibiotic stewardship. Targeting these overprescribing patterns for improvement, the American Academy of Family Practice, the American Academy of Asthma, Allergy and Immunology, the American Academy of Otolaryngology, and the American College of Emergency Physicians listed overuse of antibiotics and imaging for treatment of acute sinusitis as part of the Choosing Wisely© campaign of the American Board of Internal Medicine, which is dedicated to reducing use of low-value, unnecessary healthcare practices [17].
Patient history and physical examination is key when making a diagnosis of acute sinusitis, requiring physicians to rely on their experience and judgment of presenting symptoms, rather than radiological tests, which are generally not recommended. This can result in diagnostic uncertainty and lead to potentially harmful care [11, 18]. Criteria for the diagnosis of acute sinusitis suggest that the determination should be made based on the primary symptoms of (1) purulent rhinitis and (2) facial pain [12, 19]. Physical history obtained during the patient exam should also rely on the length of symptoms, and current guidelines for the management of acute sinusitis emphasize that uncomplicated acute sinusitis lasting less than 10 days (in the absence of severe symptoms, such as facial swelling, severe fever/worsening fever) should not result in computerized tomography imaging nor an antibiotic prescription [18, 20–22]. Despite these recommendations over the last decade, there has not been much change in providers’ practice patterns and antibiotic prescribing rates [11, 14, 15].
In this manuscript, we present the qualitative results of a mixed methods study examining patterns and reasons for use of non-recommended antibiotics for treatment of acute sinusitis. Furthermore, we highlight how we used these qualitative methods to elucidate the results of a quantitative analysis using electronic medical record (EMR) data indicating widespread overuse of antibiotics for treatment. We discuss how quantitative and qualitative findings were used sequentially, and then together, to inform the design of a care improvement intervention within the context of a large healthcare system. Finally, the core components of the resulting intervention are presented. This process (using sequential mixed methodologies to inform intervention development) is an exemplar, offering a pragmatic approach for other organisations to use in understanding current best practices and potential barriers to improving healthcare within their own specific settings.
Methods
We used an explanatory, sequential, mixed methods design in this inquiry [23]. Quantitative data were collected and analysed in phase 1, followed by qualitative data collection and analysis in phase 2. Ultimately, this rich mix of quantitative and qualitative findings from our two-phase approach informed the development of a staged, multi-faceted healthcare intervention that improved care for patients with acute sinusitis.
In the first phase of research, we conducted a retrospective, observational study of all acute sinusitis encounters (ICD-9 code 461.x) for adult health plan members [24]. Findings indicated that (1) inappropriate antibiotic prescribing was common; (2) computed tomography imaging was infrequent; and (3) emergency department encounters were less likely to result in antibiotic prescriptions compared to primary care and urgent care patient visits. Phase 1 data spurred interest in understanding the drivers of unwarranted antibiotic prescribing and processes, as well as points for intervention within the context of our large health system. The phase 1 quantitative results shaped the planned qualitative activities in phase 2 of the study in two important ways. First, because inappropriate prescribing of antibiotics was common, but computed tomography imaging use was infrequent, the focus of phase 2 shifted toward the identification of specific drivers of antibiotic prescribing patterns, as opposed to use of imaging. Second, because emergency encounters were a small proportion of encounters and less likely to result in antibiotic prescriptions, primary care and urgent care physicians were targeted for qualitative interviews.
In phase 2, the primary objective of the qualitative research was to improve our understanding of physicians’ beliefs and practices related to overuse of antibiotics for treatment of acute sinusitis, and to elicit recommendations for encouraging guideline-consistent care within the context of our large, integrated healthcare delivery system. The semi-structured interview guide (Additional file 1) examined physician stakeholders’ perspectives on the barriers and facilitators to treating patients with acute sinusitis and elicited their suggestions about how to help clinicians avoid unwarranted antibiotic prescribing and increase provider recommendation of over-the-counter symptom relief in lieu of antibiotics (as consistent with medical evidence). The qualitative research utilized a purposive sampling approach to target and recruit primary care and urgent care physicians (n = 9) for individual, face-to-face interviews [25, 26]. The Principal Investigator met with the Chiefs of Service for primary and urgent care within the health system to introduce the research and then recruited eligible providers through email. Volunteer participants met with a member of the study team trained in qualitative interviewing procedures. Each interview lasted approximately 30–45 minutes, and was audio-recorded and transcribed verbatim.
Analytical plan for qualitative interview data
Qualitative data collection, transcription and analyses occurred iteratively. Transcription and review of early transcripts by multiple team members provided an opportunity for quality control, leading to adjustments in the interview guide questions and transcription approach. Initial coding categories were generated from the preliminary research questions and key domains of inquiry to create a ‘start list’ of structural codes [27]. The primary coder reviewed transcripts using a constant comparative method of analysis [28, 29]. During the first coding cycle, start list codes were applied to source text and then, during secondary and tertiary coding cycles of the transcripts, ‘open’ codes were applied, whereby new emergent coding categories considered salient to the goals of the research were captured. These newly emergent codes were unexpected, but grounded in the source text and, therefore, viewed as representative of important patterns and themes. Through the merging of inductive and deductive approaches, additional coding categories were generated throughout the analysis. To ensure clarity in code definitions and to improve reliability in the coding process, all codes were reviewed by study team members, revised through team consensus, and ultimately documented in a final codebook [27]. All data were managed using ATLAS.ti [30].
Characteristics of interview participants
The characteristics of the participating physicians are summarized in Table 1. Mean age of the physicians was 45 years (SD, 8.9), and just over half were female (56%) and White (56%). Approximately three-quarters (78%) received their residency training in family medicine, with the remaining (22%) trained in internal medicine; mean years of practice experience was 15 (SD, 7.6).
Results
Stakeholder interviews
Interviews were conducted with nine clinical stakeholders (six primary care physicians and three urgent care physicians). Data from the qualitative interviews were organised into domains, themes and sub-themes, presented in Table 2. The two domains included primary drivers of unwarranted prescribing patterns and physicians’ recommendations to improve practice patterns. The themes encompassed clinical and non-clinical factors influencing treatment decisions and patterns of care, perceptions of current guidelines, as well as multi-level (e.g. patient-, provider- and system-level) suggestions for improvement. We begin the presentation of results with illustrations of the first domain: primary drivers of unwarranted antibiotic prescribing patterns.
Primary drivers of unwarranted antibiotic prescribing patterns
Physicians underscored their desire to help patients better understand their symptoms and reasons why alternative treatments are often recommended in lieu of antibiotics for uncomplicated sinusitis. Despite this intention, physicians noted multiple barriers to following best practices within the clinical setting. These included patient expectations and perception of the relationship between prescribing patterns and patient satisfaction ratings.
Patients’ expectations and satisfaction
Physicians reported that patients’ expectations, and the internal and external pressures to meet those expectations, are the most important non-clinical factors driving antibiotic prescribing patterns.
Patients are described as wanting tangible action (often in their minds a ‘prescription’) when they come in for an appointment and providers report struggling with the desire to meet these expectations. The following physician shared her perspective, stating:
“…a lot of people here are working class people, they can’t be taking off [work] a lot. They pay a lot of money to come in here and they feel very pissed off when they come and spend sometimes even $50 …And then they are kind of coming out like, you know, do sinus washes and stuff…So it is a real conundrum and I don’t know what to do about it.”
Physicians pointed out that routine evaluation of their services through the Member Appraisal of Physician/Provider Services surveys distributed to patients after medical encounters are, in part, responsible for driving unwarranted antibiotic prescribing patterns as well. Speaking on this issue, one provider commented:
“…there is one person here who is…a very big over prescriber of antibiotics, in addition to other things. Like over ordering of a lot of things. [That physician] probably has the best [Member Appraisal of Physician/Provider Services] scores in the region… Patients love [Dr. X]!”
Another physician echoed the sentiment of her peers, who admitted they sometimes provide antibiotics against current guidelines because they suspect patients will seek antibiotics from another provider (and often threaten to do so):
“A lot of times I give antibiotics against my…better judgment…either the patient has an expectation and they are very angry and I know they are just going to…see somebody else until somebody gives it to them, which is really… common.”
These reports reveal that pressure to meet prevailing social norms and patients’ expectations (both in the office encounter and as reflected in patient satisfaction assessments) may supersede the desire to provide guideline-concordant care. This concern may be further influenced by the financial incentive providers are given to receive favourable satisfaction scores in our system.
Patient and provider communication
Providers spoke at length about the challenges they face communicating with patients who expect antibiotics, but do not clinically need them. This perception exists despite specific courses offered to providers designed to educate and train in effective communication skills. Limited time during patient visits to effectively convey the current recommendations is a reported challenge, complicated by the degree to which there is a provider and patient connection – physicians find it difficult to influence newer patients toward alternative treatments when they do not already have a strong, established relationship with the patient:
“Yeah, [patients coming in wanting antibiotics]…that is common. I think I would say it is much, much easier to [convince]…my own patients, because I know them and they trust me and [they] know I am not withholding anything from them, you know, there is a just a better level of trust there.
Access to guidelines
While physicians reported a basic understanding of the current acute sinusitis guidelines and sources available, staying abreast of the most current recommendations is perceived as challenging. In particular, physicians highlighted that integration of guidelines into practice can be slow and it can be difficult to keep up with the rapid pace at which new guidelines are published.
Physician recommendations to improve antibiotic prescribing patterns
The second domain in this study included physicians’ recommendations to improve antibiotic prescribing patterns, which included patient-, provider- and system-level strategies.
Patient education
There was consensus among providers that patients need additional education on the natural course of acute sinusitis and recommended treatment options. Providers stressed the importance of being able to effectively communicate that the length of symptoms is critical and that decongestants, antihistamines and nasal washes are typically the first line of defence. In particular, providers value patient education and resource materials from trusted sources external to our health system, because they can provide ‘back up’ during discussions with antibiotic-seeking patients when watchful waiting and/or alternative treatments are the preferred course. Several providers mentioned a previous campaign in their acute care settings that placed large posters in exam rooms to educate patients about the symptomology and recommended treatment for upper respiratory infections. The following provider emphasized that similar types of educational resources could also be helpful for educating patients during acute sinusitis encounters:
“…[but] my chart helps…you should make that in every office, to have a huge chart [on the wall] like that even bigger than the ones they make [for upper respiratory infections], because that has a lot of impact on patients, and maybe some other large print things about maybe the dangers of antibiotics…It is really, it has helped me a lot in arguments with patients.”
Provider education
In addition to patient education, physicians would like to see greater emphasis placed on provider education, including in-service education opportunities and improved access to guidelines. While physicians do have protected independent education time (one half day per week), several providers expressed disappointment at the loss of protected group education time within the organisation because it offered opportunities for delivering more formal curricula:
“But there used to always be a Tuesday afternoon [group] education [session]. And I found those really valuable. You know, you get your [Continuing Medical Education] credit and to hear things that are updated…so we don’t have that anymore, but I found that to be the best…we get a lot of things emailed, and… we have this primary care website that I am sure has all of this stuff on there… I can’t say it is really useful.”
Clinical decision aids within the EMR system
Providers generally support the idea that integrating clinical decision aids into the EMR system can be an effective way to affect antibiotic prescribing, with the caveat that a systems-level intervention of this type should also include components that target other levels of influence (e.g. patient- and provider-level components). Physicians shared numerous stories and examples of how they have used the EMR system as a direct education tool with patients, such as the primary care provider who instructed, “I am very into the patient being a part of the decision, so I’ll turn the screen and say…these are the criteria for antibiotics…” Thus, best-practice alerts may serve as an intervention with physicians and reinforcement for providing guideline concordant care, but also as a teaching tool for patients (e.g. by spurring physician–patient discussion and or offering quick access to patient-friendly education materials by embedding them in the best practice advisory system).
Core components of a staged, multi-faceted healthcare intervention
Informed by the findings from phases 1 and 2 of the study, the following core intervention components were designed and implemented in order to reduce use of non-recommended antibiotics for treatment of acute sinusitis: (1) an educational presentation targeting acute care providers; (2) a system-level best practice alert within the EMR; and (3) a patient-friendly publication embedded within the best practice advisory.
Provider education – “Much Ado about Snot: Evaluating and Treating Acute Sinusitis”
This educational presentation targeted primary care and urgent care providers and was initially launched on two separate dates as an interactive web-based application. Prior to the launch of the web-based training for the broader Kaiser Permanente Southern California audience of providers, the content was presented to the clinical leaders in our health system. Based on our qualitative findings that physicians find it challenging to stay abreast of current recommendations and desire better integration of guidelines into practice, learning objectives of the course were to (1) understand current recommendations for diagnosing acute sinusitis; (2) identify how best to treat acute sinusitis; and (3) incorporate best evidence into current practice. After the original web-based presentation sessions were delivered, the training was uploaded onto the physicians’ online education portal, where it remains available to providers.
System-level EMR best practice alert (BPA)
The BPA is triggered within the EMR for patients over 18 years of age with an encounter diagnosis of acute sinusitis, when antibiotics are prescribed during the clinical encounter. The EMR-based electronic alert acts as a reminder to clinicians about guideline recommendations for treatment of acute sinusitis, and requests information about the requested antibiotic prescription. The system alert also guides providers to quickly select the recommended antibiotic when appropriate. The alert audience includes all providers with prescribing authority in the outpatient setting. Rollout of the BPA was staggered across medical centres over several months to facilitate comparisons between pre- and post-implementation groups.
BPA-embedded patient education materials
The BPA was designed based on information gathered during the qualitative interviews. Providers requested support from reputable, nationally recognized, third-party entities to help with patient discussions with those seeking unwarranted antibiotics. The research team identified the Choosing Wisely® publication entitled, Treating Sinusitis: Don’t Rush to Antibiotics, as an effective patient education resource made available during the clinical encounter in the BPA. This Choosing Wisely® publication, which is a joint publication with Consumer Reports® and the American Academy of Family Physicians, emphasizes current recommendations and outlines a number of over-the-counter and home remedy options as alternatives to taking antibiotics for a likely viral infection. This publication remains embedded within the BPA, such that providers can readily access the handout and provide it to patients during an acute sinusitis encounter.
Intervention implementation
The design of the intervention was based on our qualitative results, which directly informed our efforts to successfully reduce use of non-recommended services at the system-level (BPA embedded in the EMR), provider-level (physician education presentation), and patient-level (BPA-embedded patient education materials). Results from our evaluation of this intervention are forthcoming, and preliminary findings are encouraging.
Discussion
Qualitative interviews with clinicians elicited valuable information and expanded our understanding of the practice context in ways that we could not have achieved exploring the structured EMR data alone. Retrospective, structured data collected during phase 1 of our research suggested an opportunity to develop targeted implementation strategies to improve antibiotic stewardship and to translate accepted antibiotic guidelines across care settings. Whereas the first phase of the research indicated a clear gap between current practice and best practices for acute sinusitis, the qualitative component of our mixed methods approach explored why the gap exists, and suggested how to bridge the gap by examining the barriers and/or facilitators to implementing these best practices in our specific care settings.
Interviews with primary and urgent care providers underscored the extent to which physicians’ concerns about patients’ expectations and satisfaction drive unwarranted antibiotic prescribing practices. Physicians emphasized the difficulty conveying the option and potential efficacy of evidence-based, medically indicated treatment alternatives (e.g. watch and wait, over-the-counter medications) to patients intent on receiving antibiotics, but for whom a prescription is not appropriate. Patient co-pays (typically ranging from $20 to $50 per visit) and the linking of patient satisfaction scores with provider incentives may further complicate communication, arguably redefining the patient–provider relationship as a customer–provider relationship. According to provider interviews, these factors systematically reinforce undesired behaviour, with physicians openly admitting that pressure from patients and incentive structures have caused them to prescribe antibiotics contrary to known guidelines.
Physicians articulated a need for easily-accessible guidelines, patient education resources, and enhancements to the EMR system as important tools to reinforce their conversations with patients. Recent literature in primary care settings suggests that provider concerns regarding the negative influence of patient satisfaction on unwarranted antibiotic prescribing may be real. In a cross-sectional analysis of national patient survey data and prescribing data within primary care practices in England, patients reported higher rates of dissatisfaction with their care if they were associated with a practice less likely to prescribe antibiotics [10]. Researchers suggest that acknowledging the extent to which trade-offs may exist between the promotion of physician–patient relationships and antibiotic prescribing is an important next step, and call for additional studies that can determine ways that patient satisfaction can be maintained during visits where providers refuse to prescribe antibiotics based on sound clinical evidence [10, 31].
Limitations
The fundamental goals for conducting interviews was to help us to (1) explain the quantitative data by providing rich context to the patterns identified; (2) focus on the departments/physicians in our setting most likely to overprescribe unwarranted antibiotics; and (3) uncover any potential leverage points for intervention. The nine interviews were detailed, generating 157 pages of written transcript containing in-depth, deeply contextualized data from providers within our health system. Spending a significant portion of our interview time with physicians discussing both the clinical and non-clinical factors that impact their treatment decisions, we found a high degree of consensus among physicians as to the primary drivers of unwarranted antibiotic prescribing behaviours. After the first two to three initial interviews, repetition of thematic classifications was evident, allowing the team to group participants’ responses into key emerging thematic categories contextualized to our specific healthcare setting, including (1) the role of patient expectations/satisfaction; (2) the value of the patient–provider relationship to antibiotic prescribing, yet the barriers to achieving effective patient–provider communication within our setting; and (3) the need for improved access to updated guidelines. Providers’ suggestions for how to address the problem also converged quickly, emphasizing the key thematic groupings as (1) a desire for patient education materials from trusted external sources to serve as ‘back up’ with patients, (2) provider reinforcement through in-service education, and (3) capitalizing on the EMR as a tool for both provider reinforcement and patient education. These themes were consistent throughout the remaining interviews, with few divergent viewpoints emerging.
However, we acknowledge that the qualitative findings reported herein may not be generalizable to all healthcare delivery systems. Physician participants were recruited to the phase 2 portion of the study using purposive, convenience sampling and the qualitative sample of interviews was relatively small. Despite these limitations, acute care providers in our integrated health system, which serves over 4 million members, maintain patient panels that are socio-demographically diverse and largely representative of the general population of Southern California [32]. While it is possible that the physicians’ views expressed during the interviews may not be representative of the perceptions of all providers within our system, we remain less concerned that they represent widely different patient populations from physicians who did not participate in the qualitative component. In addition, we achieved reasonable variation in our provider sample, in terms of their characteristics across gender, age, race/ethnicity, and years of experience.
Given our team’s embedded nature within the organisation, it is also possible that the physicians we interviewed gave socially desirable responses out of self-awareness or an attempt to please the interviewer. However, two observed factors are likely to mitigate these concerns. First, the major qualitative themes, namely that physicians report difficulty communicating with antibiotic-seeking patients about evidence-based alternatives and that physicians face pressure to meet patient expectations, are consistent with the findings from our quantitative data, which revealed high rates of antibiotic prescribing, and only about one-third of patients receiving recommended care for acute sinusitis. Second, as with any qualitative research, the overall aim in terms of sample size is to reach saturation; namely, the point at which no new information is emerging from the data [33]. As we noted above, despite our relatively small sample size for interviews, the suggested drivers that physicians ascribed to inappropriate antibiotic prescribing patterns, and their subsequent recommendations for potential intervention, became quickly saturated during the interview process.
Further, it was not feasible in this small, exploratory study to interview patients in addition to physicians (both those who did or did not receive antibiotics during an acute sinusitis encounter). Certainly, speaking with patients would provide an opportunity to examine their opinions and the extent to which they agree or disagree with physicians’ perceptions of their needs and satisfaction as drivers of unnecessary antibiotic prescribing. Realistically, however, we believe that many health systems similar to ours will also lack resources to conduct large numbers of interviews due to funding and time constraints; this is another reason that our study is likely to be a useful, pragmatic example worthy of sharing outside of our health setting.
Finally, a pragmatic consideration when embedding qualitative research into a learning health system is to recognize and address when provider perceptions are at odds with evidence. For example, we found that some providers support integrated decision support, but experience and evidence suggests that these interventions are commonly disregarded and may offer limited benefits [34, 35]. Within learning healthcare organisations, the perception of issues raised through qualitative investigation must be balanced with sound evidence and previous experience. Regardless, even when stakeholders’ perceptions may be false, identifying, understanding and addressing them through informed design and implementation may facilitate future success.
Conclusion
Our work offers an example of the benefits of embedded research, defined as work using operational funds to integrate scientific methods into projects addressing the practical needs of an organisation to provide generalizable knowledge. Therefore, although antibiotic stewardship and mixed methods studies are not alone novel, this paper presents a new model of research that incorporates providers’ viewpoints and contributes to the interpretation and validation of quantitative findings. We found that embedded qualitative research not only improved our understanding of quantitative findings, but efficiently moulded our intervention in ways that would not have been possible otherwise. Phase 2 of our research contributed to our ability “…to understand and work in ‘real world’ or usual practice settings, paying particular attention to the audience that will use the research, the context in which implementation occurs, and the factors that influence implementation” [36, Summary Points figure, point 3].
We postulate that qualitative research can provide a clearer understanding of factors influencing practice patterns at the point of care and lead to more effective healthcare delivery interventions [7, 37–41]. Furthermore, as ubiquitous use of EMR systems offers improved opportunities for quantitative analytics, embedding complementary qualitative methods into improvement efforts is possible, necessary, and likely to lead to more efficient interventions and better care delivery. Collaboration with clinical and operations leaders throughout this process offered rapid ability to apply findings within improvement efforts. Ultimately, our experience leveraging qualitative methods to target low-value antibiotic use can be a model for other learning healthcare systems undertaking care improvement efforts, since these methods can be readily applied to other diseases, specialties and stakeholders to successfully broaden the evaluation of clinical challenges at the point of care.
References
Institute of Medicine. The learning healthcare system. Washington: National Academies Press; 2007.
Institute of Medicine, Committee on the Learning Healthcare System in America. In: Smith M, Saunders R, Stuckhardt L, McGinnis JM, editors. Best care at lower cost: The path to continuously learning health care in America. Washington, DC: National Academies Press; 2013.
Academy Health. Using Evidence to Build a Learning Health Care System. 2012. https://www.academyhealth.org/files/publications/AHUsingEvidence2012.pdf. Accessed 6 Jan 2016.
Academy Health. Early Glimpses of the Learning Health Care System: The Potential Impact of Health IT. 2012. http://www.academyhealth.org/files/publications/HIT4AKPotential.pdf. Accessed 6 Jan 2016.
Greene SM, Reid RJ, Larson EB. Implementing the learning health system: from concept to action. Ann Intern Med. 2012;157(3):207–10.
Baker R, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;3:CD005470.
Flottorp SA, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35.
InsideKP. Kaiser Permanente Southern California. About Kaiser Permanente – Southern California. https://share.kaiserpermanente.org/article/fast-facts-about-kaiser-permanente/. Accessed 6 Jan 2016.
Masood A, Moumoulidis I, Panesar J. Acute rhinosinusitis in adults: an update on current management. Postgrad Med J. 2007;83(980):402–8.
Ashworth M, et al. Antibiotic prescribing and patient satisfaction in primary care in England: cross-sectional analysis of national patient survey data and prescribing data. Br J Gen Pract. 2016;66(642):e40–6.
Jorgensen LC, et al. Antibiotic prescribing in patients with acute rhinosinusitis is not in agreement with European recommendations. Scand J Prim Health Care. 2013;31(2):101–5.
Rosenfeld RM, et al. Clinical practice guideline: adult sinusitis. Otolaryngol Head Neck Surg. 2007;137(3 Suppl):S1–31.
Rosenfeld RM, et al. Clinical practice guideline (update): adult sinusitis. Otolaryngol Head Neck Surg. 2015;152(2 Suppl):S1–S39.
Anand VK. Epidemiology and economic impact of rhinosinusitis. Ann Otol Rhinol Laryngol Suppl. 2004;193:3–5.
Gonzales R, et al. Decreasing antibiotic use in ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA. 1999;281(16):1512–9.
Fleming-Dutra KE, et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011. JAMA. 2016;315(17):1864–73.
Morden NE, et al. Choosing Wisely – the politics and economics of labeling low-value services. N Engl J Med. 2014;370(7):589–92.
Lemiengre MB, et al. Antibiotics for clinically diagnosed acute rhinosinusitis in adults. Cochrane Database Syst Rev. 2012;10:CD006089.
Wilson JF. In the clinic. Acute sinusitis. Ann Intern Med. 2010;153(5):ITC31–15; quiz ITC316.
Broder J, Fordham LA, Warshauer DM. Increasing utilization of computed tomography in the pediatric emergency department, 2000–2006. Emerg Radiol. 2007;14(4):227–32.
Hicks LA, et al. US outpatient antibiotic prescribing variation according to geography, patient population, and provider specialty in 2011. Clin Infect Dis. 2015;60(9):1308–16.
Mettler Jr FA, et al. CT scanning: patterns of use and dose. J Radiol Prot. 2000;20(4):353–9.
Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks: Sage; 2011.
Sharp AL, et al. Low-value care for acute sinusitis encounters: who’s choosing wisely? Am J Manag Care. 2015;21(7):479–85.
Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42(5):533–44.
Patton MQ. Qualitative Research and Evaluation Methods. 3rd ed. Thousand Oaks: Sage; 2002.
Miles MB, Huberman AM, Saldaña J. Research Design and Management. In: Qualitative Data Analysis: A Methods Sourcebook. 3rd ed. Thousand Oaks: Sage; 2014. p. 17–54.
Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine De Gruyter; 1967.
Strauss A, Corbin J. Basics of qualitative research: grounded theory procedures and techniques. Newbury Park: Sage Publications; 1990.
ATLAS.ti. Scientific Software Development GmbH, Berlin. 2011. www.atlasti.com. Accessed 6 Jan 2016.
Lundkvist J, et al. The more time spent on listening, the less time spent on prescribing antibiotics in general practice. Fam Pract. 2002;19(6):638–40.
Koebnick C, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Perm J. 2012;16(3):37–41.
Dicicco-Bloom B, Crabtree BF. The qualitative research interview. Med Educ. 2006;40(4):314–21.
Kawamoto K, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.
Linder JA, et al. Documentation-based clinical decision support to improve antibiotic prescribing for acute respiratory infections in primary care: a cluster randomised controlled trial. Inform Prim Care. 2009;17(4):231–40.
Peters DH, et al. Implementation research: what it is and how to do it. BMJ. 2013;347:f6753.
Grol R. Personal paper. Beliefs and evidence in changing clinical practice. BMJ. 1997;315(7105):418–21.
Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–30.
Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. 2008;34(4):228–43.
Flottorp S, Oxman AD. Identifying barriers and tailoring interventions to improve the management of urinary tract infections and sore throat: a pragmatic study using qualitative methods. BMC Health Serv Res. 2003;3:3.
Rabin BA, et al. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008;14(2):117–23.
Acknowledgements
We would like to acknowledge the internal research funding support of Ben Chu, MD, for the vision to support the Care Improvement Research Team, as well as the Southern California Physicians’ Medical Group (SCPMG) physician partners who participated in interviews for this study.
Funding
Internal Funding, Southern California Permanente Medical Group.
Authors’ contributions
CEMP participated in the acquisition of data, analysis and interpretation of data, and drafting the manuscript. CP participated in the analysis and interpretation of data and was involved in critically revising the manuscript for important intellectual content. EH TT, HN, MG and MK participated in critically revising the manuscript for important intellectual content. AS contributed to the conception and design, acquisition of data, analysis and interpretation of data, and drafting the manuscript. All authors read and approved the final manuscript.
Competing interests
Authors CEMP, CP, EEH, TT, HQN, MKG, MHK, and ALS declare that they have no competing interests.
Ethics approval and consent to participate
All procedures performed in studies involving human participants were in accordance with the ethical standards of the Kaiser Permanente Southern California institutional research committee and with the 1964 Helsinki declaration and its later amendments, or comparable ethical standards (the research reported herein was approved by the Kaiser Permanente Southern California Institutional Review Board on October 16, 2013).
Author information
Authors and Affiliations
Corresponding author
Additional file
Additional file 1:
Semi-structured interview guide. (DOC 32 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Munoz-Plaza, C.E., Parry, C., Hahn, E.E. et al. Integrating qualitative research methods into care improvement efforts within a learning health system: addressing antibiotic overuse. Health Res Policy Sys 14, 63 (2016). https://doi.org/10.1186/s12961-016-0122-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12961-016-0122-3