A Quality Improvement Intervention to Improve Inpatient Pediatric Asthma Controller Accuracy
- aDepartment of Pediatrics, Connecticut Children’s Medical Center, Hartford, Connecticut; and
- bChildren’s Hospital at Montefiore, Bronx, New York, New York
- Address correspondence to Alexander H. Hogan, MD, MS, Department of Pediatrics, Connecticut Children’s Medical Center, 282 Washington St, Hartford, CT 06106. E-mail:
Dr Hogan designed the study, conducted all analyses, and drafted the initial manuscript; Dr Rastogi aided in study design, conducted validating chart review, and critically reviewed the manuscript; Dr Rinke mentored throughout the project, aided in study design and data analysis, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
OBJECTIVES: Our objective was to investigate if a rigorous quality improvement (QI) intervention could increase accuracy of pediatric asthma controller medications on discharge from an inpatient hospitalization.
METHODS: Our interprofessional QI team developed interventions such as improving documentation and creating standardized language to ensure patients were discharged on an appropriate asthma controller medication and improve assessment of asthma symptom control. Each week of 2015–2016, the first 5 patients discharged with status asthmaticus from the pediatric wards were reviewed for documentation of the 6 asthma control questions and accuracy of the discharge controller therapy. Correct discharge medication was defined as being prescribed the age-appropriate medication and dose on the basis of baseline controller therapy, compliance with baseline medication, and responses to asthma control assessment. The weekly proportion of control questions that were accessed and correct controller medications that were prescribed were analyzed by using Nelson rules and interrupted time series.
RESULTS: A total of 240 preintervention and 252 postintervention charts were reviewed. The primary outcome of the median proportion of patients discharged on appropriate controller therapy improved from 60% in preintervention data to 80% in the postintervention period. The process measure of proportion of asthma control questions that were assessed improved from 43% in the preintervention period to 98% by the final months of the intervention period. Both of these changes were statistically significant as per Nelson’s rules and interrupted time series analyses (P = .02 and P < .001, respectively, for postintervention break).
CONCLUSIONS: An interdisciplinary QI team successfully improved the accuracy of asthma controller therapy on discharge and the inpatient assessment of asthma control questions.
Asthma is the most common chronic pediatric condition, affecting 7 million children in the United States and causing 190 000 hospitalizations annually. This morbidity can be reduced through appropriate preventative or “controller” medications, especially inhaled corticosteroids (ICS).1–4 The current asthma management guidelines from the National Heart, Lung and Blood Institute’s National Asthma Education and Prevention Program (NAEPP) recommend that severity be classified for all children with asthma (intermittent, mild persistent, moderate persistent, severe persistent), and then appropriate controller medications should be prescribed, with attention to “stepping up” or “stepping down” medication dosages on the basis of disease control.4 Adherence to the NAEPP guidelines increases ICS use and decreases asthma-specific emergency department visits and hospitalization rates.5 Unfortunately, these guidelines are often not followed and controller medications are often not appropriately prescribed.6 Much of this research has been conducted in the ambulatory setting,7–11 and it is unclear how it translates to the pediatric inpatient arena where the sickest asthmatic patients are admitted.
In the inpatient setting, researchers of quality improvement (QI) studies have predominately evaluated the effect of clinical pathways on asthma process measures and outcomes.12–14 Compliance with Joint Commission–mandated measures such as the percent of patients receiving albuterol and systemic steroids are not associated with improved outcomes, likely because compliance is generally high at baseline.15 In terms of ICS prescriptions, the authors of a systematic review found that only half of the studies in which the impact of inpatient asthma protocols was assessed contained reports of the proportions of patients on any ICS at discharge. The studies in which ICS prescriptions were reported had wide variability, with the mean percent of patients discharged on any medication ranging from 0.54% to 92% in preintervention data.16 Although some studies were focused on improving ICS use on discharge,17,18 assessing the accuracy of these ICS prescriptions has been hampered by the lack of baseline asthma severity and control assessments.19 Methods to quantify appropriate controller prescription at time of discharge by inpatient providers in an environment in which ICS prescribing rates are already high are also not known.
Our purpose with this project was to investigate if a rigorous QI intervention could improve compliance with NAEPP guideline–based classification of asthma control in the inpatient setting, and thereby improve patient discharge on the appropriate controller medication, not just any controller medication. Our specific aim with the project was to discharge 75% of patients 5 to 18 years of age admitted in status asthmaticus on an appropriate controller medicine as defined by the NAEPP guidelines by 1 year of project implementation. An interdisciplinary QI team created a series of interventions centered around improving assessments of asthma control in an inpatient setting.
Our hospital is a 132-bed, urban, academic children’s hospital serving the Bronx, New York, with ∼1100 asthma admissions annually. Patients are primarily of minority race and ethnicities (40% African American; 30% Hispanic), and 73% are on public insurance. The 3 inpatient wards are staffed by teams consisting of a mix of providers, including medical students, physician assistants, and residents who rotate every 2 to 4 weeks, with teams typically consisting of an attending physician, 3 residents, a physician assistant, and 2 medical students. Seventy-eight percent of asthma patients were cared for by attending physicians from the pediatric hospital medicine division, with the remainder cared for by the adolescent medicine division (19%) and pediatric pulmonary division (3%). Our hospital had a baseline rate of 88% of patients being discharged on any asthma controller medication.
Planning the Intervention
An interdisciplinary QI team was assembled, with representatives from key stakeholders: residents, chief residents, physician assistants, respiratory therapists, nursing leadership, QI leaders, and attending physicians from pulmonary, adolescent, and pediatric hospital medicine divisions. Using the Model for Improvement and plan-do-study-act cycles,20 the QI team identified interventions to improve documentation of asthma control questions and standardize communications between providers (Fig 1). This team met monthly with multiple informal contacts in between meetings. At the conclusion of the project, a control plan was designed to sustain the gains of the interventions. There was no outside funding or other standard institutional and/or divisional support for this project.
The QI team created a bundle of interventions initially focused on improving documentation of inpatients’ asthma control questions. This area was targeted because, as per the NAEPP guidelines, without knowledge of asthma symptom control and medication compliance, it is impossible to discharge a patient on the correct controller medication. The symptom questions from the NAEPP guidelines are listed in Table 1. These interventions can be broadly categorized as visual aids, technological aids, and incentives. For a visual aid, we created a simplified flowchart summarizing the NAEPP control questions, corresponding asthma severity and recommended controller therapy with age-appropriate dosing (Supplemental Fig 4). To improve access to the control questions via technological aids, we encouraged providers to download a free iPhone application (STAT Asthma NHLBI Guidelines; Austin Physician Productivity, LLC, Austin, TX) and had control questions available on the desktop of all provider computers to aid in standardizing language. Both of these tools also aided providers in using the asthma symptom control assessment to choose the correct controller medication. QI team members oriented medical teams to the current interventions in person for the first 6 months of the project.
After a month of education and encouragement around provider assessment of asthma control, the team identified provider buy-in as a barrier to participation and awareness. Therefore, we created a food-based incentive in which if all patients were discharged on the appropriate controller medication and all patients had all control questions documented in a given week, then the QI team would bake specialty cookies for the successful team. As part of this incentive effort, providers were also shown their team’s data via weekly feedback e-mails. Weekly feedback emails highlighted current interventions, progress of the project thus far, and gave feedback on the causes of failures from the previous week’s data.
The final intervention, again with technological aids being used, occurred halfway through the project period when a new electronic medical record (EMR) was introduced. The original EMR allowed only for documentation in free text form. The new EMR allowed multiple methods for documentation of asthma control: free text, a documentation form with “yes/no” clickable dialogues, and a text macro with selectable text, commonly called a “smartphrase.” Both the documentation form and smartphrase contained the appropriate asthma control questions, which aided in standardizing documentation. The smartphrase was easier to adapt to resident feedback, was made accessible via expected phrases (ie, “AsthmaControlQuestions”) and also memorable phrases (ie, “Cookie” and “GetThatCookie”), and anecdotally had more uptake than the clickable form. A month after the EMR introduction, we adapted the weekly reminder e-mails to be focused on encouraging providers to use the smartphrase because those using the tool were documenting more questions and getting more children on the correct medications.
Methods of Evaluation
All patients 5 to 18 years old who were discharged from the children’s hospital with an All Patients Refined Diagnosis Related Group code for asthma were considered for inclusion in the study. To approximately coincide with the calendar year, the preintervention period was defined from January 15, 2015 to January 14, 2016, and the postintervention period was defined from January 15, 2016 to February 1, 2017. The postintervention period was slightly longer to correspond with the end of a resident team work cycle. The first 5 patients who met inclusion criteria and were discharged from the inpatient wards each week were identified by All Patients Refined Diagnosis Related Group code by using Looking Glass Clinical Analytics (Streamline Health, Atlanta, GA), and the project leader (A.H.H.) reviewed their charts. In the preintervention period, this review was done retrospectively, and in the postintervention period, the review was done in real time to give immediate feedback to providers. Chart review included all inpatient provider notes for that admission, including discharge summaries, to maximize the capture of asthma control question documentation both in the preintervention and postintervention data.
The primary outcome was the percentage of patients discharged on the correct controller medication, as documented in the discharge summary, on the basis of their responses to the NAEPP control questions. To our knowledge, there are no standardized tools to adjudicate correct controller medication prescription outside of the NAEPP guidelines themselves. We found that many adjudication situations were clear, such as a patient documented as noncompliant on their controller medication who is restarted on their home medication, or a patient newly started on a controller medication who had persistent symptoms. However, some adjudication situations were less clear, as when few or no control questions were documented and a patient was discharged on their home medication. For these reasons, we defined a patient as “discharged on the correct controller medication” when they had an appropriate change or maintenance in controller medication on the basis of documentation of control questions in any provider note. To evaluate external validity of these adjudications, a local asthma expert who is the director of the Pediatric Asthma Center (D.R.) conducted a chart review of 25 charts before the project began, and assessed agreement with adjudications and gave feedback on any discrepant charts.
The process measure of the proportion of patients with asthma control assessed was examined similarly by using chart review of every provider note and tracking whether all 6 of the control questions (symptom frequency, nighttime awakenings, albuterol use for symptom control, interference with normal activity, asthma exacerbations requiring orally systemic corticosteroids, and compliance with current controller therapy, if prescribed) were documented. Balancing measures included length of stay, the proportion of patients discharged after 1 pm, and the proportion of patients discharged on any ICS. At completion of the intervention period, a control plan was implemented with assessment frequency decreased from weekly to monthly, with a plan to increase reminder e-mails if documentation of asthma control questions dropped below 80%. The Albert Einstein College of Medicine Institutional Review Board approved this study, and it was deemed exempt from written consent.
Demographics and balancing measures were assessed by χ2 for proportions and Wilcoxon rank sum for nonnormally distributed data. To track the effect of our interventions and give real-time feedback to provider teams, run charts were created by plotting the median proportion of 5 patients reviewed each week who were discharged on the appropriate controller medication or the median proportion of control questions that were documented. Medians were calculated for the preintervention period and new medians were calculated in the postintervention period when a run of 8 or more weeks were greater than the previous mean per Nelson’s rules.21 We also conducted an interrupted time series analysis using segmented logistic regression to compare pre- versus postintervention means for both outcomes.22 The utility of standardized documentation was analyzed by χ2. Data were analyzed by using Stata 14.1 (StataCorp, College Station, TX).
There were 240 charts reviewed in the preintervention and 252 in the postintervention periods. The number of charts was unbalanced because of 2 additional weeks in the intervention period and because the intervention period had more weeks during the summer months during which the full 5 charts were available to review. There was 85% agreement between the primary reviewer and the asthma expert on 25 charts, with a Cohen’s κ of 0.65, which is statistically significant and substantial.23 There was likely additional improvement after these 25 charts because feedback was given to the primary reviewer (A.H.H.) after chart review to increase agreement. Comparing the pre- versus postintervention periods, there were no significant differences in age or race of the patients; however, there were significantly more girls and patients identifying as Hispanic in the postintervention group (Table 2). Pre- and postintervention groups were not significantly different with regards to chronic asthma severity classification, the proportion that were already on an ICS, the proportion admitted to the ICU, or the proportion discharged from the hospital medicine, adolescent, or pulmonary teams (Table 2).
The primary outcome of median proportion of patients on appropriate controller therapy on discharge was 60% in the preintervention data. There was a significant shift in the data after March 2016, 12 weeks postintervention, leading to a new median of 80% of patients discharged on the correct medication in the postintervention period (Fig 2). The process measure of proportion of patients with asthma control severity assessed in the preintervention period was 43%. Control assessment improved to a median of 83%, with a shift in the data immediately after interventions began. A second shift occurred after the new EMR implementation and the change in reminder emails, focusing on using smartphrases, to 98% of patients receiving symptom control assessments (Fig 3). Balancing measures of length of stay (1.67 vs 1.71 days; P = .45), proportion of patients discharged after 1 pm (66% vs 73%; P = .13), and proportion of patients discharged on any ICS (85% vs 84%; P = .6) were not significantly different comparing the pre- and postintervention periods.
Time series analysis for the primary outcome was statistically significant in the upward break (ie, a change in the absolute proportion of patients discharged on the correct medication in the postintervention period; P = .02). The trend was also statistically significant (ie, the change in proportion correct per week; P = .01) after the intervention. The proportion of NAEPP control questions assessed was also statistically significant for the upward break (P < .001) and trend (P = .01) in the postintervention period.
Use of standardized documentation templates increased from 40% pre-EMR rollout to 60% post-EMR rollout (P < .001). Those using either the smartphrase or documentation form in the post-EMR period were significantly more likely to discharge patients on the correct medication (24% vs 77%; P = .02) and document all asthma control questions (8% vs 91%; P < .001).
Most patients adjudicated as discharged on the correct medication were restarted on the controller therapies they were noncompliant with (48% preintervention, 43% postintervention), newly started on a controller medication (32% preintervention, 20% postintervention), or were stepped up on their current controller therapy (12% preintervention, 17% postintervention) (Table 3). Conversely, most patients (78% preintervention, 58% postintervention) adjudicated as discharged on an incorrect controller medication had inadequate medical documentation to justify the medication (Table 4).
An interdisciplinary QI team successfully increased the median proportion of patients discharged on the correct asthma controller medication from 60% to 80%, surpassing the 75% aim of our project. By engaging with frontline providers and incorporating their feedback, we were able to create interventions that directly affected our key drivers and made a significant change in our asthma care. To our knowledge, this is the first project to be focused on and successfully and consistently improve discharging of pediatric patients on the correct controller medication, not just on any controller medication.
Multiple interventions and drivers contributed to this project’s success. Our main driver was improving assessment of asthma control through documentation of the NAEPP control questions. Although pocket cards have been shown to be moderately effective in improving ICS prescriptions,10 our frontline providers rejected having another item on their badge or in their pocket. Our solution used a tool already in most of their pockets (a smartphone) and facilitated understanding via a flowchart in the resident workrooms and on every computer. Our second most important driver was standardizing communication around asthma care. Before these interventions, triggers for a patient’s current exacerbation and their exposures, such as secondhand smoke, animals, and carpets, were almost always documented. Unfortunately, the documentation of NAEPP asthma control questions (the key component to decide the appropriate controller prescription) was present in only 43% of the patients. By making these questions accessible on every computer, we had moderate improvement. The EMR intervention, which allowed for easy to remember smartphrases, markedly improved documentation of control questions and thereby the percentage of children receiving the correct controller medication at discharge. Although some of these interventions were institution specific (ie, those without smartphrases in their EMR could not use this intervention), most were either low cost (food incentives, e-mail reminders) or are provided here as Supplemental Fig 4. The next steps of this project are to sustain the improvements we have made locally and disseminate our work to other institutions through presentations and this publication.
Our findings are consistent with other studies that have revealed that guideline compliance can be improved. The authors of a recent systemic review reported that the authors of 15 studies used decision support and found up to a 34% increase in prescription of any ICS medicines after interventions. Multicomponent interventions using combinations of feedback, decision support, and other techniques demonstrated significant increases in prescribing of ICS by 25% to 49% but the success varied between the studies.10 Whereas the authors of 3 studies found between 25% and 49% increases in ICS prescriptions, the authors of another 4 studies did not find any difference in prescription rates.10 Because our baseline frequency of ICS prescription was already 85%, with 95% of persistent asthmatics on any controller, it was not our goal to increase rates but rather to improve accuracy of those and other prescriptions for controller medications. As a balancing measure, we tracked ICS prescribing rates because we did not want to inadvertently decrease our high ICS rates by accidentally discouraging ICS in favor of alternate controllers, such as leukotriene receptor antagonists. We did not find a significant difference in ICS rates (P = .61), or prescription of any controller (P = .95).
Assessing accuracy of controller medications is difficult because of a lack of standardized metrics. The National Committee for Quality Assurance identifies appropriate medications for children with asthma as those with persistent asthma prescribed any controller medication.24 Using this metric, we found that 98% of our patients were “appropriate” before the start of the project, which makes this metric of questionable utility for our institution. In their review of quality metrics, Nkoy et al19 determined that there is evidence level “A” for use of “proportion of patients discharged with proper controller medications, according to their chronic asthma severity classification.”10 However, the metric was rejected by the study’s authors because “data on chronic asthma severity…were rarely available in the medical record and therefore were not feasible to obtain.”19 We have shown that an interdisciplinary QI team can increase the asthma severity data needed to make this integral quality metric feasible to evaluate.
There are limitations to our project. Our QI team did not include a family member, a key stakeholder. This was an oversight, and we encourage all QI teams to include patient and family representatives on QI projects. We made the adjudication of controller medication accuracy feasible; the determination of the primary outcome was made by 1 reviewer who was not blinded to the study’s time period. Although most of the determinations (85% preintervention, 76% postintervention) were not subjective (eg, a patient who was not compliant with ICS was restarted on the same medication), there were a proportion of determinations that could have been subjective (15% preintervention, 23% postintervention). Tables 3 and 4 enumerate the subjective and not subjective determinations used in our analysis. We attempted to mitigate concerns of adjudication subjectivity by calculating an interrater reliability between the primary rater and an asthma expert. Another limitation is that our primary outcome was not a patient outcome, such as readmissions or recurrent needs for steroid use. These outcomes were beyond the scope of this inpatient-focused study, and the low rate of some of these patient outcomes necessitates multicenter studies to evaluate them. In addition, we did not have access to the insurance claims data that would make the evaluation of readmission at any hospital or prescription fill rates possible. However, given the overwhelming evidence of the benefits of ICS and guideline adherence, we believe our primary outcome is justified.2–4,25–27 Finally, because this is a pre-post outcome study, we cannot account for temporal trends that may have influenced the change in our outcomes, although our interrupted time-series analysis tested for this and revealed the interventions themselves had a significant effect on our primary outcome.
An interdisciplinary QI team can successfully improve the accuracy of asthma controller therapy for children with acute asthma exacerbations on discharge by improving assessment of asthma control. As documentation can be improved to make accuracy determinations feasible, standardized tools must be developed to assure the validity of this metric for quality tracking.
The authors acknowledge all the residents, chief residents, physician assistants, and attending physicians who participated in this project, as well as the leadership of the Children’s Hospital at Montefiore.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Copyright © 2018 by the American Academy of Pediatrics