Implementation of the Sepsis Risk Calculator at an Academic Birth Hospital
Abstract
BACKGROUND: Multivariate predictive models for estimating the risk of neonatal early-onset sepsis (EOS) are available as a Web-based sepsis risk calculator (SRC) and may reduce the proportion of newborns empirically treated with antibiotics after birth. EOS risk assessment based on such models would require workflow changes at most birth hospitals.
METHODS: A multidisciplinary team of obstetric, neonatal, and information technology staff at a large, academic, birth hospital collaborated to implement the SRC. The obstetric electronic medical record was modified to provide a link to the SRC. Labor and delivery nurses calculated the sepsis risk at birth and alerted neonatal clinicians for risk estimates ≥0.7 cases per 1000 live births. Subsequent interventions were based on the risk estimate and newborn clinical examination. We compared the proportion of infants born at ≥36 weeks’ gestation with laboratory testing and empirical antibiotics for risk of EOS during the 15-month periods before (n = 5692) and after (n = 6090) implementation. EOS cases were reviewed to assess for safety.
RESULTS: Empirical antibiotic use among newborns ≤72 hours old declined by 42% (6.3% to 3.7%; relative risk 0.58 [95% confidence interval, 0.50–0.69]), and laboratory testing declined by 82% (26.9% to 4.9%; relative risk 0.18 [95% confidence interval, 0.16–0.21]). The EOS incidence was not different between the study periods, and no safety concerns were identified.
CONCLUSIONS: The SRC was integrated into the workflow of a large, academic perinatal center, resulting in significant reductions in antibiotics and laboratory testing for EOS and demonstrating the potential for this approach to impact national practice.
The management of newborns for risk of early-onset sepsis (EOS) is one of the most common clinical tasks conducted by neonatal clinicians. Preventive intrapartum antibiotic therapies are now used in ∼30% of US births, with an associated decline in the incidence of EOS among term infants from 3 to 4 cases per 1000 live births (LBs) to 0.5 cases per 1000 LBs occurring at ≥37 weeks’ gestation.1–7 Those at most centers providing newborn care screen infants at birth for the presence of specific, identifiable risk factors for EOS and use different combinations of these factors as well as newborn clinical condition and laboratory tests to determine the need for empirical antibiotic administration.8 Although these are undertaken to protect the newborns, such processes expose ∼5% to 10% of newborns to broad-spectrum antibiotics, separate mothers and newborns, negatively impact breastfeeding initiation, and incur an estimated $400 million to $500 million in associated costs to newborn care.8–12
Multivariate models have been developed that provide an estimate of sepsis risk for the individual newborn.6,13 These models are based on risk factors present at birth and the evolving newborn clinical condition and are available in the format of a Web-based sepsis risk calculator (SRC). Retrospective application of the SRC to infants born to mothers with the clinical diagnosis of chorioamnionitis reveals decreases in antibiotic use compared with current recommended approaches.14–16 Recently, the clinical use of the SRC has been prospectively validated in a large, self-insured, integrated health care system that provides comprehensive newborn and pediatric care using a common inpatient and outpatient electronic medical record (EMR) and is characterized by high rates of prenatal care and comprehensive postbirth discharge care.12 Therefore, integrated health care systems differ from most birth centers in the United States, raising questions about the practicality and impact of using the SRC at more traditional perinatal centers. We changed the local approach to newborn EOS risk assessment at our large, academic birth hospital from one based on the dichotomous consideration of clinical risk factors and laboratory test results to one based on risk estimates generated by the SRC. Our objective in this study was to describe the implementation of the SRC in obstetric and newborn care practice and quantify the proportion of infants born at ≥36 weeks’ gestation who were administered empirical antibiotics and/or subjected to laboratory testing for risk of EOS before and after the use of the SRC-based approach.
Methods
Study Design and Setting
This was a retrospective cohort study that included all infants born at ≥36 0/7 weeks’ gestation at the study hospital over two 15-month time periods: March 1, 2014, to May 31, 2015, and July 13, 2015, to October 13, 2016. Infants born during a 6-week washout period (June 1, 2015–July 12, 2015) were excluded. The study site is a 534-bed teaching hospital within a university health care system with ∼5000 annual deliveries, 52 postpartum mother-infant rooms, and a 50-bed, tertiary-care NICU. Obstetric care is provided by physicians in both academic and private practice, obstetric residents, midwives, and maternal-fetal medicine specialists. Board-certified neonatologists and neonatal advanced-practice clinicians are present on-site 24 hours per day, providing all care in the NICU and ∼80% of well-baby nursery (WBN) care; the remaining WBN care is provided by private-practice pediatricians. SRC planning began in January 2015, before the publication of the Kaiser Permanente Northern California (KPNC) prospective validation study.12 We integrated the SRC approach, retrospectively validated and published in 2014, into our existing EOS risk assessment workflow.13 We focused our study of the outcomes of SRC use on infants with gestational age at birth of ≥36 weeks because these infants are eligible for WBN care at our center; all others are admitted to the NICU from birth. Data collection and data analysis were approved by the university’s institutional review board.
Study Definitions
EOS was defined by blood or cerebrospinal fluid cultures with positive results for pathogenic bacteria or fungi. Common skin-commensal organisms, including coagulase-negative staphylococci, were considered contaminants. Risk estimates from the SRC are expressed as the number of EOS cases per 1000 LBs. Empirical antibiotic exposure for EOS was defined as any antibiotics administered at ≤72 hours of age that were initiated before culture results were known. EOS evaluation was defined as any combination of complete blood cell (CBC) count with white blood cell differential, C-reactive protein (CRP), and blood culture performed at ≤72 hours of age. Testing with the CBC count alone was excluded. Antibiotic and laboratory testing up to 7 days was used as a balancing measure to capture a delayed presentation of illness and EOS evaluation.
Sepsis Risk Assessment Before and After Use of the SRC
During the preimplementation period (pre-SRC), the local policy for neonatal risk assessment was based on guidelines provided in the Centers for Disease Control and Prevention (CDC) group B streptococcus (GBS) prevention guidelines16,17 and those recommended by the American Academy of Pediatrics (Table 1).18,19 The pre-SRC policy was posted on the walls of all labor and delivery rooms. Labor and delivery nurses contacted the NICU team regarding any infant who required EOS evaluation as per the posted policy. Symptomatic newborns and all newborns requiring antibiotics were transferred to the NICU for further care. Well-appearing infants requiring screening laboratory tests at 12 hours of age were cared for in the WBN, and NICU clinicians were called with laboratory results. A multidisciplinary team consisting of obstetric and neonatal nursing and medical caregivers as well as hospital information technology staff developed the post-SRC policy (Table 1). A slide presentation was developed for educating labor and delivery staff on use of the SRC; nurse educators documented individual competency. The obstetric EMR was modified to provide a link to the SRC Web site (https://neonatalsepsiscalculator.kaiserpermanente.org). Labor and delivery nurses were provided guidelines for use of the SRC (Supplemental Table 6) and instructed to (1) calculate the sepsis risk estimate at birth, (2) record the value in the labor and delivery EMR, and (3) contact the NICU team to evaluate any infant with a sepsis risk estimate at birth of ≥0.7 per 1000 LBs. NICU clinicians examined the infant to incorporate the clinical examination in accordance with the definitions of clinical status provided in the SRC.13 For infants with a sepsis risk estimate at birth of <1.5 per 1000 LBs, NICU clinicians could choose to observe symptomatic infants for up to 6 hours after birth before determining the final clinical status. For quality-assurance purposes, a daily report was generated from the EMR containing the risk data and calculated risk estimate for all LBs of ≥36 weeks’ gestation. In addition, if they did not attend the infant’s delivery, neonatal clinicians recalculated the sepsis risk estimate at birth at the time of WBN or NICU admission.
Approach to EOS Risk Assessment
Data Sources and Analysis
Demographic data were obtained from the local administrative database. Maternal and infant clinical data were obtained by review of EMRs and paper medical records. Univariate comparisons were performed with a χ2 test and Student’s t test as appropriate. To control for the potential effect of secular trends on antibiotic initiation, we conducted an interrupted time series analysis using segmented regression. The periods before and after SRC implementation formed the 2 segments in the model. The model included monthly data on the baseline trend, change in level, and trend after SRC implementation for the proportion of eligible infants exposed to antibiotics. We tested for autocorrelation in up to 6 orders using the Durbin Watson statistic. The Dickey-Fuller unit root test was used to establish the stationarity nature of the data series, and therefore, no adjustment was made for seasonality. Antibiotic exposure rates were also compared by using statistical process control analysis. Analysis was performed by using Stata 14 (StataCorp, College Station, TX).
Results
A total of 11 782 infants were included in the study: 5692 infants during the pre-SRC period and 6090 during the post-SRC period. Delivery characteristics were comparable during the 2 time periods, with statistically significant but clinically minor differences noted in birth gestation and delivery mode (Table 2). There was no significant difference in the proportion of study infants admitted to the NICU during the 2 periods.
Study Cohort Characteristics
Implementation of the SRC Protocol
The sepsis risk estimate at birth was recorded for 6077 of 6090 (99.8%) newborns during the SRC period (Table 3). Five infants were born outside the hospital and ineligible for SRC, and 8 infants had missing data. More than half of all infants (3269 of 6090; 53.7%) had risk estimates of ≤0.1 per 1000 LBs. Only 1.6% of infants met the criteria for the empirical administration of antibiotics solely on the basis of the estimated risk of EOS at birth. Of the 173 infants with a risk estimate at birth of 0.7 per 1000 LBs to 1.49 per 1000 LBs, 134 (77.5%) were well appearing and managed with enhanced clinical surveillance in the WBN. Among these 134 infants, the median maternal peak temperature was 100.5°F (interquartile range, 100.1–101°F), and the median duration of rupture of membranes (ROM) was 15.2 hours (interquartile range, 8.8–23.0 hours). Only 2 of the 134 infants were admitted to the WBN and later transferred to the NICU, and in both cases, it was for reasons unrelated to infection. A chart review for a 20% random sample revealed good compliance with the required 36-hour vital sign protocol, with 90% of the vital signs collected every 4 hours being documented in the EMR within 5 hours of the preceding vital sign data.
Distribution of Sepsis Risk Estimates at Birth in the Post-SRC Period
Antibiotic Use
The proportion of newborns administered empirical antibiotics at ≤72 hours of age declined during the post-SRC period, from 6.3% in the pre-SRC period to 3.7% in the post-SRC period (P <.001). The relative risk of antibiotic exposure in the post-SRC period was 0.58 (95% confidence interval [CI], 0.50–0.69), which is a 42% reduced incidence of antibiotic initiation in the post-SRC period (Fig 1, Table 4). The decline in empirical antibiotic use was also significant when using interrupted time series analysis. A change in level was observed for antibiotic use after SRC implementation (estimate = −4.5; P = .0007); the baseline trend and trend after implementation were not significant. The same decline was observed for antibiotic administration through 7 days of age.
Monthly rates of antibiotic administration. The monthly rate of empirical antibiotic administration was defined as the number of infants born at ≥36 0/7 weeks’ gestation who were administered antibiotics at ≤72 hours of age divided by the total number of infants born at ≥36 0/7weeks’ gestation per calendar month. The 6-week washout period is not included. Statistical process control using p-chart methodology is shown, with the upper and lower control limits having been determined by the pre-SRC period data. The criteria for significant change in process were met after SRC implementation, with >8 consecutive data points falling below the pre-SRC mean.
Antibiotic Use and Laboratory Testing
Laboratory Test Use
The overall proportion of infants evaluated for EOS with any combination of laboratory testing at ≤72 hours of age declined by 82% in the post-SRC period (pre-SRC 26.9% versus post-SRC 4.9%; relative risk 0.18 [95% CI, 0.16–0.21]; Table 4). Proportionally, more infants were tested with blood culture alone and blood culture in combination with CBC count and white blood cell differentials in the post-SRC period. The decline was observed among all infants as well as among those admitted to the NICU for any reason. Similar results were obtained for laboratory testing performed through 7 days of age (pre-SRC 27.3% versus post-SRC 5.2%; P < .001).
EOS Cases
Four infants had blood culture–confirmed EOS during the study period, 1 during the pre-SRC period and 3 during the post-SRC period (pre-SRC incidence, 0.20 per 1000 LBs; post-SRC incidence, 0.49 per 1000 LBs; P = .67; Table 5). None had meningitis, and none required mechanical ventilation or pressor support. All were discharged from the hospital after appropriate durations of antibiotic therapy.
EOS Cases During the Study Periods
Discussion
We demonstrate that the SRC can be adopted as the primary means of EOS risk assessment in s large, academic, perinatal center, with a significant impact on antibiotic and laboratory test use and with short-term safety. We were able to leverage existing resources to make the transition from the use of a categorical risk-based algorithm to the use of the SRC as the primary means of assessing risk of EOS among term and late-preterm infants. Our approach was different from that validated in the birth centers of the KPNC system.12 We adopted the SRC approach published by Escobar et al,13 who used recursive partitioning and split validation to provide estimates of the impact of care algorithms based on estimated risk at birth <0.65, 0.65 to 1.54, and >1.54 per 1000 LBs. For practical purposes, we based our local algorithm on risk estimate cutoffs of 0.7 per 1000 LBs and 1.5 per 1000 LBs. Despite this difference in approach, the 41% decrease in empirical antibiotic use is similar in magnitude to that observed in the KPNC study (48%), although an overall lower rate of empirical antibiotic use was achieved in the KPNC hospitals.
Once we developed the local SRC algorithm described in Table 1, we needed to determine how it would be integrated into the daily workflow of a center with >5000 annual deliveries. The multivariate models that are in the SRC were based on objective data for calculation within an EMR and were meant to be applied to all infants in the targeted birth population. Because our local EMR could not accommodate the model equations, we partnered with our information technology personnel to provide a link from our existing EMR to the public SRC Web site. We also modified existing documentation tools to record the SRC variables and the risk estimate obtained at birth. To ensure that a risk estimate at birth would be calculated for every newborn of ≥36 weeks’ gestation, the SRC was placed in the workflow of the labor and delivery nursing staff. Labor and delivery nurses at our center are the only type of perinatal caregivers who are universally in attendance at all births, and they are the same caregivers who were charged with ensuring compliance with the pre-SRC policies. Nurse educators provided SRC education to each labor and delivery and postpartum nurse in small-group sessions. Finally, we ensured that the protocol was straightforward; after calculating the risk estimate at birth, the labor and delivery nurse simply had to call the NICU team to assess any infant with a value of ≥0.7 per 1000 LBs if the team was not already in attendance. All subsequent assessments of clinical status and antibiotic decisions were made by the NICU team. The robustness of these processes is illustrated by the fact that the sepsis risk estimate at birth was calculated and communicated for ∼99% of the >6000 infants born at ≥36 weeks’ gestation during the post-SRC period.
After implementation, we received feedback from the nursing staff about 2 concerns. First, the staff sought clarification about how to categorize administered intrapartum antibiotic therapy. Second, staff observed that it was not uncommon for elevated maternal temperature to be documented shortly after delivery but not recognized during delivery, often because of delays in obtaining routine vital signs toward the end of delivery. In response, we developed detailed instructions that helped address these and other issues frequently encountered by staff as they familiarized themselves with the SRC (Supplemental Table 6).
One unique aspect of our local approach to SRC implementation should be emphasized. We eliminated the routine use of CBC counts and CRP levels and based empirical antibiotic decisions solely on the risk estimate at birth and the newborn clinical condition. We made this decision on the basis of data demonstrating the poor sensitivity of these laboratory tests in predicting EOS among term and late-preterm infants.20–22 Clinicians could choose to obtain these tests on the basis of their clinical judgment in specific situations and often did so; nonetheless, laboratory testing for EOS declined by ∼80% among WBN infants and by ∼50% among infants admitted to the NICU.
The decrease in antibiotic use in our center primarily resulted from changes in the care of 2 specific types of infants. The first type included infants with risk estimates at birth in the range of 0.7 to 1.49 per 1000 LBs. These infants were often born in the setting of risk factors that were considered in our pre-SRC policy, such as maternal fever >100.5°F or ROM >18 hours. We required close surveillance of these infants in the WBN, aware that these infants may have been administered antibiotics or subjected to laboratory testing in the pre-SRC period. Such infants comprised <3% of the infants admitted to WBN, which may have contributed to the observed compliance with enhanced surveillance requirements. Neonatal medical providers had a low threshold of concern for these moderate-risk infants when they did exhibit signs of clinical instability; nearly one-quarter of the infants in the range of 0.7 to 1.49 per 1000 LBs were admitted to the NICU for observation, and half of those admitted received empirical antibiotic treatment in concert with SRC recommendations. The second type that contributed to the observed decrease in empirical antibiotic administration included infants with risk estimates at birth of <0.7 per 1000 LBs who were admitted to the NICU with initial equivocal clinical status. This was reflected in the decrease in empirical antibiotic use and laboratory testing among infants admitted to the NICU for any reason (Table 4). Even among infants admitted to the NICU for whom blood cultures were obtained, 19% were never administered antibiotics because symptoms improved or were attributed to noninfectious causes (data not shown).
The goal in all approaches to neonatal sepsis risk assessment is to ensure safety by assessing the likelihood of infection and administering empirical antibiotics to prevent the development or progression of clinical illness. The performance of CDC- and American Academy of Pediatrics–recommended algorithms in achieving such goals has never been quantified, nor was such performance the basis of those recommendations. The safety of using the SRC to assess newborn risk of EOS was addressed in the prospective validation study done at KPNC birth centers. As assessed by late presentation of EOS during the birth hospitalization; by outcomes among those with culture-confirmed EOS; and by the incidence of re-hospitalization with culture-confirmed EOS after birth hospital discharge, the SRC approach was found to be as safe as previous CDC-recommended algorithms.12 In our center, we found no evidence of later presentation of illness as assessed by the use of laboratory testing and antibiotics beyond 72 hours of age. However, a limitation of our study is that as a perinatal birth center located in a hospital that provides adult medical care, we do not readmit newborns after discharge for any indication other than hyperbilirubinemia, and we cannot assess neonatal outcomes postdischarge. Infants born at our birth center come from a large urban and suburban metropolitan area and seek pediatric care at a variety of public and private practices. Furthermore, the rate of rehospitalization for EOS in the KPNC study in both the pre- and post-SRC periods (∼5 per 100 000 LBs) reveals that it is a rare outcome that is unlikely to be observed by any individual perinatal center.
A review of the EOS cases that occurred during our study period revealed a final, important aspect of neonatal risk assessment. Recent attention has been given in the media and in state legislatures to the need for pediatric sepsis policies and protocols.23 We believe that specific policies addressing neonatal EOS risk assessment, ongoing attention to compliance with such policies, and periodic review of the outcomes of such policies are also critical to neonatal care. However, case 2 in our study reveals the limits of neonatal risk assessment. This case of GBS sepsis occurred in an infant who was born without any specific risk factors that would have flagged the infant in our pre-SRC period, with a risk estimate of 0.3 per 1000 and reassuring clinical condition at birth. The infant was found to have new-onset tachypnea, lethargy, and poor feeding at ∼36 hours of age, prompting transfer to the NICU. This infant was born to a mother who was GBS-negative. GBS-specific EOS among term infants born to mothers who are GBS-negative is well documented.1,24 This case illustrates the importance of combining universal risk-assessment protocols with ongoing clinical assessment. No protocol can function without excellent clinical care.
Conclusions
The SRC was implemented as the primary means of EOS risk assessment among infants born at ≥36 weeks’ gestation at our large perinatal center by using a multidisciplinary management approach. Our study reveals the feasibility and safety of integrating this risk model into clinical newborn care. Together with the experience in the Kaiser Permanente hospitals, our experience can inform SRC implementation efforts at diverse birth centers. The significant declines in empirical antibiotic administration and laboratory test use we observed suggest that widespread use of the SRC could have a major impact on the care of newborns across the United States. As with all protocols for managing the risk of neonatal sepsis, the SRC approach should be combined with systems for the ongoing clinical assessment of all newborns.
Acknowledgments
We thank Gabriel Escobar, MD, whose foresight and leadership led the original studies in which the SRC was developed, and Allen Fischer, MD, and Michael Kuzniewicz, MD, MPH, who led the implementation of the SRC at KPNC and critically reviewed the article. We also thank our physicians, advance-practice clinicians, Jessie George, MSN, RN, Pamela Power, RN, DPN, and all the women’s program nursing leadership and staff members who collaborated with energy and enthusiasm.
Footnotes
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2018 by the American Academy of Pediatrics
