Comparison of Administrative Data Versus Infection Control Data in Identifying Central Line–Associated Bloodstream Infections in Children’s Hospitals
Objective: As of July 2012, the Centers for Medicare and Medicaid Services prohibited state Medicaid programs from paying for medical care related to certain provider-preventable conditions. The most prevalent provider-preventable condition in pediatrics is central line–associated bloodstream infections (CLABSIs), which cause significant morbidity and mortality. The objective of this study was to compare the uses of administrative data and infection control data in measuring CLABSIs.
Methods: Retrospective chart reviews were performed in 3 children’s hospitals to compare CLABSIs identified according to administrative data diagnostic coding versus infections identified by hospital infection control departments. Clinical criteria from the Centers for Disease Control and Prevention and reported to the National Healthcare Safety Network were used.
Results: A total of 166 CLABSIs were identified in 35 698 discharges in the 3 children’s hospitals in 2010. Using the Centers for Disease Control and Prevention criteria as the standard, administrative data had 34.78% sensitivity and 99.92% specificity. The positive predictive value was 63.16% whereas the negative predictive value was 99.75%.
Conclusions: Administrative data and National Healthcare Safety Network criteria identify discordant numbers of CLABSIs.
- billing and compliance
- infection control
- medical error
- patient safety
- vascular catheter–related infections
Central line–associated bloodstream infections (CLABSIs) are a significant source of morbidity, mortality, and added medical costs for hospitalized children and adults.1,2 Documentation of these complications in the literature spurred national quality improvement efforts aimed at reducing CLABSIs. These efforts, mainly aimed at decreasing the rates through standardizing practice, resulted in dramatic reductions in CLABSIs among both adults and children.3,4 For instance, in a collaborative study in 29 children’s hospitals sponsored by the Children’s Hospital Association, Miller et al5 reported a drop in infection rates from 5.2 infections per 1000 line days to 2.3 infections per 1000 line days, with sustained and continuously decreasing rates over 3 years.
Because CLABSIs occur commonly, are potentially preventable, and are harmful to patients, the Centers for Medicare & Medicaid Services (CMS), the Joint Commission, and the Agency for Healthcare Research and Quality (AHRQ) sought to design methods to quantify the number of CLABSIs in individual hospitals for use as measures of quality. AHRQ included the measure “infection due to medical care” in their patient safety indicators (PSIs), which originally identified CLABSIs via administrative data by using a nonspecific diagnostic code that included infections associated with any device such as pacemakers, venous lines, and arterial lines.6 The AHRQ pediatric quality indicators were created in 2006 in response to many issues in PSIs that were not appropriate for children7 but maintained the use of the nonspecific diagnostic code for “infection due to medical care.” Both PSIs and pediatric quality indicators had risk adjustment methods that were helpful in comparative reporting. In October 2007 a specific International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code (999.31) was created to identify “infections due to central venous catheter,” with the intent of more accurately identifying CLABSIs by using administrative data. The code included all venous catheters that were placed centrally, including classically placed central lines such as subclavian and femoral lines, peripherally inserted central lines, and umbilical venous catheters, but specifically excluded arterial and peripheral venous catheters that are not threaded into a central venous vessel.8,9 In 2008, Medicare initiated a nonpayment policy for hospitalizations with a documented hospital-acquired condition to provide incentives to hospitals to reduce CLABSIs and other iatrogenic conditions in adults.10 The Patient Protection and Affordable Care Act (ACA) of 2010 mandated that Medicaid adopt a similar nonpayment policy. The Medicaid provider-preventable conditions (PPCs) were created in 2011 by CMS as a set of hospital-acquired complications for which hospitals would not be reimbursed for Medicaid patients.11
There is a lack of data regarding the accuracy of ICD-9-CM code 999.31. Therefore, researchers at 3 children’s hospitals began working with the Children’s Hospital Association to compare administrative reporting of CLABSI to infection control clinician reporting of CLABSI to the Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN). Although the NHSN reporting is voluntary, all 3 of these hospitals report all CLABSIs into the system. The authors tested the hypothesis that the PPC criteria would identify more CLABSIs than those identified by using the strict NHSN criteria because the NHSN criteria were more restrictive than the PPC criteria.
Patient discharges from January 1 to December 31, 2010, at University of Michigan CS Mott Children’s Hospital, Monroe Carell Jr. Children’s Hospital at Vanderbilt, and Seattle Children’s Hospital were examined for occurrences of CLABSIs. Identification of CLABSIs was done by using: (1) hospital administrative data and CMS PPC criteria: and (2) hospital infection control data and NHSN criteria (Table 1).
All 3 hospitals use medical record coders to assign ICD-9-CM codes to the diagnoses documented by clinicians as part of their standard universal billing submission process. Medical record coders have specialized training in coding and must follow strict standards and are audited for accuracy. Coders review the hospital chart and assign codes according to the clinician notes. They can only code what a clinician has specifically noted as a diagnosis and cannot extrapolate from laboratory values or vague references. In the case of a CLABSI (ICD-9-CM code 999.31 during our study), clinicians must designate that a bloodstream infection is “line associated” or use terminology that specifically associates the infection with the central catheter. If only terms such as “sepsis,” “bacteremia,” or “bloodstream infection” are designated, the code 999.31 cannot be used. Once patient chart coding was completed at the individual hospitals, copies of the information were submitted to the Children’s Hospital Association Comparative Case Mix Data Program. CLABSIs were identified from the Case Mix database according to CMS PPC methods, which use the 999.31 ICD-9-CM diagnosis code.
All the children’s hospitals involved in the study also report CLABSIs to the NHSN in a similar manner. The children’s hospitals’ microbiology laboratories all produce automated reports of positive blood culture results. Lists of these results are sent to the infection control professionals, who then review the cultures and the charts of the patients identified to judge whether the cultures fulfill the standardized NHSN criteria for CLABSI infection (Table 1). Data for patients discharged in 2010 were also collected from the infection control departments by the investigators in each hospital identifying CLABSIs submitted to the NHSN.
The number of CLABSIs identified by both methods were entered in a 2 × 2 table and analyzed for sensitivity (ie, the probability that the test result will be positive when the condition is present), specificity (ie, the probability that the test will be negative when the condition is absent), positive predictive value (ie, the probability that the condition will be present when the test is positive), and negative predictive value (ie, the probability that the condition will be absent when the test is negative). For comparison purposes, hospital CLABSIs identified according to infection control by using the NHSN criteria were used as the standard; identification via ICD-9-CM administrative coding and PPC criteria was the test being examined.
As part of the study, additional chart reviews were done by the 3 investigators on the patients who had been designated as having a CLABSI. For those infections identified according to NHSN criteria, reviews independently documented whether the positive culture results were from a patient with a central line and whether the NHSN criteria were fulfilled and matched the infection control designation. Reviewers documented when a non-“line-associated” term was used to identify the infection (eg, “bacteremia,” “sepsis,” “bloodstream infection”) in the discharge summary. However, because chart review of all clinical notes was not performed, only qualitative statements can be made concerning use of other terms for infection. Patients with a CLABSI identified by using PPC criteria were also reviewed and evaluated as to whether documentation could be found to fulfill the NHSN criteria. Descriptive analyses were performed on the identified CLABSIs to better understand patient demographic and clinical characteristics.
Total discharges from the 3 children’s hospitals in 2010 were 35 698. The number of CLABSIs reported via ICD-9-CM code 999.31 by using the CMS PPC method was 76. CLABSIs reported by hospital infection control departments and NHSN criteria were 138.
Among the 166 total CLABSIs identified (according to either method) in the study, 48 were identified by both administrative data and infection control, 90 by infection control only, and 28 by administrative data only. Using infection control NHSN reporting as the standard, the analysis illustrates that the CMS PPC method using administrative data was 99.92% specific and 34.78% sensitive, with a positive predictive value of 63.16% and a negative predictive value of 99.75% (Table 2).
Patients identified as having CLABSIs were remarkably similar, regardless of identification method. The “both administrative and infection control” and “infection control only” groups had more male subjects (56% and 52%, respectively) whereas the “administrative data only” group was equally split. All 3 groups were mainly white (60%, 57%, and 54%, respectively) and had Medicaid (48%, 57%, and 68%) as their most prevalent source of insurance. The median age for all groups was ∼1 year (Table 3). The primary reason for hospitalization was chemotherapy for the “both administrative and infection control” and “infection control only” groups (14% and 12%, respectively). For the “administrative data only” group, the primary reason for hospitalization was hematologic diagnosis except sickle cell (14%), followed closely by chemotherapy (11%). A majority of patients identified by using all 3 methods were clinically high risk and included those who were immunocompromised (42%, 42%, and 57%), malignancy (25%, 32%, and 43%), cardiac surgery (29%, 28%, and 11%), and chemotherapy (25%, 28%, and 46%). The median hospital length of stay for these cases was very high (56, 52, and 26 days), and hospital mortality rates ranged from 10% for the “both administrative and infection control” group to 18% for the “administrative data only” group (Table 4).
The PPC criteria, with its seemingly more subjective definition, did not identify more CLABSIs than the strict NHSN criteria as originally hypothesized. Qualitative review of the medical record revealed 3 major themes associated with the 90 instances of ICD-9-CM undercoding of infection control identified CLABSI. The first is that as part of the hospital’s infection control process, positive blood culture results were automatically noted and sent to infection control professionals to review against NHSN criteria; no such trigger exists to assist coders in their infection identification process. Second, physicians directly responsible for the patient often used terms such as “sepsis,” “bacteremia,” or “blood stream infection” in the discharge summaries, not specifically noting “line-associated bloodstream infection,” which is required for coders to assign ICD-9-CM code 999.31. Third, CLABSIs are counted only once in the administrative record even if a patient has multiple CLABSIs during the hospitalization. In contrast, NHSN reporting captures each CLABSI as a single event; therefore, multiple CLABSI may be captured during a single hospitalization.
Although there were fewer cases identified by using administrative data that were not identified according to NHSN criteria (n = 28), all instances seem related to the NHSN’s more restrictive definition. Some had to do with timing; for instance, a patient had a line placed while in the hospital, was discharged, and then readmitted a day later with a line infection that was identified before the 48-hour time interval required for NHSN to attribute the CLABSI to that hospitalization. There were also cases in which infections in other body sites ruled out an NHSN CLABSI but were identified as a CLABSI by the treating physicians and coded as such.
Our data indicate that administrative reporting of CLABSIs using ICD-9-CM code 999.31 and the CMS PPC method is discordant compared with CLABSIs identified by using NHSN criteria.
Results support previous work that shows most patients with CLABSIs are complex, with immunosuppression from malignancy, transplant, chronic gastrointestinal disease, underlying severe chronic illnesses, severe prematurity, or congenital heart disease who have very long hospitalizations. It is less likely that an acute patient who is otherwise healthy and has a length of stay <7 days will experience a CLABSI.14 These findings are significant because the CMS PPCs are not risk adjusted, and any comparative reporting may inappropriately represent hospitals with large oncology, transplant, neonatology, or congenital heart programs as having much higher rates compared with acute care hospitals without these programs. Similarly, when establishing CLABSI rates, currently available administrative data do not have the ability to eliminate patients without a central line in the denominator or to calculate the number of line days a patient has. Administrative data’s inability to standardize infections by device utilization may put hospitals that provide care to complex patients at a disadvantage given that this population of patients may require a high rate of central line use.
Given the similarity in demographic and clinical characteristics of patients identified according to both CLABSI methods, we conjecture that the identification differences are probably due to the identification processes themselves. The NHSN method is objective in that positive blood culture results are automatically reported and reviewed according to strict standardized criteria by infection control clinicians. In contrast to CLABSIs reported in administrative data, there are specific exclusions for the time of the infection compared with admission time, whether there is another infection present, and whether the bacterial cause has been sufficiently documented to fulfill NHSN criteria.
Clinicians providing medical treatment to patients use blood culture results to determine therapy; these results are not used to decide whether the patient has a NHSN-defined CLABSI. They similarly use terms such as “sepsis,” “bacteremia,” or “bloodstream infection” in the discharge summaries to describe the patient’s condition and treatment and may not specifically note that the infection is associated with a central line. However, phrasing identifying the infection as a “line-associated bloodstream infection” is required for hospital medical coders to assign ICD-9-CM 999.31 to the case, and this code is essential for any administrative data–based method of identifying CLABSIs, such as the CMS PPCs. Although the current study did not specifically collect quantifiable data to support our conjecture, the theory is supported by another study focusing on neonatal infections.15
Educational programs in the hospital may improve clinician documentation of CLABSIs in discharge summaries, but it is unlikely this method would reach the consistency and objectivity of trained infection control personnel who directly report to the NHSN. Another approach would be to have the infection control professionals document in the chart that criteria for a CLABSI has been fulfilled, which would then allow the coders to use 999.31. However, this option might be difficult to implement because infection control designation of a CLABSI event purposely has been independent of clinical designations by the attending physicians to avoid any possible influence on event reporting.
The alternative method for reporting CLABSIs to appropriately document epidemiology and comparative rates is through the NHSN, which is a secure Internet-based surveillance system set up in 2008 for voluntary reporting of all hospital-associated infections. This system is dependent on automated reporting of positive blood culture results, standard definitions, numerators and denominators, and calculation of rates with risk adjustment applied.16 Hospitals who participate are guaranteed confidentiality, and there is no fee required. Participants receive routine reports showing their own data compared with aggregate data from all other similar facilities. According to the NHSN, there are currently 52 children’s hospitals participating in their CLABSI event-reporting program.
Our study does have some potential limitations that are important to discuss. Only 3 children’s hospitals participated in this study, which could affect the generalizability of the results. Similar patterns of CLABSI identification by using the 2 methods was seen at each hospital, but it is possible that this pattern would not be present at all children’s hospitals in the United States. Research bias could also be present because clinicians at each of the children’s hospitals were reviewing their data. A common abstraction form and frequent communications were used to minimize this potential issue and to increase consistency in the chart review process.
Finally, the study compared CLABSIs identified by 2 distinct methods, and the authors assumed that the individuals charged with collecting, reviewing, and entering the information are doing so as specified by their specific guidelines. The coding of infections could have been biased by the movement toward pay for performance. Once clinicians and coders knew that there would be financial implications for reported CLABSIs, it is possible that there was a choice to not specifically designate and code them.17 Our research cannot answer that question because we did not survey coders or clinicians to try to ascertain potential bias. However, it should be noted that the study was done on 2010 discharges, and the regulations for withholding payment for Medicaid were not in effect until 2012.
It is our opinion that CLABSI comparisons in facilities that care for children should emphasize use of information reported to the NHSN. Given the impact of CLABSIs in the pediatric population, we feel that the reporting of CLABSIs to the NHSN is preferable with its ability to provide rates per line days versus counts of events which could bias hospitals that have more complex patients with lines in place. Farmer et al17 advocate for use of the NHSN along with suggestions on how to improve the system, including selective auditing of reports and research access to the data. In an earlier article, Perla et al18 also support the NHSN but have a number of suggestions to improve reliability and validity of the data. NHSN data are also reported according to patient characteristics, which could help institutions identify certain clinical groups in need of specific interventions. Practitioners would also have real-time access to clinical data that could inform them of pathogen patterns, resistance to antibiotics, and to hospitals’ best practices. As state Medicaid programs begin to implement policies for nonpayment of CLABSIs, they should be mindful that administrative data alone do not reliably report actual CLABSIs.
The authors acknowledge the following individuals for their project support: Jackie Smith, RN, and Tanya Boswell, RN (Infection Control and Prevention, Monroe Carell Jr. Children’s Hospital at Vanderbilt), Joan Heath, RN (Infection Prevention, Seattle Children’s Hospital), Jackie Shaffer, MPH (Department of Infection Control and Epidemiology, University of Michigan Health System), Terri Stillwell, MD (Department of Pediatrics, University of Michigan Health System), and Josh Wenk (Children’s Hospital Association).
FINANCIAL DISCLOSURE: Drs Gay, Neff, and Sedman serve as medical consultants to the Children’s Hospital Association and receive funding support for their activities in this role; the other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
This study is solely the work of the authors and does not represent the opinion of the Children’s Hospital Association.
- Patient Protection and Affordable Care Act
- Agency for Healthcare Research and Quality
- Centers for Disease Control and Prevention
- central line–associated bloodstream infections
- Centers for Medicare & Medicaid Services
- National Healthcare Safety Network
- International Classification of Diseases, Ninth Revision, Clinical Modification
- present on admission
- provider-preventable conditions
- patient safety indicators
- Richards MJ,
- Edwards JR,
- Culver DH,
- Gaynes RP
- Miller MR,
- Griswold M,
- Harris JM II.,
- et al
- Miller MR,
- Niedner MF,
- Huskins WC,
- et al
- Sedman A,
- Harris JM II.,
- Schulz K,
- et al
- Scanlon MC,
- Harris JM,
- Levy F,
- Sedman A
- 8.↵CMS Manual System. Pub 100–04 Medicare Claims Processing, Transmittal 1269. Centers for Medicare and Medicaid Services Web site. Available at: www.cms.gov/Regulations-and-Guidance/Guidance/Transmittals/downloads/R1269CP.pdf. Accessed December 18, 2012.
- 9.↵National Quality Measures Clearinghouse (NQMC). Central venous catheter-related blood stream infections (provider-level): rate per 1,000 discharges. Agency for Healthcare Research and Quality Web site. Available at: www.qualitymeasures.ahrq.gov/content.aspx?id=26522. Updated June 12, 2010. Accessed December 18, 2012.
- 12.↵Conditions H-A. Centers for Medicare and Medicaid Services Web site. Available atwww.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/HospitalAcqCond/Hospital-Acquired_Conditions.html. Updated September 20, 2012. Accessed December 18, 2012.
- 13.↵Patient Safety Component Manual NHSN. Central line-associated bloodstream infection (CLABSI) event. Centers for Disease Control and Prevention Web site. Available at: www.cdc.gov/nhsn/pdfs/pscmanual/4psc_clabscurrent.pdf. Updated February 27, 2012. Accessed December 18, 2012.
- Patrick SW,
- Davis MM,
- Sedman AB,
- et al
- 16.↵National Healthcare Safety Network (NHSN). Centers for Disease Control and Prevention Web site. Available at: www.cdc.gov/nhsn/. Updated September 18, 2012. Accessed December 18, 2012.
- Copyright © 2013 by the American Academy of Pediatrics