Abstract
OBJECTIVES: Efforts to study potential overuse of NICU admissions and hospital variation in practice are often hindered by a lack of an appropriate data source. We examined the concordance of hospital-level NICU admission rates between birth certificate data and California Children’s Services (CCS) data to inform the utility of birth certificate data in studying hospital variation in NICU admissions.
METHODS: We analyzed birth certificate data from California in 2012 and hospital-specific summary data from CCS regarding NICU admissions. NICU admission rates were calculated for both data sets while using CCS data as the gold standard. The difference between birth certificate–based and CCS-based NICU admission rates was assessed by using the Wilcoxon signed rank test, and concordance between the 2 rates was evaluated by using Lin’s concordance correlation coefficient and Kendall’s W concordance coefficient.
RESULTS: Among a total of 103 hospitals that were linked between the 2 data sets, birth certificate data generally underreported NICU admission rates compared with CCS data (median = 7.72% vs 11.51%; P < .001). However, in a subset of 35 hospitals where the difference in NICU admission rates between the 2 data sets was small, the birth certificate–based NICU admission rate showed good concordance with the rate from CCS data (Lin’s concordance correlation coefficient = 0.91; 95% confidence interval: 0.84–0.95; Kendall’s W concordance coefficient = 0.99; P < .001). Hospitals with good-concordance data did not differ from other hospitals in the institutional characteristics assessed.
CONCLUSIONS: For a selected subset of hospitals, birth certificate data may offer a reasonable means to investigate hospital variation in NICU admissions.
The rate of newborn admissions to NICUs increased from 64.0 per 1000 live births in 2007 to 77.9 per 1000 live births in 2012 in the United States.1 Although reasons for this increase remain largely unknown, a recent study showed that 79.2% of NICU admissions in California in 2015 were infants of ≥34 weeks’ gestation, and only 11.9% of these were classified as high illness acuity.2 Such high use rates among late-preterm infants and low illness acuity raise concerns for possible overuse of NICUs and potential unnecessary increases in health care costs, stress to families, and likelihood of iatrogenic problems.1–5
Furthermore, NICU inborn admission rates differ considerably across hospitals. For instance, among California hospitals, the rate of NICU admissions varied by 34-fold in 2015, ranging from 1.1% to 37.7%.2 Reasons for this variation are not yet clear, but the magnitude of the variation suggests that it is unlikely to be solely due to differences in infants’ illness acuity across hospitals. Research to examine and explain such variation in NICU admissions may inform opportunities to reduce misuse.
These research efforts have been hindered by limited availability of data measuring NICU admissions across institutions. Birth certificates contain a data element measuring NICU admission and may offer a potential means for research in this area because birth certificate data can often be linked to other data sources containing diagnoses and procedures. Although multiple studies have evaluated the quality of birth certificate data elements,6–11 the accuracy of the NICU admission measure on birth certificates remains largely unknown. To address this knowledge gap, we examined the concordance of hospital-level NICU admission rates between birth certificate data and California Children’s Services (CCS) data to inform the utility of birth certificate data in studying hospital variation in NICU admissions.
Methods
We obtained linked birth cohort files from the California Office of Statewide Health Planning and Development, which contained Vital Statistics birth (VSB) (ie, birth certificate) data elements for all live births in the state with linked information from hospital discharge records. Data from 2012 were used for this analysis because it was the latest year the linked birth cohort file was available. We used a standard birth certificate data element in VSB reporting “NICU admission,” which indicates whether an infant was admitted to a level III or IV NICU during the birth hospitalization.12
All hospitals with an accredited NICU in California are required to report data on live births and NICU admissions to CCS. Hospitals self-report these data to the California Perinatal Quality Care Collaborative (CPQCC), which facilitates the data collection for CCS. We acquired hospital-specific summary data from the CPQCC regarding the number of inborn live births and NICU admissions among inborn live births in 2012. We assumed that the CCS data provided more accurate information on NICU admissions and used CCS data as the gold standard in analysis.
The VSB and CCS data were linked by hospital. There were a total of 103 hospitals in the CCS data with level III or IV NICUs that had at least 1 inborn live birth and confirmed information on birth volume. We successfully linked all 103 hospitals between the 2 data sources. This study was approved by the Yale University Human Investigation Committee, Stanford University Administrative Panel for the Protection of Human Subjects, and California Committee for the Protection of Human Subjects.
For each hospital, we calculated its NICU admission rate: (number of NICU admissions among inborn live births at this hospital/number of inborn live births at this hospital) × 100. Separate calculations were made for VSB data and CCS data. To evaluate the accuracy of the VSB-based admission rate, we calculated the difference between the VSB-based NICU admission rate and CCS-based rate and assessed its statistical significance using the Wilcoxon signed rank test. Additionally, to evaluate whether the VSB-based rate may reasonably inform a hospital’s performance on NICU admissions, we examined Lin’s concordance correlation coefficient and Kendall’s W concordance coefficient. Lin’s concordance correlation coefficient measures how well the VSB-based NICU admission rate agrees with the CCS-based NICU admission rate (ie, concordance between rates),13 whereas Kendall’s W concordance coefficient is a nonparametric statistic reflecting how well hospital ranks that are based on the 2 rates agree with each other (ie, concordance between ranks).
We considered Lin’s concordance correlation coefficient ≥0.90 as the threshold for good concordance (versus poor concordance) between the VSB- and CCS-based NICU admission rates14 and identified a subset of hospitals that demonstrated good concordance between the 2 rates. This was achieved by calculating and comparing Lin’s concordance correlation coefficients for subsets of hospitals when incrementally restricting to hospitals that had a smaller difference between VSB- and CCS-based rates (eg, ≤4 percentage points, ≤3 percentage points, etc). We compared institutional characteristics between good- and poor-concordance hospitals using χ2 tests and Wilcoxon rank sum tests. Institutional characteristics assessed included teaching status, ownership type, multihospital system affiliation, NICU level, and delivery volume. NICU level was based on hospital self-designated information submitted to the CPQCC and reflected the highest-acuity nursery within each hospital. Delivery volume was based on CCS data. Other measures were obtained from the 2012 American Hospital Association Annual Survey Database.15
P <.05 was considered statistically significant. Analyses were performed by using SAS version 9.4 (SAS Institute, Inc, Cary, NC) and SPSS version 26 (IBM SPSS Statistics, IBM Corporation).
Results
Among the 103 hospitals in the overall sample, NICU admission rates from the VSB data (median = 7.72%; interquartile range [IQR]: 5.27%–10.93%) were lower than those from the CCS data (median = 11.51%; IQR: 8.80%–14.64%; P < .001; Table 1). NICU admission rates based on these 2 data sources had a Lin’s concordance correlation coefficient of 0.33 (95% confidence interval [CI]: 0.19–0.46) and Kendall’s W concordance coefficient of 0.78 (P < .001).
Comparison of NICU Admission Rates Between VSB and CCS Data Among 103 Hospitals in California That Had a Level III or Level IV NICU in 2012
When restricting to a subset of 35 hospitals where the difference in NICU admission rates between VSB and CCS data was no more than 2.1 percentage points, there was good concordance between the 2 data sources. Among these hospitals, Lin’s concordance correlation coefficient was 0.91 (95% CI: 0.84–0.95) and Kendall’s W concordance coefficient was 0.99 (P < .001; Table 1, Fig 1).
Concordance in NICU admission rates between VSB and CCS data. A, All hospitals. B, Good-concordance hospitals. C, Poor-concordance hospitals.
The 35 hospitals with good-concordance data are diverse in characteristics, with 51.4% being teaching hospitals, 77.1% having private nonprofit ownership, 77.1% being affiliated with a multihospital system, and 88.6% and 11.4% having a level III and IV NICU, respectively (Table 2). Their annual delivery volume ranged from 1277 to 6465. These hospitals did not differ significantly from poor-concordance hospitals in the institutional characteristics assessed.
Comparison of Institutional Characteristics Between Hospitals With Good Versus Poor Concordance in NICU Admission Rates Based on VSB and CCS Data
Discussion
Our data showed that birth certificates tended to underreport NICU admissions. This may be because of late-onset NICU admissions after birth certificates have been completed, poor recording of transient NICU care, or possible differences in the definition of NICU bed levels across hospitals. The large variation across hospitals in the accuracy of the NICU admission measure found in our study is consistent with other research showing variation in the sensitivity of this measure in identifying NICU admissions across states.16
However, in a selected subset of hospitals, we found good concordance between VSB-based and CCS-based NICU admission rates. Birth certificates from these hospitals may provide a reasonable means for studying NICU admissions. Although these good-concordance hospitals only reflect one-third of all hospitals assessed, they have diverse characteristics that are comparable to other hospitals. This finding is important because birth certificate data are often accessible across hospitals and linkable to diagnosis and procedure information in hospital discharge records. Linked birth certificate and hospital discharge data from these good-concordance hospitals can offer an important opportunity for studying infant diagnoses associated with NICU use and help identify possible overuse, underuse, or misuse.
Despite poor accuracy in documenting the exact rate of NICU admissions, hospital rank based on birth certificate data demonstrated reasonable concordance with CCS data. Kendall’s W concordance coefficient (measuring the extent of agreement between hospital ranks that are based on NICU admission rates from birth certificates and the gold standard CCS data) was 0.78 in the overall sample and 0.99 in the subset of good-concordance hospitals. This supports another potential use of birth certificate data: to reflect the relative performance of a hospital on NICU admissions across institutions. The data may be used as a performance metric for benchmarking hospitals or facilitating more targeted research (eg, in-depth evaluation of hospitals with high versus low NICU use).
Our analysis was based on data in 2012 from a single state. Because data quality may have improved in more recent years or differ across geographic regions, continued monitoring and evaluation of birth certificate data are warranted. We also recognize that our analysis was an ecological evaluation of NICU admission at the hospital level, not at the individual patient level. Although further efforts are needed to enhance accuracy, birth certificate data provide important information and opportunities for studying newborn care. Identifying hospitals with a good-quality birth certificate measure of NICU admissions and using their data to understand differences in hospital practices may offer a reasonable means to inform strategies for improving NICU services.
Footnotes
Dr Lin’s current affiliation is Division of Nursing Science, Rutgers School of Nursing, Rutgers University, Newark, NJ
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This project was supported by grant number R01HS023801 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Dr. Haidari was supported by the Stanford Maternal and Child Health Research Institute.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2020 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.