Skip to main content

Advertising Disclaimer »

Main menu

  • Journals
    • Pediatrics
    • Hospital Pediatrics
    • Pediatrics in Review
    • NeoReviews
    • AAP Grand Rounds
    • AAP News
  • Authors/Reviewers
    • Submit Manuscript
    • Author Guidelines
    • Reviewer Guidelines
    • Editorial Policies
    • Open Access
  • Content
    • Current Issue
    • Online First
    • Archive
    • Topic/Program Collections
    • Blog
  • Subscribe
  • Alerts
  • Careers
  • Other Publications
    • American Academy of Pediatrics

User menu

  • Log in
  • My Cart

Search

  • Advanced search
American Academy of Pediatrics

AAP Gateway

Advanced Search

AAP Logo

  • Log in
  • My Cart
  • Journals
    • Pediatrics
    • Hospital Pediatrics
    • Pediatrics in Review
    • NeoReviews
    • AAP Grand Rounds
    • AAP News
  • Authors/Reviewers
    • Submit Manuscript
    • Author Guidelines
    • Reviewer Guidelines
    • Editorial Policies
    • Open Access
  • Content
    • Current Issue
    • Online First
    • Archive
    • Topic/Program Collections
    • Blog
  • Subscribe
  • Alerts
  • Careers
American Academy of Pediatrics
Brief Reports

Utility of Birth Certificate Data for Evaluating Hospital Variation in Admissions to NICUs

Eman S. Haidari, Henry C. Lee, Jessica L. Illuzzi, Haiqun Lin and Xiao Xu
Hospital Pediatrics February 2020, 10 (2) 190-194; DOI: https://doi.org/10.1542/hpeds.2019-0116
Eman S. Haidari
aNeonatal-Perinatal Medicine Fellowship Program, Maternal and Child Health Research Institute, and
bDepartment of Pediatrics, Stanford Medicine, Stanford University, Stanford, California; and
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Henry C. Lee
bDepartment of Pediatrics, Stanford Medicine, Stanford University, Stanford, California; and
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jessica L. Illuzzi
cDepartment of Obstetrics, Gynecology, and Reproductive Sciences, School of Medicine and
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Haiqun Lin
dDepartment of Biostatistics, School of Public Health, Yale University, New Haven, Connecticut
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiao Xu
cDepartment of Obstetrics, Gynecology, and Reproductive Sciences, School of Medicine and
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • Comments
Loading
Download PDF

Abstract

OBJECTIVES: Efforts to study potential overuse of NICU admissions and hospital variation in practice are often hindered by a lack of an appropriate data source. We examined the concordance of hospital-level NICU admission rates between birth certificate data and California Children’s Services (CCS) data to inform the utility of birth certificate data in studying hospital variation in NICU admissions.

METHODS: We analyzed birth certificate data from California in 2012 and hospital-specific summary data from CCS regarding NICU admissions. NICU admission rates were calculated for both data sets while using CCS data as the gold standard. The difference between birth certificate–based and CCS-based NICU admission rates was assessed by using the Wilcoxon signed rank test, and concordance between the 2 rates was evaluated by using Lin’s concordance correlation coefficient and Kendall’s W concordance coefficient.

RESULTS: Among a total of 103 hospitals that were linked between the 2 data sets, birth certificate data generally underreported NICU admission rates compared with CCS data (median = 7.72% vs 11.51%; P < .001). However, in a subset of 35 hospitals where the difference in NICU admission rates between the 2 data sets was small, the birth certificate–based NICU admission rate showed good concordance with the rate from CCS data (Lin’s concordance correlation coefficient = 0.91; 95% confidence interval: 0.84–0.95; Kendall’s W concordance coefficient = 0.99; P < .001). Hospitals with good-concordance data did not differ from other hospitals in the institutional characteristics assessed.

CONCLUSIONS: For a selected subset of hospitals, birth certificate data may offer a reasonable means to investigate hospital variation in NICU admissions.

The rate of newborn admissions to NICUs increased from 64.0 per 1000 live births in 2007 to 77.9 per 1000 live births in 2012 in the United States.1 Although reasons for this increase remain largely unknown, a recent study showed that 79.2% of NICU admissions in California in 2015 were infants of ≥34 weeks’ gestation, and only 11.9% of these were classified as high illness acuity.2 Such high use rates among late-preterm infants and low illness acuity raise concerns for possible overuse of NICUs and potential unnecessary increases in health care costs, stress to families, and likelihood of iatrogenic problems.1–5

Furthermore, NICU inborn admission rates differ considerably across hospitals. For instance, among California hospitals, the rate of NICU admissions varied by 34-fold in 2015, ranging from 1.1% to 37.7%.2 Reasons for this variation are not yet clear, but the magnitude of the variation suggests that it is unlikely to be solely due to differences in infants’ illness acuity across hospitals. Research to examine and explain such variation in NICU admissions may inform opportunities to reduce misuse.

These research efforts have been hindered by limited availability of data measuring NICU admissions across institutions. Birth certificates contain a data element measuring NICU admission and may offer a potential means for research in this area because birth certificate data can often be linked to other data sources containing diagnoses and procedures. Although multiple studies have evaluated the quality of birth certificate data elements,6–11 the accuracy of the NICU admission measure on birth certificates remains largely unknown. To address this knowledge gap, we examined the concordance of hospital-level NICU admission rates between birth certificate data and California Children’s Services (CCS) data to inform the utility of birth certificate data in studying hospital variation in NICU admissions.

Methods

We obtained linked birth cohort files from the California Office of Statewide Health Planning and Development, which contained Vital Statistics birth (VSB) (ie, birth certificate) data elements for all live births in the state with linked information from hospital discharge records. Data from 2012 were used for this analysis because it was the latest year the linked birth cohort file was available. We used a standard birth certificate data element in VSB reporting “NICU admission,” which indicates whether an infant was admitted to a level III or IV NICU during the birth hospitalization.12

All hospitals with an accredited NICU in California are required to report data on live births and NICU admissions to CCS. Hospitals self-report these data to the California Perinatal Quality Care Collaborative (CPQCC), which facilitates the data collection for CCS. We acquired hospital-specific summary data from the CPQCC regarding the number of inborn live births and NICU admissions among inborn live births in 2012. We assumed that the CCS data provided more accurate information on NICU admissions and used CCS data as the gold standard in analysis.

The VSB and CCS data were linked by hospital. There were a total of 103 hospitals in the CCS data with level III or IV NICUs that had at least 1 inborn live birth and confirmed information on birth volume. We successfully linked all 103 hospitals between the 2 data sources. This study was approved by the Yale University Human Investigation Committee, Stanford University Administrative Panel for the Protection of Human Subjects, and California Committee for the Protection of Human Subjects.

For each hospital, we calculated its NICU admission rate: (number of NICU admissions among inborn live births at this hospital/number of inborn live births at this hospital) × 100. Separate calculations were made for VSB data and CCS data. To evaluate the accuracy of the VSB-based admission rate, we calculated the difference between the VSB-based NICU admission rate and CCS-based rate and assessed its statistical significance using the Wilcoxon signed rank test. Additionally, to evaluate whether the VSB-based rate may reasonably inform a hospital’s performance on NICU admissions, we examined Lin’s concordance correlation coefficient and Kendall’s W concordance coefficient. Lin’s concordance correlation coefficient measures how well the VSB-based NICU admission rate agrees with the CCS-based NICU admission rate (ie, concordance between rates),13 whereas Kendall’s W concordance coefficient is a nonparametric statistic reflecting how well hospital ranks that are based on the 2 rates agree with each other (ie, concordance between ranks).

We considered Lin’s concordance correlation coefficient ≥0.90 as the threshold for good concordance (versus poor concordance) between the VSB- and CCS-based NICU admission rates14 and identified a subset of hospitals that demonstrated good concordance between the 2 rates. This was achieved by calculating and comparing Lin’s concordance correlation coefficients for subsets of hospitals when incrementally restricting to hospitals that had a smaller difference between VSB- and CCS-based rates (eg, ≤4 percentage points, ≤3 percentage points, etc). We compared institutional characteristics between good- and poor-concordance hospitals using χ2 tests and Wilcoxon rank sum tests. Institutional characteristics assessed included teaching status, ownership type, multihospital system affiliation, NICU level, and delivery volume. NICU level was based on hospital self-designated information submitted to the CPQCC and reflected the highest-acuity nursery within each hospital. Delivery volume was based on CCS data. Other measures were obtained from the 2012 American Hospital Association Annual Survey Database.15

P <.05 was considered statistically significant. Analyses were performed by using SAS version 9.4 (SAS Institute, Inc, Cary, NC) and SPSS version 26 (IBM SPSS Statistics, IBM Corporation).

Results

Among the 103 hospitals in the overall sample, NICU admission rates from the VSB data (median = 7.72%; interquartile range [IQR]: 5.27%–10.93%) were lower than those from the CCS data (median = 11.51%; IQR: 8.80%–14.64%; P < .001; Table 1). NICU admission rates based on these 2 data sources had a Lin’s concordance correlation coefficient of 0.33 (95% confidence interval [CI]: 0.19–0.46) and Kendall’s W concordance coefficient of 0.78 (P < .001).

View this table:
  • View inline
  • View popup
TABLE 1

Comparison of NICU Admission Rates Between VSB and CCS Data Among 103 Hospitals in California That Had a Level III or Level IV NICU in 2012

When restricting to a subset of 35 hospitals where the difference in NICU admission rates between VSB and CCS data was no more than 2.1 percentage points, there was good concordance between the 2 data sources. Among these hospitals, Lin’s concordance correlation coefficient was 0.91 (95% CI: 0.84–0.95) and Kendall’s W concordance coefficient was 0.99 (P < .001; Table 1, Fig 1).

FIGURE 1
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 1

Concordance in NICU admission rates between VSB and CCS data. A, All hospitals. B, Good-concordance hospitals. C, Poor-concordance hospitals.

The 35 hospitals with good-concordance data are diverse in characteristics, with 51.4% being teaching hospitals, 77.1% having private nonprofit ownership, 77.1% being affiliated with a multihospital system, and 88.6% and 11.4% having a level III and IV NICU, respectively (Table 2). Their annual delivery volume ranged from 1277 to 6465. These hospitals did not differ significantly from poor-concordance hospitals in the institutional characteristics assessed.

View this table:
  • View inline
  • View popup
TABLE 2

Comparison of Institutional Characteristics Between Hospitals With Good Versus Poor Concordance in NICU Admission Rates Based on VSB and CCS Data

Discussion

Our data showed that birth certificates tended to underreport NICU admissions. This may be because of late-onset NICU admissions after birth certificates have been completed, poor recording of transient NICU care, or possible differences in the definition of NICU bed levels across hospitals. The large variation across hospitals in the accuracy of the NICU admission measure found in our study is consistent with other research showing variation in the sensitivity of this measure in identifying NICU admissions across states.16

However, in a selected subset of hospitals, we found good concordance between VSB-based and CCS-based NICU admission rates. Birth certificates from these hospitals may provide a reasonable means for studying NICU admissions. Although these good-concordance hospitals only reflect one-third of all hospitals assessed, they have diverse characteristics that are comparable to other hospitals. This finding is important because birth certificate data are often accessible across hospitals and linkable to diagnosis and procedure information in hospital discharge records. Linked birth certificate and hospital discharge data from these good-concordance hospitals can offer an important opportunity for studying infant diagnoses associated with NICU use and help identify possible overuse, underuse, or misuse.

Despite poor accuracy in documenting the exact rate of NICU admissions, hospital rank based on birth certificate data demonstrated reasonable concordance with CCS data. Kendall’s W concordance coefficient (measuring the extent of agreement between hospital ranks that are based on NICU admission rates from birth certificates and the gold standard CCS data) was 0.78 in the overall sample and 0.99 in the subset of good-concordance hospitals. This supports another potential use of birth certificate data: to reflect the relative performance of a hospital on NICU admissions across institutions. The data may be used as a performance metric for benchmarking hospitals or facilitating more targeted research (eg, in-depth evaluation of hospitals with high versus low NICU use).

Our analysis was based on data in 2012 from a single state. Because data quality may have improved in more recent years or differ across geographic regions, continued monitoring and evaluation of birth certificate data are warranted. We also recognize that our analysis was an ecological evaluation of NICU admission at the hospital level, not at the individual patient level. Although further efforts are needed to enhance accuracy, birth certificate data provide important information and opportunities for studying newborn care. Identifying hospitals with a good-quality birth certificate measure of NICU admissions and using their data to understand differences in hospital practices may offer a reasonable means to inform strategies for improving NICU services.

Footnotes

  • Dr Lin’s current affiliation is Division of Nursing Science, Rutgers School of Nursing, Rutgers University, Newark, NJ

  • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

  • FUNDING: This project was supported by grant number R01HS023801 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Dr. Haidari was supported by the Stanford Maternal and Child Health Research Institute.

  • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

References

  1. ↵
    1. Harrison W,
    2. Goodman D
    . Epidemiologic trends in neonatal intensive care, 2007-2012. JAMA Pediatr. 2015;169(9):855–862
    OpenUrl
  2. ↵
    1. Schulman J,
    2. Braun D,
    3. Lee HC,
    4. et al
    . Association between neonatal intensive care unit admission rates and illness acuity. JAMA Pediatr. 2018;172(1):17–23
    OpenUrl
    1. Hynan MT,
    2. Mounts KO,
    3. Vanderbilt DL
    . Screening parents of high-risk infants for emotional distress: rationale and recommendations. J Perinatol. 2013;33(10):748–753
    OpenUrlCrossRefPubMed
    1. Wigert H,
    2. Johansson R,
    3. Berg M,
    4. Hellström AL
    . Mothers’ experiences of having their newborn child in a neonatal intensive care unit. Scand J Caring Sci. 2006;20(1):35–41
    OpenUrlCrossRefPubMed
  3. ↵
    1. Polin RA,
    2. Denson S,
    3. Brady MT
    ; Committee on Fetus and Newborn; Committee on Infectious Diseases. Epidemiology and diagnosis of health care-associated infections in the NICU. Pediatrics. 2012;129(4). Available at: www.pediatrics.org/cgi/content/full/129/4/e1104
  4. ↵
    1. Josberger RE,
    2. Wu M,
    3. Nichols EL
    . Birth certificate validity and the impact on primary cesarean section quality measure in New York state. J Community Health. 2019;44(2):222–229
    OpenUrl
    1. Northam S,
    2. Knapp TR
    . The reliability and validity of birth certificates. J Obstet Gynecol Neonatal Nurs. 2006;35(1):3–12
    OpenUrlCrossRefPubMed
    1. Reichman NE,
    2. Schwartz-Soicher O
    . Accuracy of birth certificate data by risk factors and outcomes: analysis of data from New Jersey. Am J Obstet Gynecol. 2007;197(1):32.e1–32.e8
    OpenUrlCrossRefPubMed
    1. DiGiuseppe DL,
    2. Aron DC,
    3. Ranbom L,
    4. Harper DL,
    5. Rosenthal GE
    . Reliability of birth certificate data: a multi-hospital comparison to medical records information. Matern Child Health J. 2002;6(3):169–179
    OpenUrlCrossRefPubMed
    1. Kjerulff KH,
    2. Attanasio LB
    . Validity of birth certificate and hospital discharge data reporting of labor induction. Womens Health Issues. 2018;28(1):82–88
    OpenUrl
  5. ↵
    1. Dietz P,
    2. Bombard J,
    3. Mulready-Ward C,
    4. et al
    . Validation of selected items on the 2003 U.S. standard certificate of live birth: New York City and Vermont. Public Health Rep. 2015;130(1):60–70
    OpenUrlCrossRefPubMed
  6. ↵
    National Vital Statistics System. Guide to completing the facility worksheets for the certificate of live birth and report of fetal death. 2016. Available at: https://www.cdc.gov/nchs/data/dvs/GuidetoCompleteFacilityWks.pdf. Accessed September 12, 2019
  7. ↵
    1. Watson PF,
    2. Petrie A
    . Method agreement analysis: a review of correct methodology. Theriogenology. 2010;73(9):1167–1179
    OpenUrlCrossRefPubMed
  8. ↵
    1. McBride RB,
    2. McBride G,
    3. McBride GB,
    4. et al
    . A proposal for strength-of-agreement criteria for Lin’s concordance correlation coefficient. 2005. Available at: https://www.scienceopen.com/document?vid=e3cfffed-a777-439b-b244-b4acbaf7f0c7. Accessed March 11, 2019
  9. ↵
    American Hospital Association. AHA annual survey database. Available at: https://www.ahadata.com/aha-annual-survey-database-asdb/. Accessed September 17, 2019
  10. ↵
    1. Martin JA,
    2. Wilson EC,
    3. Osterman MJ,
    4. Saadi EW,
    5. Sutton SR,
    6. Hamilton BE
    . Assessing the quality of medical and health data from the 2003 birth certificate revision: results from two states. Natl Vital Stat Rep. 2013;62(2):1–19
    OpenUrlPubMed
  • Copyright © 2020 by the American Academy of Pediatrics
View Abstract
PreviousNext
Back to top

Advertising Disclaimer »

In this issue

Hospital Pediatrics: 10 (2)
Hospital Pediatrics
Vol. 10, Issue 2
1 Feb 2020
  • Table of Contents
  • Table of Contents (PDF)
  • Index by author
View this article with LENS
PreviousNext
Email Article

Thank you for your interest in spreading the word on American Academy of Pediatrics.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Utility of Birth Certificate Data for Evaluating Hospital Variation in Admissions to NICUs
(Your Name) has sent you a message from American Academy of Pediatrics
(Your Name) thought you would like to see the American Academy of Pediatrics web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Request Permissions
Article Alerts
Log in
You will be redirected to aap.org to login or to create your account.
Or Sign In to Email Alerts with your Email Address
Citation Tools
Utility of Birth Certificate Data for Evaluating Hospital Variation in Admissions to NICUs
Eman S. Haidari, Henry C. Lee, Jessica L. Illuzzi, Haiqun Lin, Xiao Xu
Hospital Pediatrics Feb 2020, 10 (2) 190-194; DOI: 10.1542/hpeds.2019-0116

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Utility of Birth Certificate Data for Evaluating Hospital Variation in Admissions to NICUs
Eman S. Haidari, Henry C. Lee, Jessica L. Illuzzi, Haiqun Lin, Xiao Xu
Hospital Pediatrics Feb 2020, 10 (2) 190-194; DOI: 10.1542/hpeds.2019-0116
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Print
Download PDF
Insight Alerts
  • Table of Contents

Jump to section

  • Article
    • Abstract
    • Methods
    • Results
    • Discussion
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • Comments

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Validation of Administrative Codes for Palliative Care Consultation Among Critically Ill Children
  • Quantifying Electronic Health Record Data: A Potential Risk for Cognitive Overload
  • Identifying Children With Medical Complexity From the National Survey of Children’s Health Combined 2016–17 Data Set
Show more Brief Reports

Similar Articles

Subjects

  • Fetus/Newborn Infant
    • Fetus/Newborn Infant
    • Neonatology
  • Journal Info
  • Editorial Board
  • Editorial Policies
  • Overview
  • Licensing Information
  • Authors/Reviewers
  • Author Guidelines
  • Reviewer Guidelines
  • Submit My Manuscript
  • Open Access
  • Librarians
  • Institutional Subscriptions
  • Usage Stats
  • Support
  • Subscribe
  • Contact Us
  • Resources
  • Media Kit
  • About
  • International Access
  • Terms of Use
  • Privacy Statement
  • FAQ
  • RSS Feeds
  • shopAAP
  • AAP.org
  • Follow American Academy of Pediatrics on Instagram
  • Visit American Academy of Pediatrics on Facebook
  • Follow American Academy of Pediatrics on Twitter
  • Follow American Academy of Pediatrics on Youtube
American Academy of Pediatrics

© 2021 American Academy of Pediatrics