The Landscape of Quality Measures and Quality Improvement for the Care of Hospitalized Children in the United States: Efforts Over the Last Decade
- aDivision of Pediatric Hospital Medicine, Department of Pediatrics, Nicklaus Children’s Hospital, Miami, Florida; and
- bDepartment of Pediatrics, Herbert Wertheim College of Medicine, Florida International University, Miami, Florida
- Address correspondence to Mario A. Reyes, MD, Division of Pediatric Hospital Medicine, Department of Pediatrics, Nicklaus Children’s Hospital, 3100 SW 62nd Ave, Miami, FL 33155. E-mail:
Dr Reyes mostly performed the literature review, contributed to the review of the government Web sites related to the topic, led the manuscript writing, editing, and submission, and conceptualized and designed the study; Mr Paulus mostly performed the review of the government Web sites related to the topic, contributed to the literature review, was instrumental in the manuscript writing and editing, made key contributions to the design of the tables, and conceptualized and designed the study and wrote the manuscript; and all authors approved the final manuscript as submitted.
Current national health care expenditures are ∼$3 trillion, with ∼12% directed toward the provision of care to children.1 With ∼6 million children hospitalized every year, ∼40% of the costs of treating children is attributable to inpatient care.2 The moral and humanistic motivation to improve the quality of care of hospitalized children is consistent with the core ethical principles of beneficence and nonmaleficence. Also, economic drivers that are focused on value and efficiency for pediatric health care exist. As a result, the government, providers, hospitals, patients, families, insurers, and accreditation organizations are all interested in clinical and other relevant measures as critical tools to improve care.3,4
Quality measures (QMs) for assessment of the care provided to hospitalized children in the United States were remarkably sparse at the beginning of the century. As of 2006, few measure sets were exclusively designated to assess the quality of care for children, and none existed for hospital care.5 By to 2008, only 5% of the available QMs for children’s health care were devoted to inpatient care.2
However, over the last decade, multiple efforts have been undertaken by various stakeholders to develop and implement standardized pediatric quality metrics and metric sets, including for the care of hospitalized children. Simultaneously, many quality improvement (QI) programs and initiatives have taken place in this field. The authors of a study published recently in Pediatrics identified and classified QMs in pediatric health care that were available as of 2015.6 The authors found 257 “unique” metrics related to children’s health care. Of them, 38 indicators (14.7%) were related to hospital-based outcomes, mostly related to adverse conditions and/or events, hospital-acquired conditions (HACs), and mortality. Additionally, 26 metrics (10.1%) were dedicated to cardiac and surgical care, and 6 assessed transitions of care and/or discharges.6
Furthermore, in 2016, the Children’s Hospital Association (CHA) conducted a national survey to explore the exposure of children’s hospitals to programs of accountability and QM sets. More than 80 children’s hospitals participated in the survey. The preliminary results are staggering: children’s hospitals report to >1450 accountability programs for quality. The vast majority of the institutions report to several types of programs, including public reporting programs (98%); patient registries (94%); certification, designation, or award (76%); or value-based purchasing programs (70%).4 Hospitals report to many programs, often the same indicator is included in >1 set and is used for a variety of purposes.4,7
Clearly, an explosion of development of QMs for the assessment of care provided to hospitalized children has taken place over the last few years. However, this dramatic expansion has not occurred in a systematic and coordinated fashion. The proliferation of these indicators has led to an unbalanced portfolio, with some aspects of care receiving more attention than others, such as “never events,” HACs, and surgical complications.8 These important fields portray a dimension of quality that is focused on safety and harm prevention and measure low-prevalence conditions.
Despite the sheer amount of measures and measure sets available, QMs for evaluation of the care provided to hospitalized children with common and/or chronic conditions, as well as those who undergo common procedures, are lacking. These diseases, conditions, and procedures include respiratory illnesses (bronchiolitis, asthma, pneumonia, croup), evaluation of infants with fever, urinary tract infections, acute gastroenteritis and complications, skin and soft tissue infections, seizures and epilepsy, mental illnesses, pediatric injuries, and care (including coordination of care) of medically complex children, among others. These represent the bulk of pediatric hospitalizations. Venipunctures, electronic monitoring, appendectomies, tonsillectomies, and adenoidectomies represent examples of common procedures performed.
Here we describe some notable efforts undertaken over the last decade for the measuring, reporting, and improvement of the quality of care for hospitalized children and conclude with some recommendations and future directions. Our intent is not to provide a complete or detailed list of all existing QMs and contributions from a variety of stakeholders but rather to summarize the most relevant sources and initiatives in this arena. For the purposes of this article, the authors reviewed the most relevant articles in the literature on the topic and the current information provided on the Web sites of the government and other institutions involved in the field. Although the content may overlap, we have divided the article into 3 sections:
Programs and organizations dedicated to the development and implementation of QMs;
Advocates for the quality, safety, and QI of children’s health care; and
Initiatives geared toward accreditation, reporting, and improvement of the quality of hospital care.
Summary of Recent Efforts for QMs and QI for the Care of Hospitalized Children
Programs and Organizations Dedicated to the Development and Implementation of QMs
The Central Role of US Government Agencies: The Agency for Health Care Research and Quality
As part of the Health Care Cost and Utilization Project, the Agency for Health Care Research and Quality (AHRQ) released the first set of Pediatric Quality Indicators (PDIs) in 2006,9 which are derived from hospital discharge data. The 2015 version contains 24 indicators, including 16 provider-level indicators (ie, hospitals) and 8 area-level indicators (ie, populations) (Table 1). The provider-level metrics screen for adverse and undesirable outcomes that occur as the result of exposure to health care (hospitalization). These indicators provide a marker for patient safety and avoiding harm. Three are neonatal indicators, and 2 are related to cardiac surgery. Eight of the indicators are endorsed by the National Quality Forum (NQF).10 The PDIs set is considered a tool for screening the quality of care for the millions of hospitalized children each year. They address problems that may be amenable to prevention by changes at the system or provider level.9 Providers and hospitals use these metrics to identify QI opportunities and for benchmarking purposes, allowing interhospital comparisons.
The Children’s Health Insurance Program Reauthorization Act of 2009 mandated the development of a children’s health care core set of QMs for voluntary reporting by state Medicaid and the Children’s Health Insurance Program. The AHRQ released a list of 24 indicators called the Initial Core Set, which undergoes annual review, with the 2017 version now containing 27 metrics.11 However, only 2 of those indicators apply to the care provided to hospitalized children: pediatric central line-associated bloodstream infections and the patient experience.
The Children’s Health Insurance Program Reauthorization Act of 2009 also led to the implementation of the Pediatric Quality Measures Program (PQMP). In 2010, the AHRQ funded and tasked 7 centers of excellence, led by or partnered with prominent children’s hospitals, with improving and strengthening the initial core set and increasing the portfolio of evidence-based, consensus pediatric QMs. The measure sets are publicly available to payers, providers, and consumers.3,7,12,13 Today, the AHRQ Web site lists over 100 metrics produced by PQMP grantees.13 It is encouraging that more than half of those indicators apply to inpatient quality of care. The current portfolio of metrics pertains mostly to newborn and perinatal medicine, resource utilization in asthma, care of patients with sickle cell disease, mental health, nutritional assessment, and sepsis syndrome. Other populations- and state-reportable indicators address patient safety, medication reconciliation, and transition to adult care. At least a few QMs resulting from the PQMP address overuse of imaging in children (Table 2). This program represents the largest investment to date on pediatric QMs.12 The PQMP is funded through September 2017, and it is currently involved in a new phase, now assessing the feasibility and usability of the newly developed measures at multiple levels of care.7
The PDIs and the “core set” do not address the quality of care for high-volume, common, and/or chronic conditions or procedures performed in hospitals or overuse of health care resources. The PQMP sets selectively pertain to some conditions with high disease burden. In general, despite their limitations, these government-led programs are valid steps in the right direction for the next generation of metrics.
The Center for Medicare and Medicaid Services
In 2011, the Center for Medicare and Medicaid Services launched a program called Partnership for Patients14 as an attempt to reduce the prevalence of HACs and readmissions in US hospitals. One of the 26 hospital engagement networks is the Solutions for Patient Safety (SPS). This program includes >100 children’s hospitals, which submit outcomes and processes data on HACs and readmissions.15 SPS measures mostly iatrogenic conditions, listed below, and is not intended for comparative health care or public reporting at the provider level.
Adverse drug events
Catheter-associated urinary tract infections
Central line-associated bloodstream infections
Injuries from falls and immobility
Surgical site infections
Peripheral intravenous infiltration and extravasations
Clostridium difficile and antimicrobial stewardship
SPS is perhaps the most herculean effort to date by children’s hospitals across the nation to assess and improve safety for all hospitalized children. As of 2016, this initiative has resulted in a 40% reduction in HACs, 10% reduction in readmissions, and 25% reduction in serious safety events.15 However, standardized measurement of care for high-volume common and/or chronic conditions and procedures is not addressed by SPS. Another limitation of SPS is that participation is limited to large, free-standing children’s hospitals, where only between one-quarter and one-third of hospitalizations of children take place.16,17
The Patient and Family Experience as a Quality Metric
Patient and Family Centeredness is 1 of the 6 aims defined by the Institute of Medicine for improving the quality of health care in the 21st century.18 Enhancing the patient experience is one of the Institute for Healthcare Improvement’s (IHI) “Triple Aim” domains.19 The Consumer Assessment of Healthcare Provider and Services-Child Hospital Survey (Child HCAHPS) aims to assess the experience of pediatric patients 17 and younger and their parents or guardians with inpatient care.20
This tool was developed as a result of the PQMP and is already in use by >80 children’s hospitals who use a list of approved vendors to deliver the survey to a sample of discharged patients. It represents the best available QMs that have been scientifically developed and validated to assess this aim. NQF endorsed the Child HCAHPS as a QM in 2015,10 but the results are not publicly available.
Role of the NQF
The NQF is a nonpartisan, public-private organization in which >400 member organizations participate.10 NQF endorses consensus standards for performance measurement proposed by its members. Although it has a “metric incubator,” NQF does not propose or develop QMs itself; however, the NQF endorsement tag is considered recognition of rigorous design. By the end of 2016, the NQF had developed a library of ∼600 endorsed QMs. One in 5 (136 QMs) pertain to children’s health, and only 42 (∼7%) are pertinent to pediatric hospital care (Table 3). Surgical, perinatal and newborn care, intensive care, and HAC measures compose the bulk of these inpatient metrics. However, the majority of the common and/or chronic conditions treated and procedures performed in children’s hospitals have no NQF-endorsed metrics. Similarly, the management or coordination of care for children with medical complexity and overuse are significantly under-represented fields.
Advocates for the Quality, Safety, and QI of Children’s Health Care
Contribution of the IHI
The IHI has the mission of redesigning health care into a system without errors, waste, delay, or unsustainable costs.21 By 2010, the IHI proposed the Triple Aim as a national framework for tackling health care issues.18 This Triple Aim (improving the health of populations and outcomes, enhancing the patient experience, and reducing per capita cost) requires stakeholders to uniformly measure health care quality as a critical step toward achieving these ambitious goals.3
The IHI recognizes that measurement is a key element in the model for improvement. Currently, the IHI has a library of sample measures, many of them applicable to hospitals that care for sick children. Each metric has specific goals.20 The areas pertinent to pediatric hospital care include hospital throughput, emergency department processes, compliance with protocols for severe sepsis and other conditions, medication safety, and surgical site infections. Many of these metrics or their equivalents are also represented in the repository of metrics for other organizations.
Role of the American Academy of Pediatrics
The American Academy of Pediatrics (AAP) and its >65 000 members are committed to the highest quality and safest health care for all children. The AAP has partnered over time with the government, the American Medical Association, NQF, AHRQ, CHA, the Hospital Quality Alliance, and other stakeholders in many initiatives that are aimed at developing QMs.7,22
The AAP serves as a home for several groups and initiatives dedicated to QI and quality measurement. The AAP Section on Hospital Medicine has >2000 members, mostly pediatric hospitalists and other health care professionals with an active interest in improving the quality, safety, and efficiency of care provided to hospitalized infants and children.23
The AAP Council on Quality Improvement and Patient Safety provides a home for pediatricians with interest in QI and patient safety efforts. The Council on Quality Improvement and Patient Safety has a Measurement Subcommittee with the explicit goal of identifying, evaluating, and using appropriate QMs and represents the AAP with external stakeholders regarding the development of QMs.24
The AAP Quality Improvement Innovation Networks25 include >700 pediatricians. The Value in Inpatient Pediatrics (VIP) Network represents the inpatient arm of the Quality Improvement Innovation Networks.26 It provides a framework for multicenter collaborative projects among hospital-based physician practices that are aimed at improving value, decreasing variation in care and waste, and standardizing best practices. Recent projects have successfully had protocols for bronchiolitis and pneumonia implemented.27,28 In addition to its emphasis on overuse, a singular distinction of the VIP Network is that most of the participating centers are community hospitals, where >70% of pediatric hospitalizations in the United States take place.17 In 2016, the VIP started sponsoring the Reducing Excessive Variation in Infant Sepsis Evaluation collaborative project29 with 133 participating centers. The VIP Network tackles the quality of care provided for many of the most common inpatient pediatric conditions: bronchiolitis, pneumonia, urinary tract infections, and neonates with fever, and is currently working on asthma. The network has created and field tested several QMs related to these common conditions. These QMs should be given strong consideration as national benchmarking measures for pediatric hospital care.
In 2015, the AAP and the American Board of Pediatrics convened a forum, “Aligning Quality Measurement to the Future of Pediatric Practice.” This event revealed the necessity of developing meaningful process and accountability metrics at a practice level, including subspecialty care, a key component of the care of hospitalized children. It also recognized that the main challenge currently is in reaching a consensus regarding specific measures and measurement domains.30
In January 2017, the AAP published a new policy statement on the development and application of QMs. This policy statement provides a roadmap for quality measurements and QI in pediatric health care in general and applies to the care of hospitalized children.7
Contributions From the CHA
CHA is a consortium of 220 children’s hospitals with the mission of advancing child health through innovation in the quality, cost, and delivery of care.16 CHA maintains a well-respected database: the Pediatric Health Information System (PHIS), in which ∼50 large, free-standing children hospitals submit quarterly clinical and administrative data. After a rigorous quality analysis of the data, PHIS regularly generates >120 “Standard Reports,” including 9 clinical “Report Cards,”31 listed here:
Choosing Wisely Campaign
These reports address multiple conditions: specific measures including hospital throughput, emergency department performance, care of common and chronic conditions, HACs, readmission rates, and other important administrative and operational metrics of children’s hospitals. This information is only available to PHIS institution members who submit data, and it is not publicly available. The PHIS database contains details of >6 000 000 pediatric admissions, representing approximately one-third of all annual hospitalizations of children in the United States.16 This database is a paramount tool for detailed, longitudinal analysis of pediatric hospital care and a valuable source for benchmarking, research, and QI.
The CHA conducted the 2016 Children’s Hospital Survey on Accountability Programs and Quality Measures.4 One of the main goals of the survey was to define and identify quality policies, measurement priorities, and gaps in pediatric hospital care.4 The results are available to CHA institution members and will hopefully be available to the public in the future.
Initiatives Geared Toward Accreditation, Reporting, and Improvement of the Quality of Hospital Care
Efforts of The Joint Commission
The Joint Commission (TJC) developed the Children’s Asthma Core measure set in 2007.32 This set of 3 process indicators represents the most uniformly adopted set of disease-specific metrics in pediatric hospital care because they were a requirement for accreditation. They were considered “accountability metrics” appropriate for public reporting and pay-for-performance.33 However, there was not a clear correlation between sustained process improvement and any measurable outcomes.34 Compliance was typically high (at or near 100% in most hospitals), making a correlation between minute changes and final results difficult to assess. By January 2016, TJC removed the entire set from its accreditation requirements because of “consistently high performance rates.”32,35 Recently, TJC launched the new “Pioneers in Quality” program, which recognizes organizations that leverage the use of information technology (eg, electronic health records) for seamless quality measurement and QI.36 No children’s hospitals have been awarded this recognition yet.
Other Multicenter Collaborative Networks and Patient Registries
In addition to the AAP VIP network, other large-scale QI projects and registries pertinent to pediatric subspecialties or specific diseases and conditions are underway. The goal of these projects is to collect high-quality data for benchmarking purposes and dissemination of best practices. These collaborative networks and registries share best practices in quality and safety, produce care guidelines, and serve as critical benchmarking tools.37 Occasionally, quality indicators are derived from these registries and networks of subspecialty care. Existing clinical registries for pediatric-specific conditions and subspecialty care include the following4:
ImproveCareNow Network (Inflammatory Bowel Disease);
Cystic Fibrosis Foundation;
Vermont Oxford Network (global database for newborn medicine);
Pediatric Cardiac Critical Care Consortium;
The Society of Thoracic Surgery Congenital Heart Surgery Database;
IMproving Pediatric and Adult Congenital Treatments Registry (IMPACT);
Global Tracheostomy Collaborative Registry;
National Surgical Quality Improvement Program;
Pediatric Heart Transplant Study;
Virtual PICU Systems, LLC;
Society for Pediatric Anesthesia Wake Up Safe;
Extracorporeal Membrane Oxygenation Registry;
Collaborative Pediatric Critical Care Research Network;
Pediatric Emergency Care Applied Research Network;
Department of Defense Trauma Registry;
American Heart Association, Get With the Guidelines Resuscitation;
Implantable Cardioverter Defibrillators Registry;
National Healthcare Safety Network; and
Society for Pediatric Sedation Registry.
Pediatric Research in Inpatient Settings
Pediatric Research in Inpatient Settings is a pediatric hospitalist research network with the mission of improving the quality of health care delivered to hospitalized children.38 The AAP and CHA, among other stakeholders, contribute to the network. The standardization of transition of care at different hospital levels (including discharges) and the development of QMs for common pediatric respiratory illnesses have been addressed in recent work.37 The Pediatric Respiratory Illness Inpatient Measurement System is a QMs tool recently developed through a multicenter collaborative for asthma, bronchiolitis, croup, and pneumonia. More than 100 indicators were developed, demonstrating significant between-hospital variation, although less than one-fourth are supported by strong evidence.39 The authors are currently involved in a study that aims at validating and assessing usability of the indicators and their relationship with health-related, patient-reported outcomes.37
US News and World Report Survey
US News and World Report annually publishes a Top-50 list for each of 10 pediatric specialties and an honor roll for children’s hospitals.40 The report is highly publicized and requires a significant effort of data collection and reporting from the participating institutions. Clinical data are gathered from ∼180 pediatric centers and the opinion of >10 000 pediatric specialists through a detailed survey with metrics generated by the magazine. The results are meant to be used as guidance when life is on the line or an uncommon condition or procedure is involved.39 However, this list does not include a measure of the quality of care for common and/or chronic conditions, common procedures, or the patient experience. Also, the metrics selected are not peer review–endorsed and have been criticized for not truly representing overall quality of hospital care.41–43 Despite the fact that the patient experience is not included in this survey, this report is paradoxically a major driver of consumer perception of care provided by children’s hospitals in the United States.
The current landscape of QMs for health care delivered to hospitalized children is extensive and complex. Still, the portfolio of existing QMs is not well balanced. Few metrics are available for the assessment of the quality of care of common and/or chronic conditions and the most common procedures performed in hospitalized children. For example, community-acquired pneumonia and bronchiolitis are underrepresented given their prevalence, high volume, and well-documented variation in care for those conditions.6
Metrics in which overuse in pediatric health care are addressed are also underrepresented.6,7 They represented only 13% of all unique pediatric QMs identified as of 2015. Considering that waste in US health care may represent up to one-third of total health care expenditures, this type of metric should be better represented.
Because >70% of pediatric hospitalizations occur in general hospitals in the United States, and less than one-third take place in free-standing children’s hospitals,17 it is expected that different patterns of care are provided in these separate settings. Also, most children are admitted to hospitals in which most condition-specific measures do not have adequate power to show modest differences in performance from average.44 Development and implementation of QMs and initiatives to improve the care for hospitalized children should take into consideration these facts.
Development of QMs for pediatric health care involves multiple challenges, including existing gaps in the scientific evidence available, the definition of the indicator and population, and selection of a reliable data source. The country still lacks robust national-level and state-level information about the health care quality for children and adolescents, including a unified database that encompasses all payers and all events of hospitalization.45
Special consideration must be given to risk adjustment, scientific testing for validity and reliability, and avoiding and/or detecting bias. Refined tools for risk- and population-adjusted comparative hospital care and benchmarking are critically important. Novel methods include the calculation of Achievable Benchmarks of Care, which are calculated from the top decile of the population corresponding to the best performing institutions of a given cohort that will represent benchmarks of excellence46 and the creation of templates of patients after multiple variable matching.47 These tools can provide more objective and achievable goals for benchmarking and a more meaningful comparison of patient populations within an institution or among institutions.
Also, children represent a unique, special population with additional challenges, quoted in the literature as the 5 “Ds”: developmental changes, dependency of caregivers, differential epidemiology, demographics, and differences in the financing of children’s health services.7,21,48 Applying quality indicators designed for the adult population to children is inadequate.
Despite these many challenges, measuring health care is recognized as a critical step toward achieving the triple aim of better care, better population health, and more affordable care.3,18 QMs are important tools for advancing high quality and safety in health care,3,12 including for hospitalized children. Measurement makes providers more accountable and can further answer critical research questions regarding disparities in health care and can document the effects of policy changes.12
However, the rapidity with which measures have been developed over the last decade has outpaced the research on the validity and impact of QMs on health outcomes.6 The sheer number of measures available is currently considered by many to be excessive.49 The recently completed CHA survey on accountability programs and measure sets used by children’s hospitals revealed that 83% to 97% of responders reported “moderate to serious” challenges in questions addressing the level of effort and dedicated resources required for gathering data and reporting.4
Moving forward, a focus of measurement should critically address a balance between the benefits and relevance of the metric versus the cost of implementation and resources needed.6,7,47 Berwick49 postulates that “intemperate measurement is as unwise and irresponsible as is intemperate health care.” He called for a reduction of 50% in 3 years and 75% in 6 years of the volume and total cost of current measurements being used and enforced in health care.49 This should be the foundation and the guiding principle for the future of quality measurements and QI in health care, including hospital care.
Measuring quality of care provided to hospitalized children requires a leaner and more balanced set of indicators. Outcome metrics related to mortality, surgical complications, and those addressing safety and avoiding harm should be combined with refined process metrics with a proven link to improved outcomes to evaluate the care of the most common and/or chronic conditions and procedures treated and performed in hospitalized children. Measures related to overuse, the patient experience, and patient-reported outcomes should be better represented.
Pediatric QMs and metric sets for hospital care should be publicly and transparently reported. The results of the Child HCAHPS survey could be the first step in this direction. This tool is ripe for public reporting and benchmarking.
Better coordination of efforts among different stakeholders caring for hospitalized children is needed in developing QMs. Harmonization and alignment of the existing programs of accountability and measure sets are required.
Children are not small adults. The time has come to think about a national clearinghouse and coordinator for pediatric-only QMs. The NQF and the US government could lead, coordinate, and provide ongoing support to this new pediatric-only organization that will include every stakeholder committed to improving the quality and safety of children’s health care.
These steps, taken together, will certainly improve the effectiveness and validity of measurement and eventually will lead to the improvement of the quality of care provided to millions of hospitalized children each year in the United States.
The authors acknowledge Dr Kristine De La Torre and Dr Jennifer McCafferty-Fernandez at the Miami Children’s Research Institute of Nicklaus Children’s Hospital for medical writing assistance and support. They also recognize Tatiana Consuegra for her clerical assistance in the preparation and submission of this article.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Copyright © 2017 by the American Academy of Pediatrics