BACKGROUND AND OBJECTIVES: Patient experience surveys provide feedback regarding the perceived quality of health care services. Unfortunately, many surveys have tended to be focused on an adult population, resulting in a lack of validated instruments for pediatric populations. In 2015, Alberta Health Services implemented the Child Hospital Consumer Assessment of Healthcare Providers and Systems (Child HCAHPS) survey. Our objectives in the current study were to describe the use of the Child HCAHPS survey in Alberta, Canada, and to present preliminary results.
METHODS: Parents and/or guardians completed a modified version of the Child HCAHPS survey via telephone within 6 weeks of their child’s discharge from 1 of 14 hospitals (2 stand-alone pediatric and 12 adult). For each of the survey composites and stand-alone items, the mean “top box” score is reported, reflecting the most positive response option. Overall results were reported, as were comparisons in top box scores between stand-alone pediatric and adult facilities.
RESULTS: From October 2015 to March 2017, 3389 surveys were completed. Overall, mean top box scores ranged from 41.5% (“preventing mistakes and helping you report concerns”) to 95.8% (“keeping you informed about your child’s care in the emergency department”). Stand-alone pediatric hospitals tended to outperform the adult ones, particularly in global ratings of care, parental involvement in decision-making, and communication between parents and providers.
CONCLUSIONS: With our findings, we indicate areas in which pediatric inpatient care is being delivered well, as well as targeted areas for quality improvement. Our results may be highlighted in future comparisons among others who have adopted the Child HCAHPS.
The Institute of Medicine considers patient-centered care (PCC) to be 1 of the 6 key elements of high-quality health care.1 An overarching goal of PCC is to encourage and foster an environment where patients are equal and active participants in their care. In doing so, the care that is provided is meant to be respectful and responsive to individual patient preferences while incorporating the patient’s needs and values within clinical decisions.1 To provide successful PCC, it is imperative to include the patient voice because clinicians must know what is important to each individual patient with respect to his or her condition, treatment, and ongoing management.
Patient-reported experience measures (PREMs) are PCC tools that are used to collect patient and family experiences with the health care received, adding the voice missing from quality improvement initiatives.2 Provided in the standardized collection of PREMs and linkage to other data sources such as administrative data and medical records is a vast amount of quantitative data to examine patterns, associations, and trends. For instance, comparisons may be made between hospitals and across various demographic and clinical comparators. Similarly, from a quality improvement perspective, the data may reveal further insights as to where care is being delivered well and where care may be augmented.3
There is an array of validated PREMs within the acute care, in-hospital setting. Examples include surveys such as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey,4 the Picker Patient Experience Questionnaire,5 and the Canadian Patient Experiences Survey–Inpatient Care.6 However, the majority of instruments that do exist have tended to be focused on the adult population. As such, the personnel of hospitals and health care systems looking to measure the experience of pediatric patients and families have had to do so using ad hoc, in-house instruments. This, of course, introduces concerns around validity and reliability, as well as a potential inability to make valid comparisons across multiple facilities. To address the need for a validated, publicly available, inpatient pediatric experience survey,7 the Child Hospital Consumer Assessment of Healthcare Providers and Systems (Child HCAHPS) instrument was developed after a literature review, expert interviews, and focus groups with patients and their family members.8 Preliminary results from the Child HCAHPS survey have been provided in the United States9 and, more recently, in Belgium.10
In October 2015, Alberta became 1 of 3 Canadian provinces to adopt the Child HCAHPS survey. To date, however, no Canadian results have been reported from its use. The current study is the first Canadian one in which the family-reported experience with the quality of pediatric care is assessed via the Child HCAHPS. Our primary objectives in this study were to describe the use of the Child HCAHPS survey in Alberta hospitals and to present preliminary results. Our secondary objective was to highlight variations in reports of family experience between stand-alone pediatric facilities and adult facilities that provide pediatric care in the province.
Ethical approval for the current study was granted by the University of Calgary Conjoint Health Research Ethics Board (file number REB17-0769). All data were provided to the research team via a research agreement with Alberta Health Services (AHS).
Description of the Child HCAHPS Measure
Respondents (parent or guardian of child) completed a modified version of the Child HCAHPS survey,8 which comprised 65 questions. The survey included the 39 experience questions and 10 screening questions from the original Child HCAHPS survey.8 A total of 9 additional patient experience questions, which were developed during focus groups and pilot testing with parents, were included as part of AHS organizational purposes. With another 6 demographic questions, we asked about the child’s overall health and number of hospitalizations in the previous year as well as the respondent’s age, level of education, relation to the child, and portion of time spent at the hospital with the child during the visit in question. In a final open-ended question, respondents were asked if there is anything additional that they would like to share about their hospital experience. Reponses to this question were transcribed verbatim.
Surveys were administrated according to standard HCAHPS practices. Every 2 weeks, a list of eligible cases was generated from administrative discharge records. This data contained up to 2 telephone contact numbers for each patient and/or family member as provided at the time of hospital admission. Telephone surveys were completed within 2 to 42 days of the child’s discharge from 1 of 14 hospitals (2 stand-alone pediatric, 12 adult sites providing pediatric care) across Alberta. The complete list of hospitals sampled, their location, setting, and the number of completed surveys from each site is provided in Table 1. Inclusion criteria for the survey were that the patient was <18 years old at discharge, had an inpatient stay of at least 24 hours (ie, overnight), and was alive at the time of discharge. Cases pertaining to care occurring in an emergency department only, healthy newborns (eg, length of stay <2 days), and any mental health unit were excluded from sampling.
A team of trained interviewers administered the survey using a standard script, a list of approved prompts, and answers to some frequently asked questions. Random dialing was conducted from 9 am to 9 pm on weekdays and between 10 am and 3 pm on Saturdays. Once selected, each eligible telephone number was dialed up to 9 times on varying days and times to increase the chance of response. Ten percent of phone calls were monitored for quality assurance. Potential respondents were asked to provide their verbal consent and were informed that their data would be used for quality assurance and/or research purposes. Respondents were asked to verify the hospital discharge in question and were asked to not consider any other health care interactions (eg, emergency department, readmission, outpatient clinic) that they and their child may have had between the time of inpatient discharge and survey completion. At the end of the survey, respondents wishing to report a compliment or complaint were provided with contact information for the organization’s patient relations department. Each survey typically required 13 to 24 minutes to complete (median = 16).
Data Source and Analysis
We examined a collection of patient and respondent demographics as well as clinical variables using descriptive statistics. Patient demographic variables included sex, age group, and health status as reported by the child’s parent and/or guardian. Respondent demographics included age group, education level, relationship to child, and portion of time spent at the hospital with the child. Clinical variables included the hospital type (stand-alone pediatric or adult) and length of hospital stay (<3, 3–7, or ≥7 days).
For each survey question, we calculated the percent of respondents reporting a “top box” response according to the Child HCAHPS scoring methodology, where “top box” represents the most positive answer choice(s) for a given question.11 For the items from the standard Child HCAHPS survey, we calculated the composite scores as in the previous investigation by Toomey et al.9 A detailed description of these composites and their calculation methodology is provided by the Agency for Healthcare Research and Quality.12 For items specific to AHS, we report the percent in top box for each individual question. The overall results from all 14 hospitals were reported. Comparisons between the stand-alone pediatric (n = 2) and adult sites providing additional pediatric care (n = 12) were also made. Raw and case-mix adjusted percentages are reported. Using predictors previously identified by Toomey et al,9 we used linear regression to perform case-mix adjustment. These included patient age, respondent age, the respondent’s relationship to the child, health status of the child (as reported by the respondent), and the respondent’s level of education. All analyses were performed by using SAS network version 9.3 for Windows (SAS Institute, Inc, Cary, NC).
Over the 18-month study period (discharges from October 2015 to March 2017), 3389 surveys were completed across the 14 sampled hospitals. The response rate, calculated as (completed surveys/[completed surveys + refusals]) × 100, was 66.0%. The characteristics of patients and respondents are shown in Table 2. The majority of patients were boys (55.1%), ages ≤4 years (66.7%), and had excellent or very good health (67.6%). Respondents were primarily ˂35 years of age (53.8%) with an education level of college or higher (76.4%). Most respondents were the child’s mother (84.6%) and had spent most or all of the time at the hospital with the child (94.0%). Clinically, 55.7% of discharges (n = 1888) came from pediatric hospitals, and the mean length of stay of all cases was 7.9 days (median = 3.9; range: 1–191). Fig 1 shows the percent of respondents who provided a “top box” rating to the overall rating of care question. Pediatric hospitals ranged from 71.0% to 74.2%, while adult hospitals ranged from 32.0% to 72.6%.
In Table 3, we present the top box composite and item results. Overall, for elements on the standard Child HCAHPS, the 3 top-performing composites pertained to “keeping you informed about your child’s care in the emergency department” (95.8% in top box), “willingness to recommend the hospital” (83.9% in top box), and “communication between you and your child’s doctors” (79.5% in top box). On the other hand, the 3 worst-performing composites dealt with “preventing mistakes and helping you reporting concerns” (41.5% in top box), “quietness of hospital room” (55.2% in top box), and “how well doctors communicated with your child” (61.1% in top box). The raw and adjusted (ie, case-mix) results were similar for all composites. When comparing the top box percentages by hospital type, the stand-alone pediatric hospitals outperformed the adult sites on all composites and items, with the exception of “quietness of hospital room.” A great deal of variability in top box percentages was observed across the pediatric and adult sites. An example of this variability at pediatric sites was the “quietness of hospital room” item, which varied from 46.5% to 60.8%. One example among adult sites was “how well nurses communicate with your child,” which varied from 41.7% to 89.6%.
With respect to the additional organization-specific items (ie, items not on the standard Child HCAHPS, as denoted by italics), the 3 top-performing ones pertained to the parent’s understanding of their role in their child’s ongoing care (87.4% in top box), providers doing everything they could to help with the child’s pain (76.7% in top box), and parents being involved in decisions about their child’s care (76.1% in top box). The 3 worst-performing organization-specific items pertained to providers having an understanding of the child’s condition (62.7% in top box), providers following-up on parent’s concerns and observations (62.7% in top box), and staff members introducing themselves and explaining their roles (72.3% in top box). Once again, little variation was observed between the raw and case-mix adjusted top box percentages. As with the standard Child HCAHPS composites, the stand-alone pediatric sites tended to have better results than the adult ones, with the exception of the question pertaining to the providers’ understanding of the child’s condition (61.5% in top box versus 64.2% among adult sites). Once again, a great deal of variability in top box percentages was observed within each site subgroup.
To our knowledge, with this descriptive study, we are the first to evaluate the use of the Child HCAHPS survey in Canada. Overall, care experiences tended to be positive, with 83.9% of respondents indicating that they would “definitely recommend” the hospital to family and friends. Additionally, respondents had high ratings of communication with providers, specifically around “keeping you informed about your child’s care in the emergency department” (95.8% in top box), “communication between you and your child’s doctors” (79.5% in top box), and “communication between you and your child’s nurses” (78.4% in top box). In contrast, our findings revealed areas that can be improved; for example, “preventing mistakes and helping you report concerns” (41.5% in top box), “quietness of hospital room” (55.2% in top box), provider understanding of the child’s condition (62.7% in top box), and providers following-up on parent’s concerns and/or observations (62.7% in top box). Across the 14 hospitals studied, we found a great deal of variability in parent-reported experience, which is highlighted in the disparities between pediatric and adult hospitals. Experiences in pediatric hospitals tended to be more positive than those of adult facilities across most composites. These differences were particularly large for the global ratings of care, parental involvement in clinical decision-making, and for elements of communication between parents and providers.
When considering the raw and case-mix adjusted findings, we found that our results are strikingly similar to those of Toomey et al9 from the United States. They too observed that “preventing mistakes and helping you report concerns” was the worst-performing area (55% in top box, compared with 41.5% in our study), whereas “keeping you informed about your child’s condition in the emergency department” was the best performing (84% in top box, compared with 95.8% in our study). Again, in agreement with our findings, the authors reported that stand-alone pediatric hospitals tended to have better results than sites where adult patients are primarily treated.
Our results also replicate those from Belgium in which Bruyneel et al10 reported poor results around “preventing mistakes and helping you report concerns” (48.2% in top box). Overall, our global results tended to be superior, as revealed in the “overall rating of hospital” (63.9% vs 51.2% in top box) as well as the “willingness to recommend the hospital” (83.9% vs 78.3% in top box). These results were surprising to us, given the Belgian study was conducted solely on pediatric wards where one may expect better hospital experience scores.
The current study is one in which we provide robust data to measure aspects of PCC among pediatric hospital patients in Alberta. A fundamental strength of the study lies within the use of the Child HCAHPS, a validated instrument for assessing the experience of pediatric inpatients. As the Child HCAHPS continues to be widely adopted, this will allow for future, valid comparisons between hospitals and health care systems for benchmarking purposes. Among adults, these types of comparisons are well underway in the United States and are a future goal of the Canadian Institute for Health Information, with use of the Canadian Patient Experiences Survey–Inpatient Care survey. The value of comparative reports of health care services has also been revealed by the Commonwealth Fund through their annual surveys that have been conducted internationally.13 Other strengths of the study lie within the comprehensive sampling strategy and standard script that was used as part of the survey. Each of these contributes to the robustness of our data as well as our findings.
The study is not without limitations. Although we performed case-mix adjustment, we did so according to a predefined list of predictors from previous pediatric9,10 and adult14,15 studies. A future study by our group is currently underway to examine the potential impacts of other predictors in our Canadian data set. A secondary limitation is the aggregated nature of the reports. Although with surveys we can easily quantify how groups of patients may rate a particular element of care, we are unable to understand the “how” or “why.” For this reason, we advocate that surveys may be complimented with personal narratives to better understand the individual experience within the context of PCC.16–18 Additionally, because we have conducted the survey using a telephone format, our results may not be generalizable to a mail-out format. Previous HCAHPS research among adults has revealed higher scores with a telephone format partly because of the possibility for social desirability.19 Lastly, at the time of this report, there were 95 hospitals in Alberta that provide overnight acute care. The 81 sites not included are primarily adult ones in which few pediatric patients are discharged annually (<48). The patient experience of children and families who receive care at these sites remains unknown.
With our results, we indicate areas in which pediatric inpatient care is being delivered well; we also provide areas of “low-hanging fruit” for quality improvement. This descriptive study is an important first step to creating a hospital environment that fosters PCC. Additionally, our results have the potential to serve as useful comparisons among others who have adopted, or who may adopt the Child HCAHPS survey. There may also be great potential in the linkage of Child HCAHPS surveys with administrative data sets to examine the interplay between patient-reported experience and other outcomes (eg, health system use).
We thank the parents and guardians who took the time to complete the survey. We also thank the team of Health Research Interviewers from Primary Data Support, Analytics, Alberta Health Services for their professionalism in conducting the surveys, as well as Ms Rose Petrovic (Lead, Primary Data Support, Analytics, Alberta Health Services) for her assistance with data collection.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- ↵Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001
- Beattie M,
- Murphy DJ,
- Atherton I,
- Lauder W
- Gleeson H,
- Calderon A,
- Swami V,
- Deighton J,
- Wolpert M,
- Edbrooke-Childs J
- ↵Canadian Institute for Health Information. Canadian patient experiences survey—inpatient care. Available at: https://www.cihi.ca/sites/default/files/patientexper_backgrounder_en_0.pdf. Accessed July 14, 2017
- Toomey SL,
- Zaslavsky AM,
- Elliott MN,
- et al
- Toomey SL,
- Elliott MN,
- Zaslavsky AM,
- et al
- Bruyneel L,
- Coeckelberghs E,
- Buyse G,
- et al
- ↵Centers for Medicare and Medicaid Services. A note about HCAHPS “boxes”. Available at: http://hcahpsonline.org/en/summary-analyses/. Accessed April 16, 2018
- ↵Agency for Healthcare Research and Quality. CAHPS child hospital survey measures. 2016. Available at: https://www.ahrq.gov/cahps/surveys-guidance/hospital/about/child-survey-measures.html. Accessed August 8, 2017
- ↵The Commonwealth Fund. Surveys. Available at: www.commonwealthfund.org/interactives-and data/surveys#/sort=%40fdate63677%20descending. Accessed July 24, 2017
- Elliott MN,
- Lehrman WG,
- Goldstein EH,
- et al
- Greenhalgh T,
- Hurwitz B
- Guney S,
- Santoro D
- Copyright © 2018 by the American Academy of Pediatrics