Background and Objective: Education of residents in academic medical centers occurs as part of clinical care, but little is known about the relationship between clinical workload and educational experiences among pediatric residents. The goal of this study was to assess residents’ perceptions of learning on inpatient services at a children’s hospital in relation to perceived workload and actual patient census.
Methods: This was a prospective cohort study of pediatric residents at 1 urban academic children’s hospital. Surveys on educational experience were administered weekly to residents on 12 inpatient units from October 2010 to June 2011. Daily peak medical inpatient census data were collected, and Pearson correlations were performed.
Results: Mean weekly response rate was 25%. Perceived workload was correlated with weekly peak of patient census for interns (r = 0.66; P = .00) and senior residents (r = 0.73; P = .00). Many aspects of perceived learning were negatively correlated with perceived workload among interns and residents in “acute care” units. Activities beyond direct patient care (eg, attending conferences, independent reading) revealed more negative correlation than educational experience during rounds. Among seniors, scores of aspects of perceived learning did not correlate with workload.
Conclusions: The study found mostly negative associations between high perceived workload and perceived learning, especially for interns. Results suggest varied impact of workload on perceived learning by training year. Although patient care is essential for resident education, higher workload may adversely affect learning opportunities for pediatric trainees. More research is needed to identify if generalizable thresholds of patient census and/or clinical workload cause declines in perceived or real education.
The core missions of academic medical centers include providing high-quality health care to patients and clinical education to trainees. The training of residents, in particular, occurs as part of daily patient care and is complemented by didactic teaching and independent reading. Although busier inpatient services may provide more cases for clinical learning, high workload may affect available time and quality of learning opportunities, as studies among internal medicine residents have demonstrated.1–4 Attending physicians, fellows, and senior residents charged with teaching may experience challenges when the workload is too high. Adverse patient safety events have also been associated with higher workload in hospitals.5–9 Given that children’s hospitals face varying volume, turnover, and acuity throughout the year,5,10–12 pediatric residents may also have varying educational experiences as workload changes. The goal of the current study was to assess the educational experience of pediatric residents at a large, academic children’s hospital and to examine the correlation between perceived patient-related workload and actual inpatient census, self-reported opportunities for educational activities, and self-directed learning among residents.
This prospective cohort study was conducted between October 2010 and June 2011 at The Children’s Hospital of Philadelphia (CHOP), a large, urban, tertiary care children’s hospital with >450 inpatient beds and a pediatric residency program of 134 residents. The study was approved by CHOP’s institutional review board.
During the study period, CHOP had ∼28 000 admissions with a median length of stay of 2.1 days. Resident teams were assigned to 12 different, geographically bound units. The teams, in most cases, were intended to cover no more than 20 patients, although this level could be exceeded by up to 2 patients during times of high volume.
For the purposes of this study, we defined 3 roles and created a survey instrument targeted at each: intern survey, “acute care” survey, and senior survey. The intern survey was given to all postgraduate year (PGY)-1s rotating on 8 inpatient teams, each of which included general pediatrics and subspecialty patients. Seven of the 8 teams had 3 interns, and 1 team had 2 interns. The acute care survey was administered to residents on the inpatient cardiology, oncology, NICU, and PICU rotations. The cardiology team consisted of 3 PGY-2s. The oncology team consisted of 4 PGY-2s. The PICU was staffed with 9 residents (mostly PGY-2 and PGY-3/4, but from January to June, 1 PGY-1 rotated in the PICU each month). The senior survey was distributed to supervising residents on 8 inpatient teams that had interns; 5 of the teams had 2 seniors (paired PGY-2 and PGY-3/4 or 2 PGY-3/4s). Two teams had a PGY-2 as the primary senior with a PGY-3/4 who supported these seniors. One team had a single PGY-3/4 in the supervisory role.
The survey instruments were created by a chief resident (DH) and a general pediatric attending physician (ESF) based on an existing instrument that assessed perceptions of learning on inpatient services.2 The questionnaires assessed self-reported educational experiences over a weekly period during rounds, outside of rounds, and overall. The surveys were piloted with a representative sample of 6 residents (2 from each year), selected by 2 chief residents who were not involved in the study. Revisions were made to the surveys based on feedback from a recorded discussion with respondents.
The final version of each survey (Supplemental Information 1) had 14 to 17 questions, with 8 to 11 questions about various aspects of perceived learning in the format of a 5-point Likert scale (1 = poor, 2 = below average, 3 = average, 4 = above average, and 5 = excellent). Residents also assessed the patient-related workload on their unit and were asked, in multiple-choice format, their ability to attend educational activities. In addition, each survey allowed free-text comments on educational experience during rounds, outside of rounds, and overall. The surveys were hosted on SurveyMonkey.com (Palo Alto, CA), and links to the questionnaires were e-mailed on Fridays. Data on daily peak medical inpatient census (including PICU and cardiac ICU but excluding surgical patients) were collected as part of regular hospital patient flow initiatives. Patients in inpatient and observation billing statuses were included.
Summary statistics were generated for responses and stratified according to role of respondent (intern, acute care, or senior). Daily peak medical inpatient census data were used to determine maximum medical census per week. The weekly maximum medical census was selected as a summary measure of workload because it is considered a better predictor of census in pediatrics, over the long-term, compared with average census, a measure that approaches minimum census.13 Ordinary least-squares regression did not support a nonparametric relationship between perceived workload and maximum inpatient census, and Pearson product-moment correlation coefficients were therefore calculated between weekly mean score of perceived workload and each week’s maximum patient census per respondent role.
A mean weekly score for each aspect of perceived learning was calculated by averaging individual responses from residents in each role for each week. Pairwise Pearson correlations of perceived workload with weekly mean scores of aspects of perceived learning were computed. Correlations with P values <.05 were considered statistically significant. Fisher’s z transformation was used to compare correlations. All analyses were conducted in Stata version 12 (StataCorp, College Station, TX).
For free-text responses, investigators grouped comments on perceived learning and workload into thematic areas. Because the comments provided were short and few compared with total responses, all authors participated in identifying major themes and classifying comments into relevant thematic concepts. We focused content analysis on open coding, which is an analytical process of examining, comparing, and grouping qualitative data to develop thematic concepts.14
A total of 211, 166, and 104 responses were received from residents in the intern, acute care, and senior roles, respectively, over the 35 weeks of the study. Mean weekly response rates were 27% for interns, 29% for acute care residents, and 20% for seniors.
Perceived Workload and Actual Inpatient Medical Census
The mean ± SD of all actual weekly maximum medical inpatient census values was 263 ± 20. The lowest medical census observed was 217 (early June), and the highest was 311 (mid-February). Mean scores of perceived patient workload by interns and seniors correlated with weekly maximum of actual medical inpatient census (interns: r = 0.67, P = .00; seniors: r = 0.73, P = .00) but was not significant for acute care residents (r = 0.25, P = .16) (Fig 1).
The winter weeks between February 7, 2011, and March 7, 2011, recorded highest patient census, with the weekly maximum averaging at 301 medical inpatients over those 4 weeks, compared with 263 in the 17 weeks before and 256 in the 13 weeks after that period. Before this high-census period, the hospital introduced a new computer order entry system and electronic health record for inpatient care, Epic (Epic Systems, Verona, WI), which was activated on January 22, 2011.
Perceived Workload and Learning
Among interns, all aspects of perceived learning were negatively and significantly correlated with perceived workload (Table 1, Supplemental Information 2). Activities outside of rounds (eg, attending educational conferences, independent reading) showed significantly more negative correlation with perceived workload than activities that were part of rounds. When scores for perceived learning during rounds were averaged over the high-census weeks of February and March, the weekly mean (3.15) was lower than for the weeks before (mean: 3.64; P = .03) and after (mean: 3.79; P = .02). As perceived workload increased, the average number of days per week when interns missed noon conferences increased from 1 when perceived workload was rated <3 (13 responses) to 2.16 when workload was rated ≥3 (198 responses) (P = .01).
For interns, mean scores of all aspects of perceived learning declined as perceived workload increased past the midpoint (3 of 5) on the Likert scale (Fig 2). At the highest level of perceived workload (5 on the Likert scale), all perceived learning measures were rated with a mean score of ≤3.5. Time for reading about patients and attending educational conferences had the lowest average assessments when the mean score of perceived workload was 5. Among 119 responses from interns that ranked workload ≥4, mean scores for time for reading and attending conferences were 2.47 ± 0.98 and 2.50 ± 1.06, respectively. For this group, the mean for perceived learning during rounds was significantly higher at 3.51 ± 0.97 (P = .00). Mean scores for perceived learning during rounds remained >3 across all values of perceived workload. As workload increased, average score of time available for teaching by attending physicians and fellows decreased at a faster rate than for quality of teaching (Fig 2A). Interns’ weekly mean scores for perceived learning during rounds (discussions on patient care) had a different association with peak census than activities outside of rounds (time for reading about patients’ conditions) (Fig 3, Supplemental Information 3).
Importantly, interns’ weekly mean scores for perceived workload reflected expected decreased efficiency from the introduction of the new computer order entry system on January 22, 2011. Two weeks before activation, mean workload score among interns was 3.75 (n = 4). There was a statistically nonsignificant increase in mean workload rating to 4.5 (n = 4, P = .44) in the week of activation and a further rise to 4.63 (n = 8, P = .83) in the week after the Epic launch (Fig 1).
Acute care residents rated all measures of perceived learning during rounds nominally higher than interns (Table 1). As with interns, aspects of perceived learning during and outside of rounds were significantly and negatively correlated with perceived workload, except for quality of teaching by attending physicians and fellows (r = –0.02; P = .84). Correlation of perceived workload with time for patient care discussion on rounds (r = –0.21) was significantly less negative than the correlation with time for reading about patients (r = –0.38; P = .05). There was a negative correlation between perceived workload and attendance at educational conferences (r = –0.31; P < .01). Among these residents, mean scores for other aspects of perceived learning also decreased as workload increased (Fig 2B, Supplemental Information 2), except for quality of teaching by attending physicians (mean between 3.71 and 4 as perceived workload ranged from 2–5).
On average, seniors rated perceived workload higher than interns (P = .00) and acute care residents (P = .02) over the study period, according to Fisher’s transformation. For seniors, none of the coefficients of correlation of perceived workload with aspects of perceived learning was significantly different from zero (Table 1, Fig 2C, Supplemental Information 2). This finding may be attributed to low absolute values of correlation coefficients coupled with small total number of responses. In Fig 2C, opportunity for teaching during rounds by seniors increased with workload, from a mean of 3.31 ± 1.12 (26 responses) when workload was rated 3 to 3.82 ± 0.95 (33 responses) when workload was rated 5 (P = .04). Average time for extracurricular activities (eg, reading about patients) among seniors was also greater than for interns (P = .02) and acute care residents (P = .00).
Of 481 completed surveys, 29 responses from interns, 39 from acute care residents, and 18 from seniors included free-text comments. Across all years, 55% of free-text responses focused on residents’ experience, with longer rounds affecting education and the factors that may influence the length of rounds, including the number of specialty services, patient census, discharge/administrative responsibilities, and complexity of patients (Table 2). A majority of acute care responses underscored circumstances that resulted in missed opportunities for learning, notably administrative responsibilities related to discharge of patients. Similarly, intern and senior responses reflected varied dimensions of workload that negatively affected perceived learning, including high number and complexity of patients, structure of rotation, and administrative work.
We examined the relationship between perceived workload and pediatric residents’ perceived learning on inpatient services. We report mixed findings, with mostly negative associations between perceived high workload and perceived learning. Findings also suggest that the impact of workload on perceived learning may vary according to year of training, as a result of experience and responsibilities. The results are in line with a study that found varying thresholds for learning during times of high workload across internal medicine training years, with interns having the lowest perceived learning in relation to high census.2 These results may reflect the unique role of interns, who have many tasks and the least experience among residents. For the intern, each additional patient provides both an opportunity for learning and an increased amount of clinical and administrative work (eg, notes, discharge summaries, telephone calls).
Although perceived learning was negatively associated with higher workload for interns, the relationship varied across types of learning. Perceived learning associated directly with clinical care (eg, time spent on patient care discussions) was least affected by increased workload, whereas opportunities for perceived learning outside of rounds were most affected. Because rounds are a necessary part of care delivery, this finding is not surprising. Likewise, a demanding clinical workload would be expected to lead to less ability to attend conferences or read independently. It is also interesting that high levels of workload for interns were associated with decreased time for reading about patients, although duty-hour restrictions would suggest that out-of-hospital time for such activities should be protected. Possible explanations include that at times of lower workload, reading is done in the hospital, or that high workload is associated with fatigue, or that duty hours are stretched (even if staying in compliance) during the busiest times. More investigation is needed. A confounding variable, of course, is that interns’ efficiency increases over time. After the winter peak and as workload decreases in spring, interns tend to have increased efficiency. This improvement seemed to translate into increased opportunities for self-directed learning.
Study data also reflected the increased perceived workload among interns, likely from the activation of a new inpatient computer order entry system (Epic) in late January 2011. Upsurge in perceived workload was observed, although the increase was statistically nonsignificant, likely due to small subsample size. This finding occurred despite institutional efforts at workload mitigation, including previous exposure of residents to Epic in the outpatient setting, extensive training, and additional staffing to assist with processing and data entry.
Acute care residents experienced similar findings as interns, although correlations were less negative, likely related to the scope of workload in the designated inpatient areas. The cardiology and oncology floors and NICU do not necessarily experience the same seasonal fluctuations in patient census as other hospital areas. In addition, although the intern noon conference is separated geographically and topically from clinical work of inpatient units, didactic education on acute care rotations is floor-based and related to that specialty, even if not directly about the patients on the service. These rotations also generally involve nonintern residents with more experience.
Among seniors, scores of aspects of perceived learning did not vary as workload increased. The near-zero correlations observed may be due to the smaller number of senior residents rotating at any time and the low response rate. Trends in senior responses, however, can be cautiously interpreted in relation to the role of supervising senior residents. These residents ensure the completion of all tasks necessary for medical care delivery by providing oversight to interns and contribute to teaching of interns and medical students. In contrast to interns, seniors focus their attention on diagnostic evaluation, differential diagnosis, overall patient care, management of the team, and teaching. Seniors may find that more patients (particularly those with new presentations or diagnoses) offer more opportunities to teach. The potential for true differences in perceptions of education between interns and seniors in the face of high workload invites more investigation. It would be important to evaluate how exposure to high levels of workload trains novice residents for later years of training and independent practice by enhancing their efficiency.
Our study has limitations. The study was conducted at 1 urban pediatric hospital, which limits its generalizability. We had low response rates for our analysis of perceived education over time and for qualitative assessment of free-text responses. However, response rates were not entirely outside of previous surveys of physicians or residents.15,16 Some surveys have achieved higher response rates,17,18 but the methods differed from the current study, particularly in repeat surveying over time. The small number of responses may be partially due to increased workload that limited time to respond to the survey, although the correlation between weekly response rate and weekly mean score of workload was not significant (r = –0.02; P = .09). This response bias would be expected to blunt the correlation between perceived workload and perceived learning, as residents without time to complete the survey could more likely miss conferences. Alternatively, nonresponders could perceive a lower level of workload than respondents, thereby affecting the relationship between perceived workload and learning if there is a higher frequency of nonresponders during periods of high workload. Although we surveyed all CHOP residents on inpatient services, residents may not have the same perceptions of education or workload. In addition, our measure of actual workload does not capture other dimensions of workload that may be relevant to pediatrics, including acuity, turnover, and variety of conditions. Nevertheless, our study demonstrated a strong relationship between peak daily medical census and perceived workload among residents, suggesting that census is a relevant, albeit incomplete, measure of workload. Similarly, we used perceived learning as a surrogate for actual learning, although we recognize this method is imperfect. A more objective measure, such as in-service test scores, was not collected.
Although we collected data throughout the year, we did not track individual residents’ perceptions over time and therefore could not account for variation within respondents. The survey was anonymous, thus limiting our ability to adjust for individual-level factors. Whereas the survey instruments assessed time for educational activities and self-directed learning, surveys did not ask about components of workload that may detract from perceived learning (eg, time spent on administrative documentation). These questions might help identify strategies to improve quality of education during times of high workload or high cognitive load.19 Although the correlation coefficients presented in this study are statistically significant, their magnitudes are relatively weak.20 Nevertheless, the findings suggest trends between high workload and learning that could be studied in future research.
We report a negative correlation between perceived inpatient workload and perceived learning, particularly among interns for activities outside of direct clinical care. Although our study demonstrates that certain aspects of learning take precedence over others in residents’ assessment, more research is needed to identify which activities are of greatest value. Further studies can answer questions about the optimal workload to achieve educational aims and the patient census threshold at which actual learning declines for different levels of trainees.
We acknowledge the contributions of Patricia Hicks, MD, to the design and implementation of this survey, and of Joan Li, for assistance in logistical aspects of its weekly delivery to the residents.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
- The Children’s Hospital of Philadelphia
- postgraduate year
- Scott LD,
- Rogers AE,
- Hwang WT,
- Zhang Y
- Fieldston ES,
- Hall M,
- Sills MR,
- et al
- Macy ML,
- Stanley RM,
- Lozon MM,
- Sasson C,
- Gebremariam A,
- Davis MM
- Hsieh HF,
- Shannon SE
- Freedman D,
- Pisani R,
- Purves R
- Copyright © 2013 by the American Academy of Pediatrics