A Comparison of Resident Self-Perception and Pediatric Hospitalist Perceptions of the Supervisory Needs of New Interns

Discussion

We found that pediatric hospitalists and residents in a large pediatric residency training program reported that new interns were not consistently able to perform the core EPAs that the AAMC has established as a goal for all new residents without direct supervision. For 6 of the 11 EPAs that we studied, less than half of pediatric hospitalists thought new interns could perform the EPAs without direct supervision. Even residents (who consistently indicated less need for supervision) reported that they could not have performed 2 of the 11 activities without direct supervision at the beginning of their intern year. These data support the AAMC’s call for significant curriculum development and increased training of medical students.

Less than one-third of pediatric hospitalists thought new residents could perform the following 2 EPAs without full supervision: (1) recommending and interpreting common diagnostic and screening tests; and (2) recognizing a patient requiring urgent or emergent care and initiating evaluation and management. Medical errors by new interns performing these activities can lead to significant patient harm. Accordingly, it may be unsafe to expect new interns to perform these tasks without direct supervision, although fostering their ability to do so would certainly be of value.

Although we primarily examined new residents’ performance of the EPAs without direct or full supervision, we also analyzed their abilities to perform the EPAs independently, without any supervision, as a secondary analysis. We hypothesize that many new residents do, in fact, perform many of the EPAs independently, and we therefore thought it would be important to include this analysis. For each of the activities studied, most pediatric hospitalists doubted that new interns could function independently. Similarly, more than half of residents reported that they were unable to perform 8 of the 11 tasks independently at the beginning of their intern year. Therefore, we agree with the AAMC that it is critical to provide at least some supervision for all of the EPAs.

Our study also revealed that for many of the EPAs, residents and pediatric hospitalists had significantly different perceptions regarding the need for supervision of new interns. This secondary finding is in many respects not surprising and likely in part stems from the inaccuracies of intern self-assessment. There is a robust literature base to suggest that physicians inaccurately self-assess their knowledge,¹² procedural skills,^12,13 and clinical competence.^12,14,15 Of further concern is that trainees who are the least competent are also the least able to self-assess competently.^12–14 Additionally, residents may mistakenly believe that less intern supervision is needed, perhaps because they may not fully appreciate the need for supervision or because they may have an interest in achieving autonomy as soon as possible.^16,17 It is critical that pediatric hospitalists are aware of the inaccuracies of trainee self-assessment. Specifically, pediatric hospitalists should rely on direct observation in place of intern self-assessment when making entrustment decisions.

An alternative explanation for the discrepancies between hospitalist and resident ratings is that pediatric hospitalists may mistakenly believe that more intensive intern supervision is needed than is in fact the case, perhaps out of an inability to adequately evaluate trainees and a desire to ensure the safest possible care in the face of uncertainty.¹⁸ It should be noted that the EPA framework was designed with the intention of having an observer assess a single trainee in a clinical context and, on the basis of those observations, make a decision about entrustment.¹¹ In contrast, in this survey we asked pediatric hospitalists to rate how much supervision the average trainee needs. Therefore, pediatric hospitalists are forced to reflect on all of their workplace observations of interns over many years. It is likely that specific intern interactions may unduly influence their perceptions, a form of availability bias. For example, if a pediatric hospitalist were to work with a struggling new intern in the week before completing this survey, they might mistakenly report that the average intern needs more supervision than they actually do need. Pediatric hospitalists should be aware of this potential bias and make decisions about entrustment based on direct observation rather than recent experiences.

Despite its limitations, the disparity between hospitalists’ and residents’ perceptions has some important implications for the organization of care systems on inpatient units. For example, 72% of residents self-reported that they could give or receive a patient handover to transition care responsibility at the beginning of their intern year without direct supervision, as compared with only 44% of hospitalists. Researchers in a previous study have demonstrated that implementation of a supervised group handover was associated with reduced medical errors.¹⁹ Therefore, if interns do, in fact, perform handoffs independently or with only moderate supervision, then adverse patient outcomes could result. Reassuringly, however, there is agreement between residents and pediatric hospitalists regarding some of the other high-stakes activities (eg, entering and discussing orders and prescriptions).

Interestingly, there was little difference in the perceived need for supervision of new interns between residents across PGY levels. One may have hypothesized that as residents progressed in training, they may have changed their self-perceptions on their need for supervision at the beginning of their intern year; however, this occurred to only a limited degree. The stability in answers across PGY levels also suggests that PGY3 residents, who are further away from the beginning of their intern year and could theoretically be at risk for recall bias, may in fact recall how much supervision they perceived they needed at the start of internship reasonably accurately.

There are several limitations of this study. First, the response rate was modest, and there may be a nonresponse bias. Additionally, we did not specify in the surveys who would be supervising the new interns (ie, residents, pediatric hospitalists, or both), and therefore the responses were based on the interpretation of the respondents. It is possible that residents might have assumed this supervision would be by residents, whereas hospitalists assumed it would be by attending physicians. We also did not define some of the terms used in the survey (eg, “average intern” and “moderate supervision”), and therefore these terms were subject to interpretation. In addition, pediatric hospitalists were asked to rate the amount of supervision that the average intern needs at the beginning of the intern year, whereas residents were asked to rate how much supervision they themselves needed at the beginning of their intern year. It is possible that residents might either under- or overestimate how much supervision they themselves needed as compared with their peers. Additionally, this study was performed in a single large pediatric program, and therefore its results may not be generalizable across other hospitals. Lastly, shifting expectations regarding intern competence and the evolution of hospitalists’ clinical expertise could affect how ratings were provided.

With further preparation at the end of medical school or the onset of internship, we believe that it will be possible to narrow the gap between the current and desired independence of graduating medical students and new interns. We are encouraged by new fourth-year medical school boot camps and other curricula that aim to better prepare new interns.²⁰ In addition, residency programs could consider implementing intensive training programs for incoming interns on their arrival.²¹ Such training programs would permit residency program leadership and pediatric hospitalists to simultaneously assess new residents’ capabilities and adjust the levels of individual supervision accordingly. Given pediatric hospitalists’ unique opportunity to assess new interns and tailor their supervision on the basis of learner competency, we should offer faculty development regarding intern assessment and feedback. By better preparing our new interns to enter residency and by improving pediatric hospitalists’ abilities to tailor supervision, we can better align intern and attending physician expectations. It is essential that residents and hospitalist physicians openly discuss these issues so that interns can be supervised appropriately and patients can receive optimal care.