hosppeds
February 2017, VOLUME7 /ISSUE 2

Improving Pediatric Rapid Response Team Performance Through Crew Resource Management Training of Team Leaders

  1. Ashley Siems, MD,
  2. Alexander Cartron, BS,
  3. Anne Watson, MSc, PhD, BSN, RN,
  4. Robert McCarter Jr, ScD and
  5. Amanda Levin, MD
  1. Children’s National Health System, Washington, DC
  1. Address correspondence to Ashley Siems, MD, Department of Critical Care Medicine, Children’s National Health System, 111 Michigan Ave NW, Washington, DC 20010. E-mail: asiems{at}cnmc.org
  1. Dr Siems assisted with designing the study, designed and executed the crew resource management intervention, assisted with data analysis, and drafted the initial manuscript; Mr Cartron designed the study's data collection instrument and database, collected data, assisted with data analysis, and critically reviewed and revised the manuscript; Dr Watson assisted with designing the study, supervised and assisted with design of the data collection instrument, and critically reviewed and revised the manuscript; Dr McCarter performed statistical analysis and critically reviewed and revised the manuscript; Dr Levin conceptualized and designed the study, supervised design of the crew resource management intervention, assisted with data analysis, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.

Abstract

BACKGROUND: Rapid response teams (RRTs) improve the detection of and response to deteriorating patients. Professional hierarchies and the multidisciplinary nature of RRTs hinder team performance. This study assessed whether an intervention involving crew resource management training of team leaders could improve team performance.

METHODS: In situ observations of RRT activations were performed pre– and post–training intervention. Team performance and dynamics were measured by observed adherence to an ideal task list and by the Team Emergency Assessment Measure tool, respectively. Multiple quartile (median) and logistic regression models were developed to evaluate change in performance scores or completion of specific tasks.

RESULTS: Team leader and team introductions (40% to 90%, P = .004; 7% to 45%, P = .03), floor team presentations in Situation Background Assessment Recommendation format (20% to 65%, P = .01), and confirmation of the plan (7% to 70%, P = .002) improved after training in patients transferred to the ICU (n = 35). The Team Emergency Assessment Measure metric was improved in all 4 categories: leadership (2.5 to 3.5, P < .001), teamwork (2.7 to 3.7, P < .001), task management (2.9 to 3.8, P < .001), and global scores (6.0 to 9.0, P < .001) for teams caring for patients who required transfer to the ICU.

CONCLUSIONS: Targeted crew resource management training of the team leader resulted in improved team performance and dynamics for patients requiring transfer to the ICU. The intervention demonstrated that training the team leader improved behavior in RRT members who were not trained.

In 2006, the Institute for Healthcare Improvement published their “100,000 Lives Campaign” and included rapid response teams (RRTs) as a modality to improve patient safety.1 The adoption of RRTs has prevented cardiac and respiratory arrest, improved survival after these events, reduced the need for ICU-level treatments soon after transfer, and lessened the time between deterioration and treatments.29 In spite of these successes, RRTs, like other complex medical teams, are challenged by difficulties with communication and collaborative problem-solving.10 Barriers to RRT performance include varying levels of confidence with one’s own skills, challenges overcoming professional hierarchies, and expectations of adverse interpersonal conflict.11 Recently, the 2016 Joint Commission National Patient Safety Goal to “improve the effectiveness of communication among caregivers” signaled the growing recognition that improvement in the “nontechnical” social interaction skills of health care providers is critical for the delivery of safe and effective patient care.12 Adverse events can often be attributed to poor interpersonal interactions and coordination rather than insufficient clinical knowledge.13,14 Communication errors have been reported to be responsible for nearly 60% of all medical errors.15 Thus, approaches to improving RRT performance are necessary and should focus on interpersonal interactions and communication.

One such approach is crew resource management (CRM), which is a strategy of training with a focus on nontechnical skills that complement a provider's clinical knowledge and promote the delivery of safe care.16 CRM training of high-performance teams in other high-risk industries, including the military, aviation, and nuclear power fields, includes adoption of standardized tools and behaviors to improve teamwork and reduce risk.1724 CRM focuses on leadership, problem-solving, situational awareness, communication skills, and resource management to enhance team performance.2527 Careful consideration of one’s own fatigue coupled with verbal and nonverbal communication are paramount. CRM training is most effective when structured around simulation exercises, in which providers have the opportunity to safely practice vital nontechnical skills and receive real-time feedback in an environment uncoupled from adverse events.28,29 Although nontechnical skills have not historically been taught in clinical medical education, implementation of CRM training in the ICU and other medical settings has been shown to reduce complications and lower mortality in critically ill patients.30,31 Due to its previous successful application in high-risk industry and preliminary data suggesting it can be applied to medical teams, we hypothesized that CRM training can help improve RRT performance. The purpose of the current study was to evaluate CRM nontechnical skills training as an improvement technique for RRT performance.

In designing this study, we recognized that training and maintaining the skills of every potential responder to an RRT (nurses, trainees, rotating trainees, and respiratory therapists) was expensive and impractical to sustain. We targeted the team leaders of our RRT to have a high impact and minimize the cost and time barriers associated with comprehensive CRM training programs.32,33 We hypothesized that training the team leader would improve the overall team performance as well as completion of ideal tasks during RRT activation.

Methods

Setting and Design

This study was conducted on the acute care floors of a single, urban, tertiary 313-bed pediatric medical center from March 2015 to December 2015. This institution averages 50 RRT activations per month. The RRT consists of personnel from critical care medicine (CCM) including a critical care fellow or nurse practitioner (NP), an ICU charge nurse, and an ICU respiratory therapist (RT) who joins the acute care floor team consisting of the bedside nurse, floor charge nurse, primary team physician or physician extender, and floor RT at the patient’s bedside. A pre- and postintervention design was used to assess leadership and team performance during RRT activations. Observations of RRT activations occurred before and after team leader training in CRM. Observations were conducted in situ to assess the impact of our training in real-life hospital environments with challenges of stress, time-pressure, and coordination of personnel unfamiliar with one another.

Team Leader Training

Our team leader training consisted of 2 sessions that were conducted by physician champions with knowledge of CRM principles. The first 90-minute training session was built on adult learning theory.34 The learners engaged in active learning and were introduced to evidence that rapid response systems reduce patient morbidity while being instructed on the ideal RRT task list and CRM principles. First-year CCM fellows and NPs attended the first session because it was designed to target junior practitioners. The second 60-minute session used simulation training to reinforce CRM principles and apply them to common RRT scenarios. All CCM fellows and NPs attended the second session. We defined junior practitioners as NPs and first-year fellows because these providers were new to both our ICU and to the team leader role on the RRT.

Observations

Members of the research team (A.C., A.W.) conducted all in situ observations based on the observer’s availability when the RRT system was activated. RRTs at our institution are activated via pagers. The observers carried pagers and would go directly to the patient room when an RRT was activated. The ICU team has 15 minutes to respond to the page allowing for the observer to be in place before the ICU team arrival. All observations were during the day and represent a convenience sampling.

The RRT observations were conducted in 3 phases. Ten RRTs were observed in the project development phase to validate and test the data collection instrument. These observations were not included in the final analysis. During this phase, both observers were present at each RRT observed. The second phase was the preintervention stage, where 36 RRT activations were observed during the spring and early summer of 2015. The intervention, team leader training, occurred during the midsummer of 2015 with 37 postintervention observations occurring in the late summer and fall. During the second and third phase, only 1 observer was required to be present, although 20% of observations in these phases had both observers present. Interrater reliability was assessed using the observations that both researchers attended.

The observers used a paper data collection tool and recorded patient demographics and characteristics of the RRT events (eg, patient age, sex, mean duration of the event, and who called the RRT). The disposition status was also recorded to determine which patients were transferred to the ICU. The observers are not members of the RRT and although they may be identified by RRT members as researchers, the nature and purpose of their observations were not disclosed until the end of the study.

Measures of Leadership and Team Performance

Two metrics were used to assess team performance: 1 to observe the completion of tasks essential to RRT activations and a second to assess the nontechnical skills of the RRT. A focus group of key stakeholders in RRTs at our institution, including ICU physicians, nurses, RTs, acute care nurses, nurse educators, and acute care physicians, identified 12 ideal tasks essential to RRT function. The task list was developed from consensus opinion and previously published research. Elements previously shown to enhance team performance and patient outcomes, including using standard safety language such as Situation Background Assessment Recommendation (SBAR) to enhance shared mental models, formulating and stating a clear plan if the patient is going to remain on the acute care floor, and using cross-check techniques to identify concerns with the plan key elements of the task list.28,3538 The 12 ideal tasks were: (1) measuring patient vital signs; (2) ICU team introduces themselves; (3) ICU team leads team introductions; (4) acute care team presents using SBAR; (5) team leader asks for additional input from the acute care team; (6) ICU team assesses patient; (7) ICU team formulates a clear plan with disposition; the plan delineates a time frame for (8) interventions, (9) task delegation, and (10) an escalation plan given with expectations of outcome; (11) acute care team repeats back plan; and (12) the ICU team asks for final concerns and questions.

The nontechnical skills of the team were scored using the Team Emergency Assessment Measure (TEAM), a 12-item tool validated to measure the performance of medical emergency teams.3941 The TEAM tool includes 12 statements and offers subscore composites in leadership, team work, and task management measured on a 0 to 4 Likert scale, ranging from “never/hardly ever” (0) to “about as often as not” (2) to “always/nearly always” (4). Overall global team nontechnical performance was measured using a Likert scale ranging from 0 to 10.

Adherence to each item on the RRT ideal task list was recorded during the event. Within 15 minutes after the event, the observer rated the nontechnical skills of the team using the TEAM tool. Study data were managed using REDCap (Research Electronic Data Capture, Vanderbilt University, Nashville, TN) tools hosted at Children's National Medical Center.42

Data Analysis

Sample size calculation was powered to detect a 15% improvement in the global rating of the TEAM score, resulting in an N = 35 per observation group. Descriptive statistics are presented as percentages for dichotomous variables, averages, and SDs. Pre and postintervention comparisons were presented as adherence frequencies ±95% confidence intervals (CI) for elements of the ideal RRT task list and as medians with 95% CIs for the overall and component TEAM scores. P values <.05 were regarded as statistically significant. We performed a quantile (median) regression model based on quantile regression in Stata (Stata Corp, College Station, TX) for the analysis of scores. This regression enabled us to evaluate interaction effects of patient severity defined by the need for ICU transfer. The distributions of study outcomes was not normal (parametric) and could not be transformed to meet the parametric assumption required for linear regression modeling. Estimating the median provided a nonbiased estimate of the central tendency of the sample distribution and by inference of the population from which the sample was drawn. When we evaluated discreet (categorical) outcomes, we relied on logistic regression, which allowed us to estimate the change in odds of the outcome associated with a unit change in the independent variable (predictor) of interest. All statistics were performed on Stata 2013 software.

Results

Interrater reliability was determined to be high in all categories, including adherence to ideal RRT tasks (intraclass correlation coefficient [ICC] = 0.97), TEAM rating in leadership (ICC = 0.98), team work (ICC = 0.99), task management (ICC = 0.96), and global rating (ICC = 0.97).

Patient demographics and characteristics of the RRT events are listed Table 1. Leader demographics pre- and postintervention were uniform with 56% of the RRT activations led by junior practitioners preintervention and 51% postintervention (P = .81).

TABLE 1

Patient Demographics and Characteristics of RRT Activations Pre– and Post–CRM Training

The regression model showed a significant interaction effect when grouping the events in terms of disposition: transfer versus no transfer to the ICU. Due to this interaction, adherence rates for items on the ideal task list pre- and postintervention are presented in Table 2 by transferred versus nontransferred. After the team leader training, the team leader introduced themselves (40% preintervention vs 90% postintervention, P = .004) and led introductions of the team more often (7% preintervention and 45% postintervention, P = .03) for patients who required ICU transfer as highlighted in Fig 1. Although no member of the floor team participated in CRM training or had been trained with the task list, floor team members presented their patient concerns using SBAR more often in both the ICU (20% preintervention to 65% postintervention, P = .01) and nontransfer (28% preintervention to 64% postintervention, P = .03) groups postintervention. For patients who were transferred to the ICU, the occurrence of cross-check behavior improved 10-fold, from 7% preintervention to 70% postintervention (P = .002).

TABLE 2

Percentage Adherence to Ideal RRT Tasks Pre– and Post–CRM Training

FIGURE 1

Improvement in RRT ideal tasks adhered to pre– and post–CRM training in patients transferred to the ICU. Error bars represent 95% CIs.

Evaluation of nontechnical skills and team dynamics using the TEAM tool is displayed in Table 3. All 3 composite measures (leadership, 2.5 [95% CI, 2.1–2.9] to 3.5 [95% CI, 3.1–3.9]; teamwork, 2.7 [95% CI, 2.3–3.1] to 3.7 [95% CI, 3.4–4.1]; task completion, 2.9 [95% CI, 2.5–3.2] to 3.8 [95% CI, 3.5–4.1]) and the global rating (6.0 [95% CI, 4.9–7.1] to 9.0 [95% CI, 8.0–10.0]) increased for the events with patient transfer to the ICU. In addition, the task management composite score for the events with no transfer to the ICU (3.3 [95% CI, 3.0–3.6] to 3.8 (95% CI, 3.5–4.2) improved significantly after the CRM intervention. The results for the events with an ICU transfer are highlighted in Fig 2.

TABLE 3

Median TEAM Composite Subscores Pre– and Post–CRM Training

FIGURE 2

Bar and whisker plot for median TEAM composite subscores pre– and post–CRM training in patients transferred to the ICU.

Discussion

In this pre- and postintervention direct observational study of team dynamics, we implemented a short, pragmatic, and cost-efficient CRM training session for the team leaders. We were able to demonstrate improved leadership and overall team performance of our RRT in the real hospital environment with the use of skills learned at the simulation center to crises in real clinical settings.43

Significant effects of this training were found primarily for RRTs caring for patients who were transferred to the ICU, with poorer performance scores before the intervention compared with events where the patient was not transferred. In other words, postintervention scores in patients transferred to the ICU reached levels similar to the preintervention, non-ICU transfer group. Higher acuity patients may increase the stress of a situation, create more chaos, and lead to team breakdown.44,45 We believe that focusing education on nontechnical skills allowed team leaders to gain insight into the impact of their tone and leadership style during events, especially when the outcome was transfer to the ICU. Concrete steps, such as leading introductions and facilitating an SBAR presentation, may have diffused the stress and chaos of a team trying to care for a rapidly deteriorating patient. The leaders may have also learned how to create a safe environment and reduce the burden of hierarchies on team performance.

Interestingly, we would have postulated that RRTs caring for patients who remain on the floor after RRT involvement might have had poorer team performance based on previous research.4650 ICU team members often perceive that floor patients do not require ICU expertise, transfer occurs primarily due to nurse staffing issues on the acute care floor, and leaving the ICU for a RRT call compromises the care of patients in the ICU. Floor team members often perceive that they have difficulty accessing the resources they need on the floor, have received punitive responses from the ICU team for activating the RRT, and experience stress involved in having to justify a transfer. More research that specifically examines the flow of the 2 types of events is needed to additionally refine nontechnical skill development interventions.

RRTs are fast-start, multidisciplinary teams composed of members from different professions and specialties who need to quickly establish trust and good communication in high pressure situations. Most implementations of CRM training occur in small fixed teams and include all team members. We adapted CRM to unfixed teams and trained only 1 vital team member, the leader. Although full-staff training in CRM principles has clear benefits for patient safety, the start-up costs balanced by concerns over efficacy make comprehensive implementation unfeasible at most institutions.3133,5153 We were able to incorporate a reproducible, short, and sustainable training program into an existing yearly critical care medicine education structure. This also helps to address concerns for training new staff and accounts for turnover. Finally, this strategy helps to reinforce the training before the skills expire.

By training the team leader in CRM principles, our data suggest that the entire team performance improved. Targeting a high impact person and affecting team behavior have also been studied in cardiac arrest teams with similar results.32 The task management composite subscore of the TEAM instrument was higher posttraining for both the patients who were transferred and those who were not. In addition, one of the ideal tasks, presentation of the event in SBAR format, improved after training of the leader without any training of the frontline providers for both groups. It may be that the leader provided prompts for this type of complete report, especially the “recommendation” component of SBAR, which may be difficult in hierarchical environments, such as a standard RRT activation. This finding suggests that training communication skills, including how to create a shared mental model, resulted in a safer environment for nurses and trainees to speak up and express their perspective.

There are certain limitations to this study. The Hawthorne effect, the potential change in behavior of the team leader due to awareness that their behavior was being observed, especially after the intervention, was a limitation that we anticipated. We aimed to minimize this by blinding RRT members to the aims of the study and specific metrics used in observation. Furthermore, it is common for our RRT team leaders to have experience with a varied number of team members with other types of observers, such as trainees, new staff, or administrative personnel, depending on the time of day. Another limitation we anticipated was the lack of independence of the ideal task list and TEAM metric, because both were assessed by the same observer. If the team completed the expected tasks on the list, it is plausible that the observers rated the TEAM metric higher. We attempted to limit this bias by using a strict data dictionary a priori for the TEAM metric Likert scores. Finally, the observations were conducted during normal business hours as part of a convenience sample due to research staff availability. Future research that observes RRT activations at night, when fatigue and decreased provider availability are likely, may give additional insight into team dynamics and performance.

Ultimately, we observed an improvement in objective task completion as well as in a validated metric for team performance. Our findings support the evidence that CRM improves nontechnical skills and add that CRM can be applied to RRTs. These findings help highlight the benefit of nontechnical skill education in the trainee curriculum.

Conclusions

Two short, successive simulation-based CRM trainings of RRT team leaders effectively improved team dynamics during RRTs for patients who were transferred to the ICU. The training also improved completion of ideal tasks during RRT activations and affected the behavior of those trained and untrained. Targeting high impact team members and training them in CRM principles improved the performance of the entire team.

Acknowledgments

We thank Abdullah Al Shammari, who worked on the manuscript, provided valuable background research, and contributed to the creation of the observation tool.

Footnotes

  • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

  • FUNDING: This work was supported by awards UL1TR000075 and KL2TR000076 from the National Institutes of Health National Center for Advancing Translational Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health. Funded by the National Institutes of Health (NIH).

  • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

References