Directly Comparing Handoff Protocols for Pediatric Hospitalists
BACKGROUND AND OBJECTIVES: Handoff protocols are often developed by brainstorming and consensus, and few are directly compared. We hypothesized that a handoff protocol (Flex 11) developed using a rigorous methodology would be more favorable in terms of clinicians’ attitudes, behaviors, cognitions, or time-on-task when performing handoffs compared with a prevalent protocol (Situation Background Assessment Recommendation [SBAR]).
METHODS: Using a between-groups, randomized control trial design (Flex 11 versus SBAR) during a pilot study in a simulated environment, 20 clinicians (13 attending physicians and 7 residents) received 3 patient handoffs from a standardized physician, managed the patients, and handed off the patients to the same standardized physician. Participants completed surveys assessing their attitudes and cognitions, and behaviors and handoff duration were assessed through observations.
RESULTS: All data were analyzed using independent samples t tests. For attitudes, “ease of use” ratings were lower for SBAR participants than Flex 11 participants (P < .01), and “being helpful” ratings were lower for SBAR participants than Flex 11 participants (P = .02). For behaviors, results indicate no significant difference in the information acquired between the SBAR and Flex 11 protocols. However, SBAR participants gave significantly less information than Flex 11 participants (P < .01). For cognitions, SBAR and Flex 11 participants reported similar workload except for frustration. For handoff duration, there were no significant differences between the protocols (P = .36).
CONCLUSIONS: The results suggest that Flex 11 is an efficient, beneficial tool in a simulated environment with pediatric clinicians. Future studies should evaluate this protocol in the inpatient setting.
Providing quality patient care depends on effective communication, particularly when patient responsibility is handed off from one provider to another. Unfortunately, adverse events emerge from communication mishaps during handoffs.1 Consequently, inadequate handoffs have been associated with multiple negative outcomes, such as work effort duplications, lowered patient face-to-face time, patient discharge hindrances, and associated medication errors.2 Given these communication breakdowns during handoffs and their subsequent consequences, administrators, policy makers, and clinicians advocate for standardized handoff protocols.3,4 However, most protocols are developed from brainstorming and consensus instead of from empirical data. Although some evidence suggests that using standardized handoff protocols are better than not using standardized protocols,5–9 to our knowledge, no studies directly compare 2 different protocols to each other. Considering that protocol use is increasingly common practice, a more pertinent question is which protocol is more effective?
Because there are dozens of protocols, it is outside the scope of 1 study to comprehensively compare them all. Thus, we elected to conduct a pilot study to compare a handoff protocol developed using a rigorous methodological approach of systematic review, interviews, and card sort (Flex 11) and a prevalent protocol (Situation Background Assessment Recommendation [SBAR]). We elected to develop a customized handoff protocol because it can be designed to meet the specific needs of providers or departments.10 In addition, employee input can enhance attitudes, behaviors, and even performance and productivity.11 We selected SBAR as the comparison handoff protocol because it is recommended by the World Health Organization12 and Institute for Healthcare Improvement,13 and some consider it the “most valuable” protocol.14 To further illustrate, the Joint Commission’s Transitions of Care report “formally recommends SBAR as the industry best practice.”15 In fact, a recent systematic review16 found that almost 70% of handoff mnemonic articles use SBAR, and the protocol has been implemented successfully in an array of institutional divisions (eg, pediatrics, emergency departments, operating rooms, adult ICUs, radiology, and heart centers).17–19 With over a decade of research,20 SBAR has been studied domestically and internationally.21 The mounting evidence suggests that SBAR is associated with improved teamwork,17 better satisfaction and safety reporting,18 streamlined work processes,22 decreased time to treatment,23 strengthened safety climate,14,17,18 and fewer unexpected deaths24 compared with not using a handoff protocol. With this foundation in mind, this study seeks to determine if there are differences in clinicians’ attitudes, behaviors, cognitions, and handoff duration when performing handoffs using the Flex 11 protocol versus SBAR. More specifically, we aim to determine whether Flex 11 will be perceived more positively (attitude), facilitate information exchange (behavior), appear less cognitively taxing (cognition), and take more time to complete compared with SBAR during handoffs. The remainder of this paper will detail a pilot study held in a simulated environment comparing the Flex 11 and SBAR handoff protocols on the outcomes of interest (ie, on physician attitudes, behaviors, and cognition as well as handoff duration).
Participants and Setting
This was a between-groups, randomized control trial design (treatment [Flex 11] versus control [SBAR]) performed at a tertiary pediatric care center in a Midwestern urban setting. Within this institution, although providers had an established time for daily handoffs, they did not use a standardized approach when conducting handoffs at the end of service. As a result, both the Flex 11 and SBAR standardized approaches were new to the participants. For this study, 20 clinicians (13 attending physicians and 7 residents) enrolled between March 31, 2014 and April 23, 2014, with 12 in the treatment group (Flex 11) and 8 in the control group (SBAR). Participants were required to be pediatric or internal medicine/pediatric residents or general pediatric attending physicians who provided inpatient clinical care for a minimum of 36 hours or ∼1 week during the last year. Participants were recruited through emails, flyers, and announcements, and received $20 and cookies if they participated.
As participants enrolled, they were assigned to a condition based on a randomized list. On arrival, each participant provided consent and immediately took a short online survey to obtain background and demographic information. After this survey, each participant received training comprising information-, demonstration-, and practice-based strategies in an isolated room away from the patient floor on their respective protocol (Flex 11 or SBAR) individually by a member of the research team. More specifically, for the information-based portion, participants reviewed a short PowerPoint presentation highlighting the importance of handoffs and providing descriptions of either the Flex 11 or SBAR protocol. For the demonstration aspect, each person viewed a tutorial depicting how to appropriately use the protocol on a paper-based patient case. Finally, for the practice-based facet of the training, every participant had the opportunity to use the protocol on a new paper-based patient case. Immediately after the training, participants completed a survey measuring their cognitions about the protocols.
After the questionnaire assessing their cognitions during training, each participant immediately received handoffs of 3 simulated patients from a standardized physician using their assigned handoff protocol. Participants next simulated a shift where they managed patient care (described below). After the simulated shift, participants passed on updated information back to the same standardized physician using the same handoff protocol. Participants were videotaped while receiving and giving the handoffs of the simulated patients, allowing us to determine the length of each handoff and analyze the transcripts of the recordings to determine the number of patient facts that were transmitted during the handoffs. Directly after the simulation, a second survey was administered to gain insight about participants’ attitudes and cognitions regarding the use of the protocol during the simulation. For a visual representation of the procedure, refer to Fig 1. The study was approved by the institutional review board of the institution where it was conducted.
The first arm of the study consisted of the Flex 11 protocol, which was developed using a methodological approach, including a systematic literature review, semistructured interviews, and an online card-sort (a technique for quantifying and understanding how people organize information).25 To elaborate, the research team interviewed 13 pediatric hospitalists and second-year residents regarding what information pediatric hospitalists include in handoffs as well as specific incidents they have encountered with bad and good patient handoffs. Additionally, 25 pediatric hospitalists and residents completed an open-card sort consisting of 119 cards composed of handoff items (eg, allergies, blood pressure, nutrition), which were gleaned from the interview data and an exhaustive literature review. Participants were able to organize the terms into any number of categories as well as to name those categories and exclude term(s). They primarily sorted the cards into 10 substantive categories and an “as needed” category. Consequently, the finished protocol consisted of 10 categories with an 11th “as needed” category to enable flexibility for cases that require additional information. See Fig 2 for the Flex 11 protocol. Each participant in the treatment condition received training on the Flex 11 protocol. The training was completed immediately before the simulation and was timed to ensure participants had equal opportunity to learn the materials. During the simulation, the standardized physician used the Flex 11 protocol to deliver the handoff, and the participants were instructed to use Flex 11 to give the handoff at the end of the simulated shift.
In the second arm of the study, participants were taught and used the SBAR format (see Fig 3). All procedures were identical to the Flex 11 arm with the exception of the use of SBAR.
Regardless of the research arm, each participant had the same standardized physician who gave and received the patient handoffs using a preestablished script to enhance standardization and minimize individual differences. Each participant received and gave handoffs because we posit that providers should be competent and actively participate in both roles (receiver and giver) during a handoff. Three patients were handed off to the participants by a standardized physician playing the role of a pediatric hospitalist finishing a day shift. The patients were created from real charts with all personal information de-identified. Participants received a paper template of the assigned handoff tool in case they elected to take notes and were allowed to ask the standardized physician any questions to acquire additional patient information.
During the simulated shift, participants indicated which tasks they would perform (eg, obtaining laboratory tests or radiologic studies), diagnosed patients based on these results, and determined treatment options to care for their patients. They also updated their diagnoses and treatment plans depending on the laboratory results and studies they had ordered. Incorporating this component of the simulation provided a richer context and enabled participants to formulate an updated handoff and care plan.
We elected to use a simulated environment because simulations offer the ability to control factors that could confound the true relationship between handoff protocols and provider attitudes, behaviors, and cognitions (eg, managing competing priorities, frequent interruptions, and distractions while under serious time constraints).26 Simulations also afford the ability to use a standardized physician during the handoff; using the same person reduces handoff variability and individual differences from one provider to another.
To measure participant satisfaction with the tool during handoffs, we adapted a 3-item measure by Sawatsky et al.27 Questions include, “The standardized format was easy to use”, “The standardized format was a helpful tool”, and “The standardized handoff improves communication.” Responses ranged from 1 (Strongly Disagree) to 5 (Strongly Agree).
Behaviors: Information Exchange
To evaluate information exchange, members of the research team transcribed all communications from video recordings. The video and audio were recorded with the assistance of Morae (TechSmith), a recording and editing software platform that enables time stamping of events, such as the start and stop of the handoff. From those transcriptions, communication was unitized (ie, broken down) into the smallest meaningful statement (eg, patient name, heart rate, blood pressure, and temperature). Trained members of the research team (E.H.L., B.P., and D.C.S.) independently unitized the transcripts. To assess rater agreement, we calculated intraclass correlation, one of the most common measures of interrater agreement,28 for the number of unitized statements made by each participant. The intraclass correlation among team members was 0.79, which indicates excellent agreement.29 Performing these unitizations enabled us to calculate the frequency of information exchange within each patient handoff, which was used for the analyses. To avoid artificially inflating the frequency, we only included perceived novel statements that were pertinent to the patient handoff in the analyses.
To assess workload, we administered the National Aeronautical Space Administration Task Load Index (NASA TLX) because it is widely used and cited by >300 publications (NASA TLX Web site).30 The NASA TLX contains 6 items that assess (1) mental demands, (2) physical demands, (3) temporal demands, (4) performance, (5) effort, and (6) frustration. See Table 1 for a mapping of items to dimensions. Individuals rated each dimension from 1 (very low) to 100 (very high).
The total time spent discussing each patient was determined using video/audio recordings with Morae video editing software. Specifically, we calculated both the time spent acquiring and then passing information to the standardized physician (in minutes).
After measurement, all attitudinal, behavioral, and cognition data as well as the handoff times were analyzed using independent samples t tests to determine differences between SBAR and Flex 11. Analyses were conducted using SPSS (IBM SPSS Statistics, IBM Corporation).
Attitudes: Reactions to Protocol During Simulation
An independent samples t test established that ease of use ratings were less favorable for SBAR participants (mean 2.38, SD ± 1.19) than Flex 11 participants (4.50 ± 0.52) (P < .01). Regarding whether the format was a helpful tool, SBAR participants rated it lower (3.25 ± 1.28) than Flex 11 participants (4.33 ± 0.65) (P = .02). Communication ratings did not significantly differ between SBAR (3.13 ± 1.13) and Flex 11 (4.00 ± 0.95) (P = .08).
Behaviors: Information Acquired and Given
Using the unitized communication statements, we calculated the amount of information acquired and given as determined by the number of perceived novel statements pertinent to handoffs. Results indicate no significant difference in the amount of information acquired during the receiving handoffs between SBAR (36.75 ± 3.05) and Flex 11 (39.50 ± 4.58) (P = .15). However, participants in the SBAR group gave significantly less information during the giving handoffs (17.41 ± 2.80) than participants in the Flex 11 group (25.32 ± 5.84) (P < .01).
Cognitions: Workload Ratings
During the training phase, SBAR participants reported significantly higher frustration (38.75 ± 29.54) than Flex 11 participants (11.08 ± 13.65) (P = .01). However, no significant differences were detected in mean reported mental demand, physical demand, temporal demand, performance, or effort.
During the handoff simulation, there were no significant differences between the conditions across the dimensions of mental demand, physical demand, temporal demand, performance, effort, or frustration. The means and SDs for each workload dimension for the training and simulation are displayed in Tables 2 and 3.
There were no significant differences between the durations of the SBAR and Flex 11 handoffs across patients for the giver or receiver. Measured in minutes, participants completed the handoffs in similar time frames when receiving a handoff (SBAR, 13.79 ± 3.11 and Flex 11, 13.80 ± 3.92) and when giving a handoff (SBAR, 6.87 ± 2.65 and Flex 11, 7.85 ± 3.05).
This study compared the empirically-derived handoff protocol Flex 11 against the widely-cited SBAR protocol on several outcomes in a simulated environment. In general, we found that clinicians had more positive attitudes toward Flex 11, experienced the same cognitive demands as SBAR, gave more information using Flex 11, and spent the same amount of time conducting the handoffs regardless of protocol. The following will discuss each of these findings in greater detail and how they align with our hypotheses.
We hypothesized that clinicians would have more positive reactions to the Flex 11 protocol because the organization and nomenclature of the categories is more specific compared with the categories within SBAR. As predicted, Flex 11 was rated as being more helpful and easier to use. These reactions suggest that the protocol has utility, which is correlated with learning and on-the-job behaviors.31 Additionally, positive reactions are pragmatically beneficial because they can garner organizational support for employing tools, such as handoff protocols, in the working environment.31
We also hypothesized that more information would be exchanged using Flex 11, and the results partially support our hypothesis. Clinicians acquired approximately the same amount of information regardless of protocol, but clinicians using Flex 11 gave more information in their handoffs compared with clinicians using SBAR. Flex 11 has more granular and specific prompts for information acquisition and transmission. Such granularity and specificity facilitates presentation and recall.32
For cognitions, we hypothesized that clinicians would have lower cognitive workload when performing handoffs with Flex 11. The specificity of the Flex 11 prompts the user to address specific categories of information, and research has demonstrated that such cues are associated with reduced workload.33 This hypothesis was largely unsupported because there were no statistically significant differences during the simulation performance. However, individuals found Flex 11 significantly less frustrating compared with SBAR during training. Although only 1 dimension was statistically significant, we should note that Flex 11 was not detrimental to workload. That is, despite more information being exchanged with Flex 11, workload did not significantly increase.
Finally, we hypothesized that Flex 11 would take more time to complete compared with SBAR given that the protocol itself is longer and that more information would be exchanged. Our hypothesis, however, was unsupported. Handoffs performed using Flex 11 did not significantly differ in duration compared with handoffs performed with SBAR. Although this finding is counterintuitive, we believe that it is actually favorable because Flex 11 fostered more thorough information exchange while taking approximately the same amount of time to complete compared with the widely established SBAR, suggesting that Flex 11 was more efficient.
This study is not without limitations. One limitation is the small sample size. Despite the small sample size, the results indicate large effect sizes suggesting that the influence of the Flex 11 protocol was quite considerable. Another noteworthy limitation is that the data were collected in a simulated environment. Indeed, the utility of simulation is an important consideration in the context of patient safety.34 According to Riesenberg et al,16 “there are risks involved in implementing interventions without evidence to support their effectiveness.” As such, some postulate that simulation is a technique to examine human performance to garner insights into potential causal pathways to enhance safety.35 Consequently, the simulated environment was a necessary, safe first step in determining the impact of using a data-driven handoff protocol while still prioritizing patient safety. A third limitation is that some of the individuals who participated in the simulation also participated in the studies that contributed to the development of Flex 11. More specifically, out of the 38 participants in the Flex 11 development studies, 4 participated in the simulation study. All participants were randomized to conditions with the intent of minimizing influence. With that being said, a few individuals participated in the development of Flex 11, leaving a majority of them unfamiliar with Flex 11. To mitigate novelty and enhance familiarity of Flex 11, the research team employed a training comprising information-, demonstration-, and practice-based strategies. A final limitation is the potential generalizability of this specific tool because Flex 11 was assessed at 1 institution with their pediatric hospitalists and residents.
First, this project addressed the gap in the literature of comparing 2 handoff protocols directly. Although there are numerous available protocols, they are rarely evaluated against other protocols, leaving practitioners to question how protocols measure up in a direct comparison. This study compared SBAR with Flex 11, and within the context of this study, Flex 11 was more favorable compared with SBAR on 4 outcomes: attitudes, behaviors, cognitions, and duration.
Second, this study supports a protocol development methodology that intersects medicine and human factors and is applicable for any medical specialty. More specifically, interviews enable researchers to obtain rich data on the complexities of handoffs; in addition, card sorts offer unbiased, quantitative insights into how providers organize the information pertaining to a handoff. Card sorts can be particularly invaluable because they do not rely on the subjective perspectives that interviews do. Instead, practitioners have to group the handoff information into categories that reflect their own mental organization of the information. Using this methodology enabled a tailored handoff protocol that was valued by the providers, and presumably this methodology could be translated to other clinical domains to meet the needs of other specific users.
Third, this study offers a handoff tool that is derived from empirical support that organizes information in a meaningful way without increasing workload or handoff duration. Reasonable workload with increased efficiency enables the handoff process to be lean. Maintaining a reasonable level of workload is crucial because it is related to interruption management,36 performance,37 and quality and safety of care.38 In addition, handoff duration is important because time is often limited, and the ever-present time demands unfortunately impact cognitive workload, decision-making, and multitasking.39 Consequently, a handoff protocol that is efficient, thorough, and not cognitively taxing is not only beneficial but also necessary in the provision of patient care.
Given that this was 1 study at 1 institution, there are multiple avenues for future research. First, future work could study the Flex 11 protocol using larger sample sizes, taking into account provider expertise and patient acuity. We saw benefits within our study, but do these benefits remain with more providers? Second, researchers could evaluate the effectiveness of Flex 11 in the clinical environment by assessing the impact on clinical workflow. Our study included a simulated shift to ensure new information was accounted for in the handoff, but clinical care is complex so more work is needed. Third, future studies could investigate other clinical care providers (eg, nurses) and expertise levels (eg, trainees versus practicing clinicians). To elaborate, Flex 11 was designed for pediatric hospitalists, so would the same approach result in similar findings with other targeted providers? Additionally, does a protocol standardize the type and amount of information exchanged despite level of experience? Considering that mandates are now requiring residents to use protocols, this area is ripe for additional research. Fourth, future work could compare Flex 11 against other handoff protocols. There are many other well-established and well-studied protocols in the literature that we could have chosen as a comparison handoff protocol; however, we chose SBAR for the aforementioned reasons and excluded I-PASS (Illness severity, Patient summary, Action list, Situation awareness and contingency planning, and Synthesis by receiver) because it is an entire quality improvement curriculum that extends beyond simply a handoff protocol.40 Comparison of Flex 11 and other protocols would be beneficial in providing more concrete information on how different handoff protocols and methodologies contribute to patient care. Finally, other studies could assess the impact of protocols on clinical processes as well as other relevant outcomes, such as care plan prioritization and safety culture. Although our study saw similar handoff durations regardless of protocol, there are other important aspects of clinical care that might change based on the implementation of a protocol.
Handoff protocols are an effort to enact standardization and ameliorate communication breakdowns and subsequent problems (eg, medication, treatment, and testing errors).41 However, little research makes direct comparisons between various protocols to evaluate their effectiveness. To address this gap, we developed an empirically derived handoff protocol (Flex 11) and evaluated it against one of the most prevalent protocols (SBAR). The results suggest that Flex 11 is an efficient, beneficial tool in a simulated environment because it strengthened positive attitudes, sustained cognitions, increased communication, and maintained handoff duration.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relative to this article to disclose.
FUNDING: This work was funded by Kansas City Area Life Sciences Institute (KCALSI). All opinions expressed in this presentation are those of the authors and do not necessarily reflect the official opinion or position of Embry-Riddle Aeronautical University, Children’s Mercy Hospital, Wichita State University, or the KCALSI.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of to disclose.
- Accepted August 9, 2016.
- ↵The Joint Commission. Joint commission center for transforming healthcare releases targeted solutions tool for hand-off communications. Available at: www.jointcommission.org/assets/1/6/tst_hoc_persp_08_12.pdf. Accessed June 30, 2014
- McSweeney ME,
- Lightdale JR,
- Vinci RJ,
- Moses J
- ↵Accreditation Council for Graduate Medical Education. Common program requirements. www.acgme.org/What-We-Do/Accreditation/Common-Program-Requirements. Accessed May 9, 2016
- ↵Joint Commission. Improving America’s hospitals: a report on quality and safety. Available at: www.jointcommission.org/assets/1/6/2006_annual_report.pdf. Accessed May 9, 2016
- Bigham MT,
- Logsdon TR,
- Manicone PE,
- et al
- Wheeler K
- Kicken W,
- Van der Klink M,
- Barach P,
- Boshuizen HPA
- ↵Institute for Healthcare Improvement. SBAR technique for communication: a situational briefing model. Available at: www.ihi.org/resources/Pages/Tools/SBARTechniqueforCommunicationAsituationalBriefingModel.aspx. Accessed June 8, 2015.
- Leonard M,
- Graham S,
- Bonacum D
- ↵The Joint Commission. Hot topics in healthcare: transitions of care: the need for a more effective approach to continuing patient care. Available at: www.jointcommission.org/assets/1/18/Hot_Topics_Transitions_of_Care.pdf. Accessed June 8, 2015
- Riesenberg LA,
- Leitzsch J,
- Little BW
- ↵Safer Healthcare. What is SBAR. Available at: http://www.saferhealthcare.com/sbar/what-is-sbar. Accessed October 22, 2016
- Randmaa M,
- Mårtensson G,
- Leo Swenne C,
- Engström M
- Dingley C,
- Daugherty K,
- Derieg MK,
- Persing R
- Fincher S,
- Tenenberg J
- Sawatsky AP,
- Mikhael JR,
- Punatar AD,
- Nassar AA,
- Agrwal N
- ↵National Aeronautical Space Administration Task Load Index. Task load index. Available at: https://humansystems.arc.nasa.gov/groups/tlx/. Accessed April 8, 2015
- Hitchcock EM,
- Dember WN,
- Warm JS,
- Moroney BW,
- See JE
- Salvucci DD,
- Bogunovich P
- Huey BM,
- Wickens CD
- Arora V,
- Johnson J,
- Lovinger D,
- Humphrey HJ,
- Meltzer DO
- Copyright © 2016 by the American Academy of Pediatrics