OBJECTIVES: Repeat visits represent up to 45% of mental health (MH) presentations to emergency departments (EDs) and are associated with higher health care costs. We aimed to synthesize available literature on predictors of pediatric MH repeat ED visits and differences between repeat visitors and nonrepeat visitors.
METHODS: A systematic review was performed using PsycINFO, PubMed, and CINAHL databases. Reporting followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement checklist. Methodologic quality was assessed using the following 8 criteria: design, generalizability, breadth of predictors, reporting of effect sizes, additional outcomes, interaction terms, confounding variables, and clear definition of repeat visits.
RESULTS: A total of 178 articles were retrieved; 11 articles met inclusion criteria. Quality assessment revealed that all studies used chart review or administrative data. Predictors were grouped into 3 categories: demographic, clinical, and MH care access and utilization factors. Common predictors associated with repeat ED MH visits included socioeconomic status, involvement with child protective services, and previous and current MH service use. For studies using a 6-month repeat window, the most common factors were previous psychiatric hospitalization and currently receiving MH services. Heterogeneity in statistical analyses and determinants explored precluded the use of meta-analysis.
CONCLUSIONS: Findings revealed that repeat visits to the ED for MH concerns is a complex phenomenon that can be attributed to various demographic, clinical, and MH care access and utilization factors. To further elucidate the strongest predictors, future prospective research should consider prospective designs and include family factors. Investigating recency and frequency outcomes can also inform clinical practice.
The emergency department (ED) plays a critical role in the care of children and adolescents with mental health (MH) concerns.1 There has been a noticeable increase in the number and proportion of pediatric ED presentations for MH problems in Canada and the United States, primarily attributable to patients without an emergent presentation.2–7 This increase may reflect unmet MH needs and barriers to accessing MH services in the community.1 For many children and families in crisis, EDs are often the first point of contact with the MH care system because they are always accessible and do not require an appointment or referral.8 Furthermore, evidence suggests that some families access emergency services before contacting their primary care physician.1
MH presentations can be unique in their level of complexity and most EDs are not as well equipped to deal with complex MH visits compared with other medical presentations.9 There is little consensus as to which protocols or models are required to best serve MH disorders within the ED.10,11 There also lacks clear standards for assessments, treatments, clinical tools, and MH training for ED physicians.2,12 Effective management and appropriate detailed discharge plans, whether inpatient or outpatient, are important for reducing symptom worsening and recurrent use of the ED.12 A substantial portion of ED MH visits made by children and adolescents (20%–45%) are return visits,6,13 possibly suggesting that patients’ needs are not being met after discharge, even though outpatient providers are often accessed.14 Within overloaded EDs, repeat visits are a significant economic and resource burden.14 Patients who revisit the ED within a short time (<6 months) incur the highest MH care costs.14 Therefore, a strong understanding of which MH patients are at risk for becoming a repeat ED visitor is needed to develop targeted interdisciplinary care management plans.15
To address this issue, the primary objective of this study was to systematically review the literature for predictors of pediatric MH repeat ED visits and differences between repeat visitors and nonrepeat visitors.
A systematic review of the literature was performed using PsycINFO, PubMed, and CINAHL databases. Search terms were related to children and adolescents, EDs, return visits, and psychiatric disorders. The reference lists of identified and related articles were screened to find potential additional publications. The initial search was conducted in November 2014 and was updated in January 2016. The protocol was developed a priori and reporting follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement checklist.16
The search was restricted to observational cohort studies published in English after 1980. Earlier studies were not included because the many changes to the MH care system and categorization of MH disorders have likely rendered them less relevant. Studies were included if the outcome analyzed was repeat pediatric presentations to the ED for MH concerns. Studies were excluded if (1) their sample included adults (>18 years of age), (2) the visits were made for non-MH reasons, (3) the outcome was a psychiatric rehospitalization (readmission to an inpatient unit), (4) the outcome was a repeat suicide attempt, or (5) the outcome was a repeat visit specific to a particular presenting problem (eg, self-harm, drug or alcohol abuse) as these studies overlooked other MH ED presentations that might have occurred during the study period.
Two reviewers (S.L., C.P.) independently screened titles, abstracts, and full texts and then extracted data from included studies. Any disagreements not resolved through consensus required consultation with a coauthor (P.C.). Data were extracted by using a standardized form, which included study and population characteristics, predictive variables, and results. In 4 instances, primary authors of included and eligible studies were contacted to clarify uncertain or unreported information. The same 2 reviewers independently rated all included articles on 8 criteria to assess methodologic quality. This study was registered with the International Prospective Register of Systematic Reviews (CRD42016032329).
The search identified a total of 176 articles: 21 from PsycINFO, 130 from PubMed, and 25 from CINAHL. Nine studies met all inclusion criteria.8,14,17–23 Two additional studies also met inclusion criteria and were identified through references and our own work.13,24 See Fig 1 for the flow diagram. Characteristics of included studies can be found in Table 1. Heterogeneity in reported statistical analyses, determinants explored, and the small number of studies meeting inclusion criteria precluded the use of meta-analyses to pool and interpret study results. Thus, a narrative and graphical synthesis of review findings is presented and grouped by study variables.
Demographic factors examined in the literature were age, biological sex, indicators of socioeconomic status, race/ethnicity, and involvement with child protective services. Nine studies considered age8,13,14,17–19,21–23; however, the use of different age ranges and comparison groups might account for the mixed findings. Nevertheless, there appears to be a trend toward older youth being at higher risk of returning to the ED. Similarly, 10 studies examined the influence of biological sex8,13,14,17–23 and found mostly nonsignificant results, although there is some evidence that suggests girls have a higher likelihood of repeating. Seven studies examined various indicators of socioeconomic status; for example, governmental assistance and insurance status.8,14,18–21,23 Results suggest that lower socioeconomic status may be a significant predictor of return visits. Six studies also considered race/ethnicity14,18–21,23 and 4 of them found that minority status increased return visits, which might be linked to cultural and/or socioeconomic differences in MH-seeking behaviors. Finally, involvement with child protective services was a significant determinant in 3 of 4 studies.13,18,20
Clinical factors examined in the literature were suicidal ideation, intentional self-harm, psychiatric and developmental diagnoses, symptom severity, and taking psychotropic medications at the time of the index visit. Seven studies considered suicidal ideation, self-harm, and/or suicide attempts and found mixed results.8,13,17,18,20–22 There appeared to be a trend toward a higher likelihood of repeating when suicidal ideation and self-harm behaviors were of greater intensity and duration.22 Previous suicide attempt was not a predictor in 1 study,18 but a suicide attempt at the index visit became a predictor in another when other variables were taken into consideration.21 Only 1 study examined homicidal thoughts and found it to be associated with repeat visits.17 Psychiatric diagnoses included in the analyses varied greatly between studies. A few psychiatric diagnoses were associated with repeat visits: mood and affective disorders,8,13 psychotic disorders including schizophrenia,8,24 and behavior or personality disorders.18,24 There were mixed results when considering presentations related to alcohol and other substance use/abuse, as 1 study found that it increased likelihood,24 another found that it decreased likelihood,8 and yet another found no association.13 Patients with dual diagnoses (defined by a co-occurrence of a mental disorder and a substance-related disorder) were at an increased likelihood of repeating.24 Two studies looking at triage level found no association with repeat visits for high severity and a decreased likelihood of presenting for low severity.8,23 Nevertheless, there was a trend toward increased presentations when severity was measured by psychiatric comorbidity.18,23 On the other hand, having a medical comorbidity appeared to decrease return visits.23 Finally, 1 study determined that children and adolescents currently taking psychotropic medication were more likely to return to the ED.13
The most common MH care access and utilization factors examined were receipt of MH services, disposition at the time of the presentation, previous psychiatric hospitalizations, and hospital characteristics. Results suggest a strong association between past and/or current MH services and repeat visits.13,14,18,21,23 Only 1 study found that it reduced repeat visits.17 One study took a more in-depth look at how patients accessed discharge recommendations and found that having a physician follow-up visit for MH; having more than 1 follow-up visit; seeing a physician other than a general practitioner, pediatrician, or psychiatrist; and seeing a physician in a hospital outpatient clinic increased their likelihood of revisiting the ED.23 Four studies13,14,18,21 found that children and adolescents with a history of psychiatric hospitalization were more likely to return to the ED. Three studies observed that living close to the hospital (city residents) compared with those living farther away was significantly associated with repeat visits.13,14,18 One study determined that those presenting to general hospitals (compared with pediatric hospitals) were more likely to have return visits.8 Finally, in terms of time of ED presentation, nighttime visits (5 pm–9 am) were more common in repeat visitors compared with daytime visits (9 am–5 pm).17
Only 1 study looked at a family-level factor21 and determined that previous parental experience seeking MH care for their child was significantly associated with repeat visits.
A visual summary of contributing factors can be found in the provided forest plots. Adjusted odds ratios (ORs), hazard ratios (HRs), and relative risk (RRs) of repeat visits are found in Figs 2-4. Unadjusted ORs, unadjusted HRs, and χ2s converted into ORs (calculated ORs) are available in Supplemental Figs 5-7. It should be noted that many studies did not report nonsignificant statistics, particularly in the adjusted and RR/HR models. Furthermore, a very limited number of statistics were not adequate for plotting and are solely described in the narrative synthesis.
Risk of Bias Assessment. Two reviewers rated the quality of included studies based on a modified version of the Newcastle Ottawa Scale for Cohort Studies criteria: (1) Design. All studies relied on medical chart reviews or administrative databases. This introduces methodological limitations because charting has been demonstrated to be less reliable for MH information.12,25 Only 1 study relied on both chart review and data collected prospectively.21 (2) Generalizability across EDs. Samples were somewhat representative, as current results reflect 7 geographical regions. Four were US urban centers, including Baltimore (John Hopkins Hospital, Maryland),14,18 Washington, DC (Children’s National Medical Center),20 Cincinnati (Cincinnati Children’s Hospital Medical Center, Ohio),21 and Ann Arbor (University of Michigan Hospital, Michigan).22 Two regions were represented in Canada: the greater Ottawa area (Children’s Hospital of Eastern Ontario, Ontario),13,17 and the province of Alberta.8,19,23 One region was represented in France (Hôpitaux de Marseille, Bouches-du-Rhône).24 Most studies included patients accessing 1 or 2 medical centers,13,14,17,18,21,22,24 whereas 3 studies included a provincial registry database of 96 to 104 medical centers.8,19,23 Research integrating data from multiple ED sites provides stronger evidence, as families might have accessed other EDs. However, we noted an overlap in the participant samples of the 3 studies that used registries from the province of Alberta8,19,23 and an overlap in the samples of the 2 studies from Baltimore city,14,18 limiting generalizability as findings might be inflated by sample specific variation. Finally, 2 studies examined samples composed of specific MH presentations (suicidal behaviors,21 anxiety and stress-related disorders23), making the predictors in these studies less relevant for predicting return visits for other MH presentations. (3) Breadth of predictors. All included studies explored demographic and clinical factors. All but 1 study24 examined MH care access and utilization predictors. Family predictors were considered in 1 study only.21 (4) Measure of effect sizes. For articles using χ2 statistics, the strength of association was not reported.14,17,18,21,22 (5) Additional outcomes. Four studies examined recency,8,13,19,23 whereas only 1 study looked at both recency and frequency of repeat visits.13 (6) Interaction terms. Three studies explored how factors might interact in their analyses.8,18,23 (7) Confounding variables. Four studies controlled for other variables in their models.8,13,18,23 Two used a washout period to ascertain the index visit.13,18 Based on a sensitivity analysis, a third study reported minimal impact on the results whether a washout period was used or not.8 Three of the studies with longer follow-up periods controlled for time-to-repeat in their analyses.8,13,19 This is important for decreasing the risk of bias, as some participants might have either turned 18 during the study period or had less time to repeat if their index visit was near the end of the study period. (8) Clear definition of repeat visits. None of the studies clearly defined what was meant by a repeat visit. In fact, repeat visit windows ranged from 2 months to 6 years, which limits our ability to draw firm conclusions from between-study differences.
This is the first published systematic review looking at predictors of pediatric MH repeat ED visits. This review uncovered 11 studies investigating predictors of repeat MH visits, repeat visit recency, and/or repeat visit frequency. Findings revealed that repeat visits to the ED for MH concerns is a complex phenomenon that can be attributed to various demographic, clinical, and MH care access and utilization factors. Although these factors do not provide information on reasons behind return visits, they can provide insights into the characteristics of patients most likely to come back.
It has been hypothesized that repeat MH visits are either a product of clinical factors (ie, worsening of symptoms) or MH care access and utilization factors (eg, difficulty accessing MH services for financial, availability, or other reasons). This systematic review suggests that traditional measures of clinical severity in the ED are not reliable indicators of future repeat visits. In fact, triage level showed either no association with repeat visits18 or that lesser severity of symptoms was associated with return visits8 matching reported increases in nonemergent presentations.2–7 Moreover, clinical severity also was not found to significantly change across repeat visits,26 indicating that traditional measures of severity based on triage may be less appropriate for predicting return pediatric MH visits or that children and their families are coming back because they are unable to access other MH services. Yet, findings from this review also showed that those currently receiving MH services had an increased likelihood of repeating.14,18,21 Currently receiving professional resources also significantly predicted recency13,23 and frequency13 of visits. Therefore, results may indicate that recent and repeated experience with the MH care system facilitate further MH-seeking behaviors, possibly because of greater knowledge, familiarity, and self-efficacy when facing the complexities of the MH care system. However, it is important to note that identifying whether the patient is currently receiving MH treatment does not give any indication as to the appropriateness of the services received or the level of the patient’s engagement/compliance with treatment. Two studies looked at the level of compliance with treatment and found conflicting results. Compliance was associated with decreased likelihood of repeat visits in 1 study,17 whereas findings from another indicated that youth who were resistant or noncompliant were less likely to repeat or repeat more frequently.13 Taken together, ED clinicians should consider indicators of clinical severity other than traditional ones, such as the presence of comorbid MH disorders.18,24 It also would be important for ED clinicians to consider the appropriateness and compliance with current MH outpatient services in determining whether further referrals or supports are needed before discharge.
Next, we examined predictors identified in studies with a 6-month repeat window,14,18,21,23 because it has been shown that 50% of psychiatric repeat visits occur within a month and 85% within 6 months of the index ED visit.18 It appeared that MH care access and utilization factors (currently receiving MH services and previous psychiatric hospitalization) were the strongest predictors of short-term repeat visits.14,18,21 Analysis of recency of repeat visit also highlighted that these variables were important in earlier return visits,13,23 which suggests that patients who repeat within a shorter time might be characterized as high service users who treat the ED as part of their MH care continuum. Comparing predictors of earlier versus later return visits might therefore be another important step in understanding how pediatric patients use the ED.
In addition, it has been suggested that to decrease families’ reliance on the ED for MH concerns, EDs should implement a multidisciplinary approach, validated pediatric MH screening tools, pediatric-trained MH consultants, and enhanced collaboration with primary care and outpatient MH services.27 Having professionals trained in pediatric MH care who work as a multidisciplinary team is desirable because they are better equipped to attend to social determinants, such as socioeconomic status, cultural factors, and involvement with child protection services, which have been shown to increase likelihood of repeat visits.8,13,14,17–20,23 This review also highlights the importance of MH care utilization factors, therefore multidisciplinary teams who can comprehensively assess, collaborate with the patient’s other providers, and advocate for patient needs might be key in reducing overuse of the ED. Furthermore, it would be important to develop memorandums of understanding with community partners, such as child protective services to establish standardized comprehensive interdisciplinary care management plans for these at-risk children and youth. Despite this evidence, current management practices in EDs continue to vary widely and there have been few evaluations of which models work best.2,9–12
Important patient-level and family-level predictors of repeat visits may have been missing in the present literature due to the limitations of retrospective designs based on chart reviews and administrative databases. In studies in which model goodness-of-fit was provided,8,18 results suggested that other variables not captured in charts or databases may have significant predictive ability. The pediatric population is unique in its dependence on caregivers to access services, and therefore family determinants should not be overlooked. Constructs such as family functioning, caregiver burden, parental stress, and parenting styles and coping strategies have been found to play a role in accessing a variety of other health and MH care services.28–32 In this review, only one such variable was examined, prior parental experience seeking MH care for their child, and found to be associated with repeat visits.21 Furthermore, the adoption of family-centered care in some EDs also underlines the recognition of the transactional nature between the child and family environment not only on the progression of MH issues, but also on access to MH services such as the ED.33 Similarly, a greater exploration of interaction terms might not only inform research, but also can be useful for decision-making. For example, in Goldstein and colleagues,18 patients who presented with disruptive behavior and were admitted at the index visit had a lower likelihood of repeating to the ED than those discharged from the hospital.
The current state of the literature is limited by the lack of consensus among the included studies on the operational definition of a repeat visit. The concept of the repeat visit is ill defined in terms of the time window between the index visit and the return visit. Some researchers have considered any visit within a few years’ time a repeat visit,8,13,24 whereas others have considered a shorter period, such as 18 months,17 12 months,19,20,22 6 months,14,18 3 months,23 or 2 months.21 In comparison, the medical ED visit literature is more concerned with the number of repeat visits and defines frequent ED use as between 2 to 20 visits per year.15,34 However, as with the MH repeat ED literature, few studies have explored the number, return window, or pattern at which striking differences in resources, demographics, or clinical factors are observed.15,34
Furthermore, it is unclear whether a repeat visit should be defined as a visit after an index visit during which the presenting symptoms were the same. The difficulty of ascertaining this lies in the lack of reliable charting and in the nature and classification of MH symptoms. For example, a child might originally present with depressive symptoms and a few months later with mainly anxiety symptoms, although both might be rooted in the same overall clinical picture. Two of the included studies have attempted to address this issue,8,14 and 1 posited that visits within a shorter time span might be more similar in their presentation.14
This article extends the pediatric MH ED literature in 3 important ways. First, our research provides an in-depth review of available evidence on repeat visitor characteristics, predictors of repeat visits, and also predictors of the recency and frequency of repeat visits, 2 new emerging outcomes. Second, we identified factors that might provide additional information for clinical decision-making and help develop optimal clinical management for MH visits. Third, we recommend that future research studies define what they consider a repeat visit (specifying and providing a rationale for recency, frequency, return-window, and washout-period decisions), report determinants in categories that are meaningful and relevant, such as the ones used in this review (demographic, clinical, MH care access, and utilization factors), and use prospective designs, not solely reliant on medical records, to elicit a greater breadth of information on factors and motivations behind return visits.
Finally, as is the case with adult repeat ED visitors, establishing the strongest determinants of return visits is necessary to develop a successful way of identifying those at risk for repeat ED visits and to reduce nonemergent visits.35 This is of importance presently, as some EDs have started to develop and evaluate new MH clinical pathways between EDs and community MH agencies to streamline pediatric MH care,36 which include standardized comprehensive interdisciplinary care management plans for children and youth.37
We acknowledge the contribution of Hiba Abdul-Fattah, a University of Ottawa Psychology undergraduate honor student, in helping to conduct the initial literature review. We also acknowledge Margaret Sampson and Janet Joyce, medical librarians at Children’s Hospital of Eastern Ontario, for recommendations and guidance of the search strategy, ratings of methodological quality, and reporting guidelines.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This study was funded by the CHEO Foundation, RBC Foundation, Fonds de Recherche du Québec en Santé (29525), and a CHEO Research Institute Resident Research Grant (RS 2014-09). These sponsors had no involvement in the study design; the collection, analysis, and interpretation of data; the writing of the report; or the decision to submit the article for publication.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- ↵Dolan MA, Fein JA, The Committee on Pediatric Emergency Medicine. Pediatric and adolescent mental health emergencies in the emergency medical services system. J Pediatr. 2011;127:e1356–e1366
- Chun TH,
- Katz ER,
- Duffy SJ,
- Gerson RS
- Mapelli E,
- Black T,
- Doan Q
- Leon SL,
- Cappelli M,
- Ali S,
- et al
- Cloutier P,
- Thibedeau N,
- Barrowman N,
- et al
- Newton AS,
- Rosychuk RJ,
- Dong K,
- Curran J,
- Slomp M,
- McGrath PJ
- Ballard ED,
- Horowitz LM,
- Jobes DA,
- Wagner BM,
- Teach SJ
- Sobolewski B,
- Richey L,
- Kowatch RA,
- Grupp-Phelan J
- Gipson PY,
- Agarwala P,
- Opperman KJ,
- Horwitz A,
- King CA
- Newton A,
- Rosychuk RJ,
- Niu X,
- Radomski AD,
- McGrath PJ
- Boyer L,
- Henry JM,
- Samuelian JC,
- et al
- ↵American Academy of Pediatrics Committee on Pediatric Emergency Medicine American College of Emergency Physicians Pediatric Emergency Medicine Committee. Pediatric mental health emergencies in the emergency medical services system. Pediatrics. 2006;118(4):1764–1767
- Brannan AM,
- Heflinger CA,
- Foster EM
- Fite PJ,
- Stoppelbein L,
- Greening L
- O’Malley PJ,
- Brown K,
- Krug SE
- ↵Provincial Council for Maternal and Child Health. Implementation toolkit: emergency department clinical pathway for children and youth with mental health conditions. Available at: http://www.pcmch.on.ca/wp-content/uploads/2015/09/1.0-Toolkit-ED_Clinical_Pathway-Tookit_Oct_29_2014_Updated-PPO.pdf. Accessed February 1, 2017
- Copyright © 2017 by the American Academy of Pediatrics