Prioritizing a Research Agenda: A Delphi Study of the Better Outcomes Through Research for Newborns (BORN) Network
Background: There is a paucity of evidence to guide clinical management for term and late preterm newborns. The Better Outcomes through Research for Newborns (BORN) network is a national collaborative of clinicians formed to increase the evidence-base for well newborn care.
Objective: To develop a consensus-based, prioritized research agenda for well newborn care.
Design: A two-round modified Delphi survey of BORN members was conducted. Round 1 was an open-ended survey soliciting 5 clinical questions identified as important and under-researched. Using qualitative methods, 20 most common themes were extracted and transformed into research questions. Round 2 survey respondents ranked the top 20 questions using a 5- point Likert scale and a quantitative analysis was conducted.
Results: Round 1 survey generated 439 unique research questions that fell into 57 themes. In the Round 2 survey, the highest rated questions were: 1) At what weight-loss percentage is it medically necessary to formula supplement a breastfeeding infant? 2) What is the optimal management of infants with neonatal abstinence syndrome? 3) How and when should we initiate a workup for sepsis, and how should these newborns be managed?
Conclusions: Research priorities of clinicians include criteria for medically indicated formula supplementation of the breastfed newborn, management of neonatal abstinence syndrome and management of newborns at-risk for sepsis.
In the United States, birth newborn hospitalizations outnumber those for all other pediatric age groups combined at a ratio of ~3 to 1.1 More than 90% of all newborns are cared for in routine birth hospital settings outside of NICUs.2,3 In a 2004 policy statement, the American Academy of Pediatrics (AAP) stated that routine newborn hospital care is essential for a safe, healthy transition home.4 Outcomes related to the newborn hospitalization such as breastfeeding rates and readmission increasingly represent areas for quality measurement in pediatric hospital care. Moreover, national trends, such as the rise in the late preterm birth rate and the more recent increase in the incidence of neonatal abstinence syndrome, underscore the importance of effective approaches to routine newborn hospital care to prevent adverse outcomes. Although routine newborn care represents such a large proportion of all pediatric hospital care, few specific routine newborn interventions have been rigorously evaluated. In the absence of a good evidence base, clinicians caring for newborns may rely on varying historical practices and individual clinical judgment. To develop a more robust and useful evidence base, it is crucial to promote research that addresses frequently occurring clinical scenarios. One way to identify such scenarios is to query clinicians who have active well-newborn practices.
Studying pediatric care through large research networks of health care providers has proved useful across a range of settings.5 The AAP and the Academic Pediatric Association have long recognized the importance of research networks and have sponsored such organizations as the Pediatric Research in Office Settings (PROS) network, Pediatric Research in Inpatient Settings, and Continuity Research Network. Other research networks, such as the Pediatric Emergency Care Applied Research Network, have developed important clinical practice guidelines (eg, the closed head trauma study).6
To this end, in 2010, the Better Outcomes through Research for Newborns (BORN) network was developed. BORN was established by pediatric clinicians and researchers with expertise in normal newborn care from hospitals with academic affiliations across the United States. BORN was developed with the support and as a core function of the Academic Pediatric Association. Currently, there are 348 BORN members representing >60 hospitals in >22 states, caring for >100 000 newborns per year.7 The mission of the BORN network is to improve the health of newborns and their families by establishing an evidence base for optimal outcomes.
Many research networks have used Delphi studies to develop a consensus-based, prioritized research agenda for their members by using qualitative methods8–12 Delphi methods are a structured approach to transforming expert opinions into group consensus for decision-making by using an iterative process of questionnaires followed by controlled opinion feedback.13,14 From interviews of pediatric experts to electronic surveys of large numbers of providers, pediatric networks such as the Australian Paediatric Research Network and PROS have used Delphi methods to query their membership and prioritize research questions of importance.
With respect to BORN’s mission, the aims of the current study were to identify clinically oriented, important questions regarding newborn hospital care for which research is needed and to rank these questions in order of importance. Such identification and prioritization of a research agenda may have a broad impact on the hospital care of US newborns by influencing the quality and breadth of, and investment of resources into, clinical investigations for this important population.
Study Design and Subjects
A modified Delphi method with electronic surveys of all BORN members was used for this study. We adapted our surveys from a similar study conducted by the PROS network in 2006 to identify research priorities of outpatient pediatricians.9 The current study used a 2-round, ranking-type Delphi method, similar to previous work by Rudolph et al11 and as recommended by Okoli and Pawlowski.15 In round 1, we elicited a broad set of research questions important to members; in round 2, we established research priorities. Institutional review board approval for this work was obtained from Children’s Mercy Hospitals & Clinics, Kansas City, Missouri. The study sample included all 348 clinicians who were members of BORN at the time of the survey. These clinicians provide routine newborn medical care during birth hospitalization and represent 60 unique newborn nursery sites across 22 states. An electronic newsletter announcing this study was sent to all members in the months leading up to the survey implementation.
Round 1 Survey
We administered the round 1 survey electronically to all BORN network members in May 2012 via SurveyMonkey. To identify the full breadth of issues important to respondents, the following open-ended question was posed to members: “Thinking about your clinical practice, please list 3 or more routine nursery decision questions which are extremely important and, to date, inadequately addressed in the medical literature.” Up to 5 responses were recorded from each respondent. We sent electronic reminder surveys in 2-week intervals to nonresponders up to 5 times. In addition, respondents were asked to provide demographic information, including age, gender, type of training/education received, and practice setting. These demographic data for respondents were not tied to the individual responses. In September 2012, the round 1 survey was closed.
A qualitative analysis of round 1 responses was conducted to identify common themes that would be used for the subsequent quantitative survey. Excel spreadsheets were used to record the individual questions and to extract themes. Two research team members (E.S., N.K.G.) independently evaluated each of the questions generated by round 1 respondents and assigned them at least 1 theme that described its topic area. As an example, 13 respondent questions were collapsed into 1 theme, “group B streptococcus [GBS] guideline adherence,” in which specific topics included length of stay and utility of screening laboratory studies for well-appearing, GBS-exposed infants. In another example, 25 respondent questions were collapsed into the theme, “defining normal glucose levels and/or hypoglycemia screening.” Specific topics included appropriate timing for glucose screening as well as differentiation of true hypoglycemia versus physiologic glucose nadir in asymptomatic newborns.
When appropriate, we assigned primary and secondary (and possibly tertiary) themes. For example, the response “How well does an infant need to be breastfeeding in order to be ready for discharge?” was assigned a primary theme of “assessing readiness for discharge” as well as a secondary theme, “assessment of feeding adequacy.” For many questions, themes emerged with varying levels of topic specificity. As an example, respondent questions touched on a variety of topics related to care of late preterm infants. These topics included vital sign monitoring, nutrition and feeding, the utility of car seat trials, and discharge criteria for this population. The question “Late preterm bilirubin management–what do levels mean in the 34 weeker?” was assigned a theme of “care of the late preterm infant” but was also grouped into a separate theme of “hyperbilirubinemia screening.”
Using an iterative process, the 2 research team members compared and combined their individual lists of theme extractions. Discrepancies in thematic assignments for each question were discussed and resolved by consensus. The full study team, which included physician members of the BORN research committee as well as a nonphysician research coordinator, reviewed the thematic extraction process, and thematic categories were edited for clarity as needed. An Excel worksheet was used to tabulate the frequencies of all unique themes, and the 20 most common were selected.
With discussion and feedback from the full study team, the coding researchers developed and refined representative research questions with a goal to achieve 20 clearly stated but not overly specific questions that reflected the nature of the original questions assigned to the theme. For example, the theme “group B streptococcus (GBS) guideline adherence” was transformed into the question: “What is appropriate management of newborns with inadequate intrapartum prophylaxis for GBS?” These questions were subsequently used for the round 2 survey.
Round 2 Survey
The round 2 survey was administered electronically to all BORN network members in November 2012 via SurveyMonkey. The survey stated: “Thinking about your membership in the BORN network and your nursery practice, please rate each research question for its importance in expanding the evidence base for clinical decision-making.” Responses were collected by using a 5-point Likert scale for each research question ranging from 1 (not important) to 5 (extremely important).14 We sent reminder surveys in 2-week intervals to nonresponders for a total of 5 times. The round 2 survey was closed in January 2013.
Mean Likert scores of responses to each of the 20 research questions included in the round 2 survey were calculated. The questions were then ranked in order of descending importance.
Of 348 total BORN members, 135 (39%) responded to the round 1 survey. Characteristics of the respondents are summarized in Table 1. The majority of respondents were female, were trained exclusively in pediatrics, and practiced in academic teaching hospitals.
The 135 respondents generated 439 unique research questions addressing a wide range of topics. These questions addressed typical clinical issues (eg, hyperbilirubinemia, cardiac screening, neonatal abstinence syndrome), as well as less common clinical scenarios (eg, care of infants born to mothers with systemic lupus). Questions also focused on psychosocial issues (eg, family-centered rounding, effective anticipatory guidance) and medicolegal aspects (eg, parental refusal of care, billing and documentation). Using the qualitative methods described earlier, a final list of 57 themes was developed. Of these, the 20 themes that emerged most frequently from respondent questions are listed in Table 2.
The most frequently occurring theme, “promotion of successful breastfeeding,” was assigned to 62 responses that encompassed a broad range of questions related to breastfeeding. Examples of these questions are listed in Table 3.
Themes not in the top 20 that emerged from several questions included “management of infants with hyperbilirubinemia,” “circumcision topics,” “vital sign monitoring,” and “workup of infants with intrauterine growth retardation/small for gestational age status.” The remaining 37 themes also included 12 respondent-generated topics that were not collapsible into another more general topic. Examples of these miscellaneous topics included: workup of bilious emesis, pain management for procedures, soap versus nonsoap cleaners for use during newborn bath, and management of upper extremity birth injuries.
Of 348 total BORN members, 150 (43%) responded to the round 2 survey. The mean Likert scale score for each of the top 20 questions ranged from 3.41 to 4.11 (Table 4). The 3 research questions generating the highest mean scores, indicating that they were of greatest importance to respondents, were: (1) What degree of weight loss requires formula supplementation in a breastfeeding newborn? (2) What is the optimal management of infants with neonatal abstinence syndrome? and (3) How and when should we initiate a workup for sepsis, and what is the optimal management for infants requiring such a workup?
The first question, generating a mean Likert score of 4.11, was generated from 12 responses in the round 1 survey. Each of the responses is listed in Table 5.
The second question regarding neonatal abstinence syndrome, generating a mean Likert score of 4.05, was generated from 35 responses in the round 1 survey. Examples of these are listed in Table 3 under the theme, “Management of neonatal abstinence syndrome.”
The third most highly ranked question, generating a mean Likert score of 4.04, focused on screening infants for neonatal sepsis. Thirty-six round 1 survey responses had been used to generate this question, examples of which are also listed in Table 3 under the theme “Workup for infection.”
To our knowledge, this study is the first to report the research priorities of clinicians caring for healthy newborns. Research concordant with these priorities is likely to have maximal utility for clinicians who otherwise may depend on guidelines and consensus statements based on observational studies or expert opinion, often extrapolated from data on premature or ill infants in ICU settings.15–17
The top 3 research topics developed by respondents from the BORN network membership are reflective of the unique nature and current realities of newborn care. First, a major goal of newborn nursery care is the promotion of successful breastfeeding. Because most normal newborns are discharged before the time when their mothers’ breast milk production is established,18 identifying mother–infant dyads at risk for breastfeeding problems during the postpartum hospitalization is challenging. Overall, research specifically focused on this population may improve evidence-based recommendations, thereby minimizing unnecessary interventions (eg, prolonged formula supplementation) and unwarranted variation in lengths of stay and costs. Second, many of the guidelines on screening for and treating an infant with neonatal abstinence syndrome are based on recommendations that are ~40 years old and based on consensus rather than data.19 The utility of these guidelines is unclear in an era of increasing neonatal abstinence syndrome resulting from both illicit drug use and prescription opiate analgesics in the setting of new treatments for addiction in pregnant women. Third, with the implementation of maternal screening and intrapartum antibiotic prophylaxis, the rate of early-onset GBS disease, the most common cause of sepsis in term neonates, has fallen by >80% and is now estimated to occur in ~0.35 per 1000 births.20 Given the low rate of sepsis in newborns receiving routine postpartum care, many healthy infants are exposed to testing and treatment. For this cohort of patients, there remains a dearth of evidence on how to minimize testing and interventions while maximizing the chances of detecting newborns with a positive blood and/or spinal fluid culture result. Current recommendations of the Centers for Disease Control and Prevention and AAP policy statements on the prevention and treatment of GBS21 may therefore benefit from supporting evidence in these low-risk settings.
These consensus-based results can help align priorities and resources to maximize the utility of research and the effectiveness of funding. Delphi studies used in pediatrics to generate consensus-based research priorities have resulted in research programs that, in turn, have transformed practice.22,23 Research has shown that uptake of new information by clinicians can be remarkably slow.24 Linking the questions and needs of clinicians to well-designed research is key to making the all-important leap across the quality chasm to translate research into practice.
Importantly, a prioritized research agenda can also enable the coordinated efforts of a clinical research network. Pediatric clinical research networks that have engaged clinicians in research generate medical knowledge which is patient-centered and outcome-oriented. Furthermore, networks can maximize efficiency as front-line clinicians and practices gain experience with clinically based research.25 One of the major challenges for a research network is funding. Developing a prioritized agenda may help networks match the interests of their members with funding opportunities.
This study had several limitations. First, the response rates to both surveys were <50%, introducing the possibility of nonresponse selection bias. However, feedback from 150 clinicians with active newborn practice is the largest sample to date on which to base conclusions about research priorities. Second, clinicians practicing in an academic setting are overrepresented in the BORN network. Research priorities of clinicians in nonacademic settings might therefore differ from those described here. Further research among nonacademic pediatricians would also be useful. Third, inherent to a Delphi study, the initial qualitative component requires expert consensus and allows some influence from the authors’ interpretations of results. However, the second quantitative component ensures that the results reflect the correct ranking of the priorities among those generated by the first round.
US clinicians providing care during routine newborn hospitalization identified a wide range of topics as research priorities. Using a 2-round Delphi process, we developed a consensus-based, ranked research agenda to inform pediatric care in this setting. The following are top priorities for research based on survey data from BORN members: (1) The percentage of weight loss at which it is medically necessary to supplement a breastfeeding infant; (2) the optimal management of infants with neonatal abstinence syndrome; and (3) evaluation and management of newborns at risk for sepsis. These data can inform the investment of resources into high-quality, impactful research that will expand the evidence base for US normal newborn care and have direct implications for clinical practice
The BORN Research Committee members thank the >150 survey participants from 59 nurseries.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Research coordination was provided by the Academic Pediatric Association.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- American Academy of Pediatrics
- Better Outcomes through Research for Newborns
- group B streptococcus
- Pediatric Research in Office Settings
- Owens P,
- Thompson J,
- Elixhauser A,
- Ryan K
- 4.↵American Academy of Pediatrics Committee on Fetus and Newborn. Hospital stay for healthy term newborns. Pediatrics. 2004;113(5):1434–1436.
- Slora EJ,
- Harris DL,
- Bocian AB,
- Wasserman RC
- Kuppermann N,
- Holmes JF,
- Dayan PS,
- et al
- 7.↵Academic Pediatrics Association. Better outcomes through research for newborns (BORN) network. Available at: www.academicpeds.org/research/research_BORN.cfm. Accessed April 21, 2014.
- Adler M,
- Ziglio E
- Likert R
- Riley LE,
- Stark AR,
- Kilpatrick SJ,
- Papile L-A
- 18.↵Adamkin DH; Committee on Fetus and Newborn. Postnatal glucose homeostasis in late-preterm and term infants. Pediatrics. 2011;127(3):575–579.
- Schanler RJ,
- Dooley S,
- Gartner LM,
- Krebs N,
- Mass SB
- Verani JR,
- McGee L,
- Schrag SJ
- Kennedy K
- Wasserman RC,
- Slora EJ,
- Bocian AB,
- et al
- Copyright © 2014 by the American Academy of Pediatrics