Tools and Links

Hospital Pediatrics
November 2014, VOLUME 4 / ISSUE 6
Research Articles

Increasing Recruitment Rates in an Inpatient Clinical Research Study Using Quality Improvement Methods

Hadley S. Sauers, Andrew F. Beck, Robert S. Kahn, Jeffrey M. Simmons
Download PDF

Abstract

Objective: One important benefit of successful patient recruitment is increased generalizability of findings. We sought to optimize enrollment of children admitted with asthma as part of a population-based, prospective, observational cohort study with the goal of enrolling at least 60% of all eligible and staffed patients.

Methods: Quality improvement methods were used to improve cohort recruitment. Weekly meetings with study staff and study leadership were held to plan and discuss how to maximize recruitment rates. Significant initial variability in recruitment success prompted the team to use small-scale tests of change to increase recruitment numbers. A number of tests were trialed, focusing primarily on reducing patient refusals and improving recruitment process efficiency. Recruitment rates were calculated by dividing eligible by enrolled patients and displayed using annotated Shewhart control charts. Control charts were used to illustrate week-to-week variability while also enabling differentiation of common-cause and special-cause variation.

Results: The study enrolled 774 patients, representing 54% of all eligible and 59% of those eligible for whom staff were available to enroll. Our mean weekly recruitment rate increased from 55% during the first 3 months of the study to a statistically significant sustained rate of 61%. This was sustained given numerous obstacles, such as departing and hiring of staff and adding a second recruitment location.

Conclusions: Implementing quality improvement methods within a larger research study led to an increase in the rate of recruitment as well as the stability in recruitment rates from week-to-week.

Clinical research benefits from successful patient recruitment to increase generalizability. Many researchers encounter recruitment challenges that limit conclusions. Recruitment in pediatrics presents unique challenges; studies must obtain consent from parents and assent, when appropriate, from children. Researchers must also determine the proper amount of reimbursement for both parents and children without coercion, balancing ethical considerations with the desire to maximize generalizability of the research.1

Quality improvement (QI) methods are gaining prominence in medicine.2–4 QI has been used in both inpatient and outpatient settings to change and improve care delivery. Although clinical research and QI have historically been seen as separate entities, there has been recent work highlighting their potential synergy. In 2009, Margolis et al detailed opportunities for the integration of QI methods into the design, conduct, and analysis of clinical research.5 To our knowledge, however, no studies have specified the use of QI to improve recruitment for a clinical research study.

The National Institutes of Health–funded Greater Cincinnati Asthma Risks Study (GCARS) was designed as a population-based, prospective, observational cohort study. GCARS’s primary aim was to identify racial disparities in subsequent asthma morbidity among children, aged 1 to 16 years, on an inpatient pediatric unit.6 Collected data included a 177-question face-to-face survey with the primary caregiver, and blood and saliva from the child. The primary outcome was time to rehospitalization with study participants followed for at least 12 months.

Given that GCARS was designed to provide population-based, generalizable conclusions, study staff sought to approach and enroll as many eligible patients as possible while minimizing selection bias. Because the average length of stay for children admitted with asthma or wheezing at our institution is 33 hours, with many clinical interventions focused on acute and chronic management occurring during this short period of time, recruitment was expected to be a challenge. Although not included as part of the initial grant application, QI methods were conceived as a potential way to achieve successful study recruitment. We therefore aimed to maximize sample recruitment using QI methods, with the a priori goal of enrolling 60% of all eligible and staffed patients.

METHODS

Setting

Cincinnati Children’s Hospital Medical Center (CCHMC) is a 512-bed, urban, academic pediatric hospital with an 8-county primary service area. Market share data suggest that CCHMC captures >80% of asthma admissions from the primary service area and ∼95% from the institution’s home county.7 Annually, there are ∼1200 admissions to CCHMC for asthma.

Planning the Intervention

CCHMC has long used QI methods in the clinical setting. CCHMC has also been a center of clinical research, and recently, there has been a push to pursue clinical research within the general pediatric inpatient setting. The CCHMC Institutional Review Board (IRB) oversees review of both QI and clinical research studies. Amendments can be made to studies after IRB approval for tests of change, such as revisions to compensation for parents and children.

A cohort study undertaken in 2008–2009 involved recruitment of children admitted with asthma.8 This was the first study to use the general pediatric inpatient unit as a research laboratory, and it became the basis on which GCARS was designed. The earlier study was not staffed for the recruitment of a population-based sample, and it only involved a face-to-face survey and hair collection, not the more invasive sampling procedures (ie, blood draws) planned for GCARS. Challenges faced during recruitment of this previous study informed design of the more ambitious on-the-ward recruitment strategies for GCARS. It also facilitated more efficient identification of potentially eligible patients and improved relationships between the study team and inpatient staff such as nurses and respiratory therapists.

Complete data collection required navigation of many processes in a relatively short, busy period of time. Complete enrollment required parental consent (and child assent if ≥10 years), face-to-face survey administration, and specimen collection. Adequate staffing was essential, so 4 Clinical Research Coordinators (CRCs) were hired and scheduled to ensure that at least 2 were present in the hospital 7 days per week. Time from consent to completion of study procedures averaged 90 minutes, dependent largely on the time of sample collection.

Although CRCs could complete surveys and obtain saliva samples, they were not trained in phlebotomy. Thus, the team decided to use a hospital-wide research resource, the Center for Clinical and Translational Science and Training (CCTST). The CCTST, funded through the National Institutes of Health’s Clinical and Translational Science Award program, employs registered nurses whose sole focus is phlebotomy, primarily for outpatient research. Although the CCTST had participated in inpatient studies before, previous studies had not aimed to recruit as many or in such a short inpatient time frame as was planned for GCARS. In initial meetings with CCTST staff (both leadership and front-line nurses), we devised a plan that we believed would be best suited to this work. Initially, we planned to page the CCTST nurses each time a family consented, but we highlighted and shared an openness to revise and improve this process through tests of change.

Evaluation

GCARS recruitment began on August 11, 2010, and the 4 CRCs, along with the principal investigator, coinvestigator, and 2 fellows began weekly meetings to discuss recruitment progress and challenges. Although the weekly meetings were conceived from the start of the grant, using QI methods to improve recruitment rates was not implemented until the first month of the project. At these meetings, both weekly and cumulative recruitment algorithms were shared, detailing the variety of reasons patients were unable to be enrolled. Aside from patients who failed to meet GCARS inclusion criteria, categories for enrollment failures included those who were unstaffed (with no CRCs present to recruit), were unable to be consented, or refused participation. Top refusal reasons included the blood draw and parents not being interested. The research team reviewed the circumstances behind each recruitment failure. The number and percentage of participants who were successfully enrolled was similarly reviewed. Within 1 month of beginning recruitment, we began to implement small tests of change to improve recruitment rate.

The primary outcome measure was the recruitment rate. This rate was calculated by dividing the number of all eligible participants, admitted when staff was able to recruit, by the number who were enrolled in GCARS. Patients could only be enrolled once, but they could be approached for recruitment, and refuse participation, more than once. This outcome, treated as a percentage, was chosen because the goal was to enroll a population sample; it was decided that excluding the few periods when staff was not available to recruit (2 weekends early in recruitment and certain hospital holidays) did not introduce selection bias. We chose 60% as our recruitment goal to provide adequate power while recognizing that certain families would refuse or would not be present to consent. Given that biological specimens were a part of data collection, secondary outcomes related to sample adequacy (ie, blood and saliva).

Analysis

Each week, an updated Shewhart control chart detailed the percentage of eligible and staffed admission events that were successfully enrolled in the GCARS cohort. The chart was reviewed by the principal investigator, coinvestigator, fellows, and CRCs at the team meeting. Given that we expected some patients to be hospitalized multiple times during the enrollment period, we opted a priori to track admission events instead of individual patients. This allowed for patients not enrolled during their first admission to have the opportunity to be enrolled during a subsequent admission. The control chart displayed week-to-week variability while enabling differentiation of common-cause and special-cause variation. Common-cause is typical variation that occurs inherent to a process, and special-cause results from variation due to a specific circumstance. We sought to identify whether special-cause could be attributed to our tests of change or to background changes in the inpatient setting. We also sought to reduce the inherent common-cause variation that stems from multiple individuals performing the same process.9

RESULTS

Overall, 774 patients were enrolled in GCARS, from August 2010 to October 2011, representing 54% of all eligible and 59% of eligible and staffed admission events (Fig 1). In the first month of recruitment, the baseline weekly recruitment rate was ∼55% (Fig 2). Given that we were below our goal of 60% and given significant initial variability in recruitment success, the team rapidly conceptualized and used small-scale tests of change aimed at improving and stabilizing recruitment numbers while focusing on potentially modifiable reasons for nonenrollment (Table 1).

FIGURE 1
FIGURE 1

Recruitment algorithm depicting asthma admission events during the course of recruitment for the GCARS.

FIGURE 2
FIGURE 2

Annotated Shewhart control chart detailing percentage of enrolled children who were both eligible and staffed as part of our population-based, prospective, observational cohort study.

View this table:
TABLE 1

Small-Scale Tests of Change Used as Part of Ongoing QI Efforts Aimed at Increasing Recruitment for a Population-Based, Prospective, Observational Cohort Study From the General Pediatric Inpatient Unit

Refusals Due to Phlebotomy

As anticipated, the blood draw was identified early on as a common reason for families to refuse to participate.10 We therefore reexamined our approach to study compensation. Although the child’s caregivers were provided with compensation for participation, initially, patients were not, even though they were experiencing the more invasive study procedures (blood draw, saliva sampling). After a budget review, and approval by the IRB, we began to offer an additional incentive to the child worth ∼$5 (either a book or cash) for completion of all study-related procedures.

We also sought ways to lessen the trauma of the blood draw. The research team drew on personal experiences and knowledge, largely derived from the clinical, nonresearch setting. For example, the team posited that families would be more likely to participate in the blood draw if a numbing agent could be used.11–13 In discussion with front-line CCTST phlebotomists, we learned that other studies had used a numbing gel that took 30 minutes to activate. Given time and staffing constraints, this was not feasible. We did, however, identify a product (Spray ‘n Stretch; Gebauer, Cleveland, OH) that worked more quickly. A CRC–parent discussion about the availability of this numbing agent was added to our recruitment processes with the goal of alleviating hesitation about blood draws. After education to ensure that the product was being applied correctly, CCTST nurses became increasingly satisfied with the product’s effectiveness. They agreed to offer it to families as part of their phlebotomy process and even began to use it in other studies.

During the recruitment period, we were also approached by CCHMC Child Life staff members who were interested in becoming involved in our study.14 Child Life works directly with patients and families to reduce stress related to procedures and hospitalization (eg, blood draws).15 Although Child Life involvement was seen as a net positive by key stakeholders, research nurses were concerned about having to deviate from scheduled blood draw times, and study staff members were concerned about the feasibility of notifying Child Life of all imminent blood draws. After a few trials with study participants, it was determined that standardized preexisting e-mails sent between study staff and research nurses would also be sent to Child Life staff. These e-mails were sent when a patient consented to inform all of the blood draw time. Once Child Life became involved, they were present for ∼60% of blood draws.

Maximizing Staffing Capability

CRC staffing schedules were also revised to provide for presence 12 hours per weekday (2–3 CRCs) because original staffing models only had 2 or 3 CRCs scheduled for 8 hours per weekday and for 6 hours most weekend days (2 CRCs). We also revised recruitment processes to enable more complete recruitment at a satellite facility. Although recruitment was initially limited to the CCHMC Base facility (site of 87% of asthma admissions), we tracked admissions at the CCHMC Liberty Campus, a satellite, suburban facility (site of remaining 13% of admissions). Our goal was to stabilize recruitment strategies at the Base before extending recruitment to Liberty. This satellite facility presented a new set of challenges, including distance (30 minutes by car), opportunity costs of missing a patient at Base, no CCTST presence for blood draws, and no additional CRCs. We trialed phone recruitment, but early failures led us to recruit in person after confirming that the parent/guardian was present. Study leadership worked with the clinical nursing and laboratory staff to arrange for reliable completion of study blood draws and saliva sampling with appropriate processing of samples.

Outcome Assessment

During the course of recruitment, we were able to increase our mean weekly recruitment rate (based on statistical process metrics of 7 points above the mean line, not accounting for special cause weeks)16 to 61% after 3 months (Fig 2). Two special cause weeks (eg, where recruitment rates were either 100% or 0%) occurred during times of low patient census.

After the mean was above goal, the focus was to maintain and, if possible, improve the rate while reducing week-to-week common-cause variability. Maintaining performance was threatened by staff transitions during the latter months of recruitment. Still, the control chart demonstrated that new staff members were quickly able to become part of the stable system, through training and attendance of weekly staff meetings and to maintain recruitment numbers through the end of the recruitment period.

At the end of the recruitment period, we reevaluated the recruitment rate variable in a way that used individual patients as the denominator instead of admission events. With this change in the outcome calculation, we found that 59% of eligible individuals, and 63% of eligible and staffed individuals were enrolled.

Although QI methods were primarily targeted at improving recruitment rates, interventions may have also improved the adequacy of specimens obtained. The blood draw assessed allergen sensitivities and cotinine levels; complete samples were obtained for 87% and 88% of participants, respectively. We also collected saliva for cotinine (obtained 96%) and DNA (obtained 91%). Overall, a full set of biological samples was obtained for 76% of participants. We expected some failures, with most due to inability to draw blood or child refusal at the time of the draw. We believe that the numbing spray and Child Life interventions had the most effect on obtaining these samples.

DISCUSSION

Researchers strive to maximize recruitment rates to maximize generalizable conclusions and minimize selection bias. We show, in this article, that QI methods can improve and stabilize recruitment rates for research conducted in a complex, busy inpatient setting. By using recruitment algorithms and control charts, reviewed by staff weekly, tests of change can be rapidly implemented, and the benefits and drawbacks of each test of change can be analyzed in close to real time. During the recruitment period, we saw an increase in the mean recruitment rate. Additionally, week-to-week enrollment variability reduced as the study progressed. This suggests that our weekly meetings and small-scale tests of change were successful in meeting our aim of both improving and stabilizing recruitment numbers.

The involvement of key stakeholders, such as front-line floor staff, research nurses, and study staff in the implementation of tests of change led to the successful and seamless integration of altered processes into study flow. Qualitatively, research staff felt empowered to speak up at weekly meetings when tests of change were not working and when new tests of change should be implemented. The input of staff is also highly valuable in the use of QI in the health care setting because many QI projects involve active participation and buy-in from a multidisciplinary staff.2,17,18 Similarly important was the desire to improve recruitment without pressuring families to participate. Thus, certain tests of change, such as compensation for the child, were discussed with the CCHMC IRB before implementation, and care was taken to ensure that there was appropriate balance between compensation and coercion.

It is difficult to conclude which of our tests of change had the most impact on changes in recruitment rates. The control chart indicates that incentives for the patient, as well as the use of the numbing spray, may have had the greatest impact on our recruitment numbers. This is corroborated by CRC impressions, who were the first to indicate, early on, the need to compensate the child, as well as identifying the blood draw as the main deterrent of participation. Furthermore, while staff transitions initially led to lower recruitment numbers, the system quickly restabilized, indicating that weekly meetings likely added stability to an evolving system.

Although some of our methods were unique to recruiting an observational cohort from the inpatient setting, we believe that many of the tests of change, along with weekly meetings to discuss recruitment strategies, could be used successfully in studies using other methods or in other settings. High recruitment numbers were important to us, as our funder expected a population sample, but high recruitment numbers are also essential to successful completion of other types of studies. For example, we believe QI methods could be applied to randomized control trials; however, adaptations may need to be made should study staff require blinding. Similarly, by meeting with recruitment staff to identify deterrents of participation, drug trials may be able to modify their recruitment strategies to improve recruitment rates.

This study carried certain limitations. Although we believe that QI methods were effective in improving recruitment numbers for our study, we acknowledge that our setting is unique. CCHMC also had preexisting resources, such as the CCTST and Child Life that may not exist at other institutions. Because we were a grant-funded study, we also had the resources to purchase incentives for the participants and the numbing agent used during the blood draws. Still, the other tests of change were essentially free of cost. Regardless of setting structure or grant funding, we believe that consistent and open engagement of front-line research staff could be widely implemented. We also believe that QI methods are most effective when adapted to the particular setting in question. Thus, the interventions successful for us may not be the most effective interventions at other sites or for other studies.

CONCLUSIONS

QI methods, implemented within the context of a larger research study, can lead to recruitment improvement and stabilization. Input from study staff implementing study processes, as well as various stakeholders, is essential, and scheduled, structured periodic meetings are an excellent way to solicit this input. By using rapid tests of change and reviewing control charts in the context of these meetings, staff members can quickly identify which tests are successful and which can be abandoned. Although the content of the tests of change will inherently vary between sites and studies, QI methods show promise for improving recruitment rates in research studies and, in turn, providing for more generalizable conclusions.

Hospitalist Haiku

Sixteen, belly pain

Thorough work up negative

Now my belly hurts.

Attending drones on

I have patients to care for

When will he shut up?

Routine RSV

Medical student presents

Prolongs length of stay.

Committee meeting

How do I get out of here?

Is that a code blue?

Discharged last Tuesday,

Bounced back seven days later

There goes my metrics.

Phone call at midnight

Crashing kid, sent to PICU

Angst replacing sleep.

Chronic pain, no source

Dementors suck happiness

Chocolate doesn’t help

Child running the halls

Mom feels eyes are just not right

How to get them home?

Dr David Pressel

Footnotes

  • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

  • FUNDING: The Greater Cincinnati Asthma Risks Study was supported by the National Institutes of Health (1R01AI88116). The Center for Clinical and Translational Science and Training was supported by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grants 8 UL1 TR000077-05 and 8 KL2 TR000078-05. Funded by the National Institutes of Health (NIH).

  • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

CCHMC
Cincinnati Children’s Hospital Medical Center
CCTST
Center for Clinical and Translational Science and Training
CRCs
clinical research coordinators
GCARS
Greater Cincinnati Asthma Risks Study
IRB
Institutional Review Board
QI
quality improvement

REFERENCES

  1. 1.↵
    1. Tishler CL,
    2. Reiss NS
    . Pediatric drug-trial recruitment: enticement without coercion. Pediatrics. 2011;127(5):949–954.
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    1. Baily M,
    2. Botrell M,
    3. Lynne J,
    4. Jennings B
    . The Ethics of Using QI Methods to Improve Health Care Quality and Safety. Garrison, NY: The Hastings Center; 2006.
  3. 3.
    1. Woods ER,
    2. Bhaumik U,
    3. Sommer SJ,
    4. et al
    . Community asthma initiative: evaluation of a quality improvement program for comprehensive asthma care. Pediatrics. 2012;129(3):465–472.
    OpenUrlAbstract/FREE Full Text
  4. 4.↵
    1. Crane S,
    2. Sailer D,
    3. Patch SC
    . Improving asthma care in emergency departments: results of a multihospital collaborative quality initiative in rural western North Carolina. N C Med J. 2011;72(2):111–117.
    OpenUrlPubMed
  5. 5.↵
    1. Margolis P,
    2. Provost LP,
    3. Schoettker PJ,
    4. Britto MT
    . Quality improvement, clinical research, and quality improvement research—opportunities for integration. Pediatr Clin North Am. 2009;56(4):831–841.
    OpenUrlCrossRefPubMedWeb of Science
  6. 6.↵
    1. Beck AF,
    2. Huang B,
    3. Simmons JM,
    4. et al
    . Role of financial and social hardships in asthma racial disparities. Pediatrics. 2014;133(3):431–439.
    OpenUrlAbstract/FREE Full Text
  7. 7.↵
    1. Bosnjakovic E
    . INSIGHT Database. Columbus, OH: Ohio Hospital Association; 2009.
  8. 8.↵
    1. Beck AF,
    2. Simmons JM,
    3. Huang B,
    4. Kahn RS
    . Geomedicine: area-based socioeconomic measures for assessing risk of hospital reutilization among children admitted for asthma. Am J Public Health. 2012;102(12):2308–2314.
    OpenUrlCrossRefPubMedWeb of Science
  9. 9.↵
    1. Burkhardt MC,
    2. Beck AF,
    3. Conway PH,
    4. Kahn RS,
    5. Klein MD
    . Enhancing accurate identification of food insecurity using quality-improvement techniques. Pediatrics. 2012;129(2):e504–e510.
    OpenUrlAbstract/FREE Full Text
  10. 10.↵
    1. McMurtry CM,
    2. Noel M,
    3. Chambers CT,
    4. McGrath PJ
    . Children’s fear during procedural pain: preliminary investigation of the Children’s Fear Scale. Health Psychol. 2011;30(6):780–788.
    OpenUrlCrossRefPubMedWeb of Science
  11. 11.↵
    1. Taddio A,
    2. Soin HK,
    3. Schuh S,
    4. Koren G,
    5. Scolnik D
    . Liposomal lidocaine to improve procedural success rates and reduce procedural pain among children: a randomized controlled trial. CMAJ. 2005;172(13):1691–1695.
    OpenUrlAbstract/FREE Full Text
  12. 12.
    1. Tak JH,
    2. van Bon WH
    . Pain- and distress-reducing interventions for venepuncture in children. Child Care Health Dev. 2006;32(3):257–268.
    OpenUrlCrossRefPubMedWeb of Science
  13. 13.↵
    1. Fetzer SJ
    . Reducing the pain of venipuncture. J Perianesth Nurs. 1999;14(2):95–101, 112.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Wilson JM
    ; American Academy of Pediatrics Child Life Council and Committee on Hospital Care. Child life services. Pediatrics. 2006;118(4):1757–1763.
    OpenUrlAbstract/FREE Full Text
  15. 15.↵
    Child Life at Cincinnati Children’s Hospital Medical Center. Supporting your child during care. 2012. Available at: http://www.cincinnatichildrens.org/service/c/child-life/default. Accessed March 20, 2012.
  16. 16.↵
    1. Perla RJ,
    2. Provost LP,
    3. Murray SK
    . The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20(1):46–51.
    OpenUrlAbstract/FREE Full Text
  17. 17.↵
    1. Jeffs LP,
    2. Lo J,
    3. Beswick S,
    4. Campbell H
    . Implementing an organization-wide quality improvement initiative: insights from project leads, managers, and frontline nurses. Nurs Adm Q. 2013;37(3):222–230.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. Lee HC,
    2. Martin-Anderson S,
    3. Lyndon A,
    4. Dudley RA
    . Perspectives on promoting breastmilk feedings for premature infants during a quality improvement project. Breastfeed Med. 2013;8:176–180.
    OpenUrlCrossRefPubMed
View Abstract

 

View this article with LENS
Email

Alerts
Citation Tools
Increasing Recruitment Rates in an Inpatient Clinical Research Study Using Quality Improvement Methods
Hadley S. Sauers, Andrew F. Beck, Robert S. Kahn, Jeffrey M. Simmons
Hospital Pediatrics Nov 2014, 4 (6) 335-341; DOI: 10.1542/hpeds.2014-0072
del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
Print
PDF
Insight Alerts

Subjects

Back to top