Statistical modeling and prediction of clinical trial recruitment

Real-time prediction of clinical trial accrual can support logistical planning, ensuring that studies meet but do not exceed sample size targets. We describe a novel, simulation-based prediction method that is founded on a realistic model for the underlying processes of recruitment. The model reflects key features of enrollment such as the staggered initiation of new centers, heterogeneity in enrollment capacity, and declining accrual within centers. The model's first stage assumes that centers join the trial (ie, initiate accrual) according to an inhomogeneous Poisson process in discrete time. The second part assumes that each center's enrollment pattern reflects an early plateau followed by a slow decline, with a burst at the end of the trial following the announcement of an imminent closing date. By summing up achieved and projected enrollment, one can predict accrual as a function of time and, thereby, the time when the trial will achieve a planned enrollment target. We applied our method retrospectively to two real-world trials: NSABP B-38 and REMATCH (Randomized Evaluation of Mechanical Assistance for the Treatment of Congestive Heart Failure). In both studies, the proposed method produced prediction intervals for time to completion that were more accurate than those from conventional predictions that assume a constant rate of enrollment, estimated either from the entire trial to date or over a recent time window. The advantage is substantial in the early stages of NSABP B-38. We conclude that a method based on a realistic accrual model offers improved accuracy in the prediction of enrollment landmarks, especially at the early stages of large trials that involve many centers.     

Effect of early patient enrollment on the time to completion and publication of randomized controlled trials.

The authors evaluated whether early enrollment affects the significance of the results and the time to completion and publication of randomized controlled trials. Seventy-seven efficacy randomized controlled trials (total enrollment, 28,992 patients) initiated by the Acquired Immunodeficiency Syndrome Clinical Trials Group between 1986 and 1996 were evaluated. After adjustment for target sample size, for each 10-fold increase in the first-month accrual, the odds of a trial reaching statistically significant results increased 2.8-fold (p = 0.040). The relative enrollment during the first month over target sample size (hazard ratio (HR) = 1.40 per 10 percent increase, p = 0.004) and masking (HR = 1.78 for double-blind vs. single or unblinded studies, p = 0.031) were the major predictors of faster completion. Rapid early accrual (HR = 1.09 per 10 additional patients accrued the first month, p = 0.011) and statistical significance in favor of an experimental arm (HR = 2.47, p = 0.004) independently predicted faster publication. Early enrollment is a strong predictor of whether a study will reach formal statistical significance, and it can offer predictive information on the time needed to complete the study and publish its findings. Ongoing unpublished studies and their enrollment rates may need to be considered when interpreting the accumulated evidence.     

Late-starter sites in randomized controlled trials

In a cohort of 14 randomized controlled trials conducted by the Adult AIDS Clinical Trials Group between 1986 and 1999 with a target sample size of >400 (total enrollment 15,531 patients), we evaluated whether "late-starter" sites can make a meaningful contribution to eventual trial accrual. The sites that started recruiting within 5 months from the time the first patient entered the trial were eventually responsible for over 90% of the total enrollment in 11 of the 14 trials. Across the 14 trials, some sites were consistently among the first to start enrollment, whereas others were routinely among the last. The late-starter sites are unlikely to make important contributions to eventual trial enrollment in large clinical trials conducted by groups with a fixed number of sites. Protracting administrative efforts to add more sites many months after a multicenter trial has started may not be useful to trial accrual.     

Modeling and validating Bayesian accrual models on clinical data and simulations using adaptive priors

Slow recruitment in clinical trials leads to increased costs and resource utilization, which includes both the clinic staff and patient volunteers. Careful planning and monitoring of the accrual process can prevent the unnecessary loss of these resources. We propose two hierarchical extensions to the existing Bayesian constant accrual model: the accelerated prior and the hedging prior. The new proposed priors are able to adaptively utilize the researcher's previous experience and current accrual data to produce the estimation of trial completion time. The performance of these models, including prediction precision, coverage probability, and correct decision‐making ability, is evaluated using actual studies from our cancer center and simulation. The results showed that a constant accrual model with strongly informative priors is very accurate when accrual is on target or slightly off, producing smaller mean squared error, high percentage of coverage, and a high number of correct decisions as to whether or not continue the trial, but it is strongly biased when off target. Flat or weakly informative priors provide protection against an off target prior but are less efficient when the accrual is on target. The accelerated prior performs similar to a strong prior. The hedging prior performs much like the weak priors when the accrual is extremely off target but closer to the strong priors when the accrual is on target or only slightly off target. We suggest improvements in these models and propose new models for future research. Copyright © 2014 John Wiley & Sons, Ltd.     

Telephone reminders are effective in recruiting nonresponding patients to randomized controlled trials

OBJECTIVE: Studies investigating means of recruiting participants to randomized controlled trials (RCTs) are sparse. We investigated the effects of telephone reminders as a recruitment strategy. STUDY DESIGN AND SETTING: Sick-listed employees received a written invitation to participate in a study comparing standard treatments with a solution-focused follow-up and were randomly allocated to an intervention or control group (n=703). Those who did not respond within 2 weeks received either 'no reminder' (n=242) or 'attempted telephone reminder' (n=256). Outcome was enrollment to the RCT. RESULTS: An intention to recruit analysis revealed no significant differences between the groups (P=.229). An intention to phone analysis among nonresponders revealed significant differences between 'no reminder' (recruited 4.5%) and 'attempted telephone reminder' (recruited 12.1%) (P=.003, odds ratio 2.89, 95% confidence interval [CI] 1.42-5.90). An analysis of numbers needed to phone showed that to recruit one more person in this group of nonresponders, we needed to phone 13 persons (95% CI=8-33). CONCLUSION: Systematic use of telephone calls can increase the recruitment rate among nonresponders in RCTs.     

An absence of pediatric randomized controlled trials in general medical journals, 1985-2004

OBJECTIVE: There are numerous potential barriers to conducting randomized controlled trials (RCTs) in children. The purpose of this study was to compare the quantity, trends over time, characteristics, and quality of pediatric RCTs published in general medical journals (GMJs) with adult RCTs. STUDY DESIGN AND SETTING: We conducted an electronic search of adult and pediatric RCTs from 1985-2004 and a manual search of published RCTs in the year 2000 in five high-impact GMJs (New England Journal of Medicine, Journal of the American Medical Association [JAMA], the Lancet, British Medical Journal [BMJ], Canadian Medical Association Journal [CMAJ]). Linear trends were identified and the 1-year sample was analyzed for publication characteristics (location of recruitment, sample size, number of centers, funding sources, and results) and quality scoring (Jadad score, intention-to-treat analysis, and citation frequency since publication). RESULTS: Adult RCTs increased by 4.71 RCTs/year (95% confidence interval (CI) 3.62-5.80; P<0.001), which was significantly higher (P<0.0001) than pediatric RCTs, which increased by 0.4 RCTs/year (95% CI -0.02 to 0.9; P=0.06). Adult RCTs were more likely to be hospital-based (P=.001) and to involve more centers in multicenter studies (P=0.02). Quality scores were similar, although adult RCTs were cited more frequently (P=0.003). CONCLUSION: There may be significant barriers to the publication of high-quality pediatric RCTs in GMJs.     

Pricing health care services: applications to the health maintenance organization.

This article illustrates how management in one type of service industry, the health maintenance organization (HMO), have attempted to formalize pricing. This effort is complicated by both the intangibility of the service delivered and the relatively greater influence in service industries of non-cost price factors such as accessibility, psychology, and delays. The presentation describes a simple computerized approach that allows the marketing manager to formally estimate the effect of incremental changes in rates on the firm's projected patterns of enrollment growth and net revenues. The changes in turn reflect underlying variations in the mix of pricing influences including psychological and other factors. Enrollment projections are crucial to the firm's financial planning and staffing. In the past, most HMO enrollment and revenue projections of this kind were notoriously unreliable. The approach described here makes it possible for HMOs to fine-tune their pricing policies. It also provides a formal and easily understood mechanism by which management can evaluate and reach consensus on alternative scenarios for enrollment growth, staff recruitment and capacity expansion.     

Undisclosed Changes in Outcomes in Randomized Controlled Trials: An Observational Study

PURPOSE We wanted to investigate the frequency of undisclosed changes in the outcomes of randomized controlled trials (RCTs) between trial registration and publication.METHODS Using a retrospective, nonrandom, cross-sectional study design, we investigated RCTs published in consecutive issues of 5 major medical journals during a 6-month period and their associated trials registry entries. Articles were excluded if they did not have an available trial registry entry, did not have analyzable outcomes, or were secondary publications. The primary outcome was the proportion of publications in which the primary outcome of the trial was, without disclosure, changed between that recorded in the trial registry and that reported in the final publication. The secondary outcome was the proportion of publications in which the secondary outcome was changed without disclosure.RESULTS We reviewed 158 reports of RCTs and included 110 in the analysis. In 34 (31%), a primary outcome had been changed, and in 77 (70%), a secondary outcome had been changed.CONCLUSIONS There are substantial and important undisclosed changes made to the outcomes of published RCTs between trial registration and publication. This finding has important implications for the interpretation of trial results. Disclosure and discussion of changes would improve transparency in the performance and reporting of trials.     

Patient advocacy and patient centredness in participant recruitment to randomized‐controlled trials: implications for informed consent

Context With the routinization of evidence‐based medicine and of the randomized‐controlled trial (RCT), more patients are becoming ‘sites of evidence production’ yet, little is known about how they are recruited as participants; there is some evidence that ‘substantively valid consent’ is difficult to achieve. Objective To explore the views and experiences of nurses recruiting patients to randomized‐controlled trials and to examine the extent to which their recruitment practices were patient‐centred and patient empowering. Design Semi‐structured in‐depth interviews; audio recording of recruitment appointments; thematic interactional analysis (drawing on discourse and conversation analysis). Setting and participants Nurses recruiting patients to five publicly funded RCTs and patients consenting to the recording of their recruitment sessions. Main outcome measures The views of recruiting nurses about their recruitment role; the extent to which nurse–patient interactions were patient‐centred; the nature of the nurses’ interactional strategies and the nature and extent of patient participation in the discussion. Results The nurses had a keen sense of themselves as clinicians and patient advocates and their perceptions of the trial and its interventions were inextricably linked to those of the patients. However, many of their recruitment practices made it difficult for patients to play an active and informed part in the discussion about trial participation, raising questions over the quality of consent decisions. Conclusion Nurses working in patient recruitment to RCTs need to reconcile two different worlds with different demands and ethics. Evidence production, a central task in evidence‐based medicine, poses a challenge to patient‐centred practice and more research and relevant training are needed.     

Use of Medicare summary notice inserts to generate interest in the Medicare stop smoking program

PURPOSE: Evaluations of outreach strategies that effectively and efficiently reach the senior population often go unreported. The Medicare Stop Smoking Program (MSSP) was a seven-state demonstration project funded by the Centers for Medicare and Medicaid Services. The 1-year recruitment plan for MSSP included a multifaceted paid media campaign; however, enrollment was slower than anticipated. The purpose of this substudy was to test the effects of including envelope-sized advertisement inserts with Medicare Summary Notices (MSNs) as a supplemental recruitment strategy. METHODS: Information obtained from enrollees on where they had learned about the program as well as overall enrollment rates were analyzed and compared with the time periods during which the inserts were included in MSN mailings. RESULTS: Average call volume to the enrollment center increased by 65.7% in Alabama, the pilot state, and by more than 200% in the subsequent demonstration states. Despite the introduction of the MSN inserts late in the recruitment period, 32.2 % of the 7354 total enrollees stated that they learned about the project through the inserts. CONCLUSIONS: This recruitment method is highly recommended as a cost-effective way to reach the senior population.     

Influence of trial registration on reporting quality of randomized trials: Study from highest ranked journals

ObjectiveTo evaluate the reporting quality of key methodological items of randomized control trials (RCTs) in 55 of the highest ranked journals.Study Design and SettingA list of the highest top ranked journals was identified, and a search for detecting RCTs in those journals was made. Two hundred sixty four journals were screened and 55 of them were identified having at least one RCT. Three RCTs were randomly selected a priori from each journal; 148 RCTs were finally included. RCTs were assessed by two reviewers using the Consolidated Standards of Reporting Trials (CONSORT) statement.ResultsOnly 11 (8%) RCTs had all items adequately reported. In addition, 36% of RCTs reported that the study was registered in any trial registry. We found a significant difference in the quality of reporting for baseline characteristics, recruitment, participant's flow, and randomization implementation between those studies having reported the registration of their RCT in a trial registry and those that have not. Adherence to key methodological items of the CONSORT statement was as follows: sample size determination (60%), sequence generation (49%), allocation concealment (40%), and blinding (25%).ConclusionsReporting of varied CONSORT items remains suboptimal. Registration in a trial registry was associated with improved reporting. Further efforts to enhance RCT registration could contribute to this improvement.     

Smokers eligible for a clinical trial. Correlates of not returning for randomization

PURPOSE: Recruitment and retention of minority participants is often the rate-limiting step towards successful completion of clinical trials. Participants who are eligible, and express interest in, enrolling into a trial will sometimes not return for their randomization visit. Factors associated with failure to return for randomization are largely unknown.METHODS: We compared 287 eligible African American (AA) smokers who enrolled, but did not return for randomization (NR), to 500 AA smokers who returned and were randomized (R) to participate in a study comparing culturally sensitive educational materials to usual care materials for smoking cessation in AAs. An analysis was conducted on variables potentially associated with not returning for randomization.RESULTS:Table 1CONCLUSIONS: Potential NR participants differed in a number of ways from those who returned for randomization. Better understanding of these factors may allow researchers to target recruitment efforts, potentially resulting in enhanced accrual and retention, and therefore, generalizability.     

Motivation for Health Information Seeking and Processing About Clinical Trial Enrollment

Low patient accrual in clinical trials poses serious concerns for the advancement of medical science in the United States. Past research has identified health communication as a crucial step in overcoming barriers to enrollment. However, few communication scholars have studied this problem from a sociopsychological perspective to understand what motivates people to look for or pay attention to information about clinical trial enrollment. This study applies the model of Risk Information Seeking and Processing (RISP) to this context of health decision making. By recognizing the uncertainties embedded in clinical trials, we view clinical trial enrollment as a case study of risk. With data from a random-digit-dial telephone survey of 500 adults living in the United States, we used structural equation modeling to test the central part of the RISP model. In particular, we examined the role of optimistic feelings, as a type of positive affect, in motivating information seeking and processing. Our results indicated that rather than exerting an indirect influence on information seeking through motivating a psychological need for more information, optimistic feelings have more direct relationships with information seeking and processing. Similarly, informational subjective norms also exhibit a more direct relationship with information seeking and processing. These results suggest merit in applying the RISP model to study health decision making related to clinical trial enrollment. Our findings also render practical implications on how to improve communication about clinical trial enrollment.     

Active recruitment increased enrollment in a hereditary cancer registry

BACKGROUND AND OBJECTIVE: The Mid-Atlantic Cancer Genetics Network (MACGN) targets individuals from cancer risk assessment clinics for recruitment into a national hereditary cancer registry. We sought to determine whether different recruitment methods used in a high-risk breast and ovarian cancer clinic yielded differences into enrollment into MACGN. METHODS: Two methods of recruitment were compared over an 8-month period. A passive recruitment technique, used during the first 4 months of recruitment, involved distribution of a brochure. An active recruitment method, used during the second 4-month period, required a MACGN recruiter to approach patients and initiate a brief discussion of the registry. RESULTS: During the first 4-month period, 158 eight patients were seen in the clinic and 142 were seen in the second 4-month period. During passive recruitment, 20% of available patients were approached, compared with 63% during active recruitment. Active recruitment also resulted in fourfold increase over passive recruitment in enrollment (from 15.6% to 67.4%). CONCLUSION: Allocating research staff specifically for recruitment and personal contact with potential participants is effective in achieving increased enrollment into a national hereditary cancer research registry.     

Factors Associated with Obstetrician-Gynecologists' Response to the Women's Health Initiative Trial of Combined Hormone Therapy.

The Women's Health Initiative trial of combined estrogen and progestin (WHI E+P) ended prematurely after preliminary evidence indicated that harms exceeded benefits, with no cardiovascular benefit. There was controversy over the results and the decision to end the trial early, with many obstetrician-gynecologists expressing reservations about the evidence. The Research Department of the American College of Obstetricians and Gynecologists conducted a study regarding the WHI E+P, sending questionnaires to 2500 randomly selected Fellows; 703 Fellows returned usable surveys (28.1%). Despite almost universal awareness of the results of the WHI E+P (> 97%), almost half of the responding physicians did not find the results convincing and disagreed with the decision to stop the trial. In this further examination of the data, we identified characteristics of the respondents who were associated with either accepting or rejecting the WHI E+P. The year residency was completed, the relative importance a respondent attributed to randomized clinical trials (RCTs), concern about harms of action, and opinion of alternative therapies were significant factors. One of 5 respondents found the results convincing and agreed with the decision to end the trial (acceptors). One of 3 respondents did not find the results convincing and disagreed with the decision to end the trial (rejectors). Acceptors had completed residency more recently (1991 v. 1985, P = 0.001), rated evidence from RCTs as more important (P = 0.006), were more concerned with harms of action (22.4% v. 10.6%, P = 0.004), and were more likely to have a favorable opinion of alternative therapies to hormone therapy (64.1% v. 44.4%, P < 0.001).     

African American smokers interested and eligible for a smoking cessation clinical trial: predictors of not returning for randomization

PURPOSE: Recruitment is often the rate-limiting step in conducting clinical trials among ethnic minorities. Little is known about participants who consent and enroll into a trial, but do not return for randomization. Why participants fail to return for randomization is largely unknown. METHODS: We compared 287 enrolled African American smokers who did not return for randomization, to the 500 who returned and were randomized to participate in a clinical trial for smoking cessation in African Americans. Analyses were conducted to identify variables associated with not returning for randomization. RESULTS: Univariate comparisons found the nonrandomized group to be significantly different from those randomized. Logistic regression showed younger age, less readiness to quit, having been proactively recruited, lacking a regular source of health care, believing that they will be smoking in 6 months, less church attendance, and a lower literacy level to be jointly related with not returning for randomization. CONCLUSIONS: African American participants who did not return for randomization into a clinical trial were different from those who did. Better understanding of these factors may allow researchers to target recruitment efforts resulting in enhanced accrual in clinical trials and increased efficiency.     

Departures from community equipoise may lead to incorrect inference in randomized trials

Objective

To assess the impact of selective enrollment on the results of randomized controlled trials (RCTs).

Study Design and Setting

We simulated an RCT of arthroscopic partial meniscectomy vs. nonoperative therapy in patients with meniscal tear and osteoarthritis (OA). We estimated efficacy with the risk ratio (RR) comparing the likelihood of clinically important improvement after surgery with that after nonoperative therapy. We assumed that efficacy differs by extent of OA. We simulated four scenarios: (1) nonselective enrollment; (2) higher likelihood of enrolling subjects with mild OA; (3) higher likelihood of enrolling subjects with severe OA; (4) much higher likelihood of enrolling subjects with severe OA. For each scenario, we simulated 100 trials with sample size 340.

Results

With nonselective enrollment, reflecting community equipoise, the results in 100 trials were consistent with those in the underlying population (mean RR: 1.87; 95% confidence interval [95% CI]: 1.57, 2.14). Selective enrollment of subjects with much higher likelihood of severe OA resulted in 28% lower efficacy of surgery (mean RR: 1.34; 95% CI: 0.93, 2.15), with 95% CI containing the true efficacy in just 25% of trials and empirical power of 44%.

Conclusion

Selective enrollment with respect to factors associated with efficacy may affect trial results and lead to inaccurate conclusions.     

Consulting parents about the design of a randomized controlled trial of osteopathy for children with cerebral palsy

Background Although the UK Department of Health has advocated the involvement of service users and carers in health research for several years, there is little evidence about their contribution to the design of randomized controlled trials (RCTs). Objective To demonstrate how consulting parents about the design of a study, including which outcomes to use, led to the design and successful delivery of a RCT of osteopathy for children with cerebral palsy (CP). Design Semi‐structured interviews were carried out with 20 parents of children with CP and other neurological conditions, asking them to choose between four different trial designs, to talk about noticeable changes in their child’s condition and their views about payment for trial treatment. Setting and participants The parents interviewed were all members of Cerebra, a charity for ‘brain‐injured’ children and young people. All interviews were carried out at the parents’ homes. Results Parents had mixed views about possible trial designs; however, a waitlist design which allowed all children eventually to receive the treatment emerged as a clear favourite. Parents did not focus on isolated outcomes, but suggested a range of factors relevant to their child’s quality of life. They expressed a clear preference for the costs of treatment to be funded by the trial. Conclusions Involvement of parents helped design a trial which was acceptable to families and addressed outcomes that mattered to them. By consulting parents about the design of the research, the subsequent trial achieved excellent recruitment and retention rates.     

Lessons learned from enrollment in the BEST study--a multicenter, randomized trial of group psychosocial support in metastatic breast cancer

The BEST study, a multicenter randomized trial of group psychosocial support in metastatic breast cancer, had several unusual features that may have influenced recruitment, notably the group nature of the intervention and the need for close collaboration between medical and psychosocial investigators. The recruitment process was examined in light of these features. Establishment of study centers was facilitated by involvement of experienced medical investigators who had successfully collaborated in previous research projects. Systematic evaluation of potential subjects or direct recruitment by psychosocial investigators optimized recruitment; however, the group nature of the intervention prolonged recruitment. Overall, 652 women were approached and 237 (43.3% of those medically eligible) randomized. Using population-based estimates, 24.3% of women with metastatic breast cancer were assessed for the study and 8.7% randomized. A randomization ratio of 2:1 was required to form and maintain groups. Competing clinical trials were the greatest barrier to recruitment. Five lessons were learned during recruitment for this trial: (1) multicenter randomized trials of psychosocial interventions are feasible, even in very ill patients, (2) the use of a group intervention effectively increased the required sample size by 50%, (3) similarity of randomization rates suggests that generalizability of study results will probably be comparable to that of other randomized cancer trials, (4) multidisciplinary collaborations and involvement of experienced researchers facilitated enrollment, and (5) most challenges encountered in recruitment were similar to those seen in all clinical trials.