Application of sequential methods in randomized clinical trials with censored response criteria

In randomized clinical trials with a censored response criterion, it is common practice to perform interim analyses, especially for reasons of medical ethics. Sequential methods allow for repeated testing. Three sequential methods are well adapted to censored data: the group sequential analysis, the sequential probability ratio test and the triangular test. Among them, the triangular test has the best statistical properties. The influence of the frequency of the analyses on the statistical properties of sequential methods has been studied by simulation: sequential analyses need not be performed more frequently than interim analyses. Sequential methods have been applied to several randomized clinical trials. The results are briefly reported for two of them. The ability of the triangular test and of the sequential probability ratio test to reach conclusions early when there is no difference between two compared treatments is supported by these two examples. Moreover, careful patient follow-up must be planned in the protocol in order to reduce the time for data updating and to perform sequential analyses at the required frequency. It appears well-founded to propose the use of the triangular test in randomized clinical trials with a censored response criterion. Four types of analyses with different aims must be planned in the protocol.     

A conditional maximized sequential probability ratio test for Pharmacovigilance

The importance of post‐marketing surveillance for drug and vaccine safety is well recognized as rare but serious adverse events may not be detected in pre‐approval clinical trials. In such surveillance, a sequential test is preferable, in order to detect potential problems as soon as possible. Various sequential probability ratio tests (SPRT) have been applied in near real‐time vaccine and drug safety surveillance, including Wald's classical SPRT with a single alternative and the Poisson‐based maximized SPRT (MaxSPRT) with a composite alternative. These methods require that the expected number of events under the null hypothesis is known as a function of time t. In practice, the expected counts are usually estimated from historical data. When a large sample size from the historical data is lacking, the SPRTs are biased due to the variance in the estimate of the expected number of events. We present a conditional maximized sequential probability ratio test (CMaxSPRT), which adjusts for the uncertainty in the expected counts. Our test incorporates the randomness and variability from both the historical data and the surveillance population. Evaluations of the statistical power for CMaxSPRT are presented under different scenarios. Copyright © 2009 John Wiley & Sons, Ltd.     

Exploring the benefits of adaptive sequential designs in time-to-event endpoint settings

Sequential analysis is frequently employed to address ethical and financial issues in clinical trials. Sequential analysis may be performed using standard group sequential designs, or, more recently, with adaptive designs that use estimates of treatment effect to modify the maximal statistical information to be collected. In the general setting in which statistical information and clinical trial costs are functions of the number of subjects used, it has yet to be established whether there is any major efficiency advantage to adaptive designs over traditional group sequential designs. In survival analysis, however, statistical information (and hence efficiency) is most closely related to the observed number of events, while trial costs still depend on the number of patients accrued. As the number of subjects may dominate the cost of a trial, an adaptive design that specifies a reduced maximal possible sample size when an extreme treatment effect has been observed may allow early termination of accrual and therefore a more cost-efficient trial. We investigate and compare the tradeoffs between efficiency (as measured by average number of observed events required), power, and cost (a function of the number of subjects accrued and length of observation) for standard group sequential methods and an adaptive design that allows for early termination of accrual. We find that when certain trial design parameters are constrained, an adaptive approach to terminating subject accrual may improve upon the cost efficiency of a group sequential clinical trial investigating time-to-event endpoints. However, when the spectrum of group sequential designs considered is broadened, the advantage of the adaptive designs is less clear.     

Simulation study comparing exposure matching with regression adjustment in an observational safety setting with group sequential monitoring

Sequential methods are well established for randomized clinical trials (RCTs), and their use in observational settings has increased with the development of national vaccine and drug safety surveillance systems that monitor large healthcare databases. Observational safety monitoring requires that sequential testing methods be better equipped to incorporate confounder adjustment and accommodate rare adverse events. New methods designed specifically for observational surveillance include a group sequential likelihood ratio test that uses exposure matching and generalized estimating equations approach that involves regression adjustment. However, little is known about the statistical performance of these methods or how they compare to RCT methods in both observational and rare outcome settings. We conducted a simulation study to determine the type I error, power and time‐to‐surveillance‐end of group sequential likelihood ratio test, generalized estimating equations and RCT methods that construct group sequential Lan–DeMets boundaries using data from a matched (group sequential Lan–DeMets‐matching) or unmatched regression (group sequential Lan–DeMets‐regression) setting. We also compared the methods using data from a multisite vaccine safety study. All methods had acceptable type I error, but regression methods were more powerful, faster at detecting true safety signals and less prone to implementation difficulties with rare events than exposure matching methods. Method performance also depended on the distribution of information and extent of confounding by site. Our results suggest that choice of sequential method, especially the confounder control strategy, is critical in rare event observational settings. These findings provide guidance for choosing methods in this context and, in particular, suggest caution when conducting exposure matching. Copyright © 2014 John Wiley & Sons, Ltd.     

Application of the triangular test to phase II cancer clinical trials

Phase II cancer clinical trials are primarily designed to determine whether the response rate p to the treatment under study is greater than a specified value p0, that is to test the null hypothesis H0: p less than or equal to p0 against an alternative hypothesis H1 : p greater than p0 specified by p = p1. As an alternative to the single and multistage procedures and to Wald's continuous sequential probability ratio test (SPRT), we applied the group sequential methods proposed by Jones and Whitehead, namely the triangular test (TT) and the discrete SPRT, to the comparison of p with p0, and we expressed H0 and H1 in terms of the log odds-ratio statistic log [p(1 - p0)/p0(1 - p)]. A stimulation study showed that both the TT and the discrete SPRT had type I error and power close to the nominal values, and they compared favourably with multistage methods in terms of the average sample size.     

Continuous sequential boundaries for vaccine safety surveillance

Various recently developed sequential methods have been used to detect signals for post‐marketing surveillance in drug and vaccine safety. Among these, the maximized sequential probability ratio test (MaxSPRT) has been used to detect elevated risks of adverse events following vaccination using large healthcare databases. However, a limitation of MaxSPRT is that it only provides a time‐invariant flat boundary. In this study, we propose the use of time‐varying boundaries for controlling how type I error is distributed throughout the surveillance period. This is especially useful in two scenarios: (i) when we desire generally larger sample sizes before a signal is generated, for example, when early adopters are not representative of the larger population; and (ii) when it is desired for a signal to be generated as early as possible, for example, when the adverse event is considered rare but serious. We consider four specific time‐varying boundaries (which we call critical value functions), and we study their statistical power and average time to signal detection. The methodology we present here can be viewed as a generalization or flexible extension of MaxSPRT. Published 2014. This article is a US Government work and is in the public domain in the USA.     

Assessing the impact of additional follow-up in cohort studies

An approach is described for predicting the statistical value of extending follow-up in a cohort study. A simple approximation to the expected number of new events of interest is given. The effect of these events on inferences for parameters such as a standardized mortality ratio is approached in two ways. The first concerns the probability of reversing the conclusion of a significance test. The second approach finds the plausible range of values for the standardized mortality ratio after further follow-up that are consistent with the currently available data. Each of these values is displayed together with the precision of the estimate. The methods are illustrated with results from the International Radiation Study of Cervical Cancer (IARC Scientific Publication No. 52, 1984).     

Analysis of phase II clinical trials in haematology and oncology: comparison of the triangular test to the usual methods

Phase II cancer clinical trials are non-comparative trials which are designed to determine whether the response rate p to the treatment under study is greater than a certain value p0, that is, to test H0, given by p less than or equal to p0 against H1 given by p greater than po. By choosing type I error alpha and the power 1-beta and by specifying H1, that is, by choosing a clinically relevant improvement p1), one can compute the number of patients N to be included for a fixed-sample approach. Various other approaches have been proposed such as multistage methods and Wald's continuous sequential probability ratio test (SPRT). As an alternative approach, we extended the triangular test (TT), proposed by Whitehead for comparative trials, to the situation of non-comparative trials with a binary outcome. We expressed H0 and H1 in terms of the log odds-ratio statistics, namely log [p(1-p0)/p0-(1-p)]. With this choice, the two statistics of interest, Z and V, have simple expressions: Z is the difference between the observed number of positive outcomes and the expected number under H0 and V is the variance of Z under H0. After every group of n patients, Z is plotted against V, and the trial proceeds until a boundary is crossed. In our simulations, type I error alpha and the power 1-beta were close to nominal values with the TT and the average sample size was close to Wald's continuous SPRT and compared favourably with the multistage methods proposed by Herson and Fleming. Given its statistical properties and its easy use, the TT should be considered for planning and analysing cancer phase II trials.     

Student t‐tests for potentially abnormal data

When the one‐sample or two‐sample t‐test is either taught in the class room or applied in practice to small samples, there is considerable divergence of opinion as to whether or not the inferences drawn are valid. Many point to the ‘Robustness’ of the t‐test to violations of assumptions, while others use rank or other robust methods because they believe that the t‐test is not robust against violations of such assumptions. It is quite likely, despite the apparent divergence of these two opinions, that both arguments have considerable merit. If we agree that this question cannot possibly be resolved in general, the issue becomes one of determining, before any actual data have been collected, whether the t‐test will or will not be robust in a specific application. This paper describes statistical analysis system software, covering a large collection of potential input probability distributions, to investigate both the null and power properties of various one‐ and two‐sample t‐tests and their normal approximations, as well as the Wilcoxon two‐sample and sign‐rank one‐sample tests, allowing potential practitioners to determine, at the study design stage, whether the t‐test will be robust in their specific application. Sample size projections, based on these actual distributions, are also included. This paper is not intended as a tool to assess robustness after the data have been collected. Copyright © 2009 John Wiley & Sons, Ltd.     

The use of the triangular test with response-adaptive treatment allocation

A clinical trial is considered in which two treatments with binary responses are to be compared. A popular sequential stopping rule, the triangular test, is studied when various response-adaptive treatment allocation rules are applied, such as the recently proposed drop-the-loser rule, an urn randomization scheme. The paper extends previous work by Coad and Rosenberger, who combined the triangular test with the randomized play-the-winner rule. The purpose of the paper is to investigate to what extent the variability of an adaptive design affects the overall performance of the triangular test. The adaptive rules under consideration are described and some of their asymptotic properties are summarized. Simulation is then used to assess the performance of the triangular test when combined with the various adaptive rules. The main finding is that the drop-the-loser rule is the most promising of the adaptive rules considered in terms of a less variable allocation proportion and a smaller number of treatment failures. The use of this rule with the triangular test is beneficial compared with the triangular test with equal allocation, since it yields fewer treatment failures on average while providing comparable power with similar expected sample size. The results of an AIDS trial are used to illustrate the performance of the triangular test when combined with the drop-the-loser rule.     

The analysis of a sequential clinical trial for the comparison of two lung cancer treatments

Methods adopted in the analysis of data from a randomized controlled trial of a new treatment for inoperable lung cancer are described. The trial design employed the sequential logrank test using length of survival from time of randomization as the principal outcome measure. We describe the practical arrangements for regular inspections of the accumulating data as required by the sequential design, and also for more extensive but less formal interim analyses, and present some of the results. On termination of the trial the survival patterns of the two treatment groups were compared using methods of analysis developed specifically to allow for the sequential nature of the trial design. The role of prognostic factors was investigated using fixed sample methods which are approximately independent of the sequential design.     

A sequential procedure for monitoring clinical trials against historical controls

In this paper, we develop a sequential procedure to monitor clinical trials against historical controls. When there is a strong ethical concern about randomizing patients to existing treatment because biological and medical evidence suggests that the new treatment is potentially superior to the existing one, or when the enrollment is too limited for randomization of subjects into experimental and control groups, one can monitor the trial sequentially against historical controls if the historical data with required quality and sample size are available to form a valid reference for the trial. This design of trial is sometimes the only alternative to a randomized phase III trial design that is intended but not feasible in situations such as above. Monitoring this type of clinical trial leads to a statistical problem of comparing two population means in a situation in which data from one population are sequentially collected and compared with all data from the other population at each interim look. The proposed sequential procedures is based on the sequential conditional probability ratio test (SCPRT) by which the conclusion of the sequential test would be virtually the same as that arrived at by a non-sequential test based on all data at the planned end of the trial. We develop the sequential procedure by proposing a Brownian motion that emulates the test statistic, and then proposing an SCPRT that is adapted to the special properties of the trial.     

Reliability of Bayesian probability analysis for predicting coronary artery disease in a veterans hospital

To assess the accuracy of Bayesian probability analysis for the prediction of coronary artery disease, post-test probabilities were generated by the application of three Bayesian algorithms to the clinical and noninvasive test results of 199 patients undergoing angiography in a veterans' hospital. All assumed conditional independence but each used different pre-test and conditional probabilities. Two statistical approaches were employed: (1) Sorting of patients in ascending deciles of probability and comparing expected and observed probabilities in each decile. (2) Calculation of normally distributed reliability statistics which do not depend on probability subsets and the comparison of resulting probability distributions using these statistics. Both statistical approaches revealed that the Bayesian algorithms overestimated disease probability when it was high and underestimated it when low. Though all three algorithms were frequently incorrect, they differed significantly in their accuracies, suggesting that errors in Bayesian analysis are caused by factors other than the assumption of independence. The errors may be due to differences in sensitivity and specificity of tests applied in different institutions.     

成组序贯试验的原理和方法

本文介绍了成组序贯试验的原理和方法。它可用于得到结果较慢（如数周或数月）及在整个试验过程中可分少数几个时间段来重复统计分析试验结果的情况。它既保留了传统序贯方法的优点又避免其局限性。当两种处理间确实存在差异时,它常可较早地得出结论,从而可减少样本量,缩短试验周期。特别在临床试验中它可尽早地使受试者停止接受较差的处理,符合伦理学的要求,同时又正好与临床试验中的期中分析相配合,可望有较大的发展前途。     

A Bayesian stopping rule for a single arm study: With a case study of stem cell transplantation

Continuous monitoring of treatment failures is an important issue in clinical studies of a single experimental treatment for high risk therapy such as hematopoietic stem cell transplantation. The sequential probability ratio test (SPRT) of Wald in 1947 and various alternative stopping rules have been proposed for sequential monitoring of adverse events. It is natural to use prior information to improve stopping rules and statistical analysis. A Bayesian stopping rule is developed and applied to an example of an umbilical cord blood transplant study performed at the University of Minnesota. Two strata, based on the number of nucleated cells per kg recipient body weight (the 'dose') are monitored separately and different rules are constructed for each stratum using different prior distributions. It is believed that patients in the lower dose group have a greater chance of graft failure than those in the higher dose group. A program, written in R, is also presented for calculating the stopping rule using the prior beliefs. The program is an improvement upon existing programs and it can be used for larger studies.     

A comparison of the randomized play-the-winner rule and the triangular test for clinical trials with binary responses.

We consider a clinical trial model comparing an experimental treatment with a control treatment when the responses are binary. For fixed significance level and power, we compare the expected number of treatment failures for two designs--the randomized play-the-winner rule and the triangular test. The former is an example of an adaptive design while the latter is an example of a fully sequential design. We show how to determine the sample size for the randomized play-the-winner rule and how to choose the stopping boundaries for the triangular test so that the two designs have similar power functions. With this choice of design parameters, simulation indicates that the triangular test is generally more effective at reducing the expected number of treatment failures, particularly when there is a large difference between the two probabilities of success. The expected number of treatment failures can be further reduced if the triangular test is applied using the randomized play-the-winner rule to assign each patient to one of the two treatments.     

Group sequential comparison of positive predictive value curves for correlated biomarker data

Clinical studies of predictive diagnostic tests consider the evaluation of a single test and comparison of two tests regarding their predictive accuracy of disease status. The positive predictive value (PPV) curve is used for assessing the probability of predicting the disease given a positive test result. The sequential property of one PPV curve had been studied. However, in later stages of diagnostic test development, it is more interesting to compare predictive accuracy of two tests. In this article, we propose a group sequential test for the comparison of PPV curves for paired designs when both diagnostic tests are applied to the same subject. We first derive asymptotic properties of the sequential differences of two correlated empirical PPV curves under the common case-control sampling. We then apply these results to develop a group sequential test procedure. The asymptotic results are also critical for deriving both the optimal sample size ratio and minimal required sample sizes for the proposed procedure. Our simulation studies show that the proposed sequential testing maintains the nominal type I error rate in finite samples. The proposed design is illustrated in a hypothetical lung cancer predictive trial and in a cancer diagnostic trial.     

On the advantage of haplotype analysis in the presence of multiple disease susceptibility alleles

We investigated the effect of multiple susceptibility alleles at a single disease locus on the statistical power of a likelihood ratio test to detect association between alleles at a marker locus and a disease phenotype in a case-control design. Using simplifying assumptions to obtain the joint frequency distribution of marker and disease locus alleles, we present numerical results that illustrate the impact of historical variation of initial associations between marker alleles and susceptibility alleles on the power of a likelihood ratio test for association. Our results show that an increase in the number of susceptibility alleles produces a decrease in power of the likelihood ratio test. The decrease in power in the presence of multiple susceptibility alleles, however, is less for markers with multiple alleles than for markers with two alleles. We investigate the implications of this observation for tests of association based on haplotypes made up of tightly linked single-nucleotide polymorphisms (SNPs). Our results suggest that an analysis based on haplotypes can be advantageous over an analysis based on individual SNPs in the presence of multiple susceptibility alleles, particularly when linkage disequilibria between SNPs is weak. The results provide motivation for further development of statistical methods based on haplotypes for assessing the potential for association methods to identify and locate complex disease genes.     

Sequential analysis of matched dichotomous data from prospective case-control studies

Sequential analysis of randomized controlled clinical trials and epidemiological prospective (matched) case-control studies can have ethical or economical advantages above a fixed sample size approach. It offers the possibility to stop early when enough evidence for an apparent effect of the risk factor or lack of the expected effect is achieved. In clinical trials it is well accepted to stop the trial early in favour of the alternative hypothesis. In epidemiological studies, in general, the need is not felt to stop early in case of a clear exposure effect. Little attention has been paid, however, to early stopping and accepting the null hypothesis. In metabolic epidemiological studies, where analysis destroys the biological material, the question of efficient use of samples, for example, those stored in a biobank, becomes crucial. Also a slow accrual of cases or the costs of follow-up of a cohort nested study can make it desirable to stop a study early once it becomes clear that no relevant exposure effect will be found. Matching can further reduce the amount of information necessary to reach a conclusion. We derived test statistics Z (efficient score) and V (Fisher's information) for the sequential analysis of studies with dichotomous data where each case can be matched to one or more controls. A variable matching ratio is allowed. These test statistics can be entered into the software PEST to monitor the course of the study. The double sequential probability ratio test and the double triangular test were evaluated with simulated data for odds ratios equal to 1.5, 2.0 and 2.5 and various type I and type II error probabilities both under H(0) and under H(1). Our simulations resulted in average and median values for the amount of information (V) that are far less than those for a fixed sample size study. Efficiency gain ranges from 32 per cent to 60 per cent. The proposed sequential analysis was applied in an investigation on the possible relationship between the polymorphism of the MTHFR-gene and rectal cancer in a cohort of women with cases matched by age to one and to three controls. A sequential analysis of matched data can lead to early stopping in favour of H(0) or H(1), thus conserving valuable resources for future testing. A sequentially designed study can be more economical and less arbitrary than a study that makes use of conditional power or conditional coverage probability calculations to decide early stopping.     

Confidence interval construction for proportion difference in small-sample paired studies

Paired dichotomous data may arise in clinical trials such as pre-/post-test comparison studies and equivalence trials. Reporting parameter estimates (e.g. odds ratio, rate difference and rate ratio) along with their associated confidence interval estimates becomes a necessity in many medical journals. Various asymptotic confidence interval estimators have long been developed for differences in correlated binary proportions. Nevertheless, the performance of these asymptotic methods may have poor coverage properties in small samples. In this article, we investigate several alternative confidence interval estimators for the difference between binomial proportions based on small-sample paired data. Specifically, we consider exact and approximate unconditional confidence intervals for rate difference via inverting a score test. The exact unconditional confidence interval guarantees the coverage probability, and it is recommended if strict control of coverage probability is required. However, the exact method tends to be overly conservative and computationally demanding. Our empirical results show that the approximate unconditional score confidence interval estimators based on inverting the score test demonstrate reasonably good coverage properties even in small-sample designs, and yet they are relatively easy to implement computationally. We illustrate the methods using real examples from a pain management study and a cancer study.