Speaker

Speaker: Rui Wang, PhD
Associate Professor of Population Medicine and Associate Professor in the Department of Biostatistics, Harvard Pilgrim Health Care Institute and Harvard Medical School

Moderator: Fan Li, PhD
Assistant Professor of Biostatistics, Yale School of Public Health

Slides

Keywords

Cluster-randomized; Intervention; Missingness; Outcomes

Key Points

• Cluster randomized trials are trials in which clusters of individuals rather than independent individuals are randomly allocated to intervention groups, and outcomes are measured for each individual in the cluster.

• Sometimes there is missingness in the outcome. Significant work has been done on individual level missingness, where there is an observed outcome for each cluster, but an outcome is missing for one individual. Less work focuses on cluster level missingness, where the outcome is missing for an entire cluster of individuals, but researchers have made some progress in analyzing these cases.

• For missing completed at random (MCAR) models, the missing process will be independent of all the covariate treatments and outcomes so that the observed data actually represent the underlying population you’re making an inference about. Most of an analysis can proceed using complete data, so this isn’t an issue. Unfortunately, this can rarely happen, so people consider using the missing at random (MAR) model, which models the missingness process using the observed data in cluster randomized trials.

• This research addresses two challenges – outcomes can be missing, and data for individuals within the same cluster are likely to be correlated. The missing data framework is focused on making inferences about aspects of the distribution of the full data based on the observed data. To do this, two approaches are commonly used: 1) mixed effect models via maximum likelihood estimation; and 2) population average models fitted with generalized estimating equation (GEE) approaches.

• The GEE estimator focuses on population average effects rather than cluster specific effects, requires fewer parametric assumptions, and is robust to misspecification of the correlation structure. In the presence of missing data, if data are MCAR, the standard GEE estimator based on complete data is consistent. However, when data are MAR, the standard GEE estimator based on complete data may provide biased estimates. Under this assumption, some possible solutions are the multiple imputation, inverse probability weighting, augmented inverse probability weighting, and a “multiply robust” version of the augmented inverse probability weighting solution. The “multiply robust” version also works for missingness of an entire cluster.

• When thinking about using the GEE, you might run into the following computational challenges with fitting GEEs: second-order GEEs include an extra set of estimating equations, involving all possible pairs of observations; the computing complexity increases quadratically as the cluster sizes increase; and solving GEEs with large cluster sizes becomes difficult due to both convergence and memory allocation issues. Stochastic algorithms, which specify using only a subsample at each iteration, can circumvent these computational issues.

• In summary, in the multilevel case, there can be either class level or individual level missingness. If it’s class level missingness and is MCAR, use the individual level missingness mechanism. If it’s also MCAR, then proceed with complete case analysis. If not, the GEE estimator in the form that fits the situation best is a plausible solution. For class level cases with outcomes MAR, use the “multiply robust” version of the GEE estimator to proceed with analysis.

 

Discussion Themes

-If the cluster size is small, we may have a propensity score estimated at zero or one. How does the EM algorithm help in this context? The EM doesn’t help if the number of clusters is small. When you fit the lambda at the probability of missingness, that doesn’t help with the extreme waste problem. When we fit different model, we would look at the tool missing, the gamma parameter, or maybe the AGA parameter depending on the model and whether the individual outcome is missing or not. Across these situations, treating certain aspects at zero would cause these examples not to be what we’re targeting. The EM algorithm helps with that problem. However, if you don’t have enough data, the estimation is not going to be stable.

-One cautionary note when creating the propensity weights using logistic regression is that there’s issues with collapsibility. It’s difficult to gauge what kind of bias this introduces. It might be better to use a marginal based, binary indicator model. That’s a great point. When you have a non-identity link, you get that non-collapsibility issue so that other models can be better in terms of estimating. At the end, we just need a model for predicting the probability of the observed. It’s important to note that these approaches are not wedded to logistic regression, as long as we have a sensible model for the missingness.

Tags

#pctGR, @Collaboratory1

Speaker

James P. Hughes, PhD
Professor Emeritus of Biostatistics, University of Washington

Slides

Keywords

Design, Analysis, Stepped-Wedge Trial

Key Points

• Stepped-wedge design is typically run by clusters that are randomized. Each of the clusters receive the control and treatment, and each cluster is sequenced to the intervention at different times.

• There are several key design considerations for stepped-wedge design, including defining the estimand or what are you trying to estimate in the analysis, the key sources of variation, and how outcome data will be collected.

• How will the treatment effect the outcome? Magnitude of effect is a key power consideration. It is important to think about the variation in effect over exposure time. Classic analyses of stepped-wedge trials assume that the effect of the treatment occurs instantaneously once the intervention is applied and that the effect of the intervention is constant over exposure time. But this assumption may not be true in all studies or for all types of treatment.

• A key question is what is the estimand and what is most scientifically meaningful? Classic stepped-wedge analyses assume an instantaneous treatment effect. What happens if you assume the treatment effect is immediate and constant, but it’s not? Researchers need to think very carefully about what is the possible exposure time curve and what is it the study is trying to estimate in the stepped-wedge trial?

• An alternative approach to estimation in stepped-wedge trials compared to the standard immediate treatment approach is to use an exposure time indicator model, wherein researchers estimate a treatment effect for every exposure time. In this case the estimated treatment effect is going to be a weighted average of the exposure times.

• Another issue to think about with stepped-wedge trials is potential sources of variation. For any cluster randomized trial, researchers are used to thinking about the cluster means, or how much variation you expect in the cluster means (the absence of treatment). For stepped-wedge trials there may be potential variation in treatment effect and variation in cluster means over time.

• Key design recommendations for stepped-wedge trials: Don’t assume the intervention is immediate and constant (IT model) unless well-justified; if a transition period is planned, include data from the transition period; if the estimand is the effect at a point in time, maximize the number of observations at that exposure time; including more variance components in the power calculation reduces the possibility of an underpowered trial; and power calculations in SW trials can sometimes seem counterintuitive.

• Key analysis recommendations for stepped-wedge trials: Fit flexible study time effect; avoid fitting IT model unless you are very confident that the intervention effect is immediate and constant; it is better to overfit than underfit random effects; use small sample correction if necessary.

Discussion Themes

-Most people have analyzed stepped-wedge designs with the immediate treatment model. Should they go back and analysis with a more flexible model? We are trying to assemble a large set of datasets to understand how often this issue occurs in practice. At this point it is an open question for how big of a practical problem this is.

-In the stepped-wedge design there are no clusters that are always treated or never treated. Do you have thoughts about adding these to a stepped-wedge design? Part of the issue is that the stepped-wedge concept arose from a need to provide the intervention to all clusters. If you had a cluster that was not treated it would probably improve your power but misses the point of the stepped-wedge design. There are other designs that work better in that instance, such as a parallel design.

Tags

#pctGR, @Collaboratory1