Intraclass Correlation

Analysis Plan


Section 2


Intraclass Correlation

Contributor

Elizabeth R. DeLong, PhD

For the NIH Health Care Systems Collaboratory Biostatistics and Study Design Core

 

Contributing Editors

Jonathan McCall, MS

Damon M. Seils, MA

A primary driver of whether a study should randomize at a cluster level is the intraclass correlation coefficient (ICC). (For other considerations in choosing between cluster and individual randomization, see "Choosing Between Cluster and Individual Randomization" under Experimental Designs and Randomization Schemes.) The ICC is a measure of how similar the outcomes of individuals within a cluster are likely to be, relative to those of other clusters. For example, an intervention designed to enhance medication adherence might be implemented in several communities, where individuals of any single community might belong to the same socioeconomic stratum and behave similarly in terms of ability to pay for and remember to take their medication. Hence, if the primary outcome of the study is a measure of compliance, there is likely to be substantial intraclass correlation.

The ICC is intimately linked to the sample size necessary to conduct the trial with adequate statistical power. The ICC ranges from completely correlated (ICC = 1) to no correlation (ICC = 0). In the extreme case of an ICC of 1, all participants in a cluster are likely to have exactly the same outcome; thus, sampling 1 participant from that cluster is as informative as sampling the whole cluster. In other words, each cluster contributes a single data point to the study, and the effective sample size for the study is the number of clusters. On the other hand, if participants in a cluster behave essentially independently of each other and their outcomes are no more related than if they were from different clusters, then the ICC is 0 and the available sample size for the study is the total number of participants. There is a substantial literature on taking account of the ICC when deriving sample size requirements for CRTs. It is critical to have preliminary data that provide some estimate of the likely ICC.

The level at which to cluster creates a trade-off between potential contamination and bias on the one hand and available sample size on the other. For example, an NIH Collaboratory Demonstration Project originally intended to randomize at the provider level. However, because providers share clinic staff and facilities, a preliminary evaluation of the ICC demonstrated more than negligible intra-clinic correlation in the potential outcomes. So the research team needed to randomize at the clinic level rather than the provider level and to recruit additional clinics to meet their expectations for statistical power. In contrast, another study team intended to randomize at the clinic level but found negligible correlation in outcomes between providers within clinics and were able to randomize providers. Much depends on the type of intervention, and it is always prudent to obtain a preliminary estimate of the ICC before planning the study.

Accounting for the ICC in the Analysis

When individual outcomes are recorded within a cluster, it is important to understand how the outcomes relate to the primary hypothesis of the study and what exactly is to be tested and/or estimated. If conclusions are to be made at a cluster level, the individual outcomes might be accumulated into a summary measure for each cluster and the analysis performed at the cluster level. This approach eliminates the need to address the ICC directly, but it assumes that either there is little variation in cluster sizes or the size of the cluster is relatively unimportant. The analysis will place equal weight on each cluster, regardless of its size, unless some weighting mechanism is used.

For analyses performed at the individual level, random-effects models or generalized estimating equations are typically used. Again, the interpretation of the results hinges on the type of analysis, and it is important for the investigators to discuss their hypotheses clearly with the statisticians.

SECTIONS

CHAPTER SECTIONS


Version History

January 16, 2019: Embedded a video on intraclass correlation, added a resource to the Resource box, and made nonsubstantive changes to the text as part of the annual content update (changes made by D. Seils).

Published August 25, 2017

Citation:

DeLong ER. Analysis Plan: Intraclass Correlation. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: https://rethinkingclinicaltrials.org/chapters/design/analysis-plan-top/intraclass-correlation/. Updated January 17, 2019. DOI: 10.28929/016.