Intraclass Correlation

Analysis Plan

Section 2

Intraclass Correlation


Elizabeth R. DeLong, PhD

For the NIH Health Care Systems Collaboratory Biostatistics and Study Design Core


Contributing Editor

Jonathan McCall, MS

A primary driver for whether a study should randomize at a cluster level is the intraclass correlation coefficient (ICC). (For other considerations in choosing cluster versus individual randomization, please see "Choosing Between Cluster and Individual Randomization" under Experimental Designs and Randomization Schemes.) ICC is a measure of how similar the outcomes of patients within a cluster are likely to be, relative to those of other clusters. For example: an intervention designed to enhance medication adherence might be implemented in several communities, where individuals of any one community might belong to the same socioeconomic stratum and behave similarly in terms of ability to pay for and remember to take their medication. Hence, if the primary outcome of the study is some measure of compliance, there is likely to be substantial intraclass correlation.

The ICC is intimately linked to the sample size necessary to conduct the trial with adequate power. The ICC ranges from completely correlated (ICC=1) to no correlation at all (ICC=0). In the extreme case of ICC=1, all participants within a cluster are likely to have exactly the same outcome; hence sampling one participant from that cluster is as informative as sampling the whole cluster. In other words, each cluster only contributes one data point to the study and the effective sample size for the study is the number of clusters. On the other hand, if participants within a cluster behave essentially independently of each other and their outcomes are no more related than if they were from different clusters, then the ICC is 0 and the available sample size for the study is the total number of participants. There is a large literature on taking account of the ICC when deriving sample size requirements for cluster randomized trials. It is critical to have preliminary data that provide some estimate of the likely ICC.

The level at which to cluster creates a tradeoff between potential contamination and bias on the one hand, versus available sample size. For example, one study funded by the NIH Collaboratory originally intended to randomize at the provider level. However, because providers share clinic staff and facilities, a preliminary evaluation of the ICC demonstrated more than negligible intra-clinic correlation in the potential outcomes. Hence, they needed to randomize at the clinic level rather than the provider level and also needed to recruit additional clinics to meet their power expectations. In contrast, another study had intended to randomize at the clinic level, but found negligible correlation in outcomes between providers within clinics, and were able to randomize providers.

Accounting for the ICC in the Analysis

When individual outcomes are recorded within a cluster, it is important to understand how the outcomes relate to the primary hypothesis of the study and what exactly is to be tested and/or estimated. If conclusions are to be made at a cluster level, the individual outcomes might be accumulated into a summary measure for each cluster and the analysis performed at the cluster level. This approach eliminates the need to directly address the ICC, but it assumes that either there is little variation in sizes among clusters or that the size of the cluster is relatively unimportant. The analysis will put equal weight on each cluster, regardless of its size, unless some weighting mechanism is used.

For analyses performed at the individual level, random effects models or generalized estimating equations are typically used. Again, the interpretation of the results hinges on the type of analysis and it is important for the investigators to discuss their hypotheses clearly with the statisticians.



Version History

Published August 25, 2017


DeLong ER. Analysis Plan: Intraclass Correlation. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: Updated November 13, 2018. DOI: 10.28929/016.