Skip to content

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
  • Home
  • About
    • NIH Collaboratory
      • Coordinating Center
      • NIH Collaboratory Trials
      • Core Working Groups
      • Steering Committee
      • Distributed Research Network
      • Our Impact
    • Living Textbook
      • Table of Contents
      • How to Use This Site
  • Resources
    • Data and Resource Sharing
    • Training Resources
    • Tools for Researchers
    • Publications
    • Knowledge Repository
  • Webinar
  • Podcast
  • News
    • News Feed
    • Calendar
    • Subscribe
return to home
Subscribe to Newsletter go to twitter feed go to linkedin go to blue sky feed
Search
NIH Collaboratory
Living Textbook of
Pragmatic Clinical Trials

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
home button

Rethinking Clinical Trials

A Living Textbook of Pragmatic Clinical Trials

  • Design
    • What is a Pragmatic Clinical Trial?
    • Decentralized Pragmatic Clinical Trials
    • Developing a Compelling Grant Application
    • Experimental Designs and Randomization Schemes
    • Endpoints and Outcomes
    • Analysis Plan
    • Using Electronic Health Record Data
    • Building Partnerships and Teams to Ensure a Successful Trial
    • Intervention Delivery and Complexity
    • Patient Engagement
  • Data, Tools & Conduct
    • Assessing Feasibility
    • Acquiring Real-World Data
    • Assessing Fitness-for-Use of Real-World Data
    • Study Startup
    • Participant Recruitment
    • Monitoring Intervention Fidelity and Adaptations
    • Patient-Reported Outcomes
    • Clinical Decision Support
    • Mobile Health
    • Electronic Health Records–Based Phenotyping
    • Navigating the Unknown
  • Dissemination & Implementation
    • Data Sharing and Embedded Research
    • Dissemination Approaches for Different Audiences
    • Implementation
    • End-of-Trial Decision-Making
  • Ethics & Regulatory
    • Privacy Considerations
    • Identifying Those Engaged in Research
    • Collateral Findings
    • Consent, Disclosure, and Non-Disclosure
    • Data and Safety Monitoring
    • Ethical Considerations of Data Sharing in Pragmatic Clinical Trials
    • Ethics for AI and ML
    • IRB Responsibilities and Procedures

Intraclass Correlation

CHAPTER SECTIONS

Analysis Plan


Section 2

Intraclass Correlation

Expand Contributors

Patrick J. Heagerty, PhD
Elizabeth R. DeLong, PhD
For the NIH Pragmatic Trials Collaboratory Biostatistics and Study Design Core

Contributing Editors
Damon M. Seils, MA
Jonathan McCall, MS

A primary driver of whether a study should randomize at a cluster level is the intraclass correlation coefficient (ICC). (For other considerations in choosing between cluster and individual randomization, see Choosing Between Cluster and Individual Randomization in the Experimental Designs and Randomization Schemes chapter of the Living Textbook.) The ICC is a measure of how similar the outcomes of individuals within a cluster are likely to be, relative to those of other clusters. For example, an intervention designed to improve medication adherence might be implemented in several communities, where individuals of any single community might belong to the same socioeconomic group and behave similarly in terms of ability to pay for and remember to take their medication. Hence, if the primary outcome of the study is a measure of compliance, there is likely to be substantial intraclass correlation.

The ICC is closely linked to the sample size necessary to conduct the trial with adequate statistical power. The ICC ranges from complete correlation (ICC = 1) to no correlation (ICC = 0). In the extreme case of an ICC of 1, all participants in a cluster are likely to have exactly the same outcome; thus, sampling 1 participant from that cluster is as informative as sampling the whole cluster. In other words, each cluster contributes a single data point to the study, and the effective sample size for the study is the number of clusters. On the other hand, if participants in a cluster behave essentially independently of each other and their outcomes are no more related than if they were from different clusters, then the ICC is 0 and the available sample size for the study is the total number of participants. There is a substantial literature on taking account of the ICC when determining sample size requirements for cluster randomized trials. It is critical to have preliminary data that provide some estimate of the likely ICC.

The level at which to cluster creates a trade-off between potential contamination and bias on the one hand and available sample size on the other. For example, in one NIH Collaboratory Trial, the researchers originally intended to randomize at the provider level. However, because the providers shared clinic staff and facilities, a preliminary evaluation of the ICC demonstrated more than negligible correlation between clinics in the potential outcomes. So the researchers needed to randomize at the clinic level rather than the provider level and to recruit additional clinics to meet their expectations for statistical power. In contrast, another NIH Collaboratory Trial study team intended to randomize at the clinic level but found negligible correlation in outcomes between providers within clinics and were able to randomize providers.

Much depends on the type of intervention, and it is always prudent to obtain a preliminary estimate of the ICC before planning the study.

Accounting for the ICC in the Analysis

When individual outcomes are recorded within a cluster, it is important to understand how the outcomes relate to the primary hypothesis of the study and what exactly is to be tested and/or estimated. If conclusions are to be made at a cluster level, the individual outcomes might be accumulated into a summary measure for each cluster and the analysis performed at the cluster level. This approach eliminates the need to address the ICC directly, but it assumes that either there is little variation in cluster sizes or the size of the cluster is relatively unimportant. The analysis will place equal weight on each cluster, regardless of its size, unless a weighting mechanism is used.

For analyses performed at the individual level, random-effects models or generalized estimating equations are typically used. Again, the interpretation of the results hinges on the type of analysis, and it is important for the investigators to discuss their hypotheses clearly with the study statisticians.

Previous Section Next Section

SECTIONS

CHAPTER SECTIONS

sections

  1. Introduction
  2. Intraclass Correlation
  3. Unequal Cluster Sizes
  4. Accounting for Residual Confounding in the Analysis
  5. Missing Data and Intention-to-Treat Analyses
  6. Electronic Health Record Data Extraction
  7. Unanticipated Changes
  8. Interim Reassessment of Sample Size in Cluster Randomized Trials
  9. Case Study: STOP CRC Trial

Resources

What Is the Intraclass Correlation Coefficient?
Two-minute training module from the NIH Pragmatic Trials Collaboratory’s video library

The Intraclass Correlation Coefficient (ICC)
Guidance document from the Biostatistics and Study Design Core

Intraclass Correlation Coefficient Cheat Sheet
Introductory description of the ICC

The Intraclass Correlation Coefficient as a Pie Eating Contest
Video tutorial by NIH Collaboratory investigator Dr. Greg Simon

Intraclass Correlation Coefficients for Cluster Randomized Trials With Pain Outcomes
Working document from the NIH-DOD-VA Pain Management Collaboratory Biostatistics/Design Work Group


The Perils and Pitfalls of Complex Clustering in Pragmatic Trials
NIH Pragmatic Trials Collaboratory PCT Grand Rounds; November 3, 2023


Version History

April 30, 2024: Made nonsubstantive changes to the text and added items to the Resources sidebar as part of the annual content update (changes made by D. Seils).

June 23, 2022: Updated the name of the NIH Collaboratory in the contributors list, added an item to the Resources sidebar, and made nonsubstantive changes to the text as part of the annual content update (changes made by D. Seils).

July 2, 2020: Minor corrections to layout and formatting (changes made by D. Seils).

May 27, 2020: Added Heagerty to the contributors list and reordered the sections of this chapter as part the annual content update (changed made by D. Seils).

May 1, 2020: Made nonsubstantive changes to the Resources sidebar as part of the annual content update (changes made by D. Seils).

January 16, 2019: Embedded a video on intraclass correlation, added a resource to the Resource box, and made nonsubstantive changes to the text as part of the annual content update (changes made by D. Seils).

Published August 25, 2017

current section :

Intraclass Correlation

  1. Introduction
  2. Intraclass Correlation
  3. Unequal Cluster Sizes
  4. Accounting for Residual Confounding in the Analysis
  5. Missing Data and Intention-to-Treat Analyses
  6. Electronic Health Record Data Extraction
  7. Unanticipated Changes
  8. Interim Reassessment of Sample Size in Cluster Randomized Trials
  9. Case Study: STOP CRC Trial

Citation:

Heagerty PJ, DeLong ER; for the NIH Health Care Systems Research Collaboratory Biostatistics and Study Design Core. Analysis Plan: Intraclass Correlation. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Pragmatic Trials Collaboratory. Available at: https://rethinkingclinicaltrials.org/chapters/design/analysis-plan-top/intraclass-correlation/. Updated October 4, 2024. DOI: 10.28929/016.

Footer Menu

  • How to Use This Site
  • About NIH Collaboratory
  • Enrollment Reporting
  • Grand Rounds
  • Funding Statement
Link to Twitter Link to LinkedIn Link to Blue Sky Link to NIH Collaboratory email

Reference in this Web site to any specific commercial products, process, service, manufacturer, or company does not constitute its endorsement or recommendation by the U.S. Government or National Institutes of Health (NIH). NIH is not responsible for the contents of any “off-site” Web page referenced from this server.

Log in
Privacy Statement
WordPress is a content management system and should not be used to upload any PHI as it is not an environment for which we exercise oversight, meaning you the author are responsible for the content you post. Please use this system accordingly. Site Map