Data Issues With Monitoring PCTs

Data and Safety Monitoring

Section 4

Data Issues With Monitoring PCTs


Susan Ellenberg, PhD

Jeremy Sugarman, MD, MPH, MA

Doug Zatzick, MD


Contributing Editor

Gina Uhlenbrauck

The source of trial data for a PCT should be considered when determining the monitoring plan. Two main issues for PCTs are data quality and timeliness.

Data Quality

When using the EHR as the source of trial data, monitoring for data quality early in the trial can help ensure that valid results will be obtained (Ellenberg et al. 2015). Data reported by clinicians in the everyday care setting may have more variability than DMCs are used to seeing with traditional trials, which have highly controlled settings, strict reporting protocols, and separate systems for documentation (e.g., case report forms). In traditional trials, every effort is made to ensure consistency of measurement across sites. More heterogeneity may be observed in PCTs due to real-world variations in populations and approaches by healthcare delivery organizations and their practicing clinicians.

“Thus, DMCs may need to pay close attention to site-specific data to determine whether an emerging result may be attributable to one or two sites and is perhaps not widely generalizable” (Ellenberg et al. 2015).

As an element of data quality, completeness of data can also be a challenge in PCTs. Pragmatic trial protocols may allow flexibility in follow-up according to standard clinical practices at participating sites. The DMC and sponsor will need to agree on necessary data for follow-up in accordance with the risks of the trial intervention. Variations in follow-up practices across sites may also need to be accounted for in the randomization process to ensure that sites with more frequent follow-up practices are balanced across study arms.

Data Timeliness

Data obtained from sources such as claims data, state mortality data, or even the EHR will often have a delay and not be available real-time. In some trials, sites may perform data analysis locally due to privacy concerns, then submit results for centralized analysis and aggregation. This process, along with necessary data quality checks, can result in additional time before data are available for review (Ellenberg et al. 2015). Some determination will need to be made regarding the amount of delay that is acceptable to ensure adequate participant protection. DMCs should be involved in these discussions.

Case Example

In a traditional study of a mental health intervention, a suicide attempt or suicide death would be considered a serious adverse event, calling for immediate reporting to the DSMB and a DSMB determination regarding relatedness of the event to study treatment. However, in SPOT, a pragmatic trial, information on deaths is obtained from state mortality data, which are delayed up to 16 months. Death was also an expected occurrence in this population at high risk for suicide. In this situation, it would not be sensible to stop the trial for investigation of a death 16 months after the death occurred. These special circumstances required negotiation with the DSMB to work out a more practical monitoring plan.




back to top

Ellenberg SS, Culbertson R, Gillen DL, Goodman S, Schrandt S, Zirkle M. 2015. Data monitoring committees for pragmatic clinical trials. Clin Trials. 12:530–536. doi:10.1177/1740774515597697.


Ellenberg S, Sugarman J, Zatzick D. Data and Safety Monitoring: Data Issues With Monitoring PCTs. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: Updated August 25, 2017.