data quality

August 28, 2020: Designing, Conducting, Monitoring, and Analyzing Data from Pragmatic Clinical Trials: Proceedings from a Multi-Stakeholder Think Tank Meeting (Trevor Lentz, PT, PhD, MHA; Lesley Curtis, PhD; Frank Rockhold, PhD)

Posted on by Gina Uhlenbrauck

Speakers

Trevor Lentz, PT, PhD, MHA
Assistant Professor in Orthopaedic Surgery
Duke Clinical Research Institute

Lesley Curtis, PhD
Chair and Professor, Department of Population Health Sciences
Duke University School of Medicine

Frank Rockhold, PhD, ScM, FASA, FSCT
Professor of Biostatistics and Bioinformatics
Duke Clinical Research Institute

Topic

Designing, Conducting, Monitoring, and Analyzing Data from Pragmatic Clinical Trials: Proceedings from a Multi-Stakeholder Think Tank Meeting

Keywords

Pragmatic clinical trials; Think tank; Risk-based monitoring; Data quality; Real-world data; Electronic health records

Key Points

  • Pragmatism in study design is not a binary concept: some trial elements are purely explanatory (to establish efficacy in ideal settings) and some elements are purely practical (to establish effectiveness in the real world). The study design must serve the research question.
  • Findings from the think tank discussions on best practices and actionable steps included:
    • Ask precise research questions, and select the appropriate degree of pragmatism.
    • Optimize data quality through study design.
    • Focus on primary endpoints in data capture to maximize likelihood of success.
    • Innovate on mechanisms for data capture.
    • Promote adherence to the study protocol.
    • Evolve trial operations staff to focus on data science and informatics.
    • Share learning experiences openly and widely.

Discussion Themes

There is a misconception that PCTs, because they pursue pragmatism, are less rigorous and conducted without proper oversight or adherence to a protocol. Quality by design and good clinical practice principles apply equally to PCTs.

Risk-based monitoring is a potentially dynamic system that could improve study safety and quality, and make better use of study resources.

There is great interest from regulators, sponsors, and the academic research community to move PCT methods forward. To achieve this, we need to see more examples of successful PCTs in a context of regulatory decision-making.

Read the proceedings from the think tank meeting published in Therapeutic Innovation & Regulatory Science.

Tags

#pctGR, @Collaboratory1

August 27, 2020: Chapter on Acquiring Real-World Data Added to the Living Textbook

Posted on by Karen Staman

The NIH Collaboratory this week published a new chapter of its Living Textbook of Pragmatic Clinical Trials. The chapter, “Acquiring Real-World Data,” outlines strategies for obtaining real-world data for use in research.

“Real-world data” include data relating to the health status of a patient or the delivery of healthcare services. Common sources include electronic health records (EHRs), administrative claims, patient-reported outcomes, patient-generated health data, medical product and device registries, and databases relating to environmental factors or social determinants of health. Real-world data can support a number of activities in pragmatic clinical trials, such as patient identification and recruitment, monitoring of outcomes, and ascertainment of endpoints.

The new chapter includes the following sections:

The new chapter updates a previous resource, one of the most popular on the Living Textbook, based on work by experts in the NIH Collaboratory’s Electronic Health Records Core Working Group.

July 31, 2020: Using Real-World Data to Plan Eligibility Criteria and Enhance Recruitment: Actionable Recommendations and Resources from the Clinical Trials Transformation Initiative (Sudha Raman, PhD, MA; John Sheehan, PhD, MBA, RPh)

Posted on by Gina Uhlenbrauck

Speakers

Sudha Raman, PhD, MA
Assistant Professor
Department of Population Health Sciences
Duke University

John Sheehan, PhD, MBA, RPh
Senior Director, Value and Evidence (HEOR) Neuroscience
Janssen Scientific Affairs, LLC

Topic

Using Real-World Data to Plan Eligibility Criteria and Enhance Recruitment: Actionable Recommendations and Resources from the Clinical Trials Transformation Initiative

Keywords

Clinical Trials Transformation Initiative (CTTI); Real-world data (RWD); Recruitment planning; EHR; Eligibility criteria; Fit-for-purpose data

Key Points

  • Real-world data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected by a variety of sources.
  • CTTI provides recent recommendations, resources, and case studies that highlight actionable tools and best practices for evaluating and using real-world data (RWD) in clinical trial recruitment activities:
    • General principles for using RWD
    • Using RWD to plan eligibility criteria
    • Using RWD to support recruitment
    • Enhancing RWD capabilities for the research enterprise
  • Using RWD from data sources such as electronic health records and claims data brings challenges for completeness, accuracy, and generalizability of the data.
  • RWD holds the potential to increase patient eligibility and enrollment as well as reduce recruitment timelines.

Discussion Themes

Insights from RWD should be sought early in the product lifecycle and include context from patients and sites.

One challenge of RWD data sources is finding appropriate databases for the disease area of interest, especially for trials of rare diseases.

Are there lessons learned about when using RWD becomes prohibitive or too expensive?

Read and download CTTI’s recommendations for using RWD. Learn more about FDA’s guidance for real-world data and real-world evidence. A publication is available about the health plan recruitment method used in the ADAPTABLE aspirin study.

Tags

#pctGR, @Collaboratory1

July 17, 2020: Living Textbook Grand Rounds Series: Choosing What to Measure and Making it Happen: Your Keys to Pragmatic Trial Success (Devon Check, PhD; Rachel Richesson, PhD)

Posted on by Gina Uhlenbrauck

Speakers

Rachel Richesson, PhD, MPH
Associate Professor, Informatics
Duke University School of Nursing

Devon Check, PhD
Assistant Professor, Population Health Sciences
Department of Population Health

Topic

Choosing What to Measure and Making it Happen: Your Keys to Pragmatic Trial Success

Keywords

Measuring outcomes; Phenotypes; Data quality; Data linkage; Endpoints; Patient-reported outcomes (PROs)

Key Points

  • Endpoints and outcomes for embedded pragmatic clinical trials (ePCTs) should be meaningful to providers and patients and be relatively easy to collect as part of routine care. Endpoints and outcomes also should be clearly defined and reproducible.
  • Patient-reported outcomes (PROs) are often the best way to measure quality of life, but come with challenges in that they are not routinely or consistently used in clinical care nor are regularly recorded in the EHR.
  • To fully capture all care—complete longitudinal data—it is often necessary to link research and insurance claims data.

Discussion Themes

Data in EHRs are an important component of ePCTs. While ePCTs strive for efficiency, there remain tradeoffs. Sometimes it will be necessary to collect data outside of the EHR to ensure important and compelling results.

It is also important that the endpoint that is conveniently available will also be accepted as influential for stakeholders when the trial results are disseminated.

In the future, it is essential that more meaningful data as well as more patient-reported outcomes are routinely collected and incentivized.

Developing a robust data quality assessment plan will improve the value of data and detect and address data issues. Read more about how to do this in Assessing Data Quality for Healthcare Systems Data Used in Clinical Research.

To learn more about measuring outcomes, visit these Living Textbook chapters:

Tags

#pctGR, @Collaboratory1

January 29, 2020: Open-Source Tool From the ADAPTABLE Supplement Enables Comparisons of EHR and Patient-Reported Data

Posted on by Damon Seils

The ADAPTABLE Supplement project team released user documentation and source code for an open-source tool that enables rapid assessment of concordance between electronic health record (EHR) data and information reported directly by patients. The tool is part of a larger effort supported by the NIH Collaboratory Coordinating Center to develop and test methods for integrating patient-reported data into the EHR and to streamline data for use in pragmatic clinical trials.

ADAPTABLE, the first major randomized comparative effectiveness trial conducted by the National Patient-Centered Clinical Research Network (PCORnet), aims to identify the optimal dose of aspirin therapy for secondary prevention of atherosclerotic cardiovascular disease. The trial relies on both existing EHR data sources and direct patient report.

The ADAPTABLE Supplement project team developed a menu-driven query (MDQ) tool to enable comparison of patient-reported data with analogous EHR data. Using data for patients enrolled in ADAPTABLE at the trial’s largest US site, the team tested the MDQ tool by using it to compare patient-reported hospitalizations with hospitalizations recorded in the EHR. In this test, 46% of the encounters recorded in the EHR were an exact match with patient-reported encounters, and 85% of the EHR-recorded encounters fell within 5 days of the patient-reported encounter dates.

The study demonstrates the feasibility of using the MDQ tool to assess concordance between patient-reported data and EHR data. Because the tool is based on the PCORnet Common Data Model, it will be useful to participating sites across the network and can be used for querying this widely available data source.

The MDQ tool user documentation describes the features of the tool and provides links to the source code. A summary of the MDQ tool’s development describes how the tool performed with data from ADAPTABLE.

This work was supported by a supplemental grant award to the NIH Collaboratory Coordinating Center from the National Center for Complementary and Integrative Health.

October 4, 2019: Ascertaining Death and Hospitalization Endpoints: The TRANSFORM-HF Experience (Eric Eisenstein, DBA, Kevin Anstrom, PhD)

Posted on by Gina Uhlenbrauck

Speakers

Eric L. Eisenstein, DBA
Associate Professor in Medicine
Duke University School of Medicine

Kevin J. Anstrom, PhD
Professor of Biostatistics and Bioinformatics
Director of Biostatistics, Duke Clinical Research Institute
Duke University School of Medicine

Topic

Ascertaining Death and Hospitalization Endpoints: The TRANSFORM-HF Experience

Keywords

Clinical endpoints: Ascertaining death; Hospitalization; TRANSFORM-HF; National Death Index

Key Points

  • When patient deaths occur outside the care setting, the cause of death may not be reliably documented. For researchers, the challenges of measuring deaths include the lack of a national death data source and incomplete or hard-to-access sources.
  • The death identification and adjudication process differs for explanatory versus pragmatic trials, and has implications for how death endpoints are acquired and measured.
  • The TRANSFORM-HF pragmatic trial is comparing the effects of treatment strategies on long-term outcomes for hospitalized patients with heart failure. The primary study endpoint is all-cause mortality, which is ascertained and verified using a hybrid approach at the clinical site and call center, and includes searching the National Death Index data.

Discussion Themes

What are the tradeoffs in making endpoint ascertainment more simple?

If using a hybrid death data collection strategy, how are discrepancies adjudicated?

Use of call centers that coordinate follow-up patient contact and data collection is a valid approach that ensures a single point of contact for patients or proxies and care providers. This approach should also be supplemented with redundant data sources.

Read more in the Living Textbook about Using Death as an Endpoint and Inpatient Endpoints in Pragmatic Clinical Trials.

Tags
#pctGR, @Collaboratory1, @DCRINews

September 27, 2019: Preparing for Clinical Trial Data Sharing and Re-use: The New Reality for Researchers (Rebecca Li, PhD, Frank Rockhold, PhD)

Posted on by Gina Uhlenbrauck

Speakers

Rebecca Li, PhD
Executive Director, Vivli
Co-Director of Research Ethics, Harvard Center for Bioethics
Harvard Medical School

Frank W. Rockhold, PhD
Professor of Biostatistics and Bioinformatics
Duke Clinical Research Institute
Duke University Medical Center

Topic

Preparing for Clinical Trial Data Sharing and Re-use: The New Reality for Researchers

Keywords

Data sharing; Individual patient data; Open access; Raw data; ICMJE; Research dissemination

Key Points

  • Open access to individual patient data from clinical trials is a critical tool for research in health care. Despite the challenges, the question is not whether data should be shared, but rather how and when access should be granted.
  • Preparing data for reuse is often an afterthought—yet it is a new reality for researchers and institutions.
  • As of January 1, 2019, the International Committee of Medical Journal Editors (ICMJE) requires registration of a trial’s data sharing plan at the time of trial registration.
  • Institutions or teams should begin their data sharing program planning at least 18 months before a major publication (or regulatory approval).

Discussion Themes

FAIR data are data that meet standards of findability, accessibility, interoperability, and reusability.

How do we manage scientific integrity, replication, and validity given that data sharing opens a study to multiple people asking the same or related questions in potentially different ways using different methods?

How do we plan for a future that rewards data quality and reuse?

Read more about data sharing from ICMJE, NIH Office of Science Policy, and the National Academy of Medicine.

Tags

#pctGR, @Collaboratory1, @VivliCenter, @FrankRockhold

September 6, 2019: Transforming Medical Evidence Generation with Technology-Enabled Trials (Matthew T. Roe, MD MHS)

Posted on by Gina Uhlenbrauck

Speaker

Matthew T. Roe, MD, MHS
Senior Investigator, Professor of Medicine
Duke Clinical Research Institute

Topic

Transforming Medical Evidence Generation with Technology-Enabled Trials

Keywords

Mobile clinical trials; Real-world evidence; Real-world data; Study design; Regulatory oversight; Digital health; Mobile health applications; Biosensors; Electronic health records

Key Points

  • Digital health applications and electronic health records provide tremendous opportunities for improving trial efficiencies, broadening patient participation, and reducing cost.
  • Novel approaches that can help reduce data collection burden for study sites include importing EHR data directly into the trial database, collecting patient-reported outcomes through web-based portals, and incorporating digital health data from wearables and biosensors.
  • To realize the potential of new technology, cross-sectional partnerships are needed among research participants, researchers, biopharma device industries, professional medical associations, insurers, FDA, clinicians, health IT, contract research organizations, and health systems.

Discussion Themes

How many potential patients might we lose if having a smart phone is an inclusion criterion for a clinical study?

How can we ensure that the clinical trial infrastructure is inclusive of minority populations, especially those in rural settings?

What is the role of physicians in reaching a large number of participants who are not near an academic research center?

Ultimately, in clinical trials, the data are what matter and what decisions are based on. We need to understand data quality and standards for the data to be accepted.

Read more about digital health at FDA’s Digital Health website.

Tags

#pctGR, @Collaboratory1, @MTRHeart

July 2, 2019: New Living Textbook Section on Inpatient Endpoints in Pragmatic Clinical Trials

Posted on by Karen Staman

A new section in the Living Textbook describes the considerations for using “real-world” data for inpatient-based event ascertainment. There are many sources for acquiring this information, and they have different time lags in their availability and varying degrees of error and bias. In order to use inpatient endpoints in pragmatic clinical trials, these factors must be understood during the design, conduct, and analysis phases of an embedded pragmatic clinical trial.

“The pragmatic trial community needs to collectively determine which endpoints are relevant for pragmatic trials, how they can be measured and validated, and how the accuracy of these measurement methods may impact hypothesis testing sample size estimates.” —Eisenstein et al 2019

Topics in the chapter include:

  • Pragmatic trial inpatient endpoints
  • Inpatient event data sources
  • Patient-reported data
  • Secondary data sources: EHR
  • Secondary data sources: claims
  • Case studies: ICD-Pieces, TRANSFORM-HF, ADAPTABLE, and TRANSLATE ACS
  • Data source accuracy

 

March 1, 2019: Approaches to Patient Follow-Up for Clinical Trials: What’s the Right Choice for Your Study? (Keith Marsolo, PhD)

Posted on by Gina Uhlenbrauck

Speaker

Keith Marsolo, PhD
Department of Population Health Sciences
Duke Clinical Research Institute
Duke University School of Medicine

Topic

Approaches to Patient Follow-Up for Clinical Trials: What’s the Right Choice for Your Study?

Keywords

Pragmatic clinical trial; Real-world data; Distributed research network; Electronic health records; EHR; Health data sources; Data standardization; Common data model; Fast Healthcare Interoperability Resources (FHIR); Application programming interface (API)

Key Points

  • Different sites have different capabilities and levels of sophistication around data. Clinical trial investigators should think from the beginning about the questions they want to answer and how much data is needed.
  • From different sources, such as the EHR, claims, or participant, data can be procured and provided in different ways, either by the patient, staff or clinician, or through IT and data experts.
  • PCTs with many sites may require a “patchwork quilt” of approaches for patient follow-up depending on the needs of the trial. Clinician-generated reports, direct from patients, and solutions involving application programming interfaces (APIs) are all good options for data exchange.

Discussion Themes

How do we think through the options for getting patient data where some sites may not be in the distributed research network or use a common data model?

Fast Healthcare Interoperability Resources (FHIR) is a draft standard describing data formats and elements and an application programming interface (API) for exchanging electronic health records. The FHIR interface requests data as an object, and for each defined domain it specifies allowable values and variables and predefines the information that you get out of the system.

Until data are collected/generated using the same standards/formats as the API, there will still be a need to understand the EHR-to-interface mapping.

For more information on using health data in embedded pragmatic clinical trials, visit the NIH Collaboratory’s EHR Core webpage.

Tags

#CommonDataModel, #RealWorldData, #FHIR, #pctGR, @Collaboratory1