February 22, 2024: Updated Template Provides Guidance for Reporting of Pragmatic Trial Results

An updated template from the NIH Pragmatic Trials Collaboratory provides guidance for the transparent reporting of the primary results of pragmatic clinical trials.

The template includes elements from the Consolidated Standards of Reporting Trials (CONSORT) statement and its extensions. It also addresses secondary use of electronic health record data, involvement of research partners and healthcare systems in the conduct of pragmatic trials, and special ethical and regulatory considerations.

Download the template.

The updated template is organized by the recommended reporting elements presented in the CONSORT checklist and draws on recent experiences and lessons learned from the NIH Collaboratory Trials. Appendices include links to CONSORT and its relevant extensions, the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) tools and resources, and examples of figures to include in pragmatic trial reports.

June 27, 2022: Special Edition of Contemporary Clinical Trials Offers New Perspectives on Pragmatic and Virtual Clinical Trials

Contemporary Clinical TrialslsA special series of articles addressing pragmatic and virtual trials appears this week in the journal Contemporary Clinical Trials and offers practical approaches to the many challenges clinical trials face.

The 14 articles bring together leaders, researchers, biostatisticians, and bioethicists—including members of the NIH Pragmatic Trials Collaboratory—who are rethinking key aspects of the development, conduct, and oversight of clinical trials.

The articles include recommendations and real-world experience on how clinical trials can be done better while maintaining high-quality results using digital technology, direct-to-participant methods, and electronic health records.

The special issue can be found on the Contemporary Clinical Trials website.

Adrian F. Hernandez, MD, MHS, a co-principal investigator for the NIH Pragmatic Trials Collaboratory Coordinating Center, and Tammy Reece, MS, PMP, CCRA, the program’s project director, served as the issue co-editors. Dr. Hernandez is also vice dean and executive director of the Duke Clinical Research Institute and professor of medicine at Duke University. Ms. Reece is a Senior Project Leader in the Government Trials and Networks Group at the Duke Clinical Research Institute.

Following an introduction written by Hernandez, the remaining 13 articles focus on topics including simplification by design, digital technologies and clinical trials, iterative approaches to the use of electronic health records data for large pragmatic studies, ethics and regulatory considerations, and lifestyle and behavioral modification trials.

This body of work offers pragmatic approaches to overcoming the many reasons clinical trials are underperforming, such as high financial costs, long duration, administrative complexity, low recruitment and retention of participants, and difficulty translating knowledge to actual medical care.

“The articles in this issue recognize advances in evidence generation while sharing the challenges and gaps that still need to be addressed,” Hernandez said. “Ultimately, the clinical research ecosystem must improve, not only to keep pace with the advances in science, but to improve public health in meaningful ways.”

“The NIH Pragmatic Trials Collaboratory has been working to re-engineer clinical trials for a decade, yet there is still progress to be made,” Reece said. “Sharing knowledge and innovations across disciplines is critical to continue the momentum of transformation toward a learning health system.”

November 3, 2020: Disseminating Trial Results: We Can Have Faster and Better

Healthcare cover imageNIH Collaboratory investigators Drs. Greg Simon, Rachel Richesson, and Adrian Hernandez published an opinion piece in Healthcare arguing that clinical trials investigators should align their dissemination processes with industry-sponsored trials to favor speed, and that years-long delays in dissemination reduce the relevance of clinical research.

“Delays reduce the ability for researchers to apply trial findings to new research questions, impede clinicians from having the most up-to-date information, and perhaps most importantly, are a disservice to patients who could benefit from the information.”

The authors use experiences with pragmatic trials supported by the NIH Collaboratory to explore faster dissemination of results, and suggest the following solutions:

  • Real-time access to outcome data
  • Continuous data curation and cleaning
  • Immediate data analysis
  • Rapid reporting of trial results

Much change is needed to reach these goals. The authors suggest that by modeling processes after industry-sponsored trials, researchers may be able to improve the speed and quality of results reporting.

“Cultural incentives are aligned in industry sponsored trials to favor speed: readiness for generalizing topline results is considered valuable to shareholders, and the culture encourages a system where data are liquid, available, and continuously cleaned and curated, such that topline results can be reported within a timespan of two weeks rather than two years.”

As part of the NIH Collaboratory’s commitment to dissemination and sharing, all NIH Collaboratory Trials are expected to share data and resources, and topline results are reported in our weekly Grand Rounds Webinars.

 

March 27, 2020: Living Textbook Grand Rounds Series: Part 3-Tips for Putting Together a Successful PCT Grant Application (Wendy Weber, ND, PhD, MPH)

Speaker

Wendy Weber, ND, PhD, MPH
Branch Chief, Clinical Research Branch
Division of Extramural Research
National Center for Complementary and Integrative Health (NCCIH)

Topic

Tips for Putting Together a Successful PCT Grant Application

Keywords

Pragmatic clinical trials; Embedded PCTs; Healthcare systems; NIH; Scientific officers; Funding opportunity announcement; Request for application

Key Points

  • Investigators interested in developing a compelling grant application for a pragmatic clinical trial (PCT) should consider these tips:
    • First, confirm that a PCT is the best study design to answer your research question.
    • From the NIH’s 27 Institutes and Centers, find the one that matches your disease area, organ system, or stage of life—and talk with the program official.
    • Also find the right funding opportunity announcement (FOA) and tailor your application to address all the FOA-specific instructions and review criteria.
  • Do not assume that the study panel is going to understand what pragmatic means; they may have a completely different definition. It is important that you get on the same page early in the application.
  • In your application, make sure you justify the research, include pilot data, address potential overlaps, reduce complexity, ensure the aims will advance the field, choose expert personnel for a multidisciplinary team, link data collection and analysis to aims, and justify the use of multiple sites and sample size.

Discussion Themes

For some urgent clinical problems, a PCT may be the only study design that can answer the research question.

Phased awards can be helpful. Determine the key elements of your trial to test in the planning phase. It’s important to “fail quickly”—not in the implementation phase. Applicants should identify milestones in the planning phase that will help convince reviewers their study will be feasible.

Because PCTs take place in changing conditions and real-world healthcare settings, they are not necessarily easier to implement than a traditional efficacy trial.

For more information, visit the Living Textbook’s Developing a Compelling Grant Application.

Tags
#pctGR, @Collaboratory1

December 18, 2019: NIH Collaboratory Shares New Findings and Fresh Insights in 2019

NIH Collaboratory researchers in 2019 continued to generate new knowledge and research methods in pragmatic clinical trials. Their work included insights from the Coordinating Center and Core Working Groups, large-scale analyses of data from the NIH Collaboratory Distributed Research Network, and results and innovative methodological approaches from the NIH Collaboratory Trials.

So far this year, the NIH Collaboratory has produced nearly 3 dozen articles in the peer-reviewed literature, including the primary results of the ABATE Infection trial, confirmation by the TiME trial of the feasibility of embedding large pragmatic trials in clinical care, and more:

NIH Collaboratory Coordinating Center

NIH Collaboratory Distributed Research Network

ABATE Infection NIH Collaboratory Trial

EMBED NIH Collaboratory Trial

PPACT NIH Collaboratory Trial

PRIM-ER NIH Collaboratory Trial

PROVEN NIH Collaboratory Trial

SPOT NIH Collaboratory Trial

STOP CRC NIH Collaboratory Trial

TiME NIH Collaboratory Trial

TSOS NIH Collaboratory Trial

December 17, 2018: NIH Collaboratory Advances Knowledge in Peer-Reviewed Literature in 2018

The NIH Collaboratory program contributed knowledge from pragmatic clinical research with 12 peer-reviewed publications in 2018, including several reports of findings from DNIH Collaboratory Trials and more:

NIH Collaboratory Coordinating Center

Identifying and supporting nonpharmacological dementia interventions ready for pragmatic trials: results from an expert workshop. J Am Med Dir Assoc. 2018;19(7):560-562.

NIH Collaboratory Distributed Research Network

Cancer screening results and follow-up using routinely collected electronic health data: estimates for breast, colon, and cervical cancer screenings. J Gen Intern Med. 2018 Oct 22. doi: 10.1007/s11606-018-4697-y. [Epub ahead of print]

STOP CRC NIH Collaboratory Trial

Effect of reminding patients to complete fecal immunochemical testing: a comparative effectiveness study of automated and live approaches. J Gen Intern Med. 2018;33(1):72-78.

Predictors of colorectal cancer screening prior to implementation of a large pragmatic trial in Federally Qualified Health Centers. J Community Health. 2018;43(1):128-136.

Effectiveness of a mailed colorectal cancer screening outreach program in community health clinics: the STOP CRC cluster randomized clinical trial. JAMA Intern Med. 2018;178(9):1174-1181.

Positive predictive values of fecal immunochemical tests used in the STOP CRC pragmatic trial. Cancer Med. 2018;7(9):4781-4790.

Effectiveness of automated and live phone reminders after mailed-FIT outreach in a pilot randomized trialPrev Med Rep. 2018;12:210-213.

LIRE NIH Collaboratory Trial

Comparison of natural language processing rules-based and machine-learning systems to identify lumbar spine imaging findings related to low back pain. Acad Radiol. 2018;25(11):1422-1432.

PROVEN NIH Collaboratory Trial

Representation in the care planning process for nursing home residents with dementia. J Am Med Dir Assoc. 2018;19(5):415-421.

A dynamic application of PRECIS-2 to evaluate implementation in a pragmatic, cluster randomized clinical trial in two nursing home systems. Trials. 2018;19(1):453.

PPACT NIH Collaboratory Trial

Interdisciplinary team-based care for patients with chronic pain on long-term opioid treatment in primary care (PPACT) – protocol for a pragmatic cluster randomized trial. Contemp Clin Trials. 2018;67:91-99.

Automating collection of pain-related patient-reported outcomes to enhance clinical care and research. J Gen Intern Med. 2018;33(Suppl 1):31-37.

December 14, 2017: NIH Encourages Authors to Publish in Reputable Journals

Publication in journals is an essential part of the scientific process, an important metric for scholarly career advancement, and an integral aspect of grant approval and funding mechanisms. Pressure to publish has engendered a global industry of open access journals, which are driven by author publication fees rather than subscription services. Some of these journals are not credible and are considered predatory; they may have misleading pricing, fraudulent editorial boards, and inadequate (or nonexistent) peer-review processes.

To help ensure the credibility of published research, the NIH released a statement on publication in scientific journals (NOT-OD-18-011).

“In support of public access to National Institutes of Health (NIH) funded research, authors are encouraged to publish their results in reputable journals. The NIH has noted an increase in the numbers of papers reported as products of NIH funding which are published in journals or by publishers that do not follow best practices promoted by professional scholarly publishing organizations.”

There are a number of resources available to authors to help identify reputable journals:

For more information on disseminating results from pragmatic clinical trials see the Living Textbook Chapters:

Journal Editors Propose New Requirements for Data Sharing

On January 20, 2016, the International Committee of Medical Journal Editors (ICMJE) published an editorial in 14 major medical journals in which they propose that clinical researchers must agree to share the deidentified data set used to generate results (including tables, figures, and appendices or supplementary material) as a condition of publication in one of their member journals no later that six months after publication. By changing the requirements for manuscripts they will consider for publication, they aim to ensure reproducibility (independent confirmation of results), foster data sharing, and enhance transparency. To meet the new requirements, authors will need to include a plan for data sharing as a component of clinical trial registration that includes where the data will be stored and a mechanism for sharing the data.

Evolving Standards for Data Reporting and Sharing

As early as 2003, the National Institutes of Health published a data sharing policy for research funded through the agency, stipulating that “Data should be made as widely and freely available as possible while safeguarding the privacy of participants, and protecting confidential and proprietary data.” Under this policy, federally funded studies receiving over $500,000 per year were required to have a data sharing plan that describes how data will be shared, that shared data be available in a usable form for some extended period of time, and that the least restrictive method for sharing of research data is used.

In 2007, Congress enacted the Food and Drug Administration Amendments Act. Section 801 of the Act requires study sponsors to report certain kinds of clinical trial data within a specified interval to the ClinicalTrials.gov registry, where it is made available to the public. Importantly, this requirement applied to any study classified as an “applicable clinical trial” (typically, an interventional clinical trial), regardless of whether it was conducted with NIH or other federal funding or supported by industry or academic funding. However, recent academic and journalistic investigations have demonstrated that overall compliance with FDAAA requirements is relatively poor.

In 2015, the Institute of Medicine (now the National Academy of Medicine) published a report that advocates for responsible sharing of clinical trial data to strengthen the evidence base, allow for replication of findings, and enable additional analyses. In addition, these efforts are being complemented by ongoing initiatives aimed at widening access to clinical trial data and improving results reporting, including the Yale University Open Data Access project (YODA), the joint Duke Clinical Research Institute/Bristol-Myers Squibb Supporting Open Access to clinical trials data for Researchers initiative (SOAR), and the international AllTrials project.

Responses to the Draft ICMJE Policy

The ICMJE recommendations are appearing in the midst of a growing focus on issues relating to the integrity of clinical research, including reproducibility of results, transparent and timely reporting of trial results, and facilitating widespread data sharing, and the release of the draft policy is amplifying ongoing national and international conversations taking place on social media and in prominent journals. Although many researchers and patient advocates have hailed the policy as timely and needed, others have expressed concerns, including questions about implementation and possible unforeseen consequences.

The ICMJE is welcoming feedback from the public regarding the draft policy at www.icmje.org and will continue to collect comments through April 18, 2016.

Resources

Journal editors publish editorial in 14 major medical journals stipulating that clinical researchers must agree to share a deidentified data set: Sharing clinical trial data: A proposal from the International Committee of Medical Journal Editors (Annals of Internal Medicine version). January 20, 2016.

A New England Journal of Medicine editorial in which deputy editor Dan Longo and editor-in-chief Jeffrey Drazen discuss details of the ICJME proposal: Data sharing. January 21, 2016.

A follow-up editorial in the New England Journal of Medicine by Jeffrey Drazen: Data sharing and the Journal. January 25, 2016.

Editorial in the British Medical Journal: Researchers must share data to ensure publication in top journals. January 22, 2016.

Commentary in Nature from Stephan Lewandowsky and Dorothy Bishop: Research integrity: Don’t let transparency damage science. January 25, 2016.

National Public Radio interview on Morning Edition: Journal editors to researchers: Show everyone your clinical data with Harlan Krumholz. January 27, 2016.

Institute of Medicine (now the National Academy of Medicine) report advocating for responsible sharing of clinical trial data: Sharing clinical trial data: maximizing benefits, minimizing risk. National Academies Press, 2015.

Rethinking Clinical Trials Living Textbook Chapter, Acquiring and using electronic health record data, which describes the use of data collected in clinical practice for research and the complexities involved in sharing data. November 3, 2015.

NIH Health Care Systems Research Collaboratory data sharing policy. June 23, 2014.

List of International Committee of Medical Journal Editors (ICMJE) member journals.

PCORnet “Writing for Research” Webinars Available via Living Textbook


“Writing for the Clinical Research Setting, “A 4-part webinar series sponsored by the National Patient-Centered Clinical Research Network (PCORnet) is now available on Rethinking Clinical Trials. Originally given in the fall of 2014 as part of the PCORnet “Office Hours” series, the recorded sessions are presented by Living Textbook managing editor Jonathan McCall, MS, and can be accessed as streaming video under Writing for the Clinical Research Setting in the Living Textbook.

The series provides a basic introduction to various facets of writing for the clinical research environment. Individual sessions, each roughly 1 hour in length, include the following topics:

  • Writing Peer-Reviewed Research Articles
  • Organizing and Writing Your White Paper
  • Writing Guidance Documents
  • Managing the Process of Writing and Publication