October 28, 2024: New Living Textbook Contribution Explains Differences Between Medicare Data Sources

Living Textbook iconResearch-identifiable Medicare data can come from traditional fee-for-service Medicare claims or from Medicare Advantage claims. A new contribution to the Living Textbook of Pragmatic Clinical Trials published this month, Use of Medicare Data in PCTs, describes the important differences between these data.

At the healthcare system level, differences in incentives for documenting diagnoses can affect the reliability and relevance of data used for pragmatic clinical trials. The populations served by fee-for-service Medicare plans and Medicare Advantage plans are also disparate, as Medicare Advantage plans include a higher proportion of patients who require chronic disease management. There are also variations in enrollment rates across states and counties that reflect characteristics of the counties themselves (urban vs rural) and the firms that offer Medicare Advantage plans across regions.

Read the new contribution.

For more on using Medicare data in pragmatic trials, see the following Living Textbook sections:

October 15, 2024: Case Study Describes a Reassessment of Sample Size in an Ongoing Cluster Randomized Trial

FM-TIPS logoA new case study from the NIH Pragmatic Trials Collaboratory highlights an interim reassessment of sample size during an ongoing cluster randomized trial. The case study was published this week in the Living Textbook of Pragmatic Clinical Trials.

Researchers in cluster randomized trials must account for potential correlation between clusters in the design and analysis of their trial by estimating the intraclass correlation when calculating the target sample size. Often they use preliminary data from the planned enrollment sites to estimate the correlation. However, when preliminary data are unavailable at the time of study design, they may use interim data collected during the trial itself to reassess the trial’s sample size.

The contributors of the case study focus on FM-TIPS, an NIH Collaboratory Trial, to describe an approach to conducting an interim reassessment of sample size in an ongoing trial. Read the full case study.

FM-TIPS is examining whether the addition of transcutaneous electrical nerve stimulation to routine physical therapy improves movement-evoked pain compared with physical therapy alone among patients with fibromyalgia. The trial is supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases through the NIH HEAL Initiative. Learn more about FM-TIPS.

The contributors of the case study include members of the FM-TIPS study team and leaders of the NIH Collaboratory’s Biostatistics and Study Design Core. David-Erick Lafontant is a statistician, Bridget Zimmerman is a clinical professor of biostatistics, and Emine Bayman is an associate professor of biostatistics—all at the University of Iowa. Megan McCabe is an assistant professor of biostatistics at the University of Alabama at Birmingham. Patrick Heagerty is a professor of biostatistics at the University of Washington. Liz Turner is an associate professor of biostatistics and bioinformatics at Duke University.

October 3, 2024: JAMA Commentary Highlights Gap Between Research Results and Implementation Decisions

Headshots of Richard Platt, Hayden Bosworth, and Greg SimonIn a JAMA Viewpoint published online this week, leaders from the NIH Pragmatic Trials Collaboratory discuss the discordance between the results of pragmatic clinical trials and the implementation of those results in healthcare settings, even in settings that championed the work.

Coauthors Richard Platt, Hayden Bosworth, and Gregory Simon posit that, to provide evidence that healthcare systems leaders will actually use, changes are necessary:

  • Trials need to be faster (2 to 3 years)
  • Trials should consider outcomes that healthcare system leaders care about (such as costs and subgroup analyses)
  • The evidence required for change should be at the level that healthcare system leaders use (such as not necessarily rejecting the null hypothesis at P < .05 but also considering results of Bayesian methods and interim analyses)

Read the full article.

“We believe it is possible to make pragmatic clinical trials of policies and procedures more useful to delivery systems by accommodating their priorities, introducing flexibility in the level of evidence trials require, shortening the duration of planning and implementation, addressing system leaders’ interest in multiple outcomes and subgroup analyses, encouraging modification of protocols during the implementation phase, and by providing timely interim analyses that can guide decisions about continuing or modifying an intervention,” the authors wrote.

This work was based on an NIH Pragmatic Trials Collaboratory workshop held in 2023, Getting the Right Evidence to Decision-Makers Faster. The workshop explored the critical cycle of evidence generation to decision by healthcare system leaders to implement the findings of pragmatic clinical trials conducted within healthcare systems.

Platt is a cochair of the NIH Collaboratory’s Distributed Research Network and the Harvard Pilgrim Health Care Institute Distinguished Professor of Population Medicine at the Harvard Pilgrim Health Care Institute and Harvard Medical School. Bosworth is a cochair of the Implementation Science Core and a professor in population health sciences at Duke University. Simon is the chair of the Health Care Systems Interactions Core and a senior investigator at the Kaiser Permanente Washington Health Research Institute.

View the full workshop materials.

September 24, 2024: GRACE Team Identifies Barriers to Integrating Acupuncture Into US Healthcare System

Cover of the Journal of Integrative and Complementary Medicine (JICM)In a systematic review of the literature, researchers from the GRACE trial identified and categorized barriers and facilitators related to integrating acupuncture into the US healthcare system.

The review was published online ahead of print in the Journal of Integrative and Complementary Medicine.

Acupuncture is widely practiced in the United States and has been shown to be effective in multiple clinical applications for a variety of conditions. Yet, its integration into the healthcare system has been uneven.

In their review of the existing evidence, the authors found:

  • Understanding and addressing individual patients’ needs, awareness, concerns, and financial constraints is essential
  • Healthcare providers play an important role in communicating information about acupuncture to their patients and building trust
  • Organization-level changes are needed to address lack of space in healthcare facilities, lack of referral systems, and lack of credentialing and privileging procedures
  • Policies related to payment by both private and public insurers have been slow to change

Read the full report.

The authors stress that studies testing implementation of acupuncture interventions in real-world settings can help to address barriers to integration.

“Results of this study can be used to optimize strategies to effectively implement acupuncture into clinical practice, and ultimately influence policy changes,” they wrote.

GRACE, an NIH Collaboratory Trial, is a pragmatic clinical trial testing real-world implementation of acupuncture and guided relaxation for patients with pain associated with sickle cell disease. The trial is supported within the NIH Pragmatic Trials Collaboratory through the NIH HEAL Initiative under an award administered by the National Center for Complementary and Integrative Health. Learn more about the GRACE trial.

August 29, 2024: HDRN Canada Pragmatic Trials Training Program Offers Open-Access Learning Modules

Schematic of the HDRN Canada Pragmatic Trials Training ProgramA virtual, asynchronous learning program that offers open-access modules to anyone interested in learning about the complex art of pragmatic clinical trial design and implementation is now available.

Every week, a new open-access module features essential resources and leading Canadian trialists discussing a key component of clinical trials.

The 2-year, pan-Canadian program—HDRN Canada Pragmatic Trials Training Program—follows a structured curriculum that covers 12 key components of clinical trials, from understanding the basics of trial design to trial implementation, statistical analysis, quality control and oversight, and other trial components.

More than 75 learners from across Canada have been admitted to this pan-Canadian program after a competitive selection process. Admitted learners enjoy access to monthly peer-group discussions, virtual conferences, and training on indigenous knowledge.

This program is funded by the Canadian Institutes of Health Research, coordinated by the Schulich School of Medicine & Dentistry at Western University, and an initiative of Health Data Research Network Canada.

Modules are available at pragmatictrialstraining.ca. The program story is available at https://tinyurl.com/2f9m9k3s. For more information, contact the Program Coordinating Team at pragmatictrialstraining@uwo.ca.

August 8, 2024: FDA Commissioner and NIH Director Share Thoughts on the Future of Pragmatic Clinical Trials

Speaking at the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting in May, Commissioner of Food and Drugs Robert Califf and NIH Director Monica Bertagnolli discussed the intersection of pragmatic research and national health priorities.

Califf and Bertagnolli spoke about the role the NIH Collaboratory can play in promoting scientifically rigorous, pragmatic clinical research that addresses the urgent need for better and faster implementation of research findings into patient care. They also addressed shortcomings of the electronic health record (EHR) and the challenges of building a learning health system.

Dr. Monica Bertagnolli and Dr. Robert Califf speak at the NIH Pragmatic Trials Collaboratory's 2024 Annual Steering Committee Meeting

Wendy Weber of the National Center for Complementary and Integrative Health moderated the discussion, which focused on 3 challenges in clinical research and their relation to the NIH Collaboratory:

  • Medical providers and patients lack scientific evidence to support medical decisions
  • A learning health system can help to develop this evidence, but challenges in developing this system remain
  • How can the NIH Pragmatic Trials Collaboratory aid in these challenges?

“The most fundamental flaw we have in general right now is that we don't tell people that their biggest risk is getting standard healthcare in the healthcare system,” Califf said.  “Without evidence, clinicians don't actually know what to do. We act like we know, but we don’t,” he added.

Medical providers and patients alike must often make underinformed decisions. The lack of scientific evidence to support these decisions poses a challenge, as it demands clinicians use their individual experience and judgment when making a recommendation. Here, the deficiencies of the EHR become most obvious, as it fails to meet the needs of the population by not providing sufficient data for providers to refer to. Bertagnolli emphasized this, stating the EHR needed to become a learning environment, facilitating both the collection and dissemination of knowledge even for busy doctors.

“We need to reach every person,” Bertagnolli challenged the researchers in the audience. “We need to get in every corner of the United States with our data, with our research. We need to make it possible for busy doctors and clinics to be in a learning environment,” she said.

“What's the one unifying bit of technology we have that is not delivering what we need? It's the electronic health record,” Bertagnolli added.

Developing a health system that fosters collaboration and constant learning is paramount to obtaining generalizable medical evidence. Califf emphasized this need, calling for a singular “voice” that can represent the population’s prevention and health issues, collectively demonstrating what works.

However, several barriers hinder the development of such a system. The healthcare industry often finds that delivery practices that optimize profit margins are not optimal for patient health outcomes, which creates an environment of ethical uncertainty. Furthermore, the United States spends more on healthcare than do comparable countries with higher life expectancies, leading to unnecessary expenditures and unwillingness to adopt new initiatives.

“We're paying for a lot of things that don't help. So how are we going to figure out what to stop doing?” Bertanolli asked. “The only way is data. And the only people who can bring that data are our care providers to collect those data and learn from them. It is imperative to get a learning health system in progress,” she said.

The NIH Pragmatic Trials Collaboratory has long played a role in establishing cost-effective, large-scale research that treats care providers as research partners to deliver generalizable, actionable data. The program’s efforts in promoting collection of data from clinical care environments ensure that research remains grounded in real-world patient outcomes. Moreover, the NIH Collaboratory's emphasis on gathering data from underrepresented populations enhances the relevance of research findings, addressing disparities in healthcare delivery and outcomes.

Along with data collection, Bertagnolli encouraged the NIH Collaboratory to be an advocate for this new health system and encourage organizations to participate in the development of this system. The NIH Collaboratory brings years of specialized clinical knowledge and input, ensuring that the system is informed by evidence-based practices and responsive to the needs of both care providers and patients.

“It's a matter of convincing people to get over the hump, embrace a learning health system, and participate—and specifically, I'd say get the wheel turning as fast as you can, answer as many questions as you can, and talk about it,” added Califf.

View the full materials from the 2024 Annual Steering Committee Meeting.

August 5, 2024: NIH Collaboratory Leaders Reflect on Healthcare System and Patient Engagement Challenges and Lessons Learned

During the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting in May, Greg Simon and Steve George sat down to discuss challenges and lessons learned from NIH Collaboratory Trials on healthcare system and patient engagement.

Simon said it is key for embedded pragmatic clinical trials to build long-term relationships with healthcare systems, starting with finding out what are the needs of the healthcare system, the clinicians, and the patients they care for, and then building trials that address the questions they care about.

But healthcare systems change rapidly, which can lead to challenges, such as turnover at different levels of leadership, including the top level of the healthcare system, the clinic management level, and the provider level.

“It is important for clinical trial investigators to think about how do I form relationships with those people who are likely to be here for a while? How do I form relationships at multiple levels with different people,” Simon said.

With patient engagement, George said the challenges have evolved with the NIH Collaboratory Trials. When the NIH Collaboratory started, the initial challenge was whether to do patient engagement or not. Today, patient engagement is more of an expectation for trials, so the questions have become more nuanced, George said. Now trials must consider at what phase of the trial are you going to do patient engagement; who are you going to involve; and are you going to do it throughout the trial?

The NIH Collaboratory’s Living Textbook includes a new chapter on Patient Engagement to help investigators think through these questions.

“One of the most effective resources is the reporting and the information sharing across trials. There are a lot of models of how this has been done. There are a lot of models of how I would do it differently if I did it again, and I think that open sharing is very powerful for people to see because people like examples,” George said.

Simon, a senior investigator at Kaiser Permanente Washington Health Research Institute, is the chair of the NIH Collaboratory’s Health Care Systems Interactions Core and a member of the Coordinating Center leadership team. George is the Laszlo Ormandy Distinguished Professor of Orthopaedic Surgery at Duke University.

Headshots of Dr. Gregory Simon and Dr. Steven George

July 23, 2024: Article From Ethics and Regulatory Core Highlights Key Challenges for Pragmatic Trials

Headshots of Caleigh Propes, Stephanie Morain, and Pearl O'RourkeIn an invited commentary published this month in Circulation: Cardiovascular Quality and Outcomes, authors from the NIH Pragmatic Trials Collaboratory’s Ethics and Regulatory Core describe the recurring and emerging ethical issues in pragmatic clinical trials.

Coauthors Caleigh Propes, Stephanie Morain, and Pearl O’Rourke discuss 3 key challenges facing pragmatic trials researchers:

  • waivers and alterations of informed consent and their implications for transparency
  • managing and responding to “collateral findings” in pragmatic trials
  • representativeness of study populations, and the risk of reinforcing existing inequalities in healthcare delivery systems

Read the full article.

Each of the 3 challenges has taken on increasing importance for the NIH Collaboratory’s Ethics and Regulatory Core. For example, the group completed a 2-year multimethod investigation of collateral findings in pragmatic trials, identifying the core themes and proposing directions for future research.

Propes is a doctoral student in bioethics and health policy, and Morain is an associate professor of health policy and management and a core faculty member of the Berman Institute of Bioethics—both at Johns Hopkins University. O’Rourke is a retired bioethicist who served as the director of human research affairs at Partners HealthCare Systems in Boston and as an associate professor of pediatrics at Harvard Medical School.

Learn more about the Ethics and Regulatory Core.

July 15, 2024: Chat 4 Heart Health Transitions to Implementation Phase

The NIH Pragmatic Trials Collaboratory Coordinating Center is pleased to announce that the Chat 4 Heart Health trial received approval to transition from the planning phase to the implementation phase of the study. Congratulations to Michael Ho and Sheana Bull, the Chat 4 Heart Health principal investigators, and their study team for reaching this important milestone!

Chat 4 Heart Health, an NIH Collaboratory Trial, will test the comparative effectiveness of 3 text messaging delivery strategies that have been shown to improve individuals’ self-management health behaviors, including physical activity and medication adherence. The study will provide evidence regarding the best population-based strategy for universal delivery to engage patients in self-management to improve the American Heart Association’s “Life’s Essential 8” measures for improving and maintaining cardiovascular health.

Bull and Ho spoke recently about the importance of the study at the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting.

Bull is a professor of community and behavioral health and the director of the mHealth Impact Laboratory at the Colorado School of Public Health. Ho is a professor of medicine at the University of Colorado School of Medicine. They also led the Nudge study, another NIH Collaboratory Trial.

Chat 4 Heart Health is supported through a cooperative agreement from the National Heart, Lung, and Blood Institute. Learn more about Chat 4 Heart Health.

July 11, 2024: PCORI Leader Discusses Challenges of Conducting Pragmatic A vs B Clinical Trials of Approved Drugs

In an interview during the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting, Tracy Wang of the Patient-Centered Outcomes Research Institute (PCORI) discussed challenges of conducting A vs B pragmatic clinical trials of approved drugs in routine clinical practice. Wang joined Adrian Hernandez, co–principal investigator of the NIH Collaboratory Coordinating Center, and Pearl O’Rourke, cochair of the program’s Ethics and Regulatory Core.

“There are a couple of key challenges, the first of which is, who’s going to pay for the drug, especially if the [drugs being compared] have differential drug costs,” Wang said. “The other aspect is, how do we preserve the rigor of that comparison, [such as] the need for blinding, which is not as pragmatic in routine practice,” she said.

Most pragmatic trials, including the NIH Collaboratory Trials and those conducted through the National Patient-Centered Clinical Research Network (PCORnet), compare standard care with proposed improvements to standard care, or “A vs A-plus trials.” Trials that compare 2 alternative treatments that are in current use, or A vs B trials, are rare.

The problem of A vs B pragmatic trials was the subject of an NIH Collaboratory workshop, as well as an article from the Coordinating Center in the New England Journal of Medicine.

Wang, a cardiologist and clinical researcher, is PCORI’s chief officer for comparative clinical effectiveness research. Hernandez is a professor of medicine and the vice dean and executive director of the Duke Clinical Research Institute in the Duke University School of Medicine. Prior to her retirement, O’Rourke was the director of human research affairs at Partners HealthCare Systems in Boston and an associate professor of pediatrics at Harvard Medical School.

Access the complete meeting materials from the 2024 Annual Steering Committee Meeting.