January 9, 2020: NIH Collaboratory Investigators Respond to Draft NIH Policy on Data Sharing

NIH Collaboratory leadership and Demonstration Project principal investigators, along with their colleagues, responded this week to the recently released Draft NIH Policy for Data Management and Sharing and supplemental draft guidance. The draft policy proposes that applicants for research funding submit a plan describing how scientific data will be managed and shared.

“We applaud the NIH’s policy and commitment to making the results and outputs of the research it funds and conducts available to the public. We enthusiastically support data sharing and agree with the principles of this policy. However, we believe more detail is warranted about the different types of research (ie, embedded pragmatic research), the associated protections, and acceptable mechanisms for sharing data, such as public and private archives and enclaves.” —Response to Draft NIH Policy for Data Management and Sharing

The main topics covered in the response are:

  • Support for the goals of the draft data sharing policy
  • Assessing and mitigating re-identification risk
  • Protecting secondary subjects
  • Use of data enclaves
  • Crediting those who share data

Other signatories include participants in the National Academy of Medicine’s Clinical Effectiveness Research Innovation Collaborative of the Leadership Consortium for Value and Science-Driven Health Care, and leaders of the Health Care Systems Research Network.

The full letter is available for download and includes the list of signatories.

Comments are due no later than January 10, 2020, and may be submitted online.

For more on data sharing, see the Living Textbook chapter, Data Sharing and Embedded Research.

December 18, 2019: National Institute on Aging IMPACT Collaboratory Launches Grand Rounds Series and New Website

The National Institute on Aging (NIA) IMbedded Pragmatic Alzheimer’s disease (AD) and AD Related Dementias (AD/ADRD) Clinical Trials (IMPACT) Collaboratory has launched a new website highlighting the project.

The IMPACT Collaboratory recently invited applications for pilot grant funding for projects that aim to generate preliminary data necessary to design and conduct full-scale embedded pragmatic clinical trials of nonpharmacologic interventions for persons living with Alzheimer’s disease and related dementias and/or their caregivers. Pilot grants will help IMPACT Collaboratory’s mission to build the nation’s capacity to conduct pragmatic clinical trials of interventions embedded within health care systems for people living with dementia and their caregivers.

Since the IMPACT Collaboratory started this September, the program has been hard at work launching its new website, as well as a series of specialized Grand Rounds focused on the topic of pragmatic research in the area of Alzheimer’s disease and related dementias.

IMPACT Collaboratory Grand Rounds occur the 3rd Thursday of each month at 12 noon ET. Join the next webinar on Thursday, December 19, “Stepped Wedge Cluster Trials: What, How, And When?,” by Monica Taljaard, PhD, Senior Scientist, Ottawa Hospital Research Institute and Associate Professor, University of Ottawa from the IMPACT Collaboratory Design and Statistics Core.

Following in the footsteps of the NIH Collaboratory, the IMPACT Collaboratory has also launched a podcast series to correspond with its Grand Rounds Series. Both the Grand Rounds webinar recordings and accompanying podcasts for all the sessions are available on the website.

Those interested in getting updates about the IMPACT Collaboratory can also join the program’s mailing list .

October 15, 2019: Postdoctoral Fellowship in the Ethics and Regulatory Aspects of Pragmatic Clinical Trials at Johns Hopkins

The Johns Hopkins Berman Institute of Bioethics invites applications for a Postdoctoral Fellowship in the Ethics and Regulatory Aspects of Pragmatic Clinical Trials. This position includes pursuing independent research, working alongside faculty members involved with the ethics and regulatory aspects of large-scale pragmatic clinical trials (PCTs), and participating in the Hecht-Levi Postdoctoral Fellowship in Bioethics. The postdoctoral fellow is expected to pursue one or more projects addressing the ethics and regulatory aspects of PCTs in collaboration with Berman Institute faculty members. The Fellow will actively engage with the Ethics and Regulatory Core of the NIH Health Care Systems Research Collaboratory and the Pragmatic and Implementation Studies for the Management of Pain to Reduce Opioid Prescribing (PRISM) Resource Coordinating Center.

Qualifications: Applications are welcome from candidates who will have an MD, PhD, or JD or their equivalent by the start date of the fellowship. Applicants should not have completed their terminal degree more than 3 years before the start date of the appointment. Physicians should not have completed a formal residency training program more than 3 years by the start date of the appointment.

Start date: September 1, 2020.

Terms of Appointment: The fellowship is guaranteed for 1 year with the expectation of a second year of funding, contingent on review. Applicants may not be employed by another institution and are expected to be in residence for the duration of the appointment.

How to apply: For details on how to apply see: https://bioethics.jhu.edu/education-training/fellowships/#fellowship-pragmatic. Applications must be submitted by December 16, 2019.

August 2, 2019: AI and the Future of Psychiatry (Murali Doraiswamy, MBBS)


Murali Doraiswamy, MBBS
Professor of Psychiatry and Behavioral Sciences
Duke School of Medicine


AI and the Future of Psychiatry


Artificial intelligence; Machine learning; Psychiatry; Ethical adoption of technologies; Mental health; Wearables; Mobile health

Key Points

  • There is growing evidence from randomized controlled trials of the efficacy of using digital tools in mental health diagnosis and treatment.
  • Could artificial intelligence (AI) and machine learning technologies be used to:
    • Reduce the stigma associated with mental health treatment?
    • Predict the risk for future suicide?
    • Detect Alzheimer’s years before diagnosis?
  • Categories of AI applications include low-risk apps that measure but do not diagnose, and apps used in diagnosis or treatment that must meet the same high standards of evidence as medications.
  • Clinicians still struggle with how to integrate patient data from wearable devices. AI technology might help if it could be used to synthesize the data into a risk profile for an individual.

Discussion Themes

What are the roles of stress, exercise, and sleep in mental health, and can autonomic data from wearables help explain the variance in mental health symptoms?

To develop evidence thresholds for AI, we need larger scale public-private partnerships as well as pragmatic trials addressing key clinical questions.

Read more from Dr. Doraiswamy in How to Use Technology Ethically to Increase Access to Mental Healthcare.

#AI, #pctGR, @Collaboratory1

July 3, 2019: New Article Describes the Public’s Beliefs Regarding Responsibility to Participate in Research

Findings from a new article suggest that the majority of patients do not feel a personal responsibility to participate in clinical research. In the article, Kevin Weinfurt, Li Lin, and Jeremy Sugarman report the results of a national survey of nearly 3000 people regarding their attitudes towards research responsibilities as well as their trust in doctors, healthcare systems, and medical research. Ethical frameworks for learning health systems have suggested that patients have a responsibility to contribute to learning activities, including research. The findings from this survey suggest that most patients in the U.S. do not currently endorse such a responsibility.

“These data provide a useful snapshot of the public’s views toward the obligation to participate in research. It is unclear how, if at all, these views will shift with increased efforts to create mature learning health systems. And if such views do not shift, it is uncertain what that would mean for the success of learning health systems.” —Kevin Weinfurt, PhD

Read the full article: Public Views Regarding the Responsibility of Patients, Clinicians, and Institutions to Participate in Research in the U.S.

For more on alternate approaches to consent, see the Living Textbook Chapter on Consent, Disclosure, and Non-Disclosure

July 2, 2019: New Living Textbook Section on Inpatient Endpoints in Pragmatic Clinical Trials

A new section in the Living Textbook describes the considerations for using “real-world” data for inpatient-based event ascertainment. There are many sources for acquiring this information, and they have different time lags in their availability and varying degrees of error and bias. In order to use inpatient endpoints in pragmatic clinical trials, these factors must be understood during the design, conduct, and analysis phases of an embedded pragmatic clinical trial.

“The pragmatic trial community needs to collectively determine which endpoints are relevant for pragmatic trials, how they can be measured and validated, and how the accuracy of these measurement methods may impact hypothesis testing sample size estimates.” —Eisenstein et al 2019

Topics in the chapter include:

  • Pragmatic trial inpatient endpoints
  • Inpatient event data sources
  • Patient-reported data
  • Secondary data sources: EHR
  • Secondary data sources: claims
  • Data source accuracy


June 25, 2019: GGC4H Awarded Continuation From Planning to Implementation Phase

The investigators of Guiding Good Choices for Health (GGC4H), an NIH Collaboratory Demonstration Project, have received approval to move from the planning phase to the implementation phase of their study. Congratulations to the GGC4H study team for their excellent work!

At the May 2019 NIH Collaboratory Steering Committee meeting, we talked with Drs. Stacy Sterling, Margaret Kuklinski, and Richard Catalano to hear about progress and challenges during the UG3 planning phase. The goal of GGC4H is to test the feasibility and effectiveness of implementing Guiding Good Choices—a universal evidence-based anticipatory guidance curriculum for parents of early adolescents—in 3 large, integrated healthcare systems serving socioeconomically diverse families.

“Guiding Good Choices is a tested and proven intervention in community and school-based settings. We think primary care is an ideal implementation setting because pediatricians have parents’ trust, and the AAP recommends that they offer anticipatory guidance to parents. Community and school settings do not often have the same advantages.” — Rico Catalano, Co-PI of GGC4H

In the UG3 phase, the study team partnered with 5 pediatric primary care clinics in Kaiser Permanente Northern California, Kaiser Permanente Colorado, and Henry Ford Health System, finalized the trial protocol, obtained IRB approval, and presented UG3 and pilot study findings at 2 recent national meetings: the Society for Prevention Research and the National Academy of Medicine Collaborative for Healthy Parenting in Primary Care.

Were there any surprises during the study’s planning phase?

“I’ve been pleased and surprised to see the near universal excitement for Guiding Good Choices across the 3 healthcare systems and in multiple clinics. My previous work has been with adolescents who have already started to develop problems and are clearly at risk. Often when you are engaging those teens and their parents, they are coming from a place of crisis and have a lot of anxiety and worry. With Guiding Good Choices, we’re offering it to all families of young adolescents in the pediatric clinics, and we’re heartened to see pediatricians and parents welcoming this.

“The parents and kids were so enthusiastic and excited in our pilot. It’s been extremely heartening, and I think it bodes well for the reception we’re going to get.” — Stacy Sterling, Co-PI of GGC4H

What is an example of a challenge that you were able to overcome with the help of a Core Working Group?

Our study has two core questions: Will Guiding Good Choices improve adolescent behavioral health when offered in a health care setting? Will parents in a health care setting actually enroll in Guiding Good Choices and to what degree? Our initial study design attended more to the second question, and in doing that, raised problems for the valid assessment of effectiveness. These issues could have prevented us from transitioning to the implementation phase, so we needed a good design for assessing both effectiveness and implementation. Our original plan would have included all adolescents who had well visits, but at some clinics, 25% of teens don’t have them. Our new design, developed with the IRB chair and the Biostatistics and Study Design Core, includes everyone who receives care at the pediatric clinic. Although we may enroll some people who don’t engage with the intervention, this will make the study results more generalizable and valid for both effectiveness and implementation.

What words of advice do you have for investigators conducting their first embedded PCT?

“It’s a really fasted-paced year. I think we have complementary strengths across the leaders at our sites, and that is important. Pay attention to the qualities of your team and how they can help you hit the ground running and eventually help you get across the finish line to the UH3 phase.”  — Margaret Kuklinski, Co-PI of GGC4H

GGC4H is supported within the NIH Collaboratory by a cooperative agreement from the National Center for Complementary and Integrative Health and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about GGC4H in the Living Textbook, and learn more about the NIH Collaboratory Demonstration Projects.

June 13, 2019: Experience With Pragmatic Clinical Trials Gains Momentum

At the NIH Collaboratory Steering Committee Meeting in May 2019, participants shared their perspectives on the evolving landscape of embedded pragmatic clinical trials (ePCTs). Three initiatives were presented: the Patient-Centered Outcomes Research Institute (PCORI), the NIH-DoD-VA Pain Management Collaboratory, and the HEAL (Helping to End Addiction Long-term) Initiative. Although many challenges remain, the conduct of ePCTs is gaining momentum, and the synergy between the initiatives, along with the fellowship they engender, will continue to help pave the way for more embedded pragmatic research in the future.

Dr. Ann Trontell, Associate Director of Clinical Effectiveness and Decision Science at PCORI, shared PCORI’s experience with pragmatic clinical studies. Since 2014, PCORI has awarded $494 million dollars for 43 pragmatic studies that range in size from 425 to 100,000 participants (median, approximately 1700). The studies include 2 observational, 27 individually randomized, and 14 cluster randomized trials in a wide range of therapeutic areas.

Dr. Trontell urged those developing proposals for pragmatic trials to make them fit for purpose, as opposed to emphasizing pragmatism, a theme echoed in the Developing a Compelling Grant Application chapter of the Living Textbook.





Dr. Robert Kerns, a director of the NIH-DoD-VA Pain Management Collaboratory, shared progress with pragmatic trials designed to evaluate whether evidence-based nonpharmacological approaches are effective for pain management among US military personnel and veterans.

Modeled after the NIH Collaboratory, the Pain Management Collaboratory is supporting 11 projects through a 2-year planning phase and a 2- to 4-year implementation phase. Subject matter experts at the Pain Management Collaboratory Coordinating Center (PMC3) support the projects by sharing tools, best practices, and resources.


Dr. Wendy Weber, Program Officer for the NIH Collaboratory Coordinating Center, introduced the HEAL initiative, which is designed to enhance pain management and improve prevention and treatment strategies for opioid misuse and addiction. The goal of the initiative is to provide scientific solutions to the opioid crisis. It includes a set of large-scale pragmatic trials that will receive logistical and technical support from the NIH Collaboratory Coordinating Center.


While experience with ePCTs is growing, many distinct challenges remain. As the conduct of ePCTs gains momentum, there is a rich opportunity to use collective experiences to refine best practices to real-world evidence generation and help solve urgent public health problems.

June 7, 2019: Meeting Materials from the 2019 NIH Collaboratory Steering Committee Meeting

The Collaboratory has made available all the presentations from their recent Steering Committee meeting held in Bethesda May 1-2, 2019.

Highlights of Day 1 included updates on the progress and sustainability of the NIH Collaboratory, perspectives on the landscape of embedded PCTs (ePCTs) and the need for real-world evidence, challenges and lessons learned from the UH3 Demonstration Projects, updates on progress and transition plans from the UG3 Demonstration Projects, and discussions on data sharing policy and planning. Day 2 featured an intensive workshop hosted by the NIH with the goal of starting discussions on statistical issues with ePCTs.

View or download the meeting materials on the website.

May 17, 2019: The VITamin D and OmegA-3 TriaL (VITAL): Design and Results of a Large Pragmatic Trial (JoAnn E. Manson, MD, DrPH)


JoAnn E. Manson, MD, DrPH
Chief, Division of Preventive Medicine, Brigham and Women’s Hospital
Professor of Medicine and the Michael and Lee Bell Professor of Women’s Health
Harvard Medical School
Professor, Department of Epidemiology
Harvard T.H. Chan School of Public Health


The VITamin D and OmegA-3 TriaL (VITAL): Design and Results of a Large Pragmatic Trial


Pragmatic clinical trial; Dietary supplements; Primary prevention; Mail-based randomized clinical trial; Cancer prevention; Cardiovascular disease prevention; vitamin D; Omega-3 fatty acids

Key Points

  • The VITAL pragmatic trial evaluated the effects of dietary supplements (vitamin D and omega-3) on reducing risk for developing cancer, heart disease, and stroke in the general population.
  • Study recruitment involved nationwide and targeted mailings, media reports, advertising, and brochures. Retention included participant newsletters, incentive gifts, and honoraria.
  • Findings included that neither omega-3s nor vitamin D significantly reduced the primary endpoints of major cardiovascular disease events or total invasive cancer. Omega-3s did reduce total myocardial infarction by 28%, with greatest reductions in those with low dietary fish intake and in African Americans.

Discussion Themes

VITAL’s hybrid design—remote or mail-based intervention plus serial in-clinic visits in a sample—has advantages in promoting quality and cost-efficiency.

Next steps for VITAL include continued follow-up for 5 years; genetic studies; and fostering new ancillary studies through nationwide collaborations.  

Visit the VITAL study website and read more about the results of VITAL (Manson et al., New Engl J Med, 2019)


#dietarysupplements, #pctGR, @Collaboratory1