August 9, 2019: Open Science: Are we there yet? (Adrian Hernandez, MD)


Adrian Hernandez, MD
Professor of Medicine
Vice Dean for Clinical Research
Duke University, School of Medicine


Open Science: Are We There Yet?


Open science; Data sharing; Secondary analyses; Research collaboration

Key Points

  • Open science involves the responsible sharing of research data for the purpose of scientific advancement, integrity, and transparency.
  • Various stakeholders have made progress toward sharing clinical trial data, including:
  • Guiding principles of open science include appropriate access to research information; proper oversight with minimum barriers to data access; maintaining utility of data; an expectation that results of shared data will similarly be shared; and acknowledgment of those who contribute original data.
  • Despite efforts at supporting open science, no academic institution has an open science policy yet.

Discussion Themes

Open science remains an important goal to build trust and expand knowledge.

Data sharing is not a traditional measure of academic success. What incentives would need to change in order to support open science?


#OpenScience, #DataSharing, #pctGR, @Collaboratory1, @texhern

July 12, 2019: medRxiv: A Paradigm Shift in Disseminating Clinical and Public Health Research (Harlan Krumholz, MD, SM, Joseph Ross, MD, MHS)


Harlan M. Krumholz, MD, SM
Harold H. Hines, Jr. Professor of Medicine and Public Health
Yale University

Joseph S. Ross, MD, MHS
Associate Professor of Medicine and Public Health
Yale University


medRxiv: A Paradigm Shift in Disseminating Clinical and Public Health Research


Open science; Clinical research dissemination; Preprints; medRxiv preprint server

Key Points

  • medRxiv (med archive) is a server for health science preprints. It is a free service to the research community, managed in partnership with BMJ and Yale.
  • Benefits of preprints in medicine include early sharing of new information; enabling less “publishable” studies to be more readily available; and facilitating replication and reproducibility studies.
  • medRxiv submissions require:
    • Following ICMJE guidance, including author names, contact info, affiliation
    • Funding and competing interests statements
    • Statement of IRB or ethics committee approval
    • Study registration ( or other ICMJE approved registry for trials, PROSPERO for reviews) or link to protocol
    • Data sharing availability statement
    • EQUATOR Network reporting guidelines checklists
  • The medRxiv preprint server urges caution in using and reporting preprints, and includes language explaining that preprints are preliminary reports of work that have not been peer-reviewed, should not be relied on to guide clinical practice or health-related behaviors, and should not be reported in news media as established information.

Discussion Themes

Preprint servers do not replace, but rather complement, peer review.

Preprint has the potential for being a vehicle for high-quality but “negative” results. If we teach students that a negative result is also a good result, providing an avenue for us to walk-the-talk more easily via open communication seems largely positive despite the limitations.

Read more about medRxiv.


#pctGR, @Collaboratory1, @jsross119, @hmkyale

May 14, 2019: NIH Announces 3 New Funding Opportunities in Dissemination and Implementation of Evidence-Based Interventions

The NIH published 3 new funding opportunity announcements to support innovative approaches to the implementation of evidence-based interventions and the de-implementation of ineffective interventions. The announcement also encourages studies that advance dissemination and implementation research methods. The 3 funding opportunities are as follows:


September 28, 2018: Assessing and Reducing Risk of Re-identification When Sharing Sensitive Research Datasets (Greg Simon, MD, MPH, Deven McGraw, JD, MPH, Khaled El Emam, PhD)


Gregory Simon MD, MPH
Investigator, Kaiser Permanente Washington Health Research Institute

Deven McGraw, JD, MPH, LLM
General Counsel & Chief Regulatory Officer, Ciitizen

Khaled El Emam, PhD
Department of Pediatrics, University of Ottawa
Children’s Hospital of Eastern Ontario Research Institute


Assessing and Reducing Risk of Re-identification When Sharing Sensitive Research Datasets


Clinical trials; Research ethics; Data security; Data sharing; Sensitive research data; De-identified data

Key Points

  • The cycle of risk de-identification involves setting a risk threshold, measuring the risk, evaluating the risk, and applying transformations to reduce the risk.
  • The Safe Harbor method of de-identification (removal of 18 categories of data) is a legal minimum standard that does not take context into account, and may not be sufficient when sharing sensitive data publicly.
  • A higher standard for de-identification is the “Expert Determination” method, whereby an expert with contextual knowledge of the broader data ecosystem can determine whether the risk is “not greater than very small.”
  • With increasing concern about the risks of sensitive data sharing, it is important to be transparent with data participants and continue to build trust for data uses.

Discussion Themes

When is a dataset safe for sharing? What is the risk of re-identification, and how can we reduce the risk? Consider who you are releasing the data to and what other kinds of data might they have access to that could potentially lead to re-identification.

For more information on the de-identification of protected health information, visit the U.S. Department of Health and Human Services’s Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule.

The Health Information Trust Alliance de-identification framework identifies 12 criteria for a successful de-identification program and methodology.


#pctGR, #PragmaticTrials, #HealthData, @HealthPrivacy @Collaboratory1, @PCTGrandRounds

January 10, 2018: NCI Announces Training Institute in Dissemination and Implementation Research in Cancer

The National Cancer Institute’s Division of Cancer Control & Population Sciences is accepting applications for a Training Institute in Dissemination and Implementation Research in Cancer (TIDIRC). Designed for investigators at all levels who conduct dissemination and implementation research in cancer control, the training institute will consist of a 4-month online course and a 2-day in-person training.

There are no costs associated with the training institute, though applicants who are accepted into the program must cover travel-related expenses for the in-person meeting in Bethesda, Maryland.

This cancer-focused training program is adapted from a broader curriculum offered in previous years, the Training Institute in Dissemination and Implementation Research in Health (TIDIRH). Read an article in Implementation Science about the development of the program: “The U.S. Training Institute for Dissemination and Implementation Research in Health.”

The application window for the training institute will close on February 9, 2018. Read the announcement and submit an application:

December 14, 2017: NIH Encourages Authors to Publish in Reputable Journals

Publication in journals is an essential part of the scientific process, an important metric for scholarly career advancement, and an integral aspect of grant approval and funding mechanisms. Pressure to publish has engendered a global industry of open access journals, which are driven by author publication fees rather than subscription services. Some of these journals are not credible and are considered predatory; they may have misleading pricing, fraudulent editorial boards, and inadequate (or nonexistent) peer-review processes.

To help ensure the credibility of published research, the NIH released a statement on publication in scientific journals (NOT-OD-18-011).

“In support of public access to National Institutes of Health (NIH) funded research, authors are encouraged to publish their results in reputable journals. The NIH has noted an increase in the numbers of papers reported as products of NIH funding which are published in journals or by publishers that do not follow best practices promoted by professional scholarly publishing organizations.”

There are a number of resources available to authors to help identify reputable journals:

For more information on disseminating results from pragmatic clinical trials see the Living Textbook Chapters:

Journal Editors Propose New Requirements for Data Sharing

On January 20, 2016, the International Committee of Medical Journal Editors (ICMJE) published an editorial in 14 major medical journals in which they propose that clinical researchers must agree to share the deidentified data set used to generate results (including tables, figures, and appendices or supplementary material) as a condition of publication in one of their member journals no later that six months after publication. By changing the requirements for manuscripts they will consider for publication, they aim to ensure reproducibility (independent confirmation of results), foster data sharing, and enhance transparency. To meet the new requirements, authors will need to include a plan for data sharing as a component of clinical trial registration that includes where the data will be stored and a mechanism for sharing the data.

Evolving Standards for Data Reporting and Sharing

As early as 2003, the National Institutes of Health published a data sharing policy for research funded through the agency, stipulating that “Data should be made as widely and freely available as possible while safeguarding the privacy of participants, and protecting confidential and proprietary data.” Under this policy, federally funded studies receiving over $500,000 per year were required to have a data sharing plan that describes how data will be shared, that shared data be available in a usable form for some extended period of time, and that the least restrictive method for sharing of research data is used.

In 2007, Congress enacted the Food and Drug Administration Amendments Act. Section 801 of the Act requires study sponsors to report certain kinds of clinical trial data within a specified interval to the registry, where it is made available to the public. Importantly, this requirement applied to any study classified as an “applicable clinical trial” (typically, an interventional clinical trial), regardless of whether it was conducted with NIH or other federal funding or supported by industry or academic funding. However, recent academic and journalistic investigations have demonstrated that overall compliance with FDAAA requirements is relatively poor.

In 2015, the Institute of Medicine (now the National Academy of Medicine) published a report that advocates for responsible sharing of clinical trial data to strengthen the evidence base, allow for replication of findings, and enable additional analyses. In addition, these efforts are being complemented by ongoing initiatives aimed at widening access to clinical trial data and improving results reporting, including the Yale University Open Data Access project (YODA), the joint Duke Clinical Research Institute/Bristol-Myers Squibb Supporting Open Access to clinical trials data for Researchers initiative (SOAR), and the international AllTrials project.

Responses to the Draft ICMJE Policy

The ICMJE recommendations are appearing in the midst of a growing focus on issues relating to the integrity of clinical research, including reproducibility of results, transparent and timely reporting of trial results, and facilitating widespread data sharing, and the release of the draft policy is amplifying ongoing national and international conversations taking place on social media and in prominent journals. Although many researchers and patient advocates have hailed the policy as timely and needed, others have expressed concerns, including questions about implementation and possible unforeseen consequences.

The ICMJE is welcoming feedback from the public regarding the draft policy at and will continue to collect comments through April 18, 2016.


Journal editors publish editorial in 14 major medical journals stipulating that clinical researchers must agree to share a deidentified data set: Sharing clinical trial data: A proposal from the International Committee of Medical Journal Editors (Annals of Internal Medicine version). January 20, 2016.

A New England Journal of Medicine editorial in which deputy editor Dan Longo and editor-in-chief Jeffrey Drazen discuss details of the ICJME proposal: Data sharing. January 21, 2016.

A follow-up editorial in the New England Journal of Medicine by Jeffrey Drazen: Data sharing and the Journal. January 25, 2016.

Editorial in the British Medical Journal: Researchers must share data to ensure publication in top journals. January 22, 2016.

Commentary in Nature from Stephan Lewandowsky and Dorothy Bishop: Research integrity: Don’t let transparency damage science. January 25, 2016.

National Public Radio interview on Morning Edition: Journal editors to researchers: Show everyone your clinical data with Harlan Krumholz. January 27, 2016.

Institute of Medicine (now the National Academy of Medicine) report advocating for responsible sharing of clinical trial data: Sharing clinical trial data: maximizing benefits, minimizing risk. National Academies Press, 2015.

Rethinking Clinical Trials Living Textbook Chapter, Acquiring and using electronic health record data, which describes the use of data collected in clinical practice for research and the complexities involved in sharing data. November 3, 2015.

NIH Health Care Systems Research Collaboratory data sharing policy. June 23, 2014.

List of International Committee of Medical Journal Editors (ICMJE) member journals.