Feasibility Assessment Scenarios From the NIH Collaboratory Trials

CHAPTER SECTIONS

Assessing Feasibility


Section 6


Feasibility Assessment Scenarios From the NIH Collaboratory Trials

Contributors

The following table gives examples of various feasibility challenges and troubleshooting approaches taken by the NIH Collaboratory Trial study teams during the trial’s planning or pilot phase.

Feasibility Assessment Examples
RAPT Domain Feasibility Challenge
 Approach
Measurement The original study design led to partial cross-nesting of intervention participants, which would have threatened valid statistical inference. Biostatisticians devised a novel analytic approach that resolved statistical concerns and, in a simulation study, showed strong power, nominal alpha levels, and adequate coverage.
Measurement EHR data did not include all adolescent outcomes and were not consistently available across the sites. Developed and tested an Adolescent Behavioral Health Survey to collect data on key adolescent outcomes.
Feasibility Federal regulations around buprenorphine (BUP) administration for opioid use disorder required physicians to have specialized, time-consuming training. Designed the intervention’s clinical decision support tool to provide flexibility for both waivered and non-waivered emergency department clinicians to use while remaining in compliance with regulatory statutes.
Feasibility Poor EHR usability was a barrier for incorporating a complex workflow in the emergency department setting. Optimized EHR usability and integration, automation of EHR workflow, and scalability across a variety of healthcare systems.
Feasibility/Cost Patient-reported outcomes, such as the Brief Pain Inventory, were not embedded in the EHR system to allow extraction from the record. Required building an enhanced infrastructure for quarterly PRO data collection designed to be as easily scalable as possible. For example, reliance on patient health record and interactive voice response systems in clinic use and reserving person-based outreach only when patient did not engage with automated outreach.
Acceptability Navigating local systems was challenging. Involved the QI infrastructure in trial planning. QI project managers were embedded in healthcare systems and guided projects.
Cost The study team did not anticipate some of the delays associated with data validation. Reallocated funds for additional IT and data analyst efforts.
Feasibility/Cost Because the primary outcome is hospitalization rate per person day-alive, the data needed to be matched between nursing homes and hospitals and Medicare vital statistics data since nursing home data alone could have biased results. Added additional IT resources to help link the systems.
Feasibility Capabilities of the EHR systems were varied with no single administrative database. Asked all level 1 and 2 trauma centers to complete a survey regarding EHR capabilities and found that while some sites were able to automate PTSD screening, other sites needed to screen manually. Developed methods to work with all sites regardless of capability and created a 10-domain EHR screen for risk factors for PTSD and other comorbid conditions.
Acceptability A small change to workflow or the IT system was often viewed as a large change by health system personnel. More activity than expected was required at the local level and with individual practitioners and administrators to engage the personnel at the facilities.
Feasibility/
Acceptability		 The study team initially planned for structured, step-wise electronic tools that were time-consuming to use but would provide a detailed therapy plan. After discussing the tool with medical directors and physicians, the team developed more user-friendly, less burdensome tools.
Feasibility/Cost/
Measurement		 Management of multiple chronic conditions varied across different healthcare systems. Study facilitators developed different workflows to accommodate the variations in resources at every site. These were roles in the healthcare systems and required more multidisciplinary review of the proposed workflows.
Feasibility/Cost/
Measurement		 Updates in real-time with the use of the EHR meant that the lists of eligible and active patients at the clinics were continuously changing, which caused discordance between the lists that had been gathered for research purposes. The team worked with the statisticians and added a secondary analysis. In another instance, much more intensive analyst staffing during participant recruitment was required to accommodate these frequent updates in provider and clinic assignment of potentially eligible patients.
Feasibility/Acceptability The study team and healthcare system partners did not want to recruit facility leadership to participate in the study and then tell them they were assigned to control since the partners felt that all facilities would want to have the intervention video. The team chose to "prerandomize" by first applying eligibility criteria to existing data on all of the partner facilities and then giving them the opportunity to exclude other facilities based on recent leadership changes. They next divided facilities into a priori strata and randomly selected the 120 treatment facilities from the pool, leaving the rest as controls. In this way, no facilities that wanted to participate were disappointed; the partners were confident that they would have a high participation rate.
Feasibility The initial sample size was based on broad estimates of the prevalence of multiple chronic conditions across the healthcare systems and was limited by lack of cluster-level detailed information. In the planning phase, the cluster units were redefined from individual practitioners to practice sites. The team queried EHR systems with the new cluster definition and collaborated with statisticians at the NIH to establish an appropriate sample size.

Previous Section Next Section

SECTIONS

CHAPTER SECTIONS
sections

  1. Introduction
  2. Developing the Trial Documentation
  3. Establishing Close Partnerships With Participating Healthcare System Leaders and Staff
  4. Delineating the Roles of All Interest Holders to Determine Training Needs
  5. Pilot Testing
  6. Feasibility Assessment Scenarios From the NIH Collaboratory Trials
  7. Spotlight on NIH Collaboratory Trials
  8. Additional Resources


Version History

October 25, 2022: Added the RAPT Domain to the feasibility assessment examples table as part of annual content update (changes made by E. McCamic).

August 27, 2020: Added four feasibility assessment examples to the table and made nonsubstantive changes to text as part of annual content update (changes made by L. Wing).

Published August 25, 2017

current section :

Feasibility Assessment Scenarios From the NIH Collaboratory Trials

  1. Introduction
  2. Developing the Trial Documentation
  3. Establishing Close Partnerships With Participating Healthcare System Leaders and Staff
  4. Delineating the Roles of All Interest Holders to Determine Training Needs
  5. Pilot Testing
  6. Feasibility Assessment Scenarios From the NIH Collaboratory Trials
  7. Spotlight on NIH Collaboratory Trials
  8. Additional Resources

Citation:

DeBar LL, Jarvik JG, Tuzzio L, Vazquez MA. Assessing Feasibility: Feasibility Assessment Scenarios From the NIH Collaboratory Trials. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Pragmatic Trials Collaboratory. Available at: https://rethinkingclinicaltrials.org/chapters/conduct/assessing-feasibility/feasibility-assessment-scenarios-from-the-collaboratorys-demonstration-projects/. Updated January 16, 2024. DOI: 10.28929/058.