Feasibility Assessment Scenarios From the Collaboratory’s Demonstration Projects

Assessing Feasibility

Section 6

Feasibility Assessment Scenarios From the Collaboratory’s Demonstration Projects


Lynn L. DeBar, PhD, MPH

Jeffrey G. Jarvik, MD, MPH

Leah Tuzzio, MPH

Miguel A. Vazquez, MD


Contributing Editors

Liz Wing, MA

Karen Staman, MS


The following table, excerpted from the Collaboratory’s Lessons Learned document, gives concrete examples of potential issues and troubleshooting approaches taken by the Demonstration Project study teams around EHR use, engaging stakeholders, and integrating study workflows.

Feasibility Assessment Examples
Assessment Approach
Patient-reported outcomes, such as the Brief Pain Inventory, were not embedded in the EHR system to allow extraction from the record. Required building an enhanced infrastructure for quarterly PRO data collection designed to be as easily scalable as possible. For example, reliance on patient health record and interactive voice response systems in clinic use and reserving person-based outreach only when patient did not engage with automated outreach.
Navigating local systems was challenging. Involved the QI infrastructure in trial planning. QI project managers were embedded in healthcare systems and guided projects.
The study team did not anticipate some of the delays associated with data validation. Reallocated funds for additional IT and data analyst efforts.
Because the primary outcome is hospitalization rate per person day-alive, the data needed to be matched between nursing homes and hospitals and Medicare vital statistics data since nursing home data alone could have biased results. Added additional IT resources to help link the systems.
Capabilities of the EHR systems were varied with no single administrative database. Asked all level 1 and 2 trauma centers to complete a survey regarding EHR capabilities and found that while some sites were able to automate PTSD screening, other sites needed to screen manually. Developed methods to work with all sites regardless of capability and created a 10-domain EHR screen for risk factors for PTSD and other comorbid conditions.
A small change to workflow or the IT system was often viewed as a large change by health system personnel. More activity than expected was required at the local level and with individual practitioners and administrators to engage the personnel at the facilities.
The study team initially planned for structured, step-wise electronic tools that were time-consuming to use but would provide a detailed therapy plan. After discussing the tool with medical directors and physicians, the team developed more user-friendly, less burdensome tools.
Management of multiple chronic conditions varies across different healthcare systems. Study facilitators developed different workflows to accommodate the variations in resources at every site. These were roles in the healthcare systems and required more multidisciplinary review of the proposed workflows.
Updates in real-time with the use of the EHR meant that the lists of eligible and active patients at the clinics were continuously changing, which caused discordance between the lists that had been gathered for research purposes. The team worked with the statisticians and added a secondary analysis. In another instance, much more intensive analyst staffing during participant recruitment was required to accommodate these frequent updates in provider and clinic assignment of potentially eligible patients.
The study team and healthcare system partners did not want to recruit facility leadership to participate in the study and then tell them they were assigned to control since the partners felt that all facilities would want to have the intervention video. The team chose to "prerandomize" by first applying eligibility criteria to existing data on all of the partner facilities and then giving them the opportunity to exclude other facilities based on recent leadership changes. They next divided facilities into a priori strata and randomly selected the 120 treatment facilities from the pool, leaving the rest as controls. In this way, no facilities that wanted to participate were disappointed; the partners were confident that they would have a high participation rate.
The initial sample size was based on broad estimates of the prevalence of multiple chronic conditions across the healthcare systems and was limited by lack of cluster-level detailed information. In the planning phase, the cluster units were redefined from individual practitioners to practice sites. The team queried EHR systems with the new cluster definition and collaborated with statisticians at the NIH to establish an appropriate sample size.




DeBar LL, Jarvik JG, Tuzzio L, Vazquez MA. Assessing Feasibility: Feasibility Assessment Scenarios From the Collaboratory’s Demonstration Projects. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: http://rethinkingclinicaltrials.org/assessing-feasibility/feasibility-assessment-scenarios-from-the-collaboratorys-demonstration-projects/. Updated October 17, 2017.