These examples illustrate what can be learned using the Intervention Delivery Complexity Tool described in the previous section.

The first example compares PROVEN (Mor et al. 2017; Mitchell et al. 2020) to ACP COVID (Volandes et al. 2022). Both trials tested an intervention comprising a video of advance care planning (ACP), and PROVEN was conducted prior to ACP COVID with some of the same co-PIs. The visualization indicates PROVEN was much more complex with intervention delivery, involving an entirely new workflow. In contrast, ACP COVID did not require a new workflow.

Interestingly, uptake of the intervention was poor in PROVEN (<40%), and the trial results were null. For ACP COVID, uptake of the intervention was better, and the trial resulted in higher rates of ACP documentation, especially for African American and Hispanic patients (Volandes et al. 2022).

  Figure 1. Intervention Delivery Complexity: PROVEN vs ACP COVID

PROVEN Pragmatic Trial of Video Education in Nursing Homes NCT02612688 Goal: Determine if showing advanced care planning videos in nursing homes affects the rates of hospitalization Setting: 2 nursing home health systems; 359 nursing homes / Nursing home health systems serving long-stay (>12 months) patients with advanced comorbid conditions (166,196 patients)	ACP COVID Advance Care Planning: Communicating with Outpatients for Vital Informed Decisions NCT04660422 Goal: Test whether clinician communication skills training and patient video decision aids increase completion of advance care planning Setting: 42,091 patients >65 years of age at risk for COVID at clinics affiliated with Northwell Health
Analysis: PROVEN was much more complex. It required an entirely new process of having a nurse show a video to residents and family members, and documenting that the video was shown. The conversation with the provider was not immediate, and the pathway from intervention to outcome (reduction in rates of hospitalization) was long. Uptake of the intervention was poor (<40%), and the results or the trial were null. The PI stated, “The pathway from intervention to outcome was almost insurmountable…show a video, have a conversation, document the conversation, refer to a physician or nurse practitioner to apply a medical order, have the order carried out in a highly regulated nursing home setting….it’s almost too hard to fathom!” Additionally, showing the video was entirely dependent on staff having the device and remembering to do so during an in-person interaction.	Analysis: ACP COVID was less complex. Videos were texted to patients, automatically given to patients upon room assignment, and shown automatically in the waiting room. Then patients immediately saw their clinicians, who were trained to have the ACP conversation. The path from intervention (video) to outcome (completion of ACP) was short and almost happened in a linear fashion. As a result, uptake of the intervention was good, and the trial results were positive.

Figure from Staman et al. 2023. Used with permission.

A second example describes PPACT (DeBar et al. 2018) vs FM-TIPS, both designed to address the opioid crisis. PPACT was funded in the first round of NIH Pragmatic Trials Collaboratory trials (2012), and demonstrated higher intervention delivery complexity than the subsequently funded (2020) FM-TIPS. Although speculative, this reduction in complexity may reflect shared learning, because PIs who are part of the NIH Pragmatic Trials Collaboratory regularly communicate about barriers and challenges to intervention delivery (Staman et al. 2023). Not surprisingly, reducing the amount of complexity is a common way to address these barriers and challenges.

Figure 2. In NIH Pragmatic Trials Collaboratory, Trials Intended to Reduce Opioid Use Have Become Less Complex Over Time

Early Trial	Later Trial
PPACT Pain Program for Active Coping and Training NCT02113592 Goal: Help patients adopt self-management skills for chronic pain, limit use of opioid medications, and identify factors amenable to treatment in the primary care setting Setting: 3 staff model health plans; 106 primary care providers / 860 patients with chronic pain on long-term opioid therapy	FM-TIPS Fibromyalgia TENS in Physical Therapy Study NCT04683042 Goal: Test the feasibility and effectiveness of adding transcutaneous electrical nerve stimulation (TENS) nonpharmacologic treatment for pain and fatigue in patients with fibromyalgia (FM) Setting: 24 routine physical therapy clinics and 5 health systems in rural and urban settings / ~600 patients with FM
Analysis: PPACT demonstrated high complexity across all internal and one external domain. Accordingly, this trial required reconfiguration of a multidisciplinary team workflow (nurse, behavioral specialist, and PT intakes [and integrating pharmacist feedback] and follow-up group and individual appointments) as well as consultation with a primary care provider that was not part of standard clinic workflow. The PPACT intervention required new tasks including an aligned framework for understanding patient problems and approach, a congruent presentation to patients by all members of the multidisciplinary clinical team, cognitive behavioral therapy treatment (with multiple core behavioral, cognitive, and relaxation skills incorporated), and a yoga-adapted physical activity component. After 12 months, the intervention group (n = 850) experienced greater reductions on all self-reported outcomes (Debar et a. 2022).	Analysis: FM-TIPs design was informed by prior trials in the NIH Pragmatic Trials Collaboratory and had lower complexity throughout internal and external domains. The intervention had one component, a TENS unit, which was provided by physical therapists who were already trained in use of the unit. The trial is ongoing.

Figure from Staman et al. 2023. Used with permission.

Application of the tool to additional clinical trials beyond the NIH Pragmatic Trials Collaboratory will inform future refinements to the calculator as appropriate. Its creators hope this free tool will be helpful to researchers in planning and assessing interventions intended for the clinical setting.