Dissemination and Implementation
Section 9
Pragmatic Implementation Process Assessments
Case Example: The Trauma Survivors Outcomes & Support (TSOS) Pragmatic Trial Research Team: Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE)
The use of qualitative/mixed methods to assess the process of implementing ePCTs can potentially be time consuming and expensive. Furthermore, implementation process assessments are often not integrated into routine workflow, and can make a trial less pragmatic (Palinkas and Zatzick 2019; Zatzick 2019). Implementation assessments may involve lengthy semi-structured interviews, central adjudication, and time-intensive qualitative coding procedures. To make implementation process assessments more pragmatic (and less resource-intensive), data could be captured reliably as part of routine pragmatic trial rollout.
One option for pragmatic implementation process assessment was developed by the Trauma Survivors Outcomes & Support (TSOS) pragmatic trial research team: Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) (Palinkas and Zatzick 2019). The RAPICE approach rolled out in the TSOS trial derives from clinical field experiences in disaster mental health contexts, including the response efforts to school shootings and the 2010 Haiti earthquake (Palinkas et al. 2004; Zatzick et al., 2010). RAPICE also derives from prior single-site comparative effectiveness trial clinical ethnographic methods that incorporate research team participant observation coupled with external mixed-method consultation (Zatzick et al. 2011). It is informed by other clinical ethnographic perspectives that advocate for researcher immersion in daily activities (Green et al. 2015), and by rapid assessment procedures, which use participant observation, non-directed interviewing, and other data collection methods, such as informal interviews, newspaper accounts, agency reports, and statistics, to verify information (Harris et al. 1997).
RAPICE and TSOS
As part of site visits and to provide training, the TSOS research team spent hundreds of hours annually immersed in trauma care system pragmatic trial rollout (Zatzick 2019). During these activities, the principal investigator and other team members logged field notes/jottings of their clinical research experiences. These data were reviewed regularly (e.g., monthly) with a mixed-method expert consultant. Themes related to intervention delivery, sustainable implementation, barriers, and facilitators were iteratively discussed and documented. When appropriate, observations were fed back to front-line providers rolling out the TSOS intervention; other implementation process observations were collected by the study team and are anticipated to be presented at the study team policy summit or other presented in other formats (e.g., peer-reviewed publications). It is noteworthy that these procedures, because they were embedded as part of the trial, did not substantially drive up trial time investments or costs.
“RAPICE applied in a pragmatic clinical trial is distinguished by the following: (1) formation of a multidisciplinary research team including a member or members with clinical and/or administrative expertise and ethno-graphic and mixed methods training, enabling efficiency in data collection and analysis through division of labor; (2) development of materials to train team members in ethnographic methods and rapid assessment procedures that minimize the burden placed on any single study participant; (3) use of several data collection methods (e.g., participant observation, informal and semi-structured interviews, field jottings and logs, quantitative surveys) to verify information through triangulation; (4) iterative data collection and analysis in real time to facilitate continuous adjustment of the research question and methods to answer that question; and (5) rapid completion of the mixed method component of the project, which may vary depending on project aims and mixed method design." (Palinkas and Zatzick 2019)
Since its development within the NIH Health Care Systems Research Collaboratory, the RAPICE method has been extended to additional investigative contexts. The study team has extended the RAPICE approach to target the RE-AIM evaluation framework honed to policy activities (Scheuer at al. 2020). Also, the RAPICE method was recently used during the Washington State COVID-19 outbreak in order to rapidly identify primary and secondary COVID-19 prevention strategies that could be readily delivered within the TSOS care management platform. The nimble RAPICE mixed method allowed initial observations of study team intervention activities to naturalistically evolve to incorporate the larger pandemic context (Moloney et al. 2020).
For more information, see the April 19, 2019 Grand Rounds: Trauma Survivors Outcomes & Support (TSOS) Pragmatic Trial: Revisiting Effectiveness & Implementation Aims (Doug Zatzick, MD)
SECTIONS
sections
- Introduction
- Dissemination and Implementation Frameworks
- Let It, Help It, Make It Happen
- Changes to Policy and Guidelines
- Legislative Changes
- Creation of Targeted Tools
- Stepped Wedge Designs
- Intervention Staffing and Training Flexibility
- Pragmatic Implementation Process Assessments
- Partnering With Quality Improvement and Population Health Initiatives
- Implementation in the Trial Versus in the Real World
- Additional Resources
- FAQ
Resources
Grand Rounds
Trauma Survivors Outcomes & Support (TSOS) Pragmatic Trial: Revisiting Effectiveness & Implementation Aims (Doug Zatzick, MD)
REFERENCES
American College of Surgeons. 2014. Resources for Optimal Care of the Injured Patient. https://www.facs.org/~/media/files/quality%20programs/trauma/vrc%20resources/resources%20for%20optimal%20care.ashx. Accessed August 1, 2017.
Green CA, Duan N, Gibbons RD, Hoagwood KE, Palinkas LA, Wisdom JP. 2015. Approaches to mixed methods dissemination and implementation research: methods, strengths, caveats, and opportunities. Adm Policy Ment Health. 42(5):508–523. doi:10.1007/s10488-014-0552-6. PMID: 24722814.
Harris KJ, Jerome NW, Fawcett SB. 1997. Rapid assessment procedures: a review and critique. Hum Organizat. 56(3):375–378. https://www.jstor.org/stable/44127200.
Moloney K, Scheuer H, Engstrom A, et al. 2020. Experiences and insights from the early US COVID-19 epicenter: a rapid assessment procedure informed clinical ethnography case series. Psychiatry. 1–13. doi:10.1080/00332747.2020.1750214. PMID: 32338566.
Palinkas LA, Zatzick D. 2019. Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: methods and applied case study. Adm Policy Ment Health. 46(2):255–270. doi:10.1007/s10488-018-0909-3. PMID: 30488143.
Palinkas LA, Prussing E, Reznik VM, Landsverk JA. 2004. The San Diego East County school shootings: a qualitative study of community-level post-traumatic stress. Prehospital and Disaster Med. 19(1):113-121. PMID: 15453168.
Scheuer H, Engstrom A, Thomas P, et al. 2020. A comparative effectiveness trial of an information technology enhanced peer-integrated collaborative care intervention versus enhanced usual care for US trauma care systems: Clinical study protocol. Contemp Clin Trials. 91:105970. doi:10.1016/j.cct.2020.105970. PMID: 32119926.
Zatzick DF. 2019. Trauma Survivors Outcomes & Support (TSOS) Pragmatic Trial: Revisiting Effectiveness & Implementation Aims. https://rethinkingclinicaltrials.org/news/april-19-2019-trauma-survivors-outcomes-support-tsos-pragmatic-trial-revisiting-effectiveness-implementation-aims-doug-zatzick-md/.
Zatzick DF, Russo J, Darnell D, et al. 2016. An effectiveness-implementation hybrid trial study protocol targeting posttraumatic stress disorder and comorbidity. Implement Sci. 11:58. doi:10.1186/s13012-016-0424-4. PMID: 27130272.
Zatzick D, Coq N, Frederic J, et al. 2010. Psychosocial support training for HIV health care providers in response to the Haitian earthquake. Consortium of Universities for Global Health Annual Meeting, University of Washington.
Zatzick D, Rivara F, Jurkovich G, et al. 2011. Enhancing the population impact of collaborative care interventions: mixed method development and implementation of stepped care targeting posttraumatic stress disorder and related comorbidities after acute trauma. Gen Hosp Psychiatry. 33(2):123-134. doi:10.1016/j.genhosppsych.2011.01.001. PMID: 21596205.
current section : Pragmatic Implementation Process Assessments
- Introduction
- Dissemination and Implementation Frameworks
- Let It, Help It, Make It Happen
- Changes to Policy and Guidelines
- Legislative Changes
- Creation of Targeted Tools
- Stepped Wedge Designs
- Intervention Staffing and Training Flexibility
- Pragmatic Implementation Process Assessments
- Partnering With Quality Improvement and Population Health Initiatives
- Implementation in the Trial Versus in the Real World
- Additional Resources
- FAQ