Real-World Evidence: Patient-Reported Outcomes (PROs)
Section 8
Incorporating PRO Data Into the EHR
Although there is increasing interest in incorporating PROs into the EHR, many obstacles remain, and best practices are still evolving (Synder and Wu 2017). Collaboratory investigators have encountered challenges with incorporating PRO data into the EHR. In a recent NIH Collaboratory study, 20 NIH Collaboratory Trials responded to a survey about challenges encountered when using the EHR for pragmatic clinical research (Richesson et al. 2021). The goal of the study was to elucidate challenges and develop solutions—or prerequisites for pragmatic research—to enable healthcare system leaders, policymakers, and EHR designers to improve the national capacity for generating real-world evidence.
“Seven projects—particularly those researching management of pain or psychological trauma—voiced a need for patient centered-data within health systems and EHRs. These studies reported having to collect primary data that were expected to be readily available, including patient reported outcomes, previously completed questionnaires, and advance care planning conversations.(Lakin et al. 2020) This led to time- and monetary-consuming adjustments.”
To counter this problem, the authors suggest the integration and collection of patient-centered data, including PROs, questionnaires, and advance care plans, into EHR systems and clinical workflows to support pragmatic research and personalized clinical care.
The process of implementing collection of PROs through the EHR may take substantial investments of time and money; current data collection approaches that support use of PROs are nascent and need better integration in clinical care (Van Der Wees et al. 2014). Additionally, due to concerns related to workflow and time constraints, uptake by clinical care staff may be slower than expected (Owen-Smith et al. 2018).
Another potential issue involves the concordance (or lack of concordance) between PROs and EHR data. For example, in the ADAPTABLE (Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-Term Effectiveness) pragmatic clinical trial, there were inconsistencies between PRO data and EHR–derived data for demographics and clinical events that occurred during follow-up (O’Brien, et al, in press), indicating that more work is needed to ensure adequate capture of information.
Burden an acceptability of PROs relate to the time it takes for the participant to complete the measure and how acceptable the measure is at capturing how the patient is feeling and functioning.
For example, PPACT was a large mixed-methods, pragmatic, cluster-randomized clinical trial conducted in 3 regions of Kaiser Permanente health systems. PPACT was designed to evaluate the integration of multidisciplinary services within the primary care environment compared with usual care in these settings. (See Section 2 for a complete list of PROs used by PPACT.)
Lessons learned from PPACT indicate several key factors for success:
- Brevity of the instrument
- Interpretability of results by those providing care
- Actionable information
- Flexible, multimodal approach (web-based personal health records, interactive voice response, live outreach with support staff as necessary) (Owen-Smith et al. 2018)
PCORI’s Users' Guide to Integrating Patient-Reported Outcomes in Electronic Health Records is designed to facilitate integration of PRO data into the EHR. The guide addresses 11 key questions for incorporating PROs into the EHR:
- What strategy will be used for integrating PROs in EHRs?
- How will the PRO-EHR system be governed?
- How can users be trained and engaged?
- Which populations and patients are most suitable for collection and use of PRO data, and how can EHRs support identification of suitable patients?
- Which outcomes are important to measure for a given population?
- How should candidate PRO measures be evaluated?
- How, where, and with what frequency will PROs be administered?
- How will PRO data be displayed in the EHR?
- How will PRO data be acted upon?
- How can PRO data from multiple EHRs be pooled?
- What are the ethical and legal issues? (Snyder et al. 2012)
Although PROs are being used in clinical research to support and inform labeling claims, clinical care, and reimbursement policy, the quality of content regarding PROs in protocols is often suboptimal (Calvert et al. 2018). Therefore, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT-PRO) Extension was developed to provide guidelines for the inclusion of PROs in clinical trial protocols. The extension provides many PRO-specific recommendations for protocols in which PRO data are a primary or secondary outcome (Calvert et al. 2018). These include:
- Specify the individual(s) responsible for the PRO content of the trial protocol
- Describe the PRO-specific research question and rationale
- State specific PRO objectives or hypotheses
- Specify any PRO-specific eligibility criteria (eg, language/reading requirements or prerandomization completion of PRO)
- Specify the PRO concepts/domains used to evaluate the intervention
- Include a schedule of PRO assessments
- When a PRO is the primary endpoint, state the required sample size and recruitment target
- Justify the PRO instrument to be used
- Include a data collection plan
- Specify whether more than 1 language version will be used and state whether translated versions have been developed using currently recommended methods
- State and justify the use of a proxy respondent, where necessary
- Specify PRO data collection and management strategies for minimizing avoidable missing data
- Describe the process of PRO assessment for participants who discontinue or deviate from the assigned intervention protocol
- State PRO analysis methods
- State how missing data will be described
- State whether or not PRO data will be monitored during the study to inform the clinical care of individual trial participants … Describe how this process will be explained to participants; eg, in the participant information sheet and consent form.
SECTIONS
sections
- Introduction
- How Are PRO Measures Used?
- Best Practices for Collecting PRO measures in Pragmatic Clinical Trials
- NIH HEAL, FDA, and Other Core Outcome Sets
- Choosing PRO Measures
- Cultural Adaptation and Linguistic Translation
- Case Study From Guiding Good Choices for Health
- Incorporating PRO Data Into the EHR
- Patient-Focused Drug Development
- Additional Resources
Resources
Design and analytic considerations for using patient-reported health data in pragmatic clinical trials: report from an NIH Collaboratory roundtable
This article describes approaches for ascertaining and classifying patient-reported health data as study endpoints and addressing issues of incomplete data, data alignment, and concordance.
Applying patient-reported outcome methodology to capture patient-reported health data: Report from an NIH Collaboratory roundtable
This article summarize key scientific literature on the accuracy of patient-reported data compared with health data
Patient-Centered Outcomes Core Working Group for the NIH Collaboratory:
Creates guidelines and defines best practices with respect to (1) selecting, compiling, and curating the most appropriate PRO measures (and stimulating the development of new instruments when new solutions are needed); (2) the creation efficient, high-quality PRO data collection systems compatible with EHRs and registries; and (3) conducting statistical analyses of PRO endpoints.
REFERENCES
Calvert M, Kyte D, Mercieca-Bebber R, et al. 2018. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: The SPIRIT-PRO Extension. JAMA. 319(5):483. doi:10.1001/jama.2017.21903. PMID: 29411037.
Lakin JR, Brannen EN, Tulsky JA, et al. 2020. Advance Care Planning: Promoting Effective and Aligned Communication in the Elderly (ACP-PEACE): the study protocol for a pragmatic stepped-wedge trial of older patients with cancer. BMJ Open. 10:e040999. doi:10.1136/bmjopen-2020-040999.
Owen-Smith A, Mayhew M, Leo MC, et al. 2018. Automating collection of pain-related patient-reported outcomes to enhance clinical care and research. J Gen Intern Med. 33(S1):31-37. doi:10.1007/s11606-018-4326-9. PMID: 29633139.
Richesson RL, Marsolo KS, Douthit BJ, et al. 2021. Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory. J Am Med Inform Assoc. 28:2626-2640. doi:10.1093/jamia/ocab202.
Snyder CF, Aaronson NK, Choucair AK, et al. 2012. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 21(8):1305-1314. doi:10.1007/s11136-011-0054-x. PMID: 22048932.
Synder C, Wu AW, eds. 2017. Users’ Guide to Integrating Patient-Reported Outcomes in Electronic Health Records. http://www.pcori.org/document/users-guide-integrating-patient-reported-outcomes-electronic-health-records.
Van Der Wees PJ, Nijhuis-Van Der Sanden MWG, Ayanian JZ, Black N, Westert GP, Schneider EC. 2014. Integrating the use of patient-eeported outcomes for both clinical practice and performance measurement: views of experts from 3 countries: patient-reported outcomes, clinical practice, performance measurement. Milbank Quarterly. 92(4):754-775. doi:10.1111/1468-0009.12091. PMID: 25492603.
current section : Incorporating PRO Data Into the EHR
- Introduction
- How Are PRO Measures Used?
- Best Practices for Collecting PRO measures in Pragmatic Clinical Trials
- NIH HEAL, FDA, and Other Core Outcome Sets
- Choosing PRO Measures
- Cultural Adaptation and Linguistic Translation
- Case Study From Guiding Good Choices for Health
- Incorporating PRO Data Into the EHR
- Patient-Focused Drug Development
- Additional Resources