Real-World Evidence: Patient-Reported Outcomes (PROs)
Section 2
How Are PRO Measures Used?
Within the context of pragmatic clinical trials embedded within health care systems (ePCTs), PRO measures that are already being collected as part of routine clinical care and/or for quality assurance purposes could easily be integrated into the trial. However, in some cases, PROs are not collected routinely, necessitating a separate data collection protocol (although this seems to be rapidly changing).
To use PRO data most effectively in pragmatic research, it is useful to understand the different roles played by PROs in clinical research, clinical care, and quality assurance.
Clinical Research
PRO Measures as Study Endpoints
PROs play a significant role as study endpoints in the development and evaluation of new therapies (Willke et al. 2004; Gnanasakthy et al. 2016; Gnanasakthy et al. 2017). PROs have not been used as frequently as study endpoints in ePCTs, in large part because PROs are often not included as part of routine clinical care and thus the electronic health record (EHR), which is typically the main source of data for many ePCTs. Still, several trials within the NIH Pragmatic Trials Collaboratory have or are assessing PROs as trial endpoints (see Table).
Table. NIH Pragmatic NIH Collaboratory Trials With PROs
Study Name |
Project Goal | Indication | Primary and Secondary Outcome | PRO Measures |
---|---|---|---|---|
BackInAction
Pragmatic Trial of Acupuncture for Chronic Low Back Pain in Older Adults |
Evaluate the safety and effectiveness of acupuncture in older adults with chronic low back pain | Chronic low back pain in adults ≥65 years of age at 4 performance sites (789 patients) | Primary: Back-related function
Secondary: Pain Intensity, Pain Interference |
Euro-QOL-5d
global impression of change (PGIC) - 1 item Pain Catastrophizing Scale (PCS) 6 item scale Patient Health Questionnaire (PHQ-2) GAD-2 PEG PROMIS Ability to participate in social roles and activities 4a PROMIS physical functioning 6b PROMIS sleep disturbance 6a Roland Morris Disability Questionnaire (RMDQ) TAPS 1 acupunture outside the study Adherence to assigned treatment Adverse events back pain history EHR High impact chronic pain Sleep duration question cognitive functioning screener Frailty profile PHQ4 screener for depression and anxiety Impact of COVID on overall health and access to healthcare Sciatica detection Pain related healthcare and self-mgmt practices NIH LBP Task Force fear avoidance (1-item) EXPECT acupuncture expectation questions (1-items) PROMIS fatigue scale Heal CDE demographic questions + BMI |
BeatPain UtahNonpharmacologic Pain Management in Federally Qualified Health Centers (FQHCs) Primary Care Clinics
|
To compare the effectiveness of nonpharmacologic intervention strategies for patients with back pain |
500 English- or Spanish-speaking patients with chronic low back pain seeking care in FQHCs throughout the state of Utah | Primary: the Pain, Enjoyment and General Activity measure of pain
Secondary: HEAL measures |
GAD-2
global impression of change (PGIC) - 1 item Pain Catastrophizing Scale (PCS) 6 item scale Patient Health Questionnaire (PHQ-2) Patient global impression change (PGIC) PEG PHQ-2 PROMIS physical functioning 6b PROMIS sleep disturbance 6a Sleep duration question TAPS 1 back pain history HICP (High Impact Chronic Pain) Pain Medications PSEQ-4 |
FM TIPS
Fibromyalgia TENS in Physical Therapy Study |
Test the feasibility and effectiveness of adding transcutaneous electrical nerve stimulation (TENS) nonpharmacologic treatment for pain and fatigue in patients with fibromyalgia (FM) | Fibromyalgia in adults at 24 routine physical therapy clinics and 6 health systems in rural and urban settings (~600 patients) | Primary: Pain
Secondary: Physical functioning |
Brief pain inventory - short form
Fibromyalgia Impact Questionnaire- Revised (FIQR) GAD-7 Pain Catastrophizing Scale (PCS) - 13 item scale global impression of change (PGIC) - 1 item Movement evoked (5x sit to stand) pain Movement evoked fatigue Multidimensional Assessment of Fatigue (MAF) PHQ-8 PROMIS physical functioning 6b PROMIS sleep disturbance 6a Rapid Assessment of Physical Activity (RAPA) Resting fatigue by NRS Resting pain by NRS TAPS 1 Adverse events Barrier to TENS Medications (targeted to pain, mood, sleep) Patient specific functional scale (psfs) Sleep duration question Symptom Severity Score Widespread pain index (WPI)
|
GRACE
Hybrid Effectiveness-Implementation Trial of Guided Relaxation and Acupuncture for Chronic Sickle Cell Disease Pain |
To assess the effects of guided relaxation and acupuncture treatments for people with sickle cell disease. | 366 people, aged 18 and up, living with chronic pain resulting from Sickle Cell Disease |
Primary: Pain interference, enjoyment of life, and physical function
Secondary: Anxiety, depression, sleep disturbance, and substance use |
GAD-7
global impression of change (PGIC) - 1 item Pain Catastrophizing Scale (PCS) 6 item scale Patient Health Questionnaire (PHQ-2) PEG PHQ-9 PROMIS GI Constipation 9a PROMIS pain interference 4a PROMIS physical functioning 6b Sleep duration question TAPS 1 Acupuncture Protocol Checklist ED Visits and Hospitalizations Non-Pharm Treatments Opioid Followback PROMIS sleep disturbance 8a Implementation Questionnaire Adverse Events Form |
Nonpharmacologic Options in Postoperative Hospital-based and Rehabilitation Pain Management |
Evaluate the feasibility of EHR-embedded patient- and clinician-facing decision support for nonpharmacologic pain care after surgery | Post-surgical pain following eligible procedures in adults at 6 large integrated health systems; 23 practice clusters (~100,000 patients) in 5 tranches | Primary: Physical function and pain interference
Secondary: Anxiety, sleep disturbance, use of opioids, and nonpharmacologic pain care modalities |
PHQ-2
TAPS1 GAD-2 Pain Catastrophizing Scale (PCS) 6 item scale Pain NRS PROMIS CAT Anxiety PROMIS CAT Pain Interference PROMIS CAT physical function |
OPTIMUM
Group-Based Mindfulness for Patients With Chronic Low Back Pain in the Primary Care Setting |
Evaluate a group-based mindfulness program (mindfulness-based stress reduction) for patients with chronic low back pain within primary care | Chronic low back pain in patients at primary care clinics in 3 large health systems (~450 patients) | Primary: Pain intensity, physical function
Secondary: Pain interference, psychological function, opioid prescriptions |
Charlson co-morbidity index
Cognitive and affective mindfulness scale - revised (cams-r) current opioid misuse measure (comm) GAD-2 global impression of change (PGIC) - 1 item healing encounters and attitudes list (heal-expectation) health care system utilization (patient report) opioid use, single item Pain Catastrophizing Scale (PCS) 6 item scale Patient Health Questionnaire (PHQ-2) PEG PHQ-2 PROMIS physical functioning 6b PROMIS sleep disturbance 6a satisfaction, single item Sleep duration question TAPS 1 COPC measure EHR promis 29 Ethics, single item Telehealth Usability Questionnaire |
PPACT
Pain Program for Active Coping and Training |
Help patients adopt self-management skills for chronic pain, limit use of opioid medications, and identify factors amenable to treatment in the primary care setting | Chronic pain in patients on long-term opioid therapy at 3 staff model health plans; involves 106 primary care clusters (860 patients) | Primary: Pain impact
Secondary: Pain-related disability, patient satisfaction, opioids and benzodiazepines dispensed and health care utilization |
Primary: PEG, a validated 3-item measure for pain (Cleeland and Ryan 1994; Keller et al. 2004)
Secondary: 24-item Roland Morris Disability Questionnaire (RMDQ) (Roland and Fairbank 2000; Jordan et al. 2006) |
TSOS
Trauma Survivors Outcomes and Support
|
To coordinate care and improve outcomes for trauma survivors with post-traumatic stress disorder (PTSD) and comorbidity and to provide the American College of Surgeons with multisite pragmatic trial evidence that could further inform regulatory policy | PTSD and comorbidity in trauma survivors at 25 US level 1 trauma centers (635 patients) | Primary: PTSD symptoms
Secondary: Depression, alcohol use, physical functioning |
Primary: 17-item PTSD checklist, civilian version (Weathers et al. 1991)
Secondary: The 9-item Patient Health Questionnaire (PHQ-9) brief depression severity measure (Kroenke et al. 2001; Arroll et al. 2010) The Alcohol Use Disorder Identification Test (AUDIT), a 10-item screening instrument for the early identification of problem drinkers (Bohn et al. 1995) The SF-12 at baseline Physical Components Summary Score and the SF-36 Physical Components Summary Score at the follow-up time points |
Abbreviations: NRS = numeric rating scale; PROMIS = Patient-Reported Outcomes Measurement Information System; CAT = computer adaptive testing; PEG = pain, enjoyment of life, and general activity; SF = short form
For NOHARM, all outcomes are collected via the EHR. For all other trials, separate mechanisms were needed to collect the measures.
As with all trial endpoints, researchers should specify in the research protocol whether a PRO endpoint will serve as a primary, secondary, or exploratory endpoint (FDA Guidance for Industry 2009). Often times there will be interest in the effect of an intervention on more than one aspect of the patient’s experience—for example, on pain severity, pain frequency, and interference in daily activities due to pain. With multiple PRO endpoints, care must be taken to create a strong a priori rationale for how the multiple endpoints will be handled at the analysis phase, because of the risk of Type I error inflation and/or challenges in interpreting patterns of results across endpoints. FDA has published guidance entitled Multiple Endpoints in Clinical Trials Guidance for Industry, which describe strategies managing multiple endpoints in a study.
It should be noted that ethical issues arise when collecting sensitive data that could signal distress, such as suicidal ideation, opioid use disorder, or depression. According to an article by Ali et al, investigators have an ethical obligation to monitor these signals and identify in advance if, when, and how such signals will trigger a response (Ali et al. 2022). Using examples from the NIH Collaboratory Trials, the authors offered preliminary recommendations and identified opportunities for future work, which include:
- Understanding and aligning stakeholder expectations
- Considering characteristics of the trial and study population to inform a response
- Defining triggers, thresholds, and responsibilities for action
- Identifying appropriate response mechanisms and capabilities
- Integrating with clinical practices and systems
- Addressing patient-subject privacy
PRO Measures as Monitoring Tools (Adverse Events and Symptoms)
PRO measures can also be used to capture or monitor the adverse effects of an intervention separately from its effectiveness. For example, clinical trial investigators collect adverse event (AE) data to ensure patient safety and inform sponsors, regulators, patients, caregivers, and clinicians about adverse effects of treatment. Clinicians typically grade AEs using the National Cancer Institute’s Common Terminology Criteria for Adverse Events (CTCAE) (National Cancer Institute 2018). In order to better capture negative, uncomfortable, and impactful symptoms from the patient’s perspective, such as nausea and anxiety, the National Cancer Institute developed the PRO-CTCAE, which was designed for adults participating in oncology trials (Basch et al. 2014; Dueck et al. 2015).
PRO Measures as the Intervention
In some cases, the PRO measure can be the intervention. A study by Basch et al. demonstrated that clinical benefits were associated with self-report of symptoms in patients receiving care for cancer, such as improved health-related quality of life, fewer emergency department visits, fewer hospitalizations, longer duration of palliative therapy, and superior survival rates (Basch et al. 2016). In the study, patients recorded symptoms on a tablet, and an automated alert was sent to clinicians when patient-reported symptoms were severe or worsening. The authors postulate that the benefits were due to increased rates of discussion between patients and clinicians resulting in intensified symptom management and improved symptom control.
Clinical Care
Ideally, a PRO instrument will not only be a valid and reliable way to collect data, but it will also make a positive contribution to clinical care (Farnik and Pierzchała 2012). Data collected from a PRO instrument can be used in longitudinal reporting at the point of care and as part of clinical decision-making and review of systems. In addition, PRO data can be used to trigger patient education and interventions and as a means to triage patients to receive other services, helping the patient understand that the information they are reporting is meaningful to their care.
One of the NIH Collaboratory Trials within the NIH Collaboratory provides a good example of how a PRO-based intervention for research can be incorporated into clinical care. The Collaborative Care for Chronic Pain in Primary Care project was a mixed-methods, cluster-randomized pragmatic clinical trial designed evaluate the integration of psychosocial services into the primary care of patients with chronic pain. The intervention, the Pain Program for Active Coping and Training (PPACT), involves behavioral skills training designed to engage patients in their own care and help them manage their pain.
The study compared the effects of the intervention versus usual care on a number of measures, including patients’ pain symptoms, functional ability, satisfaction with healthcare services, and receipt of opioids and benzodiazepine medication. As part of the project, the patient completed a brief pain inventory (online using the EHR patient portal, using interactive voice response technology, or via a call with a medical assistant). For patients randomized to the active intervention, a more extensive intake evaluation was completed and compiled into an electronic summary outside the firewall of the EHR and sent to participant’s primary care physician through the EHR. The report incorporated real-time analysis and scoring of the data and presented the information in clinical context; as a result, it provided the physician with easily interpretable, actionable information derived from PRO data in order to promote discussion with the patient, trigger educational interventions, and aid in clinical decision-making (Debar et al. 2022).
Patient Satisfaction and Quality Assurance
A systematic review of 27 studies in the cancer setting suggests that PROs improved patient-provider communication and patient satisfaction, in part, because clinicians talked to patients about their feelings and health status and were able to develop a shared view of treatment goals, health status, or reason for the visit (Chen et al. 2013). Online patient self-reporting of toxicity symptoms during chemotherapy has been shown to be feasible, even among patients with advanced cancer and high symptom burdens (National Quality Forum 2013). PROs can also be used as reliable measures of healthcare performance; for example, the National Quality Forum endorses the use of PRO-based performance measures for the purposes of performance improvement and accountability (National Quality Forum 2013).
SECTIONS
sections
- Introduction
- How Are PRO Measures Used?
- Best Practices for Collecting PRO measures in Pragmatic Clinical Trials
- NIH HEAL, FDA, and Other Core Outcome Sets
- Choosing PRO Measures
- Cultural Adaptation and Linguistic Translation
- Case Study From Guiding Good Choices for Health
- Incorporating PRO Data Into the EHR
- Patient-Focused Drug Development
- Additional Resources
Resources
Interviews
AcuOA to Inform Medicare Coverage Decisions on Acupuncture: An Interview With Dr. Karen Sherman and Dr. Lynn Debar
FM TIPS Explores Novel Fibromyalgia Treatment in Physical Therapy Practices: An Interview With Dr. Kathleen Sluka and Dr. Leslie Crofford
NOHARM to Test EHR-Embedded Clinical Decision Support for Postoperative Pain Care: An Interview With Dr. Andrea Cheville
OPTIMUM to Bring Mindfulness-Based Stress Reduction to the Clinic: An Interview With Dr. Natalia Morone
Study Design Papers
REFERENCES
REFERENCES
Ali J, Morain SR, O’Rourke PP, et al. 2022. Responding to signals of mental and behavioral health risk in pragmatic clinical trials: Ethical obligations in a healthcare ecosystem. Contemp Clin Trials. 113:106651. doi:10.1016/j.cct.2021.106651.
Arroll B, Goodyear-Smith F, Crengle S, et al. 2010. Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Ann Fam Med. 8(4):348-353. doi:10.1370/afm.1139. PMID: 20644190.
Basch E, Abernethy AP, Mullins CD, et al. 2012. Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. J Clin Oncol. 30(34):4249-4255. doi:10.1200/JCO.2012.42.5967. PMID: 23071244.
Basch E, Deal AM, Kris MG, et al. 2016. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 34(6):557-565. doi:10.1200/JCO.2015.63.0830. PMID: 26644527.
Basch E, Reeve BB, Mitchell SA, et al. 2014. Development of the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). J Natl Cancer Inst. 106(9):dju244-dju244. doi:10.1093/jnci/dju244. PMID: 26644527.
Bohn MJ, Babor TF, Kranzler HR. 1995. The Alcohol Use Disorders Identification Test (AUDIT): validation of a screening instrument for use in medical settings. J Stud Alcohol. 56(4):423-432. doi:10.15288/jsa.1995.56.423. PMID: 7674678.
Chen J, Ou L, Hollis SJ. 2013. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 13:211. doi:10.1186/1472-6963-13-211. PMID: 23758898.
Cleeland CS, Ryan KM. 1994. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singap. 23(2):129-138. PMID: 8080219.
Dueck AC, Mendoza TR, Mitchell SA, et al. 2015. Validity and reliability of the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 1(8):1051-1059. doi:10.1001/jamaoncol.2015.2639. PMID: 26270597.
Farnik M, Pierzchała WA. 2012. Instrument development and evaluation for patient-related outcomes assessments. Patient Relat Outcome Meas. 3:1-7. doi:10.2147/PROM.S14405. PMID: 22915979.
FDA Guidance for Industry. 2009. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Accessed September 25, 2013. http://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf.
Gnanasakthy A, DeMuro C, Clark M, Haydysch E, Ma E, Bonthapally V. 2016. Patient-reported outcomes labeling for products approved by the Office of Hematology and Oncology Products of the US Food and Drug Administration (2010-2014). J Clin Oncol. 34(16):1928-1934. doi:10.1200/JCO.2015.63.6480. PMID: 27069082.
Gnanasakthy A, Mordin M, Evans E, Doward L, DeMuro C. 2017. A review of patient-reported outcome labeling in the United States (2011–2015). Value Health. 20(3):420-429. doi:10.1016/j.jval.2016.10.006. PMID: 28292487.
Jordan K, Dunn KM, Lewis M, Croft P. 2006. A minimal clinically important difference was derived for the Roland-Morris Disability Questionnaire for low back pain. J Clin Epidemiol. 59(1):45-52. doi:10.1016/j.jclinepi.2005.03.018.
Keller S, Bann CM, Dodd SL, Schein J, Mendoza TR, Cleeland CS. 2004. Validity of the brief pain inventory for use in documenting the outcomes of patients with noncancer pain. Clin J Pain. 20(5):309-318. doi:10.1097/00002508-200409000-00005. PMID: 16360560.
Kroenke K, Krebs EE, Turk D, et al. 2019. Core outcome measures for chronic musculoskeletal pain research: recommendations from a Veterans Health Administration Work Group. Pain Med. 20(8):1500–1508. doi:10.1093/pm/pny279. PMID: 30615172.
National Cancer Institute. 2018. Common Terminology Criteria for Adverse Events (CTCAE). Accessed August 5, 2019. https://ctep.cancer.gov/protocoldevelopment/electronic_applications/ctc.htm#ctc_50.
National Quality Forum. 2013. Patient Reported Outcomes (PROS) in Performance Measurement. Accessed December 5, 2013. http://www.qualityforum.org/Publications/2012/12/Patient-Reported_Outcomes_in_Performance_Measurement.aspx.
Roland M, Fairbank J. 2000. The Roland-Morris Disability Questionnaire and the Oswestry Disability Questionnaire. Spine. 25(24):3115-3124. doi:10.1097/00007632-200012150-00006. PMID: 11124727.
Weathers F, Huska J, Keane T. 1991. The PTSD checklist-civilian version. Accessed May 5, 2020. https://www.mirecc.va.gov/docs/visn6/3_PTSD_CheckList_and_Scoring.pdf.
Willke RJ, Burke LB, Erickson P. 2004. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials. 25(6):535-552. doi:10.1016/j.cct.2004.09.003. PMID: 15588741.
current section : How Are PRO Measures Used?
- Introduction
- How Are PRO Measures Used?
- Best Practices for Collecting PRO measures in Pragmatic Clinical Trials
- NIH HEAL, FDA, and Other Core Outcome Sets
- Choosing PRO Measures
- Cultural Adaptation and Linguistic Translation
- Case Study From Guiding Good Choices for Health
- Incorporating PRO Data Into the EHR
- Patient-Focused Drug Development
- Additional Resources