Real-World Evidence: Patient-Reported Outcomes (PROs)
Section 7
Case Study From Guiding Good Choices for Health
Part of the mission on the NIH Pragmatic Trials Collaboratory is to share lessons learned from the NIH Collaboratory Trials, and to this end we asked the principal investigators of the Guiding Good Choices for Health (GGC4H) NIH Collaboratory Trial (Scheuer et al. 2022) to give us their most critical advice and share their expertise and lessons learned regarding using PROs.
GGC4H tests the feasibility and effectiveness of implementing Guiding Good Choices, a parenting program for parents of early adolescents ages 11-14 that has been shown to reduce adolescent alcohol, tobacco, and marijuana use; depression; and delinquent behavior (Montero-Zamora et al. 2021). The trial is being conducted in 3 geographically and socioeconomically diverse large integrated healthcare systems and will compare GGC parenting intervention to usual pediatric primary care practice, and involve approximately 3750 adolescents. The primary outcome is substance use initiation, and the study team also asks adolescents to report on substance use frequency and amount. Secondary outcomes include antisocial behavior and depression (PHQ-9).
Table. Domains Assessed in Youth Surveys for GGC4H
Primary Outcomes | Secondary Outcomes | Exploratory Outcomes |
Substance use | Mental health | Anxiety (GAD-7) |
Age of initiation | Depression (PHQ-9) | Screen & social media time, sexting |
Lifetime frequency | Antisocial Behavior | |
Past-year, past 30-day use, past 30-day use amount | Ever | |
Substances examined | Past-year | |
Alcohol, marijuana, cigarettes, e-cigarettes, inhalants, opioids, other drugs |
From the GGC4H Grand Rounds on December 14, 2018.
Because the GGC4H study team has extensive experience collecting and using behavioral health PROs with adolescents, we asked the PIs of GGC4H to share their lessons learned.
What is your most critical advice regarding use of PROs?
Healthcare systems need to shift toward routinely collecting PRO data via the EHR. We had initially planned to use EHR data for our study, but because these data are not routinely collected during routine clinical encounters, we switched to RedCap, which is more involved and required more resources. Healthcare systems should collect behavioral, mental health, and substance abuse information more systematically and consistently, as these data are important to the care of their patients.
What tips would you share with other study teams?
- Use validated measures.
- Have an experienced data collection team
- Persistence is key. Reach out in various ways (eg, phone, email, text) to encourage responses.
Why use PROs in this population?
PROs provide a rich data source that complements EHR data and enables the use of quantitative and qualitative analysis methods. In behavioral health studies with a pediatric population (like ours), PROs are likely to be the primary source of outcomes information because (a) these outcomes are not necessarily collected routinely in pediatrics practice, (b) even when collected, they are not necessarily saved routinely in EHRs, and (c) even when collected, there may be measurement differences across settings (eg, PHQ-2 v. PHQ-9).
Data on risk and protective factors (eg, peer substance use, family conflict, attachment to parents) are also key predictors of study outcomes, and the study team collects these from adolescents as these data are also not available in EHRs.
Can you tell us what you learned about optimal administration formats within this study (internet vs telephone)?
We offered participants the option to complete online surveys or telephone interviews by trained interviewers. Most adolescents opted for self-administration over the internet, and tend to be more frank answering questions about substance use in a computerized format.
After the child assents, the data collection team contacts the teenager directly using their preferred method of contact. Researchers might nudge the parent if the child does not complete a questionnaire.
You collected the PHQ-9, which has questions about suicide ideation. Did you have a protocol in place for when these surveys signal potential distress?
When we assented the children and consented the parents, we let them know that if they express anything that makes us think they might be a danger to themselves or others, we will take action. For example, if an adolescent scores over a threshold on the PHQ-9, it triggers one of the interviewers to follow up with the Columbia Suicide Severity Rating Scale to assess degree of risk and respond accordingly. Adolescents commonly endorse distress, so the follow up with Columbia is important. If on assessment, they have suicide ideation, we have an IRB-approved protocol in place where they and their families are given resources to address the issue.
The expertise of your data collection team seems really critical to your project’s success. What skills did you include in the job description for the interviewers?
We have embedded teams within the heath system at each of our 3 sites of well-trained, skilled and personable interviewers who are experienced in collecting data from teenagers. Our research group deals exclusively with mental health issues and substance abuse. Adolescents are not always comfortable dealing with adults, and interviewing this population takes specific skills.
If the PROs were not routinely saved in the EHR, how did you save them, and how did you manage security concerns?
We used a separate independent data collection platform (REdCap) for the surveys. We do not go into the EHR, and as part of the consent/assent process we let families know that the data would not be put into the EHR. We link to EHR data using unique study IDs.
For more information on GGC4H, visit the GGC4H NIH Collaboratory Trial page on the Living Textbook and watch their Grand Rounds Presentation from 2018.
SECTIONS
sections
- Introduction
- How Are PRO Measures Used?
- Best Practices for Collecting PRO measures in Pragmatic Clinical Trials
- NIH HEAL, FDA, and Other Core Outcome Sets
- Choosing PRO Measures
- Cultural Adaptation and Linguistic Translation
- Case Study From Guiding Good Choices for Health
- Incorporating PRO Data Into the EHR
- Patient-Focused Drug Development
- Additional Resources
REFERENCES
Montero-Zamora P, Brown EC, Ringwalt CL, et al. 2021 Oct 9. Predictors of engagement and attendance of a family-based prevention program for underage drinking in Mexico. Prev Sci. doi:10.1007/s11121-021-01301-z.
Scheuer H, Kuklinski MR, Sterling SA, et al. 2022. Parent-focused prevention of adolescent health risk behavior: study protocol for a multisite cluster-randomized trial implemented in pediatric primary care. Contemporary Clinical Trials. 112:106621. doi:10.1016/j.cct.2021.106621. https://linkinghub.elsevier.com/retrieve/pii/S1551714421003578.
current section : Case Study From Guiding Good Choices for Health
- Introduction
- How Are PRO Measures Used?
- Best Practices for Collecting PRO measures in Pragmatic Clinical Trials
- NIH HEAL, FDA, and Other Core Outcome Sets
- Choosing PRO Measures
- Cultural Adaptation and Linguistic Translation
- Case Study From Guiding Good Choices for Health
- Incorporating PRO Data Into the EHR
- Patient-Focused Drug Development
- Additional Resources