This section details methods for obtaining data from participants. See the Patient-Reported Outcomes chapter of the Living Textbook for information about how patient-reported outcomes can be used as part of core outcome sets in pragmatic trials and strategies for choosing appropriate measures.

The participant can be a valuable source of data. Patient-reported data offer a relatively low-cost option and help keep participants engaged in the research (Marsolo 2019). However, the accuracy of patient-reported outcomes, including outcomes as notable as hospitalizations, has been questioned (Klungel et al 1999; Barr et al 2009; Krishnamoorthy et al 2016). One study compared the cumulative incidence of rehospitalizations identified by patient report vs medical bills and found that 10% of patients overreported and 18% underreported hospitalizations (Krishnamoorthy et al 2016). Of note, there were significant differences in patient characteristics associated with under- or overreporting: underreporting was more likely among patients who were older, women, African American, unemployed, or non–high school graduates; overreporting was more likely among patients who were women and unemployed (see figure below).

The authors of this study concluded that pragmatic research studies should not rely on patient report alone to identify rehospitalizations, and they suggested that additional mechanisms are needed. All sources of data have flaws—this is especially true of real-world data. Using multiple sources is a common technique for addressing this.

A major challenge of pragmatic research trial design is to navigate the tension between the uncertain accuracy of patient self-report and the need to efficiently utilize study resources and lower trial costs. (Krishnamoorthy et al 2016)

There are a number of methods to obtain information about patient-reported hospitalizations, and some are more passive (and less expensive) than others. The 3 most frequently used methods are patient portals, mobile apps, and call centers.

Portal

As part of a clinical study, participants may be asked to self-report events through either a portal or a mobile app. For example, the ADAPTABLE trial studied the use of low-dose aspirin vs a regular dose of aspirin in individuals with heart disease on outcomes such as myocardial infarction, mortality, and hospitalizations. To report their health status, participants were prompted to login to a patient portal. However, because this method might miss some patients, a call center contacted patients who did not login to the portal (National Patient-Centered Clinical Research Network [PCORnet] 2015).

Mobile Apps

Mobile apps can be used as standalone methods for patient data collection or in combination with other technologies. See the Mobile Health chapter of the Living Textbook for more information about opportunities and advantages and considerations for using mobile apps in clinical trials.

Some “enhanced recovery after surgery” programs use text messaging for extended surveillance. For example, text messaging has been used for home-based surveillance after colorectal surgery, where text message questionnaires led to follow-up care, including  in-hospital care, rehospitalizations, and unplanned surgeries (Carrier et al 2016).

Smartphones and other mobile technologies can be used to report events. For example, the National Evaluation System for Health Technology (NEST) was developed to ensure the postmarket safety and effectiveness of medical devices (Fleurence and Shuren 2019). A NEST feasibility/pilot NIH Collaboratory Trial was conducted to evaluate a mobile health platform for obtaining real-world postmarket surveillance data for patients after either bariatric surgery (sleeve gastronomy or gastric bypass) or catheter-based atrial fibrillation ablation (Dhruva 2020). Sixty participants were enrolled at their preoperative appointment for 8 weeks of postprocedural follow-up. The investigators used the Hugo app to obtain and aggregate electronic medical record data, pharmacy data, personal digital device data, and patient-reported outcomes through questionnaires. Hugo uses the sync-for-science model: people grant permission for applications like Hugo to access their her and pharmacy data and to sync these data (and their wearable device data) for research purposes through a common platform (Dhruva 2018).

It is important to note that a comprehensive picture can only be obtained if patients link their data from all the places where they receive care, and vendor support may be required the first time a connection is made to a given healthcare system. Prior to about 2020, the EHR data that could be consumed in an app like Hugo were summary documents in a structured format, such as Continuity of Care Documents. As more healthcare systems make EHR information available via Fast Healthcare Interoperability Resources (FHIR) application programming interfaces (APIs) and the associated technology becomes more widespread (for example, integrated into iOS and Android smartphones), app-based methods of obtaining data from participants may become more feasible.

Call Center

In a pragmatic trial, a call center can also be used as backup for other secondary information sources (as with the ADAPTABLE example above), or it can be the primary source of information (as with the TRANSFORM-HF example). The important point is that a call center can be used to validate inpatient events that are triggered by other means. For example, the Influenza Vaccine to Effectively Stop Cardiothoracic Events and Decompensated Heart Failure (INVESTED) trial was a pragmatic trial comparing the effectiveness of high-dose flu vaccine vs the standard dose in patients with a history of recent heart failure or myocardial infarction hospitalization (Vardeny et al 2021). Participants were asked to inform site personnel of hospitalizations and were then called during influenza season and during the summer to ascertain outcomes.