There are many different advantages and challenges associated with the use of digital technologies in ePCTs. In this section, we describe some of the special considerations.

Mobile Apps

Mobile app-based studies can be built with existing mobile frameworks like Apple ResearchKit for iPones and iPads and ResearchStack for Android. Potential participants can download an app in order to participate, and typically receive a secure token that links them to a particular study (Dameff et al. 2019). These apps provide a foundation for efficiently building mobile-based studies, which have been gaining traction in recent years. For example, a PubMed search on the keywords ResearchKit returned 15 clinical studies (Chan et al. 2017; Goyal et al. 2017; Webster et al. 2017; Zens et al. 2017; Crouthamel et al. 2018; Egger et al. 2018; Hausmann et al. 2018; Radin et al. 2018; Hershman et al. 2019; Yamaguchi et al. 2019; Yoshimura et al. 2019; Rubin et al. 2019; Ahmad et al. 2020; Wang et al. 2020; Inomata et al. 2020). In order to successfully use mobile apps in pragmatic clinical trials, investigators need to consider whether or not prospective patients own a smartphone capable of operating the app, and whether they are willing to download, authenticate, provide consent as necessary, and actually use the app. Mobile apps have to potential to be used for standalone data collection, and they can be linked to existing real-world data sources, such as the electronic health record.

The FDA MyStudies app is and example of a mobile app that can be used for research. It is a customizable and reusable application for mobile devices that links electronic health data with the patient perspective for use in clinical research. Potential users can configure the app for specific research studies through a web-based configuration portal, and use a secure patient data storage environment that is compliant with Federal Information Security Management Act (FISMA) security standards, HIPAA, and 21 CFR Part 11. The open source code is available on GitHub, and more information can be found at the MyStudies Mobile App Quick Overview for Research. In response to the COVID-19 pandemic, the COVID MyStudies app replaced the FDA MyStudies app to allow for remote patient enrollment and informed consent for studies of the disease. However, effective September 2022, the COVID MyStudies app will no longer be available as funding expires. For more information, refer to Phasing Out of the COVID MyStudies Application (App) on the FDA website.

Eligibility

Eligibility criteria outlined in exploratory clinical trials often fail to fully encapsulate the actual demands placed on individuals asked to participate. For example, randomization procedures for an explanatory trial of an mHealth intervention meant to improve medication adherence cannot be feasibly conducted with patient blinding. Further, it would need to account for or assume some background use of other common tools, such as pharmacy reminders, digital health assistants, and others. Pragmatic trials attempt to expand the pool of eligible participants to more closely match the clinical population seen in real-world settings. It bears noting that pragmatic mHealth trials still rely on a network of physical, social, and institutional assumptions about participant abilities and burdens, chiefly, consistent access to and comfort with the requisite technology. mHealth interventions, as well as many digital health interventions in the broad sense, seek to close this gap further by engaging patients/consumers through platforms they already use, on technology which is otherwise ubiquitous and familiar to even technologically-naive individuals.

The same is true of pragmatic evaluations of these interventions, where recruitment and retention exist within a two-stage framework: 1) recruitment and retention of the individual, and 2) their requisite technology. When done successfully, the opportunities for evaluation of a demonstrably representative study population and dissemination to other contexts and health systems are vastly improved.

Patient Engagement and Recruitment

Mobile apps and other digital technologies can facilitate remote engagement and recruitment and have the potential to reduce barriers to engagement and increase efficiency for enrollment. For example, Hugo is a freely available app that allows the combination of clinical data from all of a person’s encounters (pharmacy, labs, health systems, and payers) with patient-generated and patient-reported data for use by the patient (Beckman and Gupta 2018). Although the use of Hugo is relatively new in “sync-for-science” research, a recent study demonstrated its feasibility for enrollment and continued engagement in 25 people after percutaneous coronary intervention (Dhruva et al. 2019). Hugo Health can be downloaded in the app store for use on a smartphone or tablet or accessed through a website, and a person can add a study code (given to them by their healthcare provider) to join a study.

The Clinical Trials Transformation Initiative (CTTI) has a set of recommendations for Optimizing Mobile Clinical Trials by Engaging Patients and Sites that includes

• Recommendations for maximizing value and minimizing burden for study participants
• Addressing challenges for sites
• A checklist for trial sponsors for selecting and equipping sites for mHealth trials
• A checklist for investigative sites intended to help with budgeting and contracting

The recruitment of a diverse set of eligible individuals represents the fundamental challenge to pragmatism in mHealth trials, as each consideration represents a trade-off between representativeness and real-world applicability versus the practical ability to deliver materials and conduct an effective evaluation.

While identifying participants administratively may be possible—and even preferable—using administrative electronic health record data, the traditional consent process often proves insufficient, as it sacrifices either pragmatism or a priori informed consent. In the first case, the team may elect to identify and approach patients in the same manner as they would for standard clinical trials (where consent occurs face-to-face, often during or adjacent to an in-person clinical encounter), simultaneously limiting the sample size, representativeness, and diversity of the resulting sample. An advantage to this strategy, however, is that study staff are available to help patients overcome any initial technologic barriers to participation (usernames and passwords, transfer syncing, etc.). In the second case, patients meeting criteria may be able to be identified administratively, and enrolled in a program using some form of technology-mediated outreach (text messaging in particular), using either an opt-in or opt-out notification procedure. This avoids the limitations associated with traditional consent, but raises concerns about the ability to provide participants with adequate notice and information necessary for consent to participate.

Conducting Interventions Remotely

For some studies, the digital technology can be used to deliver the intervention. In our case example of the Nudge study, the intervention is a text message reminding patients with chronic cardiovascular conditions to refill their medications (described in detail in Case Example From the Nudge Study). Another recent example is a pragmatic clinical trial conducted to determine if a smartphone app is effective in reducing menstrual pain (Wang et al. 2020).

Completing Follow-up Remotely or Research at Home

Patients can connect their personal devices to healthcare systems to both retrieve information for personal use and continuity of care, or to deliver information for clinical care and/or research, such as glucose levels, levels of pain after surgery, or information about hospitalizations or readmissions (Dameff et al. 2019).

Ascertainment of a Range of Outcomes

Digital technology can be used to assess a range of outcomes, such as patient health status, daily activities, sleep, hospitalization, or patient-reported outcomes.

Setting

The setting domain provides perhaps the greatest advantage to mHealth interventions, particularly for pragmatic trials. With common methods of evaluating pragmatism (Thorpe et al. 2009), trial settings are described according to how similar or different the intervention context is than where patients typically receive care. In the case of mHealth, this comparison is not made between the message or application and office visit, but rather between the types of health-related activities patients are apt to engage in with their personal electronics. Text messaging, push notification, web-enabled treatment or symptom monitoring, and other related interventions all seek to structure the manner in which patients interact with healthcare resources in the social environment patients already inhabit, piggybacking on platforms and technology-use behaviors already deeply ingrained in daily routines.

Organization

The organizational setting or context is another highly variable consideration for mHealth interventions and pragmatic trials. While some interventions may exist on ubiquitous platforms (e.g., SMS text messaging, email), the integration of data sent to and received from patients may rely on an IT infrastructure that only exists within highly specialized medical systems, academic settings, or worse yet—at the study site itself. This issue is magnified with interventions using less common or bespoke platforms, where the expertise necessary to maintain and navigate these systems is highly localized.

Authenticity

Authenticity of the data being captured in pragmatic mHealth trials deserves special consideration. When patients share devices, or when caregivers or other loved ones primarily manage the devices being used to gather patient data, it is critical to confirm who is entering data at any given time. This is particularly pertinent to studies targeting older, multimorbid, or individuals with lower SES. In this case, consideration of data authenticity has both scientific and practical implications, as older and low SES populations are those not typically receiving outreach or education in other formats, and are therefore theoretically more likely to benefit from an intervention if it is efficacious. Some methods of authentication, including biometric authentication, metadata, and time stamps, may be viable in some cases. However, individuals who share or do not manage their own devices are less likely to own devices that are capable of sophisticated biometric authentication. In these cases, it may be advisable to simply include a question on each observation about whether the patient or someone else is entering data.