As part of its framework for using real-world evidence derived from real-world data to support regulatory decision making, the US Food and Drug Administration (FDA) has identified several potential sources of real-world data and information (FDA 2017):

Electronic health records: Electronic health records (EHRs) contain information collected during the course of clinical care. They may include multiple care settings—outpatient visits, inpatient stays, emergency and urgent care visits, home health, etc. EHRs can include a variety of data from structured domains, including diagnoses, procedures, laboratory results, vital signs, medication orders, and medication administrations. They may also include less standardized data, such as information captured in inpatient flowsheets, questionnaires and surveys completed directly by patients, signs and symptoms recorded by clinicians, data on surgical care and anesthesia, and provider and nursing documentation.

Administrative claims: Administrative claims are insurance claims related to services from healthcare providers. In the United States, federal insurance programs include Medicare and Medicaid. The Medicare population includes adults 65 years and older, patients with certain disabilities, and patients with end-stage renal disease. Medicaid is an insurance program for people with low income. Private insurance claims include those in employer-sponsored health plans, insurance claims for those who are self-employed, and claims for insurance plans administered on behalf of the federal government. These administrative data can include information about physician services, institutional costs, demographic characteristics, deaths, dispensed medications, home health services, and skilled nursing facilities.

Patient-reported outcomes: Patient-reported outcomes (PROs) are defined by the FDA as any report of the status of a patient's health that comes directly from the patient, without interpretation of the patient's response by a clinician or anyone else (FDA Guidance for Industry 2009). PROs might include information about symptoms, functioning, satisfaction with care or symptoms, adherence to prescribed medications or other therapy, and perceived value of treatment. Typically captured in the form of surveys or questionnaires, PROs may be obtained via paper forms, online portals, or mobile apps. See the Patient-Reported Outcomes chapter of the Living Textbook for more information.

Patient-generated health data: Patient-generated health data are data generated from devices that provide information on a patient’s status (for example, internet-connected scales, pedometers, home blood pressure monitors). These data may be obtained directly from the device via a mobile application or through some other type of instrument. Patient-generated health data can include the raw sensor values and summary statistics calculated from the underlying data.

Medical product/device registries: Registries are typically created after a product or device has been approved in order to support postmarketing surveillance. These registries often contain rich information about the product or device but limited data on the characteristics or health status of patients, generally far less data than what is available in EHRs.

Condition-specific or disease registries: Registries contain information from patients who have a specific condition or disease. These patient-focused registries often include information about disease onset, symptoms, changing phenotypes, treatments, and outcomes. Because they are designed for research or targeted care, condition or disease-specific registries often have more condition-specific data than is collected in EHRs.

Environmental factors and social determinants of health: Environmental factors and social determinants of health (for example, food insecurity, access to transportation) are increasingly being captured in EHRs as healthcare systems focus more on population health. The data may be collected directly from patients through surveys or derived from community or geographically organized resources (for example, the American Community Survey) based on a patient’s current or historical address. Environmental sources can also be used in this manner, such as to estimate exposure to pollution based on distance to a freeway, power plant, or other industrial source.

In most cases, PROs and patient-generated health data are obtained directly from patients as part of a specific trial or study. In this context, PROs and patient-generated health data are collected prospectively using the procedures that govern prospective data collection, such as patient consent. PROs and patient-generated health data collected in the EHR or as part of a registry for other purposes (such as follow-up to a surgical procedure or monitoring of patients with a chronic disease) will typically be treated like the rest of the data that are contained in that source.

Identifying the Appropriate Source

Since data in secondary real-world data sources were collected or generated for purposes other than research, they include gaps and biases that reflect the nature of the underlying activity (Kahn and Ranade 2010; Hersh et al 2013; Weiskopf et al 2013; Raebel et al 2014; Rusanov et al 2014). Therefore, given a specific research question or study, it is important to assess whether the real-world data source is relevant and can reliably fulfill its intended purpose (FDA 2017; FDA 2021), whether it is for patient identification or recruitment, monitoring outcomes, or assessing endpoints. (See the Assessing Fitness-for-Use of Real-World Data chapter of the Living Textbook.)

In many cases, the same study concept can be present in multiple real-world data sources. For example, a disease diagnosis can be identified through a query of the EHR, administrative claims, a patient-reported medical history, or a disease registry. When designing a study, investigators should understand the trade-offs between different sources in terms of completeness across a potential study population, length of follow-up, etc. Depending on how data are captured, multiple sources may be needed to adequately support a study. In this case, an adjudication process is often necessary to decide what to do if there is discordance between sources (Rockhold et al 2020). Investigators should prepare to implement such a plan.

In addition, when combining study data with real-world data sources, some form of record linkage is usually required to match patients across sources. There are a number of techniques that can be used. Some rely on deterministic matches of clear-text identifiers, while others rely on probabilistic weighting of encrypted tokens generated from combinations of identifiers (such as first and last names, date of birth, and current zip code) (Grannis et al 2002; Durham et al 2010; Kum et al 2014; Setoguchi et al 2014; Durojaiye et al 2018; Karr et al 2019). Not all data holders are able to support all of these methods, so it is important to understand their capabilities, as well as the identifiers to which they have access. Study teams may then need to collect these same identifiers to allow for linkage to occur.