Choices regarding what data are procured and how they are used to build algorithmic models are critical to the integrity of the resulting AI/ML system. To be more equity-enhancing, researchers should develop AI systems with the practical realities of point of care in mind (McCradden et al 2022) and therefore confirm that training data are representative of the intended populations and—if not—assess any biases that might result. Researchers must also ascertain who is (and is not) represented in training data and the effects, if any, this will have on scientific endpoints.

Investigator tip: If patient data from electronic health records are used, consider whether healthcare access points affect representation. Biases can arise because training datasets generated from “patients” only represent people with access to healthcare, and are much more likely to be drawn from major academic medical centers with the most up-to-date data technology. Likewise, datasets generated from “research participants” reflect only those recruited to enroll and who subsequently consent to participate. Studies show there are known differences in consent rates between historically included and excluded populations (Spector-Bagdady et al 2021).