The quality of the data in healthcare information systems has the potential to affect the results of phenotype-based queries in such a way that the resulting data may not be useful. Secondary use of healthcare data is defined as use of the data for a purpose other than that for which the data were originally collected (Safran et al 2007). This means that secondary users should not expect the data to meet their needs. For these reasons, data quality assessment should accompany phenotype validation.

Using healthcare data in the absence of an understanding of their accuracy, consistency, missingness, and possible biases can lead to misleading answers. The capacity of the data to support research conclusions is so important that requests for applications for the NIH Pragmatic Trials Collaboratory Trials require that data validation be addressed. A recent methodology report from the Patient-Centered Outcomes Research Institute (PCORI) (Kahn et al 2018) recommends reporting of data quality along with study results for observational and comparative effectiveness research. The report also provides a data quality assessment model and framework. Other guidelines from research networks provide practical advice for data quality checks and reporting (Brown, Kahn, and Toh 2013; Kahn et al 2015).

The NIH Pragmatic Trials Collaboratory has developed a data quality assessment framework to help investigators and research teams identify and implement necessary assessments. (See “Assessing Data Quality for Healthcare Systems Data Used in Clinical Research.”) There are few validated electronic methods for data quality assessment that can be executed on a dataset. Instead, current methods for data quality assessment are comparison-based, involving comparison of chart review to data returned from a phenotype-based query, or comparison of 2 datasets to quantify the number and type of discrepancies and understand how they might be distributed in a dataset.