Comparison of health measures between survey self-reports and electronic health records among Millennium Cohort Study participants receiving Veterans Health Administration care

Abstract: Background: Surveys are a useful tool for eliciting self-reported health information, but the accuracy of such information may vary. We examined the agreement between self-reported health information and medical record data among 116,288 military service members and veterans enrolled in a longitudinal cohort. Methods: Millennium Cohort Study participants who separated from service and registered for health care in the Veterans Health Administration (VHA) by September 18, 2020, were eligible for inclusion. Baseline and follow-up survey responses (2001-2016) about 39 medical conditions, health behaviors, height, and weight were compared with analogous information from VHA and military medical records. Medical record diagnoses were classified as one qualifying ICD code in any diagnostic position between October 1, 1999, and September 18, 2020. Additional analyses were restricted to medical record diagnoses occurring before survey self-report and using specific diagnostic criteria (two outpatient or one inpatient ICD code). Positive, negative, and overall (Youden's J) agreement was calculated for categorical outcomes; Bland-Altman plots were examined for continuous measures. Results: Among 116,288 participants, 71.8% self-reported a diagnosed medical condition. Negative agreement between self-reported and VHA medical record diagnoses was > 90% for most (80%) conditions, but positive agreement was lower (6.4% to 56.3%). Mental health conditions were more frequently recorded in medical records, while acute conditions (e.g., bladder infections) were self-reported at a higher frequency. Positive agreement was lower when analyses were restricted to medical record diagnoses occurring prior to survey self-report. Specific diagnostic criteria resulted in higher overall agreement. Conclusions: While negative agreement between self-reported and medical record diagnoses was high in this population, positive and overall agreement were not strong and varied considerably by health condition. Though the limitations of survey-reported health conditions should be considered, using multiple data sources to examine health outcomes in this population may have utility for research, clinical planning, or public health interventions.