Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure

https://doi.org/10.1016/j.jclinepi.2004.04.005Get rights and content

Abstract

Objectives

Questionnaires are used to estimate disease burden. Agreement between questionnaire responses and a criterion standard is important for optimal disease prevalence estimates. We measured the agreement between self-reported disease and medical record diagnosis of disease.

Study design and setting

A total of 2,037 Olmsted County, Minnesota residents ≥45 years of age were randomly selected. Questionnaires asked if subjects had ever had heart failure, diabetes, hypertension, myocardial infarction (MI), or stroke. Medical records were abstracted.

Results

Self-report of disease showed >90% specificity for all these diseases, but sensitivity was low for heart failure (69%) and diabetes (66%). Agreement between self-report and medical record was substantial (kappa 0.71–0.80) for diabetes, hypertension, MI, and stroke but not for heart failure (kappa 0.46). Factors associated with high total agreement by multivariate analysis were age <65 years, female sex, education >12 years, and zero Charlson Index score (P < .05).

Conclusion

Questionnaire data are of greatest value in life-threatening, acute-onset diseases (e.g., MI and stroke) and chronic disorders requiring ongoing management (e.g.,diabetes and hypertension). They are more accurate in young women and better-educated subjects.

Introduction

Epidemiologic studies and surveys often rely on self-administered questionnaires to obtain information on subject health status [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. The agreement between questionnaire data and a criterion standard, such as the medical record, is critical for obtaining meaningful estimates of disease prevalence.

A number of studies have attempted to assess the value of a self-reported disease by comparing self-reports with a criterion standard, such as the medical record, for a range of cardiovascular conditions [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19]. When comparing results among different studies, it is important to bear in mind that the characteristics of the cohort and study methodology vary widely. Some studies did not evaluate both men and women [3], [7], [8], [9]. Other investigations limited the target population to those who had been hospitalized due to the disease of interest [13], [16]. Most studies imposed a limit on the recall period for a particular diagnosis [6], [7], [8], [9], [13]. Questionnaire instruments have also varied; in many studies subjects have been asked if they have had a condition but have not been asked whether a medical professional had informed them of the diagnosis [1], [2], [3], [4], [5], [15]. Moreover, information obtained by interview [12], [13], [14], [15], [16], [17], [18], [19] may differ from that gained by self-administered questionnaire [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. The form of analysis and presentation of findings also varies among studies: Some authors focused on subjects who reported the presence of disease and those who denied disease [1], [2], [3], [4], [5], [6], [8], [9], [10], [11], [12], [14], [15], [18], [19], whereas other authors limited their investigation to those who reported the presence of disease [7], [13], [16], [17]. Limiting evaluation of a criterion standard (e.g., chart abstraction) only to persons who report the presence of disease may result in underestimation of disease prevalence because false-negative reports are not taken into account.

Criterion standards for the presence of disease have also been inconsistent. Some investigators have used physician's diagnosis in medical records, whereas others have required the presence of a morbidity endpoint at the time of clinical measurement or evaluation [2], [19]. Harlow pointed out that the lack of consistency in analytic methods to measure agreement makes comparison across studies difficult and suggested that investigations should incorporate standardized methodologies to enable comparison of results across studies [20]. However, study limitations, such as referral bias, insufficient sample size, nonuniform archival style, short archival period, or incomplete archive, may limit the ability to achieve interpretable results.

The purpose of this study was to measure the agreement between self-reported cardiovascular disease and extensive medical record documentation of disease in a well-characterized, population-based cohort with a long record archival period. We also sought to determine the subject characteristics that are associated with agreement.

Section snippets

Study setting

The study was conducted in Olmsted County, Minnesota using the resources of the Rochester Epidemiology Project. In 1990, the population of Olmsted County was 106,470 (96% white). The proportions of the population over age 45, 65, and 75 years were 28%, 11%, and 5%, respectively. Other characteristics of this population and the unique aspects of population-based epidemiologic research in Olmsted County have been previously described [21], [22], [23], [24].

Population sampling, subject recruitment, and enrollment

This study was approved by the Mayo

Results

The characteristics of the cohort are described in Table 2. Of 2,037 participants, 1,950 (95.7%) filled out the questionnaire without assistance. The median participant age was 61 years, and the median length of the patient medical record archive was 36 years.

The agreement between self-report and the medical record for heart failure, diabetes, hypertension, MI, and stroke is presented in Table 3. Self-report was most sensitive and specific (89.5% and 98.2%, respectively) for MI. In contrast, a

Discussion

Self-administered health-status questionnaires continue to be important tools in epidemiology and public health research. This study of agreement between questionnaire responses and medical record diagnosis was conducted in a population-based cohort of persons ≥45 years of age. It showed that there was substantial agreement between questionnaire responses and medical records for diabetes, hypertension, MI, and stroke but not for heart failure. Factors associated with higher agreement were age

Acknowledgments

We thank Tammy Burns for expert preparation of this manuscript for publication. This study was funded by grants from the Public Health Service NIH HL-55502 (R.J.R.) and NIH AR-30582 (S.J.J.), by Merck-Banyu fellowship award (Y.O.), and by the Mayo Foundation.

References (34)

  • W.D. Thompson et al.

    A reappraisal of the kappa coefficient

    J Clin Epidemiol

    (1988)
  • N. Haapanen et al.

    Agreement between questionnaire data and medical records of chronic diseases in middle-aged and elderly Finnish men and women

    Am J Epidemiol

    (1997)
  • T. Engstad et al.

    Validity of self-reported stroke: the Tromso Study

    Stroke

    (2000)
  • S. Tretli et al.

    Reliability of questionnaire information on cardiovascular disease and diabetes: cardiovascular disease study in Finnmark county

    J Epidemiol Community Health

    (1982)
  • K. Midthjell et al.

    Is questionnaire information valid in the study of a chronic disease such as diabetes? The Nord-Trondelag diabetes study

    J Epidemiol Community Health

    (1992)
  • P.G. O'Mahony et al.

    Validation of a population screening questionnaire to assess prevalence of stroke

    Stroke

    (1995)
  • A. Paganini-Hill et al.

    Accuracy of recall of hip fracture, heart attack, and cancer: a comparison of postal survey data and medical records

    Am J Epidemiol

    (1993)
  • Cited by (0)

    View full text