Review and Special Articles
The accuracy of self-reported health behaviors and risk factors relating to cancer and cardiovascular disease in the general population1: A critical review

https://doi.org/10.1016/S0749-3797(99)00069-0Get rights and content

Abstract

Objective: To critically review the literature concerning the accuracy of self-reported health behaviors and risk factors relating to cancer and cardiovascular disease among the general population.

Method: A literature search was conducted on three major health research databases: MEDLINE, HealthPLAN, and PsychLit. The bibliographies of located articles were also checked for additional relevant references. Studies meeting the following five inclusion criteria were included in the review: • They were investigating the accuracy of self-report among the general population, as opposed to among clinical populations. • They employed an adequate and appropriate gold standard. • At least 70% of respondents consented to validation, where validation imposed minimal demands on the respondent; and 60% consent to validation was considered acceptable where validation imposed a greater burden. • They had a sample size capable of estimating sensitivity and specificity rates with 95% confidence intervals of width ± 10%. • The time lag between collection of the self-report and validation data for physical measures did not exceed one month.

Results: Twenty-four of 66 identified studies met all the inclusion criteria described above. In the vast majority, self-report data consistently underestimated the proportion of individuals considered “at-risk.” Similarly, community prevalences of risk factors were considerably higher according to gold standard data sources than they were according to self-report data.

Conclusions: This review casts serious doubts on the wisdom of relying exclusively on self-reported health information. It suggests that caution should be exercised both when trying to identify at-risk individuals and when estimating the prevalence of risk factors among the general population. The review also suggests a number of ways in which the accuracy of individuals’ self-reported health information can be maximized.

Introduction

I n the field of health promotion, we need to assess both individuals’ and populations’ health risk status for a variety of risk factors and behaviors. Reliable, valid, and appropriate measures are prerequisites to any descriptive research or intervention assessment.1 Without appropriate measures, it is impossible to accurately assess the prevalence of a target behavior, to identify and describe the characteristics of those individuals at greater risk, or to assess the efficacy of interventions aimed at reducing risk factors.

Self-report is one of the easiest, cheapest, and most widely used methods of collecting data about individuals’ health and risk factor status.2, 3, 4, 5, 6 Many health studies use self-reported data to assess the prevalence of given risk factors (e.g., smoking or lack of exercise) or health behaviors (e.g., having Papanicolaou (Pap) tests or wearing sunscreen) in the community or to evaluate the success or failure of health promotion interventions. For example, a MEDLINE7 search for 1997 located 31 studies that reported rates of Pap testing: 22 (71%) described the prevalence or predictors of screening8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29; 10 (32%) assessed the effectiveness of interventions to increase screening rates26, 29, 30, 31, 32, 33, 34, 35, 36, 37; and one (3%) assessed the accuracy of self-reported screening rates.38 Excluding the one validation study, 19 studies (63%) used self-report as the screening outcome measure, highlighting that almost two thirds of current research into the prevalence and predictors of Pap testing and the effectiveness of interventions to increase Pap testing rates rely on self-report data.8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 25, 28, 29, 30

Because Pap testing is a health behavior with clear recommendations, where alternative, objective measures exist in the form of health insurance records, and where questions about the accuracy of self-report have been raised for a number of years, the authors consider it likely that at least a similar degree of reliance on self-report data is likely to be found in the literature investigating many other health risk factors and behaviors. With such a reliance on self-reported health data, it is imperative that these data are assessed to be valid and reliable measures.

For such prevalence estimates and outcome measures to be useful, the self-report items must provide an accurate measurement of that which they are supposed to be measuring. Inaccurate self-report could lead to underestimation or overestimation of the prevalence of risk factors or health behaviors in the community or to the misclassification of risk status at the individual level, which could obscure causal relationships between risk factors and subsequent disease.

Of the Pap testing studies mentioned above, the validation study found that self-reported screening prevalence did not accurately reflect the prevalence obtained from pathology records.38 These findings are consistent with those of earlier studies investigating the accuracy of self-reported Pap testing history.39, 40 Serious questions were raised about the accuracy of self-reported health data nearly 30 years ago.41, 42, 43 Despite such doubts about self-report accuracy, the 19 prevalence and intervention studies published in 1997 that employed self-reported Pap testing as an outcome measure, treated self-report accuracy as follows: nine (47%) did not raise the issue of self-report accuracy at all10, 18, 19, 20, 21, 22, 25, 29, 30; one (5%) raised the issue, attempted no validation but assumed it was sufficiently accurate16; eight (42%) raised the issue and accepted it as a limitation but did not discuss to what extent it may have affected the results8, 9, 11, 13, 14, 15, 17, 28; and one (5%) validated the self-report of respondents against their health billing records, finding that many women had overestimated their Pap screening frequency.12

The potential consequences of this avoidance of the issue of self-report accuracy are significant. Previous validation studies have indicated that approximately 45%–65% of women inadequately screened for cervical cancer are not identified by self-report.3, 39, 40 These studies have also found that only 70%–78% of self-report claims of being adequately screened could be verified. Assuming similar inaccuracy rates in the Pap testing studies described above, it would seem likely that any conclusions drawn about the prevalence of Pap testing, the predictors of being adequately screened, or the effectiveness of interventions to increase screening rates should be treated with a high degree of skepticism.

The failure of these studies to adequately address the issue of self-report accuracy could be due partly to the fact that many studies have investigated the accuracy of self-reported health information and drawn varying conclusions, depending on the behavior being investigated, the quality of the methodology employed to investigate them, and the “gold standard” against which the self-report was compared. Whatever the explanation, it is of concern that an extensive body of literature regarding the accuracy of self-reported health data is seldom addressed in the subsequent descriptive and intervention-based research. Therefore, the authors considered that conducting a critical review that attempted to establish some consensus from the literature in this area would serve to emphasize the importance of considering accuracy issues when conducting health research using self-report data. As secondary aims, this review also explores the characteristics found to be predictive of increased accuracy of self-report and of consenting to have one’s self-report validated.

Section snippets

Search strategy

This paper represents an extensive review of data-based studies, published between 1983 and 1997, that investigated the accuracy of self-report of health-risk, preventive, screening, or early detection behaviors for cancer and cardiovascular disease in the general population. These diseases were selected because: they are responsible for approximately three quarters of all deaths44, 45; they represent major target areas for current health promotion initiatives45, 46; and individuals’ health

The accuracy of individuals’ risk factor status derived from self-report

The next stage was to review the accuracy of individuals’ self-report found in these studies. Self-report data can be used in two ways: to establish an individual’s risk factor status or to establish the prevalence of a given risk factor in the community. One of the problems with the literature in this area is that a number of different statistical methods have been employed when reporting on the accuracy of self-report. Some of these methods can lead to erroneous conclusions about the accuracy

Discussion

With the exception of current smoking and diabetic status, this review found significant, and often substantial, differences between community prevalences of health behaviors and risk factors estimated from self-report data and from corresponding gold standard data. When assessing individuals’ risk-factor status, the proportion of individuals at risk who would not have been identified by their self-report data was of concern for all the behaviors explored and quite substantial for many

Conclusions

This review casts serious doubt on the wisdom of relying exclusively on self-reported health information and concludes that caution should be exercised both when trying to identify at-risk individuals and when estimating the prevalence of risk factors in the general population. In summary, this review has suggested a number of ways in which the accuracy of individuals’ self-reported health information can be maximized, but strongly recommends the use of validation techniques wherever possible,

Acknowledgements

The preparation of this paper was supported jointly by the National Health and Medical Research Council and the New South Wales Cancer Council Cancer Education Research Program, directed by R. Sanson-Fisher until November 1997. The views expressed are not necessarily those of the Cancer Council.

References (134)

  • H. Mitchell et al.

    Effect of ethnic media on cervical cancer screening rates

    Aust J Public Health

    (1997)
  • J.A. Bowman et al.

    The accuracy of self-reported pap smear utilization

    Soc Sci Med

    (1997)
  • A. Furnham

    Response bias, social desirability and dissimulation

    Person Indiv Diff

    (1986)
  • D.T. Lackland et al.

    The need for accurate nutrition survey methodologythe South Carolina experience

    J Nutr

    (1990)
  • P. Tienboon et al.

    Self-reported weight and height in adolescents and their parents

    J Adolesc Health

    (1992)
  • S.J. Bowlin et al.

    Reliability and changes in validity of self-reported cardiovascular disease risk factors using dual responsethe behavioral risk factor survey

    J Clin Epidemiol

    (1996)
  • J.P. Pierce et al.

    Cotinine validation of self-reported smoking in commercially run community surveys

    J Chron Dis

    (1987)
  • S.P. Fortmann et al.

    Indirect measures of cigarette useexpired air carbon monoxide versus plasma thiocyanate

    Prev Med

    (1984)
  • M.L. Slattery et al.

    Validity of cigarette smoking habits in three epidemiologic studies in Utah

    Prev Med

    (1989)
  • W.H. Giles et al.

    The validity of self-reported hypertension and correlates of hypertension awareness among blacks and whites within the stroke belt

    Am J Prev Med

    (1995)
  • C.M. Vargas et al.

    Validity of self-reported hypertension in the national health and nutrition examination survey III, 1988–1991

    Prev Med

    (1997)
  • S.M. Horwitz et al.

    The influence of psychological and social factors on accuracy of self-reported blood pressure

    J Clin Epidemiol

    (1997)
  • E.S. Ford et al.

    Test characteristics of self-reported hypertension among the Hispanic populationfindings from the Hispanic health and nutrition survey

    J Clin Epidemiol

    (1990)
  • S.D. Walter et al.

    A comparison of physician and patient reports of pap smear histories

    J Clin Epidemiol

    (1988)
  • R.A. Hiatt et al.

    Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California

    Prev Med

    (1995)
  • J.A. Bowman et al.

    The accuracy of self-reported pap smear utilization

    Soc Sci Med

    (1997)
  • L. Suarez et al.

    Validity of pap smear and mammogram self-reports in a low-income Hispanic population

    Am J Prev Med

    (1995)
  • P. Hawe et al.

    Evaluating health promotiona health worker’s guide

    (1990)
  • R.A. Washburn et al.

    The validity of self-reported exercise-induced sweating as a measure of physical activity

    Am J Epidemiol

    (1990)
  • N.P. Gordon et al.

    Concordance of self-reported data and medical record audit for six cancer screening procedures

    J Natl Cancer Inst

    (1993)
  • V.J. Strecher et al.

    Using patients’ descriptions of alcohol consumption, diet, medication compliance and cigarette smokingthe validity of self- reports in research and practice

    J Gen Intern Med

    (1989)
  • T. Baranowski

    Methodologic issues in self-report of health behavior

    J Sch Health

    (1985)
  • United States’ National Library of Medicine. MEDLINE on Silver Platter database. 1994. Bethseda, Maryland: United...
  • N.P. Peragallo et al.

    Cervical cancer screening practices among Latino women in Chicago

    Public Health Nursing

    (1997)
  • K.F. Strauss et al.

    The health of Navajo womenfindings from the Navajo health and nutrition survey

    J Nutr

    (1997)
  • G.C.E. Stuart et al.

    Review of the screening history of Alberta women with invasive cervical cancer

    Can Med Assoc J

    (1997)
  • S.D. Cassard et al.

    Physician gender and women’s preventive services

    Journal of Women’s Health

    (1997)
  • A.S. O’Malley et al.

    Continuity of care and the use of breast and cervical cancer screening services in a multiethnic community

    Arch Intern Med

    (1997)
  • P.M. Lantz et al.

    Education and income differentials in breast and cervical cancer screening

    Med Care

    (1997)
  • R. Branoff et al.

    A family practice residency cervical screening projectperceived screening barriers

    Fam Med

    (1997)
  • J.F. Sung et al.

    Knowledge, beliefs, attitudes, and cancer screening among inner-city African-American women

    Journal of the National Medical Association

    (1997)
  • E.C. Lazcano-Ponce et al.

    The cervical cancer screening program in Mexicoproblems with access and coverage

    Cancer Causes and Control

    (1997)
  • P. Fox et al.

    An examination of differential follow-up rates in cervical cancer screening

    J Community Health

    (1997)
  • L.J. Burak et al.

    Using the health belief model to examine and predict college women’s cervical cancer screening beliefs and behavior

    Health Care Women Int

    (1997)
  • M.M. Miller et al.

    Enhancing adherence following abnormal pap smears among low income minority womena preventive telephone counseling strategy

    J Natl Cancer Inst

    (1997)
  • N. Lurie et al.

    Why do patients of female physicians have higher rates of breast and cervical cancer screening?

    J Gen Intern Med

    (1997)
  • V. Sanchez et al.

    Migration within Spain, level of education, and cervical cancer screening

    Eur J Cancer Prev

    (1997)
  • Reding DJ, Lappe K, Krueger M, Kolehouse BL, Steneil DLRA. Cancer screening and prevention in rural Wisconsin: the...
  • P. Pengsaa et al.

    A self-administered device for cervical cancer screening in northeast Thailand

    Acta Cytol

    (1997)
  • A.C. Kant et al.

    General practitioner based screening for cervical cancerhigher participation of women with a higher risk?

    J Med Screen

    (1997)
  • Cited by (0)

    1

    The full text of this article is available via AJPM Online at http://www.elsevier.com/locate/ajpmonline.

    View full text