Review and Special ArticlesThe accuracy of self-reported health behaviors and risk factors relating to cancer and cardiovascular disease in the general population1: A critical review
Introduction
I n the field of health promotion, we need to assess both individuals’ and populations’ health risk status for a variety of risk factors and behaviors. Reliable, valid, and appropriate measures are prerequisites to any descriptive research or intervention assessment.1 Without appropriate measures, it is impossible to accurately assess the prevalence of a target behavior, to identify and describe the characteristics of those individuals at greater risk, or to assess the efficacy of interventions aimed at reducing risk factors.
Self-report is one of the easiest, cheapest, and most widely used methods of collecting data about individuals’ health and risk factor status.2, 3, 4, 5, 6 Many health studies use self-reported data to assess the prevalence of given risk factors (e.g., smoking or lack of exercise) or health behaviors (e.g., having Papanicolaou (Pap) tests or wearing sunscreen) in the community or to evaluate the success or failure of health promotion interventions. For example, a MEDLINE7 search for 1997 located 31 studies that reported rates of Pap testing: 22 (71%) described the prevalence or predictors of screening8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29; 10 (32%) assessed the effectiveness of interventions to increase screening rates26, 29, 30, 31, 32, 33, 34, 35, 36, 37; and one (3%) assessed the accuracy of self-reported screening rates.38 Excluding the one validation study, 19 studies (63%) used self-report as the screening outcome measure, highlighting that almost two thirds of current research into the prevalence and predictors of Pap testing and the effectiveness of interventions to increase Pap testing rates rely on self-report data.8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 25, 28, 29, 30
Because Pap testing is a health behavior with clear recommendations, where alternative, objective measures exist in the form of health insurance records, and where questions about the accuracy of self-report have been raised for a number of years, the authors consider it likely that at least a similar degree of reliance on self-report data is likely to be found in the literature investigating many other health risk factors and behaviors. With such a reliance on self-reported health data, it is imperative that these data are assessed to be valid and reliable measures.
For such prevalence estimates and outcome measures to be useful, the self-report items must provide an accurate measurement of that which they are supposed to be measuring. Inaccurate self-report could lead to underestimation or overestimation of the prevalence of risk factors or health behaviors in the community or to the misclassification of risk status at the individual level, which could obscure causal relationships between risk factors and subsequent disease.
Of the Pap testing studies mentioned above, the validation study found that self-reported screening prevalence did not accurately reflect the prevalence obtained from pathology records.38 These findings are consistent with those of earlier studies investigating the accuracy of self-reported Pap testing history.39, 40 Serious questions were raised about the accuracy of self-reported health data nearly 30 years ago.41, 42, 43 Despite such doubts about self-report accuracy, the 19 prevalence and intervention studies published in 1997 that employed self-reported Pap testing as an outcome measure, treated self-report accuracy as follows: nine (47%) did not raise the issue of self-report accuracy at all10, 18, 19, 20, 21, 22, 25, 29, 30; one (5%) raised the issue, attempted no validation but assumed it was sufficiently accurate16; eight (42%) raised the issue and accepted it as a limitation but did not discuss to what extent it may have affected the results8, 9, 11, 13, 14, 15, 17, 28; and one (5%) validated the self-report of respondents against their health billing records, finding that many women had overestimated their Pap screening frequency.12
The potential consequences of this avoidance of the issue of self-report accuracy are significant. Previous validation studies have indicated that approximately 45%–65% of women inadequately screened for cervical cancer are not identified by self-report.3, 39, 40 These studies have also found that only 70%–78% of self-report claims of being adequately screened could be verified. Assuming similar inaccuracy rates in the Pap testing studies described above, it would seem likely that any conclusions drawn about the prevalence of Pap testing, the predictors of being adequately screened, or the effectiveness of interventions to increase screening rates should be treated with a high degree of skepticism.
The failure of these studies to adequately address the issue of self-report accuracy could be due partly to the fact that many studies have investigated the accuracy of self-reported health information and drawn varying conclusions, depending on the behavior being investigated, the quality of the methodology employed to investigate them, and the “gold standard” against which the self-report was compared. Whatever the explanation, it is of concern that an extensive body of literature regarding the accuracy of self-reported health data is seldom addressed in the subsequent descriptive and intervention-based research. Therefore, the authors considered that conducting a critical review that attempted to establish some consensus from the literature in this area would serve to emphasize the importance of considering accuracy issues when conducting health research using self-report data. As secondary aims, this review also explores the characteristics found to be predictive of increased accuracy of self-report and of consenting to have one’s self-report validated.
Section snippets
Search strategy
This paper represents an extensive review of data-based studies, published between 1983 and 1997, that investigated the accuracy of self-report of health-risk, preventive, screening, or early detection behaviors for cancer and cardiovascular disease in the general population. These diseases were selected because: they are responsible for approximately three quarters of all deaths44, 45; they represent major target areas for current health promotion initiatives45, 46; and individuals’ health
The accuracy of individuals’ risk factor status derived from self-report
The next stage was to review the accuracy of individuals’ self-report found in these studies. Self-report data can be used in two ways: to establish an individual’s risk factor status or to establish the prevalence of a given risk factor in the community. One of the problems with the literature in this area is that a number of different statistical methods have been employed when reporting on the accuracy of self-report. Some of these methods can lead to erroneous conclusions about the accuracy
Discussion
With the exception of current smoking and diabetic status, this review found significant, and often substantial, differences between community prevalences of health behaviors and risk factors estimated from self-report data and from corresponding gold standard data. When assessing individuals’ risk-factor status, the proportion of individuals at risk who would not have been identified by their self-report data was of concern for all the behaviors explored and quite substantial for many
Conclusions
This review casts serious doubt on the wisdom of relying exclusively on self-reported health information and concludes that caution should be exercised both when trying to identify at-risk individuals and when estimating the prevalence of risk factors in the general population. In summary, this review has suggested a number of ways in which the accuracy of individuals’ self-reported health information can be maximized, but strongly recommends the use of validation techniques wherever possible,
Acknowledgements
The preparation of this paper was supported jointly by the National Health and Medical Research Council and the New South Wales Cancer Council Cancer Education Research Program, directed by R. Sanson-Fisher until November 1997. The views expressed are not necessarily those of the Cancer Council.
References (134)
- et al.
Validity of cardiovascular disease risk factors assessed by telephone surveythe behavioral risk factor study
J Clin Epidemiol
(1993) - et al.
Pap smears in general; a secondary analysis of the Australian morbidity and treatment survey 1990 to 1991
Aust J Public Health
(1997) - et al.
Are physicians less likely to recommend preventative services to low-SES patients
Prev Med
(1997) - et al.
Integrating pros and cons for mammography and pap testingextending the construct of decisional balance to two behaviors
Prev Med
(1997) - et al.
Barriers to breast and cervical cancer screening among Vietnamese-American women
Am J Prev Med
(1997) Prevalence and correlates of breast and cervical cancer among older women
Obstet Gynecol
(1997)- et al.
Knowledge, behavior and fears concerning breast and cervical cancer among older low income Mexican-American women
Am J Prev Med
(1997) - et al.
Follow-up of abnormal Papanicolaou smears in a hospital based adolescent clinic
J of Pediatr Adolesc Gynecol
(1997) - et al.
Effect of a cancer screening intervention conducted by lay health workers among inner city women
Am J Prev Med
(1997) - et al.
Effects and sustainability of a cervical screening program in remote aboriginal Australia
Aust J Public Health
(1997)
Effect of ethnic media on cervical cancer screening rates
Aust J Public Health
The accuracy of self-reported pap smear utilization
Soc Sci Med
Response bias, social desirability and dissimulation
Person Indiv Diff
The need for accurate nutrition survey methodologythe South Carolina experience
J Nutr
Self-reported weight and height in adolescents and their parents
J Adolesc Health
Reliability and changes in validity of self-reported cardiovascular disease risk factors using dual responsethe behavioral risk factor survey
J Clin Epidemiol
Cotinine validation of self-reported smoking in commercially run community surveys
J Chron Dis
Indirect measures of cigarette useexpired air carbon monoxide versus plasma thiocyanate
Prev Med
Validity of cigarette smoking habits in three epidemiologic studies in Utah
Prev Med
The validity of self-reported hypertension and correlates of hypertension awareness among blacks and whites within the stroke belt
Am J Prev Med
Validity of self-reported hypertension in the national health and nutrition examination survey III, 1988–1991
Prev Med
The influence of psychological and social factors on accuracy of self-reported blood pressure
J Clin Epidemiol
Test characteristics of self-reported hypertension among the Hispanic populationfindings from the Hispanic health and nutrition survey
J Clin Epidemiol
A comparison of physician and patient reports of pap smear histories
J Clin Epidemiol
Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California
Prev Med
The accuracy of self-reported pap smear utilization
Soc Sci Med
Validity of pap smear and mammogram self-reports in a low-income Hispanic population
Am J Prev Med
Evaluating health promotiona health worker’s guide
The validity of self-reported exercise-induced sweating as a measure of physical activity
Am J Epidemiol
Concordance of self-reported data and medical record audit for six cancer screening procedures
J Natl Cancer Inst
Using patients’ descriptions of alcohol consumption, diet, medication compliance and cigarette smokingthe validity of self- reports in research and practice
J Gen Intern Med
Methodologic issues in self-report of health behavior
J Sch Health
Cervical cancer screening practices among Latino women in Chicago
Public Health Nursing
The health of Navajo womenfindings from the Navajo health and nutrition survey
J Nutr
Review of the screening history of Alberta women with invasive cervical cancer
Can Med Assoc J
Physician gender and women’s preventive services
Journal of Women’s Health
Continuity of care and the use of breast and cervical cancer screening services in a multiethnic community
Arch Intern Med
Education and income differentials in breast and cervical cancer screening
Med Care
A family practice residency cervical screening projectperceived screening barriers
Fam Med
Knowledge, beliefs, attitudes, and cancer screening among inner-city African-American women
Journal of the National Medical Association
The cervical cancer screening program in Mexicoproblems with access and coverage
Cancer Causes and Control
An examination of differential follow-up rates in cervical cancer screening
J Community Health
Using the health belief model to examine and predict college women’s cervical cancer screening beliefs and behavior
Health Care Women Int
Enhancing adherence following abnormal pap smears among low income minority womena preventive telephone counseling strategy
J Natl Cancer Inst
Why do patients of female physicians have higher rates of breast and cervical cancer screening?
J Gen Intern Med
Migration within Spain, level of education, and cervical cancer screening
Eur J Cancer Prev
A self-administered device for cervical cancer screening in northeast Thailand
Acta Cytol
General practitioner based screening for cervical cancerhigher participation of women with a higher risk?
J Med Screen
Cited by (0)
- 1
The full text of this article is available via AJPM Online at http://www.elsevier.com/locate/ajpmonline.