Using a single question to assess physical activity in older adults: a reliability and validity study

When selecting measures for a study, investigators usually need to strike a balance among several factors such as required sample size, the level of detail needed, the resources available, and the burden posed by their measurement protocol on research participants. In particular, for a given research budget, there is typically a trade-off between measurement detail and sample size [1].

Each of these issues is present in the study of physical activity (PA) in older adults. Regular PA assists with maintaining independence and preventing disability among older adults and it is associated with a decreased risk of morbidity and all-cause mortality [2]. PA is defined as "any bodily movement produced by skeletal muscles that results in energy expenditure" [3]. PA is a complex behavioral construct that can be categorized and quantified in many ways. For example, PA can be broken down into routine activities such as housework, and those done for exercise, such as swimming. Each specific type of PA can be quantified in terms of frequency, intensity, and duration [4].

These complexities are reflected in the many methods used to measure PA or related energy expenditure in older adults [4‐7]. Measurement of PA can be categorized into direct and indirect methods. Direct methods are defined as those that measure movement as it occurs and indirect methods provide indicators of PA and energy expenditure [5]. Examples of direct methods include motion sensors, such as pedometers, accelerometers, and Global Positioning Systems, whereas indirect methods include daily PA records or log books and self-report questionnaires.

There are a number of recall questionnaires that have been used in older adult populations, with varying degrees of evidence for reliability and validity [8]. While these questionnaires provide a more detailed picture of an individual's PA, global questionnaires or single-item questions may be favored in certain situations. When investigators have a choice of questions, they might compare the evidence of validity and reliability in making their selection. Reliability and validity results from existing single-item PA questions [9‐12] indicate a need to evaluate additional single-item questions as possible measures of PA under certain conditions (i.e., when PA is not the primary focus of a study but a quick approximation of activity levels is of interest as a covariate or possible confounding factor, when the sample size is large, when resources are limited, and when more complex methods would add to respondent burden).

Similar research has been done with general health measures. A previous study found that two single-item general self-rated health measures showed good measurement properties when compared to a multi-item instrument, thus providing a less burdensome alternative [13]. In that study, researchers compared "standard" and "comparative" versions of general self-rated health measures, where the comparative version referred to a question that had respondents compare their general health to a reference group. Findings indicated that both questions represented reasonably similar assessments of health. Another study in this area, which compared three different single-item questions of self-rated health (two "standard" questions and one "comparative" question) found similar results [14].

In a previously completed fall risk factor modification trial, two single-item questions of PA were included, both intended to easily classify activity levels of participants. Similar to the self-rated health literature, one question was a "standard" measure and one was a "comparative" measure. Specifically, one question measured absolute PA (seldom, moderately, vigorously active) and the other measured relative PA (more, about as, less active than peers). Using the self-rated health literature as a model, since both PA questions have the same intent (i.e., to quickly classify PA levels), it is of interest to determine if properties of reliability and validity are similar between these questions and whether they could be used interchangeably. Thus, the aim of this study was to evaluate the test-retest reliability, convergent validity and discriminant validity of an absolute PA question and a relative PA question, in a sample of community-dwelling older adults.

The characteristics of participants who took part in the reliability sub-study are described in detail elsewhere [15]. Briefly, the mean age was 79 (standard deviation (SD) 2.9) years and approximately one-half were male.

For the validity sample, the mean age was 80 (SD 3.9) years, and close to two-thirds were males. Other characteristics of participants included in the validity sample are presented in Table 1. About 20% self-reported fair or poor health, 38% reported one or more falls in the past 12 months, 49% reported that their memory was worse than five years ago, 15% reported being seldom active and 12% reported that they were less active compared to their peers. In comparison to women, men were older (mean age 81 years, SD 3.5), and a slightly higher percentage reported fair or poor health, one or more falls in the past month and worse memory compared to five years earlier. Men and women provided similar responses in regard to their PA compared to their peers. The median time between administration of validation measures and PA questions ranged between 33 days for the subset of participants who had been administered both the earlier version of the interRAI Community Health Assessment (interRAI) and Veterans' Comprehensive Assessment (VCA), and 37 days when considering all participants in the validation sample (also see Figure 1).

Results indicated that the absolute PA question had better test-retest reliability than the relative PA question. The weighted kappa value for absolute PA was 0.75 (95% confidence interval (CI): 0.60 to 0.91) whereas for relative PA, the weighted kappa value was 0.56 (95% CI: 0.30 to 0.82). For the absolute PA question, the weighted kappa value indicated substantial agreement whereas for relative PA, the weighted kappa value indicated agreement in the moderate range.

The validation results for both the absolute and relative PA questions are presented in Table 2. For both PA questions, correlation coefficients were in the expected directions according to the type of validity being assessed. For the relative PA question, there was greater contrast between values obtained for convergent validity and discriminant validity. For absolute PA, correlations with convergent validation measures ranged from 0.10 (95% CI: 0.00 to 0.26) to 0.33 (95% CI: 0.19 to 0.49), indicating relationships ranging from little to a fair degree of association. For relative PA, correlations were consistently higher, with most comparisons indicating fair to moderate or good associations. Specifically, correlations ranged from 0.28 (95% CI: 0.15 to 0.44) to 0.57 (95% CI: 0.38 to 0.78). The total score on the balance assessment, limiting PA due to fear of falling, and all of the gait measures (unsteady gait, gait-path, gait-trunk, and gait abnormality) had stronger positive correlations with relative PA, compared to absolute PA.

For discriminant validity, the magnitude of correlation coefficients was similar between absolute PA and relative PA. For absolute PA, correlations with discriminant validation measures ranged from 0.04 (95% CI: 0.00 to 0.19) to 0.29 (95% CI: 0.15 to 0.50) and for relative PA, correlations ranged from 0.02 (95% CI: 0.00 to 0.12) to 0.24 (95% CI: 0.00 to 0.44). In general, comparisons of discriminant validity measures and PA questions led to correlations that indicated little to no association.

The absolute PA question had better test-retest reliability than the relative PA question. Paradoxically, evidence for convergent validity was stronger for relative PA compared to absolute PA. For both questions, results indicated evidence for discriminant validity. The relative PA question had the best combination of test-retest reliability, convergent validity and discriminant validity. Specifically, there was moderate agreement when this question was re-administered seven days later, fair to moderate or good associations when compared with indicators of physical function, and little to no associations when compared with measures hypothesized to be theoretically not related to PA. Although we were unable to evaluate the five-level form of the relative PA question, a previous study examining the validity of a similar question from the National Health Interview Survey (NHIS) found that very little was gained with the 5-level question compared to the 3-level question [12].

Indicators of physical function, often referred to as indirect measures of PA, have not been commonly used to evaluate the convergent validity of single-item PA questions in older adults, despite recommendations for their use [16, 17]. One study, evaluating two different PA questions in older adults, examined convergent validity against indicators of health (i.e., health conditions such as heart attack, stroke, and diabetes). This study did not report any type of validity coefficients, making comparisons with our findings difficult [9].

Two other studies that evaluated an additional four PA questions in populations of older adults, examined validity by comparing questions with summary measures from PA recall questionnaires. In the first study, a PA question designed to be used as a screening question in primary care was evaluated in a population of older women [11]. This question, "As a rule, do you do at least half an hour of moderate or vigorous exercise (such as walking or sport) on five or more days of the week?", was compared to two summary scores from the New Zealand Physical Activity Questionnaire - Long Form. Results indicated moderate agreement (κ = 0.46 to 0.56). In the second study, three PA questions from the NHIS (job-related activity, main daily activity, and activity compared to peers) were compared with summary measures from a detailed PA question set [12]. The main daily activity question asked, "How much hard physical work is required in your main daily activity? Would you say a great deal, a moderate amount, a little, or none?" The activity compared to peers question, "Would you say that you are physically more active, less active, or about as active compared to other persons you age?", was also expanded to a 5-level question with the following response options: a lot more, a little more, about the same, a little less, a lot less. For participants 65 years of age or older, correlation coefficients ranged from 0.17 to 0.21 for the main daily activity question and from 0.24 to 0.28 for the activity compared to peers question. The validity results from the present study, in particular for the relative PA question, have been similar or better than previous studies of single-item PA questions in older adults.

At least two studies have evaluated test-retest reliability of single-item PA questions in older populations. In the first study, researchers found intraclass correlation coefficients (ICCs) ranging from 0.75 to 0.80 for two PA questions that asked regular exercisers about their frequency and intensity of activity [9]. Another study evaluated the test-retest reliability of three different PA questions (work PA, strenuous PA, and moderate PA) in a sample of participants from the Canadian Mulitcentre Osteoporosis Study [10]. The kappa statistic was 0.57 (0.47 to 0.68) for the strenuous PA question and 0.30 (0.23 to 0.37) for the moderate PA question.

Reliability results achieved in the present study for the relative PA question were similar or better than those reported by Nadalin et al. [10] but worse than those reported by Davis et al. [9]. Comparing the results in the present study to those reported by Davis et al. [9] is also problematic, since the PA questions evaluated in that study were only posed to participants who had already reported engaging in regular exercise.

Indicators of physical function have been used to evaluate the convergent validity of many PA recall questionnaires designed for older adults. For a number of the most well-known questionnaires, evidence for convergent validity is not substantially stronger than that obtained in this study; in fact, in some instances, the relative PA question evaluated in this study, performed better. For example, correlations between summary scores from the Community Healthy Activities Model Program for Seniors (CHAMPS) Physical Activity Questionnaire and various measures of physical functioning ranged between 0.10 and 0.54 [16, 18‐20]. For the CHAMPS Physical Activity Questionnaire and the Yale Physical Activity Survey, test-retest reliability was evaluated over a similar interval to this study (one to two weeks), and ICCs ranged from 0.55 to 0.79 [18, 19, 21].

The intent of both the absolute and the relative PA questions was to quickly and easily classify older adults by their activity level. Since specific details related to frequency, duration and intensity are not referenced within the relative PA question, this question will remain accurate for assessment even when PA recommendations for older adults are revised, such as was done in the United States in 2007 [22] and in Canada in 2011[23]. The relative PA question may also be less prone to recall errors, compared to the absolute PA question, since participants do not need to remember the duration or frequency of their typically performed activities.

It is known that in general, people tend to over-report PA levels [24]. In the self-reported health literature, it was noted that with increasing age, people tended to overestimate their health when comparing themselves to others or alternatively, they underestimated the health of others [14]. Thus, it is plausible that the participants in this study may have overestimated their PA, and perhaps to a greater extent when responding to the relative PA question. This should be kept in mind when interpreting the results of this study and when considering the merits of measuring PA using an absolute or a relative question.

Other limitations exist for the present study. Participants included in this study were Canadian veterans of World War II or the Korean War and their caregivers, a highly selected group of older adults. In addition, some of the validation measures were only available on participants in the study who had reported at least one modifiable fall risk factor. The Project to Prevent Falls in Veterans (PPFV) began as a randomly selected sample; however, only 13% of the original participants were included as part of the risk factor modification trial and a smaller percentage completed the second clinical assessment and the final telephone interview. As a result, it is likely that the participants included in this study are different than the general population of older Canadian veterans and their caregivers. Caution should be taken in generalizing results from our study to populations that may differ clinically and demographically.

The present analyses were done because we had data that allowed us to do these comparisons, but were not part of a validation study planned a priori. It is therefore possible that the modest validity correlations achieved may be partially due to the measures selected for validation. Since there is no widely accepted criterion of PA [24], we chose to evaluate the convergent validity of two single-item PA questions, by comparing them with indicators of physical functioning. We recognize, however, that capacity to perform PA does not equal actual performance. As a result, correlation coefficients indicating more than a moderate association may not be possible when using indicators of physical functioning as validation measures. A related limitation is that while some of the indicators of physical function were objectively measured performance-based outcomes, others were measures of self-reported functional ability. Self-reported measures can be affected by factors such as cognitive impairment and guessing among older populations [25]. Additionally, it would have been preferable if the indicators of physical functioning were measured at the same time as the PA questions. Even so, we hypothesize that any resulting bias is likely toward the null, indicating that correlations may have been stronger if these measures had been conducted closer in time.

In large sample research, there is a trade-off between the intensity of measurement of a single variable and the comprehensiveness of all variables. In this study, a relative PA question had the best combination of test-retest reliability, convergent validity and discriminant validity. The magnitude of the reliability and validity coefficients achieved for this question are similar, and in some cases better, than those previously reported for other single-item PA questions evaluated in older adults. Reliability and validity results of many recall questionnaires for older adults have also not substantially exceeded the results obtained in this study. This simple PA question may be useful in studies of older adults where PA is not the primary focus, but a brief classification of activity levels is needed.

In this study, we have taken an initial step in evaluating convergent validity of a relative PA question using indicators of physical functioning as validation measures. Future research should evaluate convergent validity using other validation measures such as accelerometers and more detailed recall questionnaires. This question or other single-item questions cannot replace recall questionnaires or other direct measures of PA when resources are available or when study objectives require more comprehensive measures. In summary, this simple PA question may provide an alternative to researchers when lengthy PA measures, which increase both cost and participant burden, are not possible or necessary.