A comparison of two time intervals for test-retest reliability of health status instruments

Abstract

Studies of test-retest reliability for health-related quality of life instruments have used varying intervals between test administrations. There is no evidence available to aid in the selection of the time interval between questionnaire administrations for a study of test-retest reliability for health status instruments. We compared the test-retest reliability at 2 days and 2 weeks for four knee-rating scales and the eight domains of the SF-36. Seventy patients with disorders of the knee who were in a stable state were randomly allocated to repeat the questionnaires at either 2 days or 2 weeks. There were no statistically significant differences in the test-retest reliability (intraclass correlation coefficient and limits of agreement statistics) for the two time intervals.

Introduction

Reliability is a critical measurement property for health-related quality of life (QOL) instruments. Reliability refers to the consistency of scores obtained by the same persons when re-examined with the same test on different occasions or with different sets of equivalent sets of items [1]. There are many techniques available to measure reliability, including internal consistency and test-retest reliability.

An instrument that has adequate test-retest reliability gives the same result if an individual is re-tested while remaining in a clinical steady state. The problem with testing reliability by the test-retest method is that there is a potential for learning, carry-over, or recall effects (i.e., the first testing may influence the second) [2]. The length of time between the two test administrations also affects the test-retest reliability. A very short time interval makes the carryover effects due to memory, practice, or mood more likely, whereas a longer interval increases the chances that a change in status could occur [2]. Measuring reliability by the internal consistency method involves dividing the instrument into two equal parts and comparing the score on both halves (i.e., split-half reliability). The Kuder Richardson formula 20 is the average of all of the split-half reliabilities, and Cronbach's α is an extension of this formula for ordinal data [3].

Test-retest reliability is more relevant in the setting of clinical medicine because the constructs we attempt to measure are heterogeneous. For example, many instruments used by physicians combine apparently diverse domains (i.e., heterogeneous) such as symptoms (e.g., pain, numbness) and disability (e.g., limited mobility, difficulty with activities of daily living) into a single score. A homogenous instrument consists of items that relate to a single domain. Thus, one may expect clinically heterogenous scales to have poor internal consistency. However, despite the heterogeneity of variables such as health-related QOL, there is evidence that the latter fulfill the criteria for internal consistency despite their apparent heterogeneity [4].

Studies of test-retest reliability for health-related QOL instruments have used varying intervals between test administrations. The interval has ranged from 10 minutes to 1 month [5], [6], [7], [8], [9], [10], [11], [12], [13]. Most investigators have chosen an interval ranging from 2 days to 2 weeks. This time frame is generally believed to be a reasonable compromise between recollection bias and unwanted (on the part of the investigator) clinical change.

There is no evidence available to aid in the selection of the time interval between questionnaire administration for a study of test-retest reliability for health status instruments. The goal of this study was to prospectively compare the 2-day and 2-week test-retest reliability of four knee-rating scales and the eight subscales of the SF-36 in a cohort of athletic patients with disorders of the knee.