Test–retest reliability and minimal detectable change of repeated sit-to-stand analysis using one body fixed sensor in geriatric patients

The BFS system was inserted in an elastic belt and positioned on the lower back at the height of the second lumbar vertebra, i.e. an approximation of the body's center of movement in the standing position (figure 1). The small and light measurement system (87 × 45 × 1 mm, 74 g) contains three pre-calibrated seismic accelerometers (STM: sensor rage ± 2 g, resolution 1 mg) and three gyroscopes (EPSON: range ± 100 deg s⁻¹, resolution 0.0069 deg s⁻¹) and has a sampling rate of 100 Hz. The accelerometer signals have been shown to be highly reproducible (ICC = 0.99) (Van Hees et al 2009). Raw data were stored on a Micro-SD card. During raw data collection, the device was connected with a computer via Bluetooth. The supporting acquisition software, which includes the repeated SiSt protocol, was used to start and stop the sensor system and send event markers to mark intervals which were stored with the raw signals.

Zoom In Zoom Out Reset image size

Repeated SiSt assessment was conducted according to a standardized written protocol. We used a modified test procedure as described by Dubost et al (2005). The patients were seated on an armless adjustable chair, with the seat placed at knee height as measured from the medial tibiae plateau. They kept their arms folded across their chest and their feet were placed flat, 10 cm apart at the heels, with the shanks positioned at a 10° angle relative to the vertical.

The patients performed a five-chair stand test which is an established functional assessment in older adults (Guralnik et al 1994). Participants were asked to stand up five times from the initial sitting position. They were instructed to come to a full upright standing position and back to the initial sitting position in between each SiSt cycle. A SiSt cycle is comprised of sitting, standing up, standing, sitting down and sitting (figure 1). BFS data were collected from initial sitting to the final standing position at the end of the fifth stand. Data of the four complete SiSt cycles were included in the analysis. SiSt movements were performed at self-selected speed and no physical assistance was allowed during testing. During an earlier practice period, participants were asked to execute two SiSt cycles. If subjects were not able to get up from regular seat height due to motor limitations, the chair was elevated to 110% (120%) of knee height.

This was a single-session test–retest study where the retest was performed approximately 5 min after the first test. The pre- and retest was conducted by one (intraobserver reliability) respectively two (interobserver reliability) therapist/s with over 2 years of experience in geriatric rehabilitation. Both therapists received 2 h of training in handling the BFS and performing the assessment. Testing was performed in a separate and quiet room in the geriatric hospital. After pretesting, patients were randomly assigned to the intra- or interobserver group using the urn design (Wei 1977). To simulate application of the BFS during routine assessment, the elastic belt containing the BFS was detached and reattached between the pre- and retest. The second rater was not allowed to enter the room during the first test to prevent observation of the assessment and the positioning of the equipment.

Trunk angle (ϕ) was calculated using the acceleration and the angular velocity in the sagittal plane (Williamson and Andrews 2001). Subsequently, the sine of the trunk angle (sin(ϕ)) was calculated and drift and noise were removed using the discrete wavelet transform dw_sin(ϕ) (Najafi et al 2002). An estimation of 'true vertical acceleration' was derived by removing the influence of ϕ from the vertical acceleration and vertical velocity was derived by integrating this signal. Successful STS movements were localized by identifying an upward movement followed by a downward movement. These were identified by finding the local maxima local minima, respectively, in the vertical velocity. Changes in the trunk rotation direction were identified by the dips in dw_sin(ϕ) (see figure 2). The beginning of the SiSt was defined as the end of the plateau before the first dip in dw_sin(ϕ). Similarly, the end of the SiSt was defined as the start of the plateau after the first dip in dw_sin(ϕ), the beginning of the StSi as the end of the plateau before the second dip in dw_sin(ϕ) and the end of the StSi as the start of the plateau after the second dip in dw_sin(ϕ). Plateaus were identified where the slope of dw_sin(ϕ) was <0.1. SiSt and StSi movements were divided into a flexion and extension phase. After automated identification of all phases, durations and maximum angular velocity were calculated. Outcomes were the duration of SiSt and StSi, duration and maximum angular velocity of the flexion and the extension phase of SiSt and StSi. Data analysis was executed automatically using on-line software (http://www.mcroberts.nl/analysis).

Zoom In Zoom Out Reset image size

Descriptive data including age, gender, cognitive status (MMSE), screening for depression (15-point geriatric depression scale (GDS) ≥5 points indicate depressive symptoms) (Yesavage et al 1982), fear of falling (single-item question) (Maki et al 1991), pain (visual analog scale, VAS) (Huskisson 1974), diagnosis (number), functional status (short-physical-performance battery, SPPB) (Guralnik et al 1994) and anthropometric data (height, weight) were documented by standardized assessment.

Statistical procedures were performed on SPSS 17.0 for Windows and Matlab 7.1. The means and standard deviations (SD) of the repetitions were calculated for the durations and maximum angular velocities of all phases. The normal distribution of the data was tested with a Kolmogorov–Smirnov test. The presence of heteroscedasticity was checked visually with Bland–Altman plots (Bland and Altman 1999) and formally tested by correlating (Pearson's r) the magnitude of the mean with the absolute difference from test to retest (Aktinson and Nevill 1998).

2.6.1. Relative reliability

2.6.2. Absolute reliability

Absolute reliability describes the within-subject variability attributable to repeated measures and was evaluated by established statistical methods used in sports medicine and physical therapy (Atkinson and Nevill 1998, Haley and Fragala-Pinkham 2006, Wagner et al 2008). SEM, a reliability statistic which quantifies measurement error in the same units as the original measurement, was calculated as

$$\begin{equation*} {\rm SEM} = {\rm SD}\;\sqrt {(1 - {\rm ICC})} . \end{equation*}$$

Measurement error also was expressed as a percentage of the mean, which was defined as

$$\begin{equation*} {\rm SEM}\% {\rm } = \frac{{{\rm SEM}}}{{{\rm mean}}} \times 100. \end{equation*}$$

The mean is the mean for all of the measures for test and retest. The SEM% indicates measurement error independent of the units of measurement and represents the limit for the smallest change that indicates a real improvement for a group of subjects.

The MDC, representing the magnitude of change necessary to exceed the measurement error of two repeated measures at a specified CI, was calculated for the 95% CI as

$$\begin{equation*} {\rm MDC}_{95} = {\rm SEM} \times 1.96 \times \sqrt 2 . \end{equation*}$$

1.96 is the two-sided tabled z value for the 95% CI and $\sqrt 2$ is used to account for the variance of two measurements.

MDC₉₅ was also expressed as a percentage, which was defined as

$$\begin{equation*} {\rm MDC}_{95} \% {\rm } = \frac{{{\rm MDC}_{95} }}{{{\rm mean}}} \times 100. \end{equation*}$$

The mean is the mean for all of the measures for test and retest.

Agreement between test and retest was plotted graphically by Bland–Altman plots for selected variables. The 95% limits of agreement were calculated as mean difference ±1.96 SD of the differences.

Of 188 patients screened for eligibility, 145 did not meet the inclusion criteria and 43 patients were enrolled. Two patients were unable/refused to perform a retest due to motor limitations/pain or lack of motivation. In one case, automated analysis of raw data by using the on-line software failed (figure 3).

Zoom In Zoom Out Reset image size

Forty patients were included in the analysis, where 19 were randomized to the interobserver group and 21 to the intraobserver group. Admission diagnoses were (1) orthopaedic (n = 11, 27.5%), (2) cardiac and respiratory (n = 10, 25%), (3) neurological (n = 6, 15%), (4) gastrointestinal (n = 4, 10%) and (5) miscellaneous diseases such as metabolic disorders or cancer (n = 9, 22.5%). The study sample comprised multimorbid, geriatric patients. Forty-five percent (n = 18) had cognitive impairment (MMSE ≤24) and forty percent had a possible depressive disorder based on the results of the screening tool (GDS). SPPB averaged 6.5 points indicating low physical performance. No significant differences in clinical characteristics were found between the intra- and interobserver groups (table 1).

Thirty-two patients performed the repeated SiSt at 100% seat height as measured from the medial tibiae plateau. For eight patients (five intraobserver, three interobserver, p = 0.527), the chair was readjusted to 110% (n = 4) respective 120% (n = 4) since patients were unable to perform a chair rise at 100% without using their arms. Results obtained for test and retest are shown in table 2. Durations ranged from 2.05 to 2.32 s for SiSt and from 2.30 to 2.43 s for StSi. For flexion and extension phases, values ranged from 0.89 to 1.33 s for durations and from 36.93 to 92.94° s⁻¹ for angular velocities. Small but significant differences from test to retest were obtained in six out of ten (60%) variables in the total group.

All data were normally distributed. Visual inspection of all Bland–Altman plots revealed a uniform variability for the majority of variables which was confirmed by low correlation coefficients between the magnitude of the mean and the absolute difference from test to retest (r ≤ .01–0.48). However, there were a few examples of nonuniform relationships indicating heteroscedasticity (SiSt-total-duration r = 0.56; SiSt-flexion-duration r = 0.73).

Relative reliability statistics (ICCs) obtained for the total sample and subgroups of intra- and interobserver are shown in table 3. In the total sample, relative reliability was good to excellent (ICC = 0.80–0.94) for all variables. Highest ICCs were found for the angular velocity of the flexion (ICC = 0.94) and extension phase (ICC = 0.91) of StSi. Good to excellent ICCs were also found for 90% of variables in the intraobserver (ICC = 0.85–0.97) and for 80% in the interobserver group (0.80–0.94). Only StSi-extension-duration (ICC = 0.60) in the intraobserver group and SiSt-extension-duration (ICC = 0.74) and StSi-flexion-duration (ICC = 0.73) in the interobserver group were below the threshold of good reliability (≤0.75). More variables with excellent ICCs (≥.9) were found in the intraobserver group (n = 5, 50%) compared to the interobserver group (n = 1, 10%). Highest ICCs were obtained for angular velocities of both SiSt (flexion ICC = 0.95, extension ICC = 0.95) and StSi (flexion ICC = 0.97, extension ICC = 0.95) in the intraobserver group.

Absolute reliability statistics are shown in table 3. In the total sample, SEM ranged 0.08–0.20 s for durations and 4.02–7.30 ° s⁻¹ for angular velocities. SEM% was low for all variables (6.9–12.7%). In 70% of variables, a SEM% ≤10% was obtained. SEM ranged 0.08–0.21 s respectively 3.06–5.36 ° s⁻¹ in the intraobserver group and 0.06–0.20 s respectively 4.93–9.09 ° s⁻¹ in the interobserver group. Subgroup comparison revealed a higher number of variables below a threshold (SEM% ≤10%) in the intraobserver group (80%) compared to the interobserver group (50%).

For the total group, MDC₉₅ ranged 0.22–0.54 s for durations and 11.14–20.23 ° s⁻¹ for angular velocities. MDC₉₅% ranged 19.2–34.4% and was ≤30% for the majority of variables (80%). MDC₉₅% was ≤30% for all angular velocities but exceeded 30% for some of the durations (SiSt-extension: 34.4%; StSi-flexion: 33.6%) for the total group. MDC₉₅ ranged 0.24–0.59 s respectively 8.47–14.85 ° s⁻¹ in the intraobserver group and 0.16–0.55 s respectively 13.66–25.20 ° s⁻¹ in the interobserver group. More MDC₉₅% ≤30% were obtained in the intraobserver group (80%) compared to the interobserver group (60%). In particular for angular velocities, the MDC₉₅% was lower in the intraobserver group (16.4–23.9%) compared to the interobserver group (22.0–38.0%).

The variables of the flexion and extension phase of the SiSt with regard to inter- and intraobserver reliability are illustrated in figure 4 by Bland–Altman plots. For durations, both single values of the intra- and interobserver group were outside the 95% limits of agreement. In contrast, for angular velocities, only interobserver values exceeded the 95% limits of agreement indicating higher within-subject variability in this group.

Zoom In Zoom Out Reset image size

The ICC is the commonly used correlation coefficient for obtaining test–retest reliability (Lexell and Downham, 2005). The ICCs found here for the total duration of the SiSt (0.85) and StSi (0.87) are higher compared to those of a previous study which evaluated the day-to-day and week-to-week reliability of similar variables (average transition duration of SiSt and StSi, ICCs: 0.37–0.78) during 11 h of ambulatory activity monitoring in older adults (de Bruin et al 2007). The higher ICCs found in the present study might be due to the standardized assessment protocol which may produce more stable results compared to measurement under free-living conditions as performed by de Bruin et al. To our knowledge, no study has yet evaluated the reliability of a BFS to assess the repeated SiSt under controlled conditions thus preventing a direct comparison of the present results with other research. One study used a similar BFS (DynaPort® Mini Mod) on elderly subjects to assess temporo-spatial parameters of functional performance (walking: step duration, speed) under controlled conditions and reported comparable ICCs for intra- (0.86–0.98) and interobserver (0.90–0.97) reliability (Hartmann et al 2009). The somewhat higher ICCs found in this study might be explained by the different functional task (walking) but also by the fact that a convenient sample of independent healthy elderly subjects (mean age: 73.4 years) generating stable motor performances was used for evaluation. In contrast, reliability in the present study was obtained from a consecutively recruited sample of multimorbid geriatric patients with motor and partial cognitive impairment. Given the motor and cognitive limitations of our patients, the present results are surprisingly good, demonstrating that movement phases during repeated SiSt can be reliably obtained during geriatric rehabilitation by using a standardized BFS-based assessment. ICCs obtained in the present study partly exceed (ICC = 0.80, Ritchie et al 2005) or are similar (ICC = 0.91, Smeets et al 2006; ICC = 0.94, Thomas and Hageman 2002) to those of earlier studies which evaluated conventional SiSt tests in older adults and patient populations. However, in contrast to previous studies which drew from data obtained by stopwatch, the present results highlight that temporo-spatial parameters of relevant phases of task execution can be reliably measured by a single BFS.

In clinical practice, intrarater as well as interrater reliability is important since different therapists might perform pre- and post-assessment. Interestingly, we found that only ICCs for angular velocities but not for durations were positively influenced by the same observer. Several reasons might account for the higher reliability of angular velocities in the intraobserver group. First, the time between detachment and reapplication was short, approximately 5 min. This might have increased the ability of the observer to reattach the belt in the same way and thus positively affected intraobserver reliability, as also found in a previous study which evaluated a similar device (DynaPort® Mini Mod) to obtain walking performance (Van Hees et al 2009). The position of the BFS might have affected the signal since in mechanics, the acceleration of a mass is related to the distance between its center of mass and the axis of rotation (Van Hees et al 2009). In the present study, a slightly altered position of the BFS during retest might have influenced the measurement of angular velocity during trunk movement but not of duration in terms of identifying the start and end of movement phases. To enhance feasibility during clinical practice, the belt was attached on top of the clothes and the exact position of the attachment relative to the axis of rotation was not verified by exact measurement. Second, when measuring functional performances in the frail elderly, in particular the cognitively impaired, the observer plays an important role in terms of giving clear instructions (Hauer and Oster 2008). Thus, higher ICCs in the intraobserver group might be related to a higher consistency in instructing the subject during test and retest. Since instrumental reproducibility of the BFS under controlled conditions was found to be excellent (ICC 0.99) (Van Hees et al 2009), sources of measurement error most likely correlate with changes in the attachment and/or instructions.

Good to excellent relative reliability does not necessarily ensure that individuals' repeated performance will be consistent from test to retest. To obtain the within-subject variability that would typically occur in the routine administration of the assessment, we calculated the SEM which is one of the most common absolute reliability statistics (Atkinson and Nevill 1998). Since probabilities of the normal curve can be applied to SEM values, values from table 3 can be translated to clinical practice using these principles. For instance, there is a 68% probability that a repeated measurement of the angular velocity of trunk flexion during the repeated SiSt will be within 7.30 ° s⁻¹ of the original value (total group). The SEM is lower if assessment is performed by the same (5.36 ° s⁻¹) compared to a different observer (9.09 ° s⁻¹). We are not aware of studies that have reported the SEM for instrumented SiSt assessment in geriatric patients. To allow a comparison with other established functional measures, we calculated the SEM% that is independent of units. SEM% obtained in the present study (6.9–12.7%, total group) was lower compared to static balance assessment in older adults (40.8%) (Goldberg et al 2011) and similar to the timed 'Up & Go' (8.2%) and comfortable gait speed (7.9%) in stroke patients (Flansbjer et al 2005).

We used the SEM to estimate the MDC₉₅, providing clinically useful information in terms of defining the minimal amount of change that is not likely to be attributable to a chance variation in measurement (Haley and Fragala-Pinkham 2006). To our knowledge, no study has examined the MDC of SiSt parameters, even though these parameters have been reported with increasing frequency in the literature (Boonstra et al 2006, Giansanti and Maccioni 2006, Janssen et al 2005, 2008a, 2008b, Manckoundia et al 2006, Scarborough et al 2007, Zijlstra et al 2010) and changes in SiSt parameters are related to effects of interventions (Alexander et al 2001) and various diseases (Manckoundia et al 2006, Scarborough et al 2007). The MDCs₉₅ reported here suggest that the BFS system can identify effects of interventions aiming at improving the SiSt in elderly populations. For example, Alexander et al (2001) obtained training-related changes in the SiSt duration of approximately 0.8 s, substantially exceeding the MDC₉₅ obtained in the present study (SiSt duration: 0.54 s, total group).

Furthermore, our results indicate that the BFS could be used to identify specific chair rise strategies; this knowledge could be helpful in developing therapeutic approaches for neurological diseases such as dementia. Manckoundia et al (2006) found approximately 0.5 s difference in trunk forward displacement during SiSt (video tracked by a shoulder marker) between healthy elderly and Alzheimer's patients. Since the MDC₉₅ obtained here for a similar parameter is considerably lower (SiSt-flexion-duration: 0.22 s, total group), dementia-related changes in SiSt pattern might be detected by the BFS. However, discriminative validity has to be proven by further research.

MDCs₉₅% obtained in this study (19.2–34.4%, total group) were lower compared to other functional measures such as static balance assessment (113.1%) in older adults (Goldberg et al 2011) and similar to the timed 'Up & Go' (23%) and comfortable gait speed (22%) in stroke patients (Flansbjer et al 2005). Higher absolute reliability (lower MDCs₉₅%) was found for angular velocities but not for durations in the intraobserver compared to the interobserver group. In line with our findings on relative reliability, this suggests that to obtain more consistent results for angular velocity during trunk movement, the repeated SiSt should be assessed by the same observer. In agreement with our statistical results, a visual analysis by Bland–Altman plots revealed lower within-subject variability in the intraobserver group for angular velocities but not for durations.

Nonuniform changes (heteroscedasticity) from test to retest were obtained for a few variables (SiSt-total-duration, SiSt-flexion-duration). For these variables, greater differences between test and retest were observed for participants with poor SiSt performance (higher duration) than for participants with better performance. In accordance with other reliability studies (Wagner et al 2008), these data suggest that the severity of motor impairment may have influenced the stability of the assessment from session to session.

There was a small systematic bias from test to retest suggesting a learning effect also found in previous reliability studies (Segura-Orti and Martinez-Olmos 2011, Steffen and Seney 2008). Learning effects negatively influence reliability and can be prevented by a larger number of practice trials (Atkinson and Nevill 1998). However, fatigue due to extensive testing can also cause systematic bias, particularly in the frail elderly. In this study, reliability was computed from within-session measurements, and is not reflective of variation in repeated SiSt performance over longer periods of time. Future studies should evaluate reliability over longer lengths of time and include several SiSt trials to control for learning effects.

The results of this study demonstrate that clinically relevant phases of the repeated SiSt can be reliably measured in geriatric patients by using one BFS and automated analysis of raw data. The study shows that the BFS system is a substantial improvement over current clinical practice where commonly only a stopwatch is used. Due to high intraobserver reliability, repeated assessment by the same observer is preferable. While further research on measurement properties of the present assessment is needed, our findings support the clinical utility of BFS-based SiSt analysis during geriatric rehabilitation.