Development and psychometric evaluation of a new patient -reported outcome measure for stroke self -management: The Southampton Stroke Self - Management Questionnaire (SSSMQ)

Stroke is a major cause of death and disability world-wide [1] with potential long-term impact for the individual, such as social isolation [2, 3], long-term depression and anxiety [4, 5] and increased physical impairment [6]. Recovery following stroke is complex and multidimensional, encompassing bio-medical, psychological and sociological elements [7, 8]. Engagement in self-management practices by individuals with long-term conditions has been suggested as key to promoting recovery [9] and is cited as a means of empowerment and a facilitator of improved health outcomes [10, 11]. Self-management, defined as those activities (examples include eating well, exercising, taking medicines, monitoring and managing symptoms) people with a long-term health condition do to stay well and maintain good physical and emotional well-being [12] is not new to western health care. Since the development of the United States based chronic disease self-management programme [13] and subsequent adaption for other western health care settings [14‐16], self-management has received growing attention as part of the management of people with long-term conditions [17, 18].

Self-management interventions (SMIs) are designed to enable people to manage their health more effectively. Stroke SMIs often aim to modify attitudes and behaviours such as goal-setting and lifestyle changes [19, 20] requiring the development or enhancement of skills to self-manage effectively. However, what elements should be included in stroke SMIs remains inconclusive [21]. Patient-Reported Outcome Measures (PROMs) measure how patients function or feel in relation to their health condition, and are recognised as fundamental health outcomes in their own right [22]. A systematic review identified that stroke SMIs have been evaluated by measurement of heterogeneous concepts such as physical function, quality of life, self-efficacy, satisfaction and mood (Boger et al. 2013). The review identified 43 different outcome measures, of which 21 measures (49 %) demonstrated some psychometric properties in stroke populations. This indicates a lack of consensus regarding the appropriate measures to assist evaluation of stroke SMIs, and/or a recognition that self-management embraces a range of differing concepts. The majority of measures scored either ‘fair’ or ‘poor’ according to the COSMIN quality criteria [23], possessing limited psychometric validity. For example, the Barthel Index [24] a measure of physical function, scored poorly due its lack of engagement with users in development and the Functional Independence Measure [25] a measure of physical and cognitive disability, scored ‘fair’ for internal consistency.

No outcome measure exists to adequately measure stroke self-management behaviours, attitudes or skills as discrete concepts, or from the patients’ perspective (Boger et al., 2013). This study used non-parametric item response theory (IRT) to investigate the psychometric properties of a new patient-reported outcome measure of stroke self-management competency to help address this knowledge gap and enhance the evidence quality in terms of the effectiveness of SMIs.

To investigate the scale structure, reliability and evidence for validity of the Southampton Stroke Self-Management Questionnaire (SSSMQ).

Two additional patient-report outcome measures were used to investigate construct validity; the Stroke Impact Scale version 3.0 (SIS) [29] and the Stroke Self-Efficacy Questionnaire (SSEQ) [30, 31]. The SIS is a measure of health status after stroke and consists of 59 items, entailing eight domains as follows: Strength; Hand function; ADL/IADL; Mobility; Communication; Emotion; Memory and thinking and Participation/Role function. Responders rate the level of difficulty pertaining to each item on a 5-point Likert response format, generating summative scores for each domain. Internal consistency reliability ranges from α = 0.83–0.90 [32]. We hypothesised that health status would be positively associated with self-management skills, attitudes and behaviours. Hence, higher scores within each domain of the SIS would correlate with higher scores on the SSSMQ.

At the time the study was conducted, the SSEQ consisted of a 13-item self-report scale measuring self-efficacy judgements in specific domains of functioning post stroke using a 0–10 visual analogue response scale. Internal consistency reliability of the SSEQ is reported as α = 0.90 [30]. Following subsequent Rasch analysis [31], the SSEQ is thought to measure two distinct unidimensional constructs of self-management self-efficacy and activity self-efficacy, and be better suited to a four-point response scale. The SSEQ was selected to examine the hypothesis that stroke specific self-efficacy was positively associated with self-management skills, attitudes and behaviours. Hence higher scores on the SSEQ would correlate with higher scores on the SSSMQ.

Participants received an information sheet about the study and could opt to complete a paper (postage paid return envelope included), or an on-line version of the questionnaires (permission was gained from the authors of the SSEQ and SIS to convert them to an online format) using survey software (www.​smartsurvey.​co.​uk). To avoid over-burdening participants during test-retest reliability evaluation, participants who completed the first questionnaire were invited to opt into completing a repeat SSSMQ (repeat testing of the other measures was not deemed necessary as these already have proven reliability.). To avoid a potential bias from this group, a sub-sample of those who opted to complete a second SSSMQ were randomly selected, two weeks after first completion, using block randomisation. Test-retest participants were grouped consecutively into a ‘block’ of five participants and randomised as follows:

Block 1 – YYNNY Block 2 – YNYNY Block 3- NNYYY

(Where, ‘Y’ received second mailing, ‘N’ did not receive second mailing).

Following the first 15 positive responders, the block allocation resumed at block 1, and repeated until close of recruitment.

During April -June 2013, 87 completed responses were received from a total of 95 returned questionnaires. Six (postal) incomplete responses were returned and were excluded from the analysis. The majority of responses were submitted via the online method (62 %). Response rate for those completing online responses was not calculated due to difficulties in accurately assessing how many people had viewed via this medium. The response rate for those returning paper versions was 62 %. Respondent characteristics are summarised in Table 1.

Conceivably, those who live with others may possess a greater level of dependency due to impairment, which may have a bearing on self-management ability. In this sample, one way analysis of variance (ANOVA) tests revealed that perceived level of recovery (as measured by the SIS) was not statistically different for those who lived alone (mean = 64.0, SD = 26.4) compared to those who lived with partners or family members (mean = 54.2, SD = 21.8) (p = 0.06). Self-efficacy (Stroke Self-efficacy Questionnaire), was also not statistically significant between those who lived alone (mean 80.6, SD = 34) or with others (mean = 90.6, SD = 42.7) (p = 0.376).

Mokken scale analysis revealed that the SSSMQ forms an acceptable unidimensional scale consisting of 28 items. Each item concerns a stroke self-management attitude, skill or behaviour. Collectively, based on the preliminary evidence of validity reported here, and subject to further validation work, the SSSMQ measures stroke self-management competency, the features an individual requires to be competent and capable of managing health and wellbeing following stroke.

Self-efficacy is the most commonly purported theoretical basis underpinning stroke self-management interventions [21, 41], whereby individuals with higher self-efficacy are thought better able to self-manage [42]. Self-efficacy is thought to mediate desirable health behaviours, such as, following a healthy lifestyle, taking prescription medication, that lead to improved motivation, treatment adherence, function and better clinical outcomes [43, 44]. Lower levels of self-efficacy are associated with lower mood and coping skills after stroke [45]. Consistent with this literature, SSSMQ scores correlated moderately with stroke self-efficacy, as measured by the SSEQ (Table 2) [46]. Higher self-efficacy scores were associated with increased SSSMQ scores and thus more successful self-management behaviours, attitudes and skills.

Physical function is likely to affect individual’s ability to perform tasks or strategies important to self-management [47, 48] along with social, emotional and cognitive factors. Total scores from the SSSMQ moderately correlated with the Strength; Hand function; ADL/IADL and Mobility domains of the SIS, although correlations with the Hand function and Strength domains were weak and not significant (Table 2). Further investigation of the relationship between self-management competency and physical function is therefore required.

Better self-management is thought to lead to improved well-being and mood [47], significant to recovery following stroke [48]. Low mood has been identified as a barrier to self-management [49, 50]. Scores from the SSSMQ correlated positively with scores on the emotion domain of the SIS, suggesting that, individuals with lower mood, exhibit fewer desirable self-management behaviours and attitudes. Improved mood may augment self-management competency, or potentially vice versa. Positive correlations were also observed with the communication, memory and participation domains of the SIS, suggesting that these elements are important to self-management competency. Effective communication, which in stroke may be hindered by the presence of aphasia, is likely to enabled successful self-management [51] as navigating services and negotiating treatment strategies with professionals is key to self-management after stroke [52]. Improvement in the communication and participation domains of the SIS have been reported following a stroke self-management intervention [53], suggesting that effective self-management may have the potential to impact upon these domains.

Correlations with the SSSMQ and the SIS subscales provide preliminary evidence that successful self-management is associated with improved quality of life. Conceptually, quality of life may be important to self-management; gaining more control over health and well-being can feasibly be considered to improve quality of life as people develop the coping skills to adjust to and manage their life post-stroke [47, 54]. Alternatively, those who possess a greater quality of life may be more likely to exhibit the skills necessary to self-manage competently.

A difference between gender and total SSSMQ scores was found. It is not surprising that gender might impact upon self-management attitudes, behaviours and skills, as women typically perform better in self-management interventions [55‐57]. Total SSSMQ scores were not statistically associated with age or living circumstances as might have been expected given that previous research indicates that older adults and people who live alone, often find self-management more difficult [50, 58, 59]. Further development of the SSSMQ with additional psychometric testing is warranted to provide continuing evidence of discriminant validity.

Investigation of reliability demonstrates that the SSSMQ is an adequately stable measurement of stroke self-management competency. Internal consistency and test-retest reliability were excellent, but must be considered in light of the limitations of sample size.

The findings provide preliminary evidence of the reliability and validity of the SSSMQ. The predicted hypotheses made with regard to the relationship of scores from the SSSMQ and additional measures were borne out, suggesting that self-management competency is consistent with previously validated measures of stroke self-management.

The optimal content, target outcomes, and mechanisms for change in stroke self-management interventions remain unclear [21, 60]. Measurement of an individuals’ self-management competency, their attitudes towards self-management and relevant behaviours relies upon patient report. The SSSMQ potentially represents an instrument, grounded in the views of patients who have had experienced stroke, with which to evaluate the impact of interventions on stroke self-management competency following stroke.

There are several limitations of the study which are acknowledged. There are no definitive answers regarding sample size requirements for IRT, however sample sizes of 100 are often adequate [61]. Therefore, the inferences drawn from the results must be considered in light of the relatively small sample size, and the possibility for type II errors in analysis. The average age of the UK stroke population is 75 years [62]. The sample in this UK based research was considerably younger at just over 58 years, which may in part explain the larger proportion of people who chose to take part on-line. Moreover, it is not possible to say if those taking part on-line had differing competency at using computers or different access to computers compared to a typical stroke population. The SSSMQ and study information were only available in English, which may have prevented or dissuaded those who do not have English as a first language from participating. Over 60 % of the sample had a moderate communication impairment, according to scores from the communication domain (<60) of the Stroke Impact Scale. This is a strength of this study since people with communication impairment are often excluded from stroke research.

It is also acknowledged that a tension potentially exists between the items, which were inductively generated and considered important to potential users, and the criteria for discarding items that do not function well in a scale [63]. Nonetheless, Mokken scaling represents a measurement model with the least criteria in this respect and is the method most likely to resolve this tension in favour of retaining items [64, 65].

Further investigation of validity, including cross-cultural applications, is necessary to provide further evidence of the psychometric properties of the SSSMQ with a larger, more diverse sample. Future studies should also include clinician/researcher obtained data regarding participants’ level of impairment. This would aid judgements about the relationship between impairment and self-management competency. In this research, selection of outcome measures with which to investigate theoretical relationships with the SSSMQ focused on the prevailing theory of self-efficacy. Further exploration of construct validity with additional measures of concepts associated with self-management, such as health literacy, decision-making and the burden of self-management may further enhance the strength of the construct of self-management competency.

Mokken scale analysis revealed a 28-itemed outcome measure with acceptable scaling properties which can potentially be used to enhance evaluation of stroke self-management interventions in research and clinical practice. Early findings suggest that the SSSMQ possesses excellent reliability and preliminary evidence of validity. Further investigation of validity and reliability of the SSSMQ is required. It follows that totalled item scores from the SSSMQ can potentially be used as an indicator of an individuals’ level of self-management competency.