Methods for assessment of patient adherence to removable orthoses used after surgery or trauma to the appendicular skeleton: a systematic review

Adherence to treatment, defined by the World Health Organization (WHO) as ’the extent to which a person’s behaviour corresponds with agreed recommendations from a health care provider’ [1], is a key determinant of healthcare outcomes. Within a clinical setting, accurate documentation of adherence aids a clinician’s judgement of whether poor adherence or ineffectiveness of a prescribed intervention may be responsible for a patient’s clinical progress. Similarly, within a research setting, measuring adherence to a specific intervention is crucial in accurately determining its efficacy and in understanding how this may affect its effectiveness in clinical practice [2]. Even the most efficacious interventions are only as effective as the degree to which patients comply with the recommendations of healthcare professionals [3, 4]. For this reason, the WHO have identified improving patient adherence as one of their key research priorities [1].

Reasons for non-adherence to medical guidance are multi-faceted, with socioeconomic, health-care system-related, condition-related, treatment-related, and patient-related factors all contributing [1]. With regard to orthosis use, specific factors such as discomfort, ill fit, inconvenience, skin irritation, and issues such as disturbed sleep have all been postulated as reasons for non-adherence [5‐8]. Although not a straightforward relationship, there is evidence to support that adherence to the prescribed use of an orthosis in acute conditions leads to superior outcomes following muscle tears [9] and tendon injuries [10‐12] and post-operatively [13].

Measuring adherence is a complex task. There are multiple definitions of the term, it can fluctuate over time, and simply measuring adherence can influence the behaviour itself [14]. Accurately and reliably measuring adherence becomes more difficult with interventions that rely on the co-operation and engagement of patients away from direct supervision of healthcare professionals. The use of removable orthoses is an ideal example of this, as these devices are often prescribed with complex treatment protocols to be followed at home.

Systematic reviews have previously described and appraised the measurement tools used for assessing adherence in home-based exercise programmes [15‐17] and in medication adherence [18], but to our knowledge none exist in patients prescribed removable orthoses.

The aim of this review was to identify, summarise, and critically appraise the methods reported in the literature for assessing patient adherence to treatment with a removable orthosis in adults following surgery or trauma to the appendicular skeleton. This review formed part of the MALIT (Mallet Injury Trial) study, preliminary work to inform the design of a future randomised controlled trial (RCT) in mallet injury splinting, to identify candidate measures of adherence for use in a future trial.

The protocol for this systematic review was registered on the PROSPERO International Prospective Register of Systematic Reviews (reference number CRD42016048462) prior to commencing data extraction.

This is, to our knowledge, the first systematic review to identify, summarise, and critically appraise methods for assessing adherence to removable orthoses used after surgery or trauma to the appendicular skeleton. In the 17 included studies, patient self-report questionnaires were the most commonly used method of measuring adherence. These were commonly multi-item questionnaires, but single-item questionnaires, home diaries, and informal history taking were also used. Overall, there was a lack of consistency in the target behaviour assessed, the timing and frequency of assessment across studies, and whether adherence overall was scored. None of the instruments was validated in the study population. Approximately half of the included studies used methods of assessing adherence that minimised response bias (e.g. postal questionnaires), but the remaining studies used methods such as direct questioning which may have impacted on participants’ responses or did not provide sufficient details (e.g. how and by whom the measure was administered) for the risk of bias in the assessment to be assessed. The lack of consistency in the way that adherence is measured and reported was unanticipated, given the importance of the adherence in orthosis use and its potential impact on outcomes. A validated approach for assessing adherence to removable orthoses is recommended to be used in studies evaluating their effectiveness and will be vital in any future splinting trial. To our knowledge, no such measures exist, and work to develop suitable tools is therefore required. Such measures may need to include reasons for non-compliance.

The challenges of measuring adherence are not unique to orthosis use. Systematic reviews have previously evaluated methods used to assess adherence to medication usage [32], home-based rehabilitation [17], and prescribed exercise [33]. The most widely studied field is that of adherence to medication, with a wide range of tools being designed and validated over the past four decades, and yet none of them can be considered the gold standard [32]. The ideal adherence measure should be economically viable, user friendly, reliable, sensitive, and practical [34, 35]. As no single measure has been found to meet all of these standards, a combination of methods is often recommended to account for the limitations of individual tools [35].

The vast majority of the measurement tools identified in this study were self-reported measures. These provide a low-cost, flexible, practical, and easily implemented measure of adherence. Given the unsupervised nature of recommended orthosis use, these measures provide a straightforward method of collecting otherwise difficult-to-obtain adherence data. However, they do have significant limitations. It has long been known that patients are often unwilling to admit non-adherence [4], and measures may overestimate the true value. Self-reported measures are vulnerable to social desirability bias, with respondents providing answers they feel will please their healthcare professionals [36]. This is a particular problem if the measure is administered by a clinician. Self-report questionnaires may reduce this, but recall bias should be considered if the adherence is assessed several weeks after the injury or intervention. These factors, combined with the study-specific non-validated nature of the measures used, call into question the reliability of the adherence data reported in these studies. These problems have also been shown in other settings. Indeed, Shi et al. [37] reviewed 41 studies assessing the agreement between self-reported measures of medication adherence and electronic drug monitoring and demonstrated a significantly lower correspondence between self-report and electronically monitored adherence when non-named, non-standardised self-reported measures were used.

The widespread use of non-standardised study-specific instruments also limits the comparability of study findings and the ability to pool data in meta-analyses, as highlighted in previous systematic reviews [38, 39]. Although improving adherence to recommended therapies is widely considered to improve outcomes [40, 41], the nature of this relationship is not always straightforward. The complexity of this association may partly be explained by the poor-quality measurement of adherence data that currently exists. There is therefore a need for a validated standardised measure of adherence for use in this setting to improve the quality and comparability of future research in this field.

There are several limitations to our systematic review. Firstly, the review was restricted to papers including the term adherence in the title or abstract. This approach may have excluded studies where relevant methods of assessing adherence were used but not reported in the abstract. The focus of this review, however, was specifically to evaluate methods of measuring adherence, and most of the studies providing a detailed description of this process measure commented on this in the abstract. Restricting the review to use of orthoses in acute settings is another potential weakness. However, the psychological underpinnings of adhering to short-term and long-term treatments with a removable orthosis in these circumstances are likely very different; thus, they potentially require different monitoring approaches for optimisation [42]. A similar rationale can be applied to restricting the review to adults. Overall, however, this study has provided a snapshot of current methods for assessing adherence to orthosis use, and has demonstrated the need for improvement.

Given the importance of adherence to orthosis use in determining outcomes in research and clinical practice, there is a need to improve the quality and consistency of the measurement and reporting of this key outcome. This is particularly relevant to ongoing trials such as Osteoarthritis Thumb Therapy (OTTER) II (ISRCTN 54744256) where the clinical and cost-effectiveness of a splint is being compared to a placebo splint and optimal self-management for the treatment of symptomatic thumb osteoarthritis [43]. A valid, robust, and reliable measure is therefore required that is acceptable to patients and can be used across all studies. This may be challenging, given the variation in types of orthoses used, recommendations regarding duration of use, and other clinical variables, and any measure would need to be sufficiently flexible to meet these needs. However, there are generic components of adherence to orthoses that could be included. We recommend that work with key stakeholders including surgeons, therapists, and patients and their carers is needed to ensure that the measure is appropriate and can be effectively used across different conditions. Key design features will include ease of use and digital solutions, as electronic patient-reported outcomes have been shown to be useful in similar patient groups [44].

However, more objective measures of adherence are also likely to be needed [45], and technological advances such as the development of sensors that can be placed in any orthosis may offer an ideal solution by allowing accurate and detailed information to be reliably collected with minimal burden to patients [46, 47]. Such sensors are amenable for use in a trial setting and are also cheap and relatively ‘low tech’, making it possible to integrate them into routine practice.

This review has highlighted the need to improve methods for assessing patient adherence to orthoses following surgery, but development of appropriate tools will take time, and simple recommendations to improve the quality and value of upcoming and ongoing work can be proposed based on the findings of this review. Firstly, it is vital that trialists select measures that capture the most important aspects of adherence for their study and aim to collect data in a way that represents a minimal burden to patients whilst minimising bias. Self-completed patient questionnaires may represent the best option, and it may be possible to validate instruments already in use.

Assessing adherence may also be more important at different stages in evaluation. Robust assessment would be particularly important in explanatory trials when determining if an orthosis was an appropriate treatment in an ideal settling with a compliant patient, but it may be less important in a large-scale pragmatic study when the aim would be to evaluate effectiveness in the ‘real world’. Similarly, reasons for non-compliance to orthosis use would have particular value in the pilot and feasibility stages of a trial. It may be that the instructions for use or the orthosis itself could be modified based on patient feedback so that the intervention would be more acceptable to patients in the main study and thus more likely to be used. At present, the best recommendations for trialists involved in studies using orthosis are that details of any methods used for assessing adherence be well described (e.g. included as a study additional file) and transparently reported so that readers can judge for themselves the methods used and the results. Transparent reporting of any statistical analysis used for assessing the impact of adherence on outcomes is similarly vital, including details of any assumptions made and any limitations of the measures used. Researchers should take care when interpreting their findings to take account of the limitations of the measures of adherence currently used and to acknowledge these issues when discussing the implications of the results for both further research and clinical practice.

Measurement and reporting of adherence to orthosis use is currently inconsistent, resulting in problems with interpretation of relevant literature and impacting on understanding of the efficacy of orthoses prescribed in the acute setting following injury or surgery. Further research is required to develop measurement tools that provide a rigorous and reproducible assessment of adherence in this acute population.