Standardised assessment of patients' capacity to manage medications: a systematic review of published instruments

Older people are commonly prescribed complex multi-drug regimens, while also experiencing declines in the cognitive and physical abilities required for medication management [1, 2]. Reduced competence in medication self-administration is associated with higher risk of hospitalisation and requirement for assisted living [3‐5].

Health professionals are not good at predicting patients' ability to perform instrumental activity of daily living (IADL), and impaired functional ability may not be detected in routine clinical examinations [6]. More accurate and reliable assessment methods are required [6]. Because high-level IADLs such as medication management are dependent on cognitive abilities, global measures of cognitive function such as the mini-mental state examination (MMSE) are sometimes used to predict functional ability. However, these measures have demonstrated inconsistent relationships with functional performance and lack sensitivity in discriminating competent from incompetent individuals [2, 7]. Most measures of IADL performance rely on self-report or informant report, which is a reasonable starting point since it is relatively quick and easy [2, 6]. However, a limitation is that these measures are subjective and prone to bias [6]. Some older people have difficulty evaluating their own competence or are reluctant to admit their inability to cope, leading to over-estimation of performance [6, 8]. Family members or caregivers may under- or over-estimate performance, or may be unable to make a determination because of a lack of opportunity to observe the patient [6, 8]. Furthermore, IADL scales are usually limited by reliance on a single item to assess functional ability within complex domains such as medication management and lack of well-defined definitions for the skill required [8]. They also provide little information as to the cause of the incapacity. Sometimes health professionals judge IADL performance based on informal observation of the patient, but these unstructured assessments are of unproven efficiency for predicting performance, and the results are usually unquantified and therefore not amenable to comparison across or within individual patients [9].

There has been increasing interest in recent years in developing standardised, structured, performance-based measures that address the high-level IADLs using materials that patients actually encounter in their daily lives [8, 10]. Such instruments can provide useful information not only in terms of a score but also in terms of qualitative data relating to the individual's specific deficits.

The ideal assessment instrument should be objective, quantitative, reliable, readily administered with minimal training, easily and immediately scorable, brief, small and portable and non-threatening to the subject. It should assess both cognitive and physical abilities, [2, 11] and demonstrate clinical utility by providing information to guide planning, application and/or monitoring of medication management interventions. It should have better (or more reliable) predictive validity than self-report, informant report and non-standardised assessment, and demonstrate sensitivity to changes in function over time as cognitive or physical function deteriorates.

A recent review identified 16 instruments designed to assess patients' capacity to manage medications [12]. However the review was limited by the omission of several instruments and validation studies, and the absence of critical appraisal of the methodological quality of validation studies. The objective of our study was to critically review the reliability and validity of published instruments designed to assess patients' capacity to self-administer medications and identify those that may be suitable for use in clinical practice or research.

Forty-nine published reports describing development and/or application of 32 instruments were identified. Eighteen did not meet the inclusion criteria (Table 1) [21‐38].

The 14 instruments included in the review fell into two broad categories: those that used the patients' own medications as the basis for assessment [39‐41], and those that used a simulated medication regimen [42‐52]. Two tools used a simulated regimen plus assessment of knowledge of the patients' own medications [42, 44]. A description of the included instruments is provided in Additional file 2[53, 54].

There have been many attempts to develop a standardised, objective, quantitative measure of patients' ability to manage their own medications, but no published instrument currently has sufficient evidence of reliability and validity to allow it to be recommended for routine use in clinical practice to predict outcomes such as medication adherence or errors. Most instruments have had limited evaluation of reliability, and although most have demonstrated an association with at least one related construct there is limited evidence of ability to predict successful independent medication management in the real world. A wide variety of reference standards and comparators has been used for validity assessment, making it difficult to compare instruments. All validation studies had methodological shortcomings and most were conducted in small homogenous populations, which can lead to overestimation of instrument validity [13, 70]. Therefore the data for most instruments needs to be regarded as preliminary, until confirmed in larger, well-designed studies. The possibility of publication bias leading to over-estimation of instrument validity also needs to be considered.

Based on current evidence, the instruments that in our opinion show most promise and may warrant further investigation are: DRUGS, MAT, MedMaIDE, MMAA, MM Test and MMEI.

DRUGS [39] and MedMaIDE [41], which both use patients' own medications, are the only instruments with evidence of both inter-rater and test-retest reliability, although this needs to be confirmed by researchers other than the instruments' developers. The MedMaIDE instrument demonstrated construct validity through moderate correlations with pill-count medication adherence and cognitive function, albeit in a single small study. A disadvantage is that it requires 30 minutes for administration. The DRUGS instrument demonstrated construct validity through its associations with cognitive function, health literacy, independent medication management and MMAA performance. It has also been shown to be responsive to change, and has the advantage that it has been validated in several patient groups by three different research groups. It requires up to 15 minutes for administration.

The MAT [51] and MMAA [49] both require patients to set-out 24 hours of a simulated medication regimen. They take around 15 minutes to administer, but the MMAA requires an additional 45–60 minute delay between explaining the regimen to the patient and task completion. Validity of both instruments is supported by moderate correlations with cognitive function and observed IADL performance. MAT performance has also been associated with level of supported care, and MMAA performance has been associated with DRUGS scores. MAT has no reliability data, and has only been validated in one small study. A strength of the MMAA is that it has been validated by multiple research groups in large numbers of patients, although mostly patients with mental health disorders. Despite its widespread use, it has never been subjected to inter-rater reliability testing. MAT uses more medications but has a simpler scoring method that incorporates task completion time, which may improve discriminative validity [11].

MM Test [46] and MMEI [43] also use simulated medications but have the advantage of being faster to administer, requiring approximately 5 minutes. MM Test is strongly associated with cognitive function and moderately associated with independent medication management. It has good internal consistency, but inter-rater and test-retest reliability is unknown. MMEI has shown weaker association with these two variables but has been evaluated by three research groups. It has not been subjected to reliability testing.

Although further research is needed before any of these instruments could be used to reliably predict medication outcomes, they could be used to qualitatively identify barriers to successful medication management. Used in this way they could assist health professionals to identify individual patients' need for education, regimen simplification, medication assistance or medication aids [44]. In this context, the DRUGS, MedMaIDE, MedTake, MAI, MMEI and PA instruments may provide the most information about physical barriers and medication knowledge, while the MM Test, MMAA and MAT instruments may provide the best assessment of medication-related cognitive abilities. A limitation of most instruments in the assessment of physical barriers is that they assess ability to access medications from vials only, whilst increasingly medications are supplied in foil or blister-packs which can also be difficult for patients to manage [1], and no instrument assesses ability to self-administer complex dose-forms such as inhalers and eye drops.

Our choice of instruments warranting further study differs from recommendations made in an earlier review by Farris and Phillips [12]. Consistent with our recommendations they selected DRUGS, MedMaIDE and MMAA, however they also selected MMPT and HMS. We did not recommend MMPT because, although it is a brief test, it assesses a limited range of medication management skills and it has limited construct validity data and no reliability data.

The HMS also has limited construct validity and reliability data and seems to offer no advantage over similar, better-validated simulated medication regimen instruments such as MMAA or MAT.

Although the choice of instruments is somewhat subjective, a strength of our review compared with that of Farris and Phillips is that methodological quality of validation studies was formally assessed and, together with psychometric data, considered in the selection process. We also considered instruments' applicability in all settings, not just primary care. A limitation of both reviews is that searches were limited to English language publications, possibly leading to some instruments being overlooked.

Farris and Phillips suggested that instruments using patient's own medications may be preferable due to greater test authenticity and ability to assess patients' medication knowledge [12]. These instruments also have the advantage of not requiring special materials or props. However they also have potential disadvantages: patients' own medications need to be available, which may be problematic in inpatient and clinic settings, and patients may be reluctant to provide their medications for the test [55]. These instruments may be best suited to situations where assessment occurs in patients' homes or as part of a 'brown bag medication review'. Instruments using simulated medications provide a higher level of test standardisation, allowing comparisons to be made between patients and within patients over time. They may be suited to settings where patients' own medications are unavailable or medications are being newly commenced, such as clinics or hospital wards. Because these tests use a medication regimen that is not familiar to the patient, they are able to assess capacity to manage new medications (e.g. after discharge from hospital). To avoid bias due to medication familiarity, it may be preferable that simulated regimens contain mock medications (e.g. as in MMAA), rather than real medications.

No study has reported on the acceptability of medication management assessment instruments to patients. Administration of these instruments in clinical practice could potentially be stressful for frail elderly patients, especially if they believe that poor performance may lead to loss of independence (e.g. placement in residential care). It is possible that instruments using patients' own medications may be less threatening compared to unfamiliar simulated regimens, but this has not been studied and is not supported by a finding that fewer patients declined to take the MMAA test than the DRUGS test in the only study that has compared two instruments [55].

Instrument development is ongoing. We are aware of three additional instruments that were not included in this review, all using simulated medications. Two had only been published in abstract or thesis form [71, 72], while the third was published after this review was completed and assessed cognitive medication management abilities only [73]. There is a need for further research to confirm the reliability and validity of published standardised medication management assessment instruments. The quality of validation studies needs to be improved, for example by blinding testers to results of reference standard assessments. When medication adherence is used for instrument validation, objective adherence measures should be used, and only patients who are responsible for managing their own medications included. However, it may be unrealistic to expect any of the current instruments to predict medication adherence with a high level of accuracy. Since most instruments assess only the cognitive and physical abilities required to manage medications, their results reflect subjects' capacity to accomplish the tasks, and performance in real life may not fulfil the potential indicated by capacity [6, 10, 46]. A range of additional factors contribute to whether or not a patient will adhere to a medication regimen, including: motivation, belief that the therapy is necessary and effective, self-efficacy, relationship with healthcare providers, lifestyle and cultural factors and financial capacity [8, 10, 20]. The MedMaIDE instrument, which had a moderate association with pill-count adherence included items related to accessing ongoing medication supplies, in addition to physical and cognitive abilities [41]. Inclusion of items addressing additional adherence-related factors could potentially increase predictive accuracy. Novel validation methods may be required in future studies, due to the lack of a 'gold standard' for medication management ability. For example, the instrument could be administered to hospital patients entering an inpatient self-administration of medication program, allowing performance on the instrument to be compared with performance on the program.

A number of performance-based instruments exist to assess older patients' ability to manage their own medications. These instruments may be useful for identifying physical and cognitive barriers to successful medication management, but further studies are needed to determine whether they are able to reliably predict medication outcomes such as adherence or errors.

RAE is a senior clinical pharmacist at Austin Health and clinical senior lecturer in aged care pharmacy practice at Monash University. JLM is a senior lecturer in clinical pharmacy at Monash University.