Validation of the Adherence Barriers Questionnaire – an instrument for identifying potential risk factors associated with medication-related non-adherence

By conducting a systematic review of the relevant literature in the years 2008–2010 (see Additional file 1), the most frequent observable factors associated with medication related NA were identified; an updated review after data analysis that had been done revealed that no additional general adherence barriers were discussed in recent years (see Additional file 2). Generally, in accordance with existing literature, the identified adherence barriers could be classified into four main groups:

The results of this review were assessed by a scientific team consisting of all authors and formed the basis for item construction. A total of 16 items addressing all described adherence barriers were considered for the initial version of the ABQ. Each item was formulated as a statement. With regards to the response structure, a rating scale was chosen by assessing the grade of information exploitation and, on the other hand, the risk of overtaxing respondents. Finally, a 4-point Likert scale was defined, which deliberately left out a mean response option to force the respondents to a decision. The possible answers were “strongly agree”, “generally agree”, “generally disagree”, and “strongly disagree”, which were given values from 1 to 4 or rather 4 to 1 depending on the formulation of each item (a higher score indicated a higher influence of a certain barrier on patient’s perceptions). Based on patients’ responses, a general as well as item- and subscale-specific ABQ scores indicating the number and strength of adherence barriers present in a patient were calculated.

In general, patients’ self-reports always face the risk that the information provided by respondents may be distorted, even with a pre-defined response scale. In particular, social desirability bias could be a problem in terms of adherence-related questions. To control for the tendency of respondents to answer in a manner that will be viewed favourably by health care providers (or others), we added five more questions at the end of the interview. For this purpose, three items of the Social Desirability Scale-17 [39] were adopted and supplemented by two other health-related items. The items were also formulated as statements and could be answered either with “yes” or “no”. All items were aligned so that the answer “yes” indicated a stronger socially desirable response bias. These additional items were not part of the ABQ, but were used for assessment of the validity of the ABQ in subgroups classified by high/low social desirability response bias.

We applied the ABQ in a multicentre, non-interventional, prospective observational cohort study addressing the general treatment of patients with atrial fibrillation (AF) in general practices in Germany (ACT-AF study). Written informed consent was obtained from each patient in this study and the study protocol was approved by an independent ethical committee. AF is the most common significant cardiac rhythm disorder that is associated with substantial lethality from stroke and thromboembolism. According to current guidelines, a treatment with oral anticoagulation is recommended for AF patients with a high risk of stroke [40]. Non-adherence to oral anticoagulation (OAC) seems to be a considerable problem and leads to an increase in adverse medical events, including stroke and bleeding events [41].

In the ACT-AF study, 71 participating general practitioners (GPs) were asked to include AF patients (ICD10 code I48; no further criteria) who were at least 18 years old and not participating in any other study (first patient in: May 2009, last patient out: May 2011). The data documented in the study included the clinical and sociodemographic background of each patient as well as anticoagulation treatment information and all international normalised ratio (INR) values related to anticoagulation therapy of the patients in the prospective observational period of 12 months; new oral anticoagulants were not available at the time of the study so that anticoagulation was based on vitamin-K-antagonists (VKA) only. Additional data were collected via written questionnaires at the end of the study period; this last survey took place on average 290.6 days after study inclusion. In our analysis, we included only patients who participated in this last survey; for the statistical analyses, only data regarding patients who completed the ABQ without any data gaps were used.

The developed ABQ was applied in its written form in a patient survey at study sites. Furthermore, during this survey, patients were asked to fill out a self-report instrument that measured the extent of potential NA (modified “Adherence to Refills and Medications Scale – ARMS” [42]). The ARMS is a validated instrument (one of the few questionnaires which were validated using at least two different external criteria and, among them, at least one was a clinical outcome). Furthermore, the ARMS contains hardly any questions which measure adherence barriers; in this respect, it focuses on assessing the level of non-adherence. Nevertheless, the original ARMS had to be reduced by two items that rather seemed to measure causes of NA (carelessness and cost) than the extent of NA. The modified ARMS consisted of 10 items (4-point Likert scales) with a total score from 10 to 40, wherein a higher score indicated higher non-adherence (see Additional file 3).

The ACT-AF study protocol, which included all mentioned questionnaires, was approved by the Ethics Commission of the University of Greifswald (Germany). The ABQ as well as all other questionnaires were applied in the German language.

All analyses were conducted using SPSS Statistics 17.0 (2008 SPSS Inc.).

The internal consistency reliability of the ABQ was examined by calculating the Cronbach’s alpha, which describes the questionnaire’s homogeneity [43]. In general, an alpha ≥ 0.8 is desirable [43,44]. Simultaneously, the item-total correlation coefficient for the different items was evaluated, where a correlation ≥ 0.3 is seen as adequate [45].

The internal structure of the ABQ was evaluated by principal components exploratory factor analysis. This analysis generates possible subscales that are represented by sets of items within the questionnaire. The Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett’s test of sphericity were used to confirm the eligibility of the database for using this factor analysis. The initial number of factors was determined by using eigenvalues > 1 as well as scree plots examination. Sets of items generated by the Promax-rotated component matrix were assessed to define whether they fitted into the identified subdomains of the questionnaire. Items with a loading of > 0.4 were considered to adequately represent a factor.

The external validity of the questionnaire was investigated by using two different criteria. First, an assessment of the Spearman’s rho correlation of the ABQ score with the score of the used self-report adherence measure (modified ARMS) was done by determining the amount of correlation as a measure of the validity. In this analysis, only patients having completed the ABQ as well as the modified ARMS scale were included.

Second, a Spearman’s rho correlation of the ABQ scale with the available clinical outcome “time in therapeutic range” (TTR) was conducted. The TTR is a patient-specific assessment of the quality of oral anticoagulation therapy, and indicates the proportion of treatment time in which a patient’s INR value was in the pre-defined therapeutic range (INR between 2.0 and 3.0). The TTR is strongly correlated with the medication intake, and therefore, a good clinical indicator of medication-related adherence. In this analysis, only patients having completed the ABQ without any data gaps, having been prescribed VKA, and with two INR values available during the prospective observational period of the study, were included.

Furthermore, the proportion of patients with good anticoagulation quality among respondents with high versus low ABQ scores (above or below the median) was compared by using a chi-square test. According to current guidelines, a good anticoagulation quality can be defined as TTR ≥ 60% (at least 60% of the observed days are within the target range) [40]. The chi-square test should confirm the hypothesis that patients with a high number and/or above-average strength of existing adherence barriers are less likely to reach this level of TTR.

To consider a potential social desirability bias, all described analyses for assessing the criterion-related validity were repeated separately in patients without a high tendency to answer in a manner that will be viewed favourably by health care providers. This tendency was appraised by an additional scale gained from the last five questions addressing social desirability with a score ranging from 0 to 5. Patients with a score of five were defined as most probably biased regarding social desirability, and were excluded.

Our developed ABQ consisted of 16 different items. Five items referred to intentional adherence barriers (items 4, 5, 6, 7, 12), four items to unintentional adherence barriers (items 2, 9, 10, 13), four items to medication-related barriers (items 11, 14, 15a, 15b), and three items to health care system-related barriers (items 1, 3, 8).

Of the 786 AF-patients registered in the ACT-AF study, 570 (72.5%) participated in the survey. However, 138 patients did not respond to all ABQ questions, so that 432 patients completed the ABQ questionnaire without any data gaps. These 432 patients who formed the basis of our analysis had a mean age of 72.7 years; 45.6% were female. These patients suffered from AF for 6.6 years on average, and took a mean number of 6.6 long-term medications as reported by the treating physician. Most frequent observed comorbidities were hypertension (83.8%) and diabetes (38.4%).

In the external validation analyses, two subsamples of the sample of 432 patients were analysed. In the first analysis (ABQ versus modified ARMS scale), 401 patients with completed responses to both the ABQ and the modified ARMS questionnaire were included. Of these patients 44.6% showed poor self-reported adherence (ARMS scores below the median of 11). In the second analysis, 371 patients with completed responses to the ABQ, having received vitamin K antagonists as anticoagulation treatment, and having at least two INR value measures available, were included. Table 1 shows the main characteristics of the different patient samples.

Each ABQ item was scored 1 to 4 with a higher score indicating a stronger influence/importance of the specific adherence barrier from a respondent’s point of view. Table 2 displays the distribution characteristics of the patients’ responses to the 16 ABQ items. Most of the items showed a right-skewed distribution of the scores and thus had similar effects on the answering pattern of the patients (particularly important with regards to the inter-item-correlation). The only exception was item 8 (“I feel that co-payments for medicines are a great burden”), with a skewness of −0.135. This indicates that it was difficult for patients to contradict this statement; which is also illustrated by the mean and median of the scores (item 8 showed the highest values with a mean of 2.66 and a median of 3.0).

Cronbach’s α for the original ABQ-scale was 0.814 and demonstrated a good internal consistency. Item-total correlations for the 16 items ranged from 0.225 to 0.634 (Table 3). Statement 6 of the questionnaire (“I feel basically healthy. Therefore I am sometimes unsure whether I really have to take my medicines daily.”) showed the lowest item-scale correlation (0.225), and Cronbach’s α increased to 0.817 if this item was deleted. Furthermore, item 5 (“Medicines are all poisonous. You should avoid taking medicines at all if possible.”) demonstrated a low item-total correlation coefficient (0.263). Deleting this item led to an increased Cronbach’s α of 0.815. Additionally, statement 15a with an item-scale correlation coefficient of 0.274 was below the critical threshold of 0.3, but in this case, removing the item led to a decreased internal consistency (α = 0.812). Consequently, the ABQ questionnaire was reduced by removing items 5 and 6, and the examination of the internal consistence reliability was revised (Table 3). Item-total correlation in the reduced 14-item-ABQ scale ranged from 0.265 to 0.640, and upon removing items 5 and 6, the Cronbach’s α increased to 0.820.

During development of the questionnaire, four subscales were originally defined (medication-related adherence barriers, health care system-related barriers, and intentional as well as unintentional barriers). The corresponding items were initially assigned to these scales on the basis of their content. Factor analysis of the original ABQ-scale (16 items) based on eigenvalues also suggested a four-factor solution, which explained 55.72% of the variance. However, factor analysis based on the reduced ABQ-scale (without items 5 and 6) identified a three-factor solution (Table 4). This three-factor solution still explains more than 50% of the variance (52.88%), which supported the decision to reduce the original scale (parallel to the advantage in terms of ease of use for respondents). Factor 1 (representing intentional adherence barriers) demonstrated an eigenvalue of 4.459 and accounted for 31.9% of the variance. In whole, five items load on this factor (Table 4). The second component containing five items can be labelled as subscale describing medication- or health care system-related adherence barriers; it had an eigenvalue of 1.864 and explained 13.3% of the variance. Finally, items 1 and 2 as well as items 9 and 10 can be summarised as unintentional adherence barriers. These items showed a maximum loading on the third factor. Factor 3 had an eigenvalue of 1.080 and explained 7.7% of the variance.

For the first subscale (intentional adherence barriers), Cronbach’s α was 0.653 and the item-total correlations ranged from 0.298 to 0.539. A range from 0.407 to 0.641 with regards to the item-total correlation coefficients of the second subscale (medication- or health care system-related adherence barriers; α = 0.749) could be observed. The third subscale (unintentional adherence barriers) demonstrated a Cronbach’s α of 0.709 and a range of item-total correlations from 0.433 to 0.551.

The observed overall ABQ score based on the reduced ABQ-scale (14 items; score of 1–4 per item) ranged from 14 to 52 with a mean of 25.35 (SD = 6.46). A range of 5 to 9 was observed with regards to the scores of the five-item intentional adherence barriers subscale (mean = 7.00, SD = 2.15). With regards to the five-item medication- or health care system-related adherence barriers subscale and the four-item unintentional adherence barriers subscale, scores from 5 to 20 (mean = 9.89, SD = 3.41) and from 4 to 16 (mean = 8.47, SD = 2.57) were observed, respectively. Figure 1 shows the proportion of patients who were affected by the individual barriers (assuming a barrier existed in case of an item score of at least 3).

Figure 1 also shows which percentage of patients could be assigned to the defined groups barriers’ groups/subscales. A patient was assigned to a barriers’ group if the average score per item belonging to this subscale was > 2, or at least one subscale item had a score of 4. In 85.9% of the patients, there was at least one adherence barrier present. More than five barriers were present in 21.1% of the patients. The most commonly mentioned adherence barrier was the feeling that co-payments were a great burden. Based on the three subscales, the following were present: in 13.2% of the patients, intentional adherence barriers; in 45.8% of patients, medication/health care system-related adherence barriers; and in 49.8% of the patients, unintentional adherence barriers.

The overall scores related to the reduced 14-item ABQ and its subscales correlated significantly with the chosen validation variables in both external validation analyses (Table 5). The ABQ score had a stronger correlation with the modified ARMS score (self-report to identify the extent of NA) than with the TTR. However, the correlations with the TTR are still significant for all scales of the ABQ, and show that an increase in number/strength of adherence barriers is associated with a decrease in the TTR, indicating a poorer quality of anticoagulation quality.

Of the 371 patients with available information on TTR, 35.0% had a poor quality of oral anticoagulation (TTR < 60%). Patients with a high ABQ score (above the median of 25), which indicates an increased number of existing adherence barriers and/or a high strength of these barriers, were significantly (p < 0.005, Pearson Chi-Square) more likely to have a poor anticoagulation quality than patients with a low ABQ score (44.6% versus 27.3%).

Based on the additional five questions with regards to social desirability, a subgroup of 322 patients with no strong tendency of responding behaviour towards social desirability (social desirability score < 5) was identified. For this subgroup of patients, the ABQ scores (reduced 14-item ABQ), as well as scores related to its subscales, showed a stronger correlation to the modified ARMS score on a remaining high level of significance (Table 6). Also, the correlates with the TTR increased, but for the third subscale of unintentional adherence barriers the correlation was on an insignificant level. In this subgroup of patients without a tendency towards socially desirable responding behaviour, patients with an overall ABQ score of > 25 were significantly more likely to experience poor anticoagulation quality (43.8% versus 24.7% affected patients with n = 275 patients, p < 0.005).

We acknowledge some limitations of our analysis. First, because of data limitations, we were only able to use AF patients treated by GPs for our validation. This may limit generalizability to other settings. Particularly, patients with AF tend to be older and more morbid. Therefore, further research is needed to show the effectiveness of the questionnaire in other settings. Nevertheless, the analysed patient samples were characterised by a high average rate of medication dependency and occurrence of chronic diseases. Second, we used the ABQ in a German treatment setting only and there probably exist country specific adherence barriers (e. g. out-of-pocket cost). It needs to be seen whether the ABQ shows similar validity and consistency when applied in other countries. Third, our survey design did not facilitate the assessment of test-retest reliability that should be proven in further investigations. Fourth, 138 out of 570 patients (24.2%) did not complete the ABQ. The characteristics of these patients do not differ from those of all ACT-AF patients or from those of the patients who completed the ABQ without any gap (Table 1). However, most of these patients (n = 76) did not complete the other parts of the survey as well so that non-response to the ABQ items seems not to be an ABQ-specific phenomenon. Of those patients who did complete the other questions of the survey but could not complete the ABQ, only one patient did not answer any of ABQ items, and a large proportion of patients only missed one (67.7%) or two (12.9%) items of the ABQ. Item 15b was the item with the highest number of gaps (35 of 62 patients 56.4%). It is possible that patients consider items 15a and 15b as alternative items and think they finished the questionnaire after answering item 15a. A renumbering of these items should be done and the effect should be proven in further investigations. Fifth, we decided to use self-reported NA and TTR as external validation criteria. So, there might be more objective measures, which are more suitable to validate our tool (e. g. data derived from medication event monitoring systems or laboratory data). However, self-reported NA and TTR were chosen because they seemed to be both objective measures of medication adherence/clinical effectiveness of medication without being influenced by patient behaviour within the setting. The ARMS was involved in a regular survey within the prospective study period and the INR values for calculating the TTR were measured in a regular care setting. Nevertheless, the ABQ correlated stronger with the ARMS than with the TTR, which may be due to several reasons. On the one hand, the TTR is influenced by other factors, like nutrition. On the other hand, it might be that both the ABQ and the ARMS scores are affected by a self-report bias. The data were collected at a scheduled GP appointment. To the extent that appointment keeping indicates compliance with health behaviour, patients who completed the questionnaire may have been more likely to be adherent to medication therapy as well. At the same time the social desirability bias may be higher when patients answer in a medical environment. However, additional questions were included to control for the tendency of patients to response in a socially desirable manner, and separate subgroup analyses were conducted. Our analysis shows that the predictive power of the ABQ improves if socially desirable responding behaviour is excluded. Based on this analysis, future research may explore whether a more sophisticated ABQ item scoring methodology adjusting ABQ scores by “socially desirability scores” improves the efficacy of the ABQ in identifying existing adherence barriers.