Background
According to the United Nations AIDS Program (2019), by the end of 2018 nearly 38 million people were living with HIV/AIDS, of whom 23 million were on antiretroviral therapy (ART) [
1]. At the same time, 63% of the nearly 700 thousand adults living with HIV in Ethiopia were women, and new infections among young women aged 15–24 years annually were more than double those of young men, 5800 compared to 2000 [
2]. HIV treatment using ART can improve functionality and decrease mortality but lapses in adherence may render treatment permanently ineffective, for example, due to drug resistance [
3]. The WHO has defined adherence as “the extent to which a person’s behavior – taking medication, following a diet, and executing lifestyle changes, corresponds with agreed recommendations from a healthcare provider” [
3]. Non-adherent patients have higher mortality rates than adherent ones with similar CD4+ counts and adherence is the critical determinant of survival among persons living with HIV [
4‐
6]. Non-adherence is also associated with poor health outcomes, increased healthcare costs and poor patient safety, due to increased risk of dependence, relapses, toxicity, to mention a few [
7]. Adherence is reported to be a major challenge in healthcare, estimated at 50% in high-income countries and even lower in some low and medium income countries [
7]. Adherence is also critical to the achievement of the third target of the UNAIDS Fast-Track Initiative goals of 2020–2030, in which 90–95% of people with HIV are diagnosed with it, 90–95% of the diagnosed receive ART, and 90–95% of those on ART achieve viral suppression [
8‐
10].
In Ethiopia, treatment adherence and retention were estimated to be on average 51–85% and 70% among those who had been initiated on ART, respectively [
11]. In addition, a meta-analysis of 27 studies conducted in 12 sub-Saharan Africa countries (not including Ethiopia) found average adherence rates of 77% among study participants who were on ART [
12]. Further, in the same meta-analysis, the authors reported average adherence of 55% among patients who participated in 24 studies in the United States and Canada [
12]. In the literature, studies comparing adherence rates by sex of participants in Ethiopia are scant, but Molla et al. (2018) found that women had 1.22 higher odds of adherence to ART than men [
13].
Accurate measurement of adherence is important for correct assessment of health outcomes and in predicting the efficacy of ART [
7]. Non-adherence compromises treatment efficacy, and without accurate treatment efficacy data, adherence rates necessary for planning and evaluation cannot be achieved [
7]. Further, accurate measurement of adherence is required for effective and efficient treatment planning, and for ensuring that changes in health outcomes can be attributed to recommended regimens. In addition, decisions to change recommendations, medications, and communication style to promote patient participation depend on valid and reliable measurement of the adherence construct [
7].
Medication adherence has been measured using several methods, including: direct measures, measures involving secondary database analysis, measures involving electronic medication packaging (EMP) devices, pill count and measures involving clinician assessments and self-report [
14]. However, there is no “gold standard” for measurement of adherence, and each method has advantages and disadvantages [
14,
15]. For example, the WHO reported that there are challenges in measurement of the adherence construct even when more objective methods are used [
7]. The report cited challenges including:
-
counting inaccuracies using the “remaining dosage units” method;
-
the inability to capture important information such as timing of dosage and pattern of missed dosage;
-
the high cost of medication event monitoring systems (MEMS);
-
the inability to tell whether patients actually use their medicine when they are removed from the bottle;
-
difficulties faced when an individual acquires medication at multiple pharmacies; and
-
inaccurate and incomplete records using the prescription refills method [
7].
Self-reports include measures such as patient-kept diaries, patient interviews and questionnaires and scales; they tend to overestimate adherence behavior compared with other methods [
15]. Despite their limitations, self-reports can significantly predict clinical outcomes and produce actionable information for patients and providers [
15]. They are also cheaper, noninvasive and easier to administer compared with other methods [
15]. Some examples of self-report questionnaires and scales for general use include: Adherence Estimator, Adherence to Refills and Medication Scale (ARMS), Brief Medication Questionnaire (BMQ), Medical Outcomes Study (MOS), Medication Adherence Scale (MAS), Medication Management Instrument for Deficiencies in the Elderly (MedMaIDE), Medical Adherence Measure, Morisky Adherence Questionnaire 4 item (MAQ) and the Morisky Adherence Questionnaire 8 item (MAQ) [
14‐
16]. In addition, there are self-report questionnaires and scales specific to measurement of adherence to ART, for example: AIDS Clinical Trials Group (ACTG) Adherence Questionnaire, Community Programs for Clinical Research on AIDS (CPCRA), Antiretroviral Medication Self-Report, Self-Rating Scale Item (SRSI), Self-Reported Adherence (SERAD) Questionnaire, Self-Reported Questionnaire Assessing Adherence to Antiretroviral Medication, Simplified Medication Adherence Questionnaire (SMAQ), Visual Analog Scale (VAS), among others [
14‐
16].
There is a dearth of literature on use of standardized scales to measure medication adherence among people on ART in Ethiopia. A systematic review of 15 ART adherence studies in Ethiopia reported that 60% of the studies used self-reports, other methods included: caregiver reports, unannounced pill counts, pharmacy refill record, medication event monitoring systems, viral load measurement, CD4 count and record review [
17]. Some studies have reported challenges associated with various methods of assessing adherence among users of ART in Ethiopia. Biressaw et al. (2013) found a discrepancy in adherence levels estimated by caregiver reports and unannounced home-based pill counts. They found adherence estimated from unannounced pill count was unacceptably low, but comparable to that of Medication Event Monitoring System reported by other studies [
18]. Amberbir et al. (2008) and Markos, Worku & Davey (2008) also reported using self-reports to assess adherence to ART among HIV positive individuals in Ethiopia. Both studies reported that self-reports overstated adherence levels compared to unannounced pill count due to social desirability bias, in addition to being susceptible to recall bias [
19,
20]. The authors reported that despite their limitations, self-reports and pill count are widely used in Ethiopia because they are cheaper and easy to implement [
17]. Self-reports have also been found to correlate with viral load and clinical outcomes [
17].
The SMAQ is one of the self-report questionnaires which is increasingly used globally to assess adherence to ART and non-HIV-related medications [
21]. It was developed and validated among a sample of predominantly male (72%) HIV-positive individuals in Spain, with 72% sensitivity, 91% specificity, and a likelihood ratio of 7.9 in identifying nonadherent patients as compared to medication event monitoring systems, the authors concluded that the SMAQ was reliable and valid for assessment of adherence among HIV-infected patients in most settings [
21]. It has been used to assess adherence to ART in at least 12 countries, including South Africa and Kenya, in at least 25 studies and interventions between 2002 and 2018 [
22‐
31,
20‐
25]. It has also been used to assess adherence to non-HIV medication in at least eight countries and 12 studies [
32‐
41].
According to the WHO, standardized multi-item scales such as SMAQ that assess specific behaviors relating to medication recommendations may be better predictors of adherence than simple yes/no responses [
7]. The underlying logic is that each indicator when used on its own may be insufficient to capture the construct, but when these indicators are combined, they represent a valid composite measure of the underlying construct of interest [
42]. While standardized scales have potential advantages in understanding perceptions about adherence, literature assessing psychometric properties including reliability, validity and measurement invariance (MI) of different scales in diverse settings is sparse. In addition, standardized scales are often used with populations that may be quite different from the one in which the scales were originally validated [
42]. Also, there is a natural desire to make group comparisons and conclusions about effects of interventions on the mean scale scores of expected patient outcomes [
43]. However, such comparisons are justified only to the extent that these comparisons approximate differences of means on the theoretical true score of the relevant constructs, and when the means are generated from data collected using questionnaires and scales exhibiting acceptable levels of reliability and validity [
15,
43]. Further, even when standardized scales are used, inferences and conclusions about observed mean differences are dependent on the between–group equivalence of the underlying measurement model [
43]. However, an investigator’s ability to assess true differences between groups or across time can be hindered by measurement errors, which can limit the ability to make accurate meaningful comparisons when determining program impacts [
42].
Measurement invariance is a statistical criterion that is used to assess the extent to which a standardized scale measures the same construct in each group and at each time point studied [
43]. It provides a way to assess whether respondents interpreted measures conceptually similarly across groups and time and whether participation in an intervention altered the conceptual frame of reference against which a group responded to an indicator over time [
42]. Measurement invariance requires that any two persons with the same level of the latent construct should obtain the same expected score on the indicators used to measure the underlying construct, regardless of the group they are in [
44]. Assessment of MI helps in determining if a scale functions equivalently for all groups defined by factors such as gender, age, education, mother tongue, socioeconomic status, regional background, among others [
44]. Demonstrating that a scale has MI allows an investigator to make valid comparison of construct scores such as means that yield meaningful interpretations and substantive inferences [
45].
To improve clinical research on ART adherence in this population, properties of measuring instruments, such as reliability, validity, and MI must be analyzed. While the importance of reliability and validity for assessing a self-report instrument is well-understood, measurement invariance is increasingly being evaluated for valid comparisons of levels of latent outcomes to be made. Despite increasing frequency of use of the SMAQ in assessment of adherence to antiretroviral therapy, to date no study has assessed its MI and other psychometric properties such as reliability and validity in sub-Saharan Africa. Using data from a pre-post quasi-experimental evaluation study of a HIV/AIDS intervention among HIV-positive women of reproductive age in Ethiopia, hereinafter referred to as the parent study (pre refers to before intervention assessment or T1, whereas post refers to post intervention assessment or T2) [
46,
47], this paper assesses the internal consistency reliability, concurrent and factorial validity, and MI for the SMAQ in this setting. These analyses build upon the parent study and add to the sparse literature about the validity of SMAQ as a HIV/AIDS treatment adherence measure.
Discussion
The purpose of this study was to assess the psychometric and related measurement properties of the six-item SMAQ using data from a quasi-experimental parent study of HIV-positive women of reproductive age in Ethiopia. Our findings indicate that the six-item SMAQ demonstrated adequate internal consistency reliability, suggesting that the six items in the questionnaire reflect the same latent construct of adherence to antiretroviral therapy. In addition, concurrent validity of the scale was moderate to excellent based on correlations between the item responses at T1 compared with T2. Further, model fit indices and significant factor loadings demonstrated factorial validity, which suggests construct validity as well.
In addition, we documented strong factorial invariance across the four independent study groups, suggesting that the SMAQ questions/items were being interpreted in an equivalent manner across groups. This finding suggests that the SMAQ performs equally well across samples and operationalizes group-specific differences in an invariant manner across groups. An important implication of this finding is that adherence scores obtained using SMAQ from the four study groups can be compared pre-and post-intervention for policy or intervention purposes [
37]. Taken together, these findings affirm that the six-item SMAQ is a valid measure of adherence to ART in this sample of women with HIV/AIDS in Ethiopia. Our findings add confidence for researchers and interventionists interested in using the SMAQ to assess adherence to ART in this setting.
We found one negative but nonsignificant correlation between the six indicators of the SMAQ suggesting that a five-item version might be more efficient [
59]. However, our findings showed no differences in measurement invariance tests when question five was included or excluded. It is possible that the lack of correlation was caused by a data entry error, but we were unable to verify this possibility. More likely, it was due to the magnitude of question five’s correlation with question three being too small to impact the results. In addition, question five was strongly and positively correlated with other items of the scale, and all its factor loadings were positive and significant. Thus, we maintained the integrity of the original six-item SMAQ scale in our final analyses.
In comparison with the validation study in Spain, the mean age of patients in the present study was slightly lower (33 years versus 36 years). All participants in our study were female compared to 28% in the Spanish study. The Cronbach’s α in the present study was lower for T1, but comparable for T2 and for the full study sample (α = 0.75) [
21]. Estimating concurrent and factorial validity was a quick way to validate our SMAQ data, although predictive validity would be a more powerful criterion for future studies predicting SMAQ scores in relation to ART interventions where the counts of HIV ribonucleic acid (RNA) or CD4 T lymphocytes (CD4 cells), for example, are available.
The SMAQ has several advantages for field studies—it is short and easier to administer, which makes identification of non-adherent patients and intervening quicker at crowded health service facilities associated with severe personnel shortages and long waiting times [
60]. Conversely, collection of HIV RNA or CD4 counts requires much more time, financial and workforce recourses which were limited in the study setting. Other studies have used data from two cross sections to assess validity of standardized scales [
52]. Our study complemented the need for assessment of validity by testing for measurement invariance of the SMAQ and found it to be invariant across groups and time, suggesting that the six items are relevant for measurement of the latent factor of adherence to ART.
The Morisky Scale and variations of the adult AIDS Clinical Trials Group (ACTG) are also used to assess self-reports of adherence [
15,
61,
62]. The SMAQ is a modified version of the original four-item Morisky Scale, which has since been modified and validated as an eight-item scale [
61,
63]. However, the Morisky Scale has been validated and is more commonly used in hypertension patients and general purpose adherence studies and interventions, compared to the SMAQ and ACTG which have been validated with persons living with HIV [
62,
64]. Compared to the SMAQ, the ACTG scale is longer and would require much more time and higher costs to administer and collate participant responses into actionable insights that can guide quick adherence improvement interventions. Self-reports of adherence seek various types of information from respondents, including: medication-taking behavior, and barriers and beliefs associated with adherence [
64]. The choice of a self-report measure depends on the goal of the study or intervention. The SMAQ seeks information about medication-taking behavior and barriers to adherence [
64]. The parent study assessed level of adherence with the goal of improving adherence by reducing ART access barriers by increasing the number of access points or service providers. In this way, the SMAQ was more appropriate to the goals of the parent study than the ACTG. Researchers and interventionists with similar needs and goals may consider using the SMAQ in their studies.
Initiation, retention and adherence to ART positively influence quality of life among persons living with HIV [
65‐
69], are required for viral suppression, and are critical to the achievement of the UNAIDS 90–90-90 goals. However, initiation and retention on ART are only meaningful to the extent to which users of ART can adhere to the regimen [
70]. Also, recent studies have shown that adherence to ART can be a successful HIV prevention strategy [
8,
71,
72]. Improving adherence may be challenging or impossible without our ability to measure it reliably, validly and consistently across groups of individuals, which makes efforts to improve measurement methods and tools an important contribution for public health. A study of strategies to improve adherence to ART in low-resource settings reported that adherence measurement was required for optimal targeting and tailoring of interventions [
73]. The present study moves the field forward by presenting reliability, validity and invariance test statistics for SMAQ from a sub-Saharan Africa setting where such HIV research is scant, yet the burden of disease and potential need for such measurement is greatest as sub-Saharan Africa bears the greatest HIV/AIDS burden. According to the WHO, nearly one in every 25 adults is living with HIV in Africa, accounting for nearly two-thirds of the global total [
74]. Providing evidentiary measurement properties for SMAQ increases practitioners’ confidence in using SMAQ, which increases its adoption in assessment of adherence.
Although we found strong invariance for the six items of the SMAQ, it is worthwhile to note that adherence is a dynamic behavior which may change over time, even without intervention. Thus, invariance can be expected for SMAQ items that assess intentional non-adherence across time, such as question three of the SMAQ: “Sometimes if you feel worse, do you stop taking your medicines?”, because such items are embedded in a patient’s beliefs and self-construct and therefore, are more robust to behavior change. Conversely, the SMAQ also has a component of unintentional non-adherence due to forgetfulness, assessed by questions one and two: “Do you forget to take your medicine?” and “Are you careless at times about taking your medicine”? The Unintentional non-adherence component may be prone to random variability, which may not be captured by invariance testing of the six items of the SMAQ together, but by testing invariance for each item using longitudinal data. Thus, attribution of changes in adherence to specific components of the SMAQ as intentional or unintentional was not possible in the present study, because of the independent cross-section design. This is an important area for future studies in which researchers may be able to identify modifiable items of non-adherence measured by the SMAQ so as to appropriately intervene to improve adherence, as was demonstrated by Mora et al. (2011) in their assessment of non-adherence among asthma patients using the Medication Adherence Report Scale (MARS-A10) [
75].
Several limitations of our study should be noted. Although we treated the samples as independent, they may not be truly independent because some participants may have participated at both T1 and T2 interviews. This limitation may manifest in repeated questions where social desirability bias is also a limitation. However, the cross-sectional design of the parent study mitigated this tendency. In addition, statistical tests showed group differences in demographic characteristics. The design limited the use of multilevel multigroup CFA, as suggested by Kim and colleagues [
76]. Ethical considerations and operational logistics were also considered in the design. The taxonomy for describing adherence to medications now suggests that results from baseline and follow-up can only be compared if the patient was already on treatment at least 3 months prior to baseline [
77]. However, the taxonomy was not in place at the time of data collection. Challenges associated with diagnosis and treatment initiation records in the study settings would also limit application of the taxonomy. Further, the lack of data on clinical methods of measuring adherence—such as a HIV RNA test (a test that checks for RNA genetic material from the virus in a sample of blood) [
78,
79] or CD4 counts (the number of CD4 T lymphocytes – a type of white blood cells -- in a sample of blood, which is used to monitor an individual’s response to ART) [
80]—limited our ability to assess the predictive validity of the SMAQ with these data. This is an important agenda for future research.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.