Validation of the HIV Pre-exposure Prophylaxis Stigma Scale: Performance of Likert and Semantic Differential Scale Versions

The use of oral emtricitabine/tenofovir disoproxil fumarate for HIV pre-exposure prophylaxis (PrEP) for men who have sex with men (MSM) has high clinical trial efficacy, providing over 90% protection when the medication is used [1, 2]. Mathematical modeling indicates the promise of PrEP, with one model finding that half of all new HIV infections over a ten-year period could be averted in a population of MSM in the United States if 60% of persons indicated for PrEP were taking the medication with high adherence [3]. Encouragingly, the population-level impact of PrEP is now visible in some settings. In New South Wales, Australia, there was a population-wide 25% decline in new HIV diagnoses after PrEP was made widely and freely available [4].

In the United States, estimates range from 12,000–200,000 individuals taking PrEP in 2017 [5, 6]. Recent gains in increasing PrEP prescriptions may be leveling off, [5] far short of the 1.1 Mio. persons the Centers for Disease Control and Prevention (CDC) estimates are indicated for PrEP [6]. Progress in scaling up PrEP has not been uniform. One metric, the PrEP-to-need ratio, describes disparities in PrEP scale-up by comparing the number of new PrEP prescriptions relative to the number of new HIV diagnoses; the metric identified women, younger persons, and the Southern US region as having low PrEP-to-need ratios [5, 7]. Structural barriers contribute to these disparities; for instance, counties with higher concentrations of residents living in poverty are less likely to have a PrEP-prescribing clinic [9]. Social factors have also been identified as key components limiting PrEP uptake, including relationships with partners, family, and HIV and PrEP stigma [8, 9].

Stigma has been seminally defined by Goffman as a discrediting attribute that leads to “a whole and usual person” to be considered “tainted (and) discounted [10].”Two recent reviews and one meta-analysis identified the stigmatization of PrEP use (hereafter referred to as PrEP stigma) as a barrier to PrEP scale-up [11‐13]. Individuals that endorsed beliefs stigmatizing PrEP, such as it being “for people who are promiscuous,” were less likely to be interested in taking PrEP [14]. Numerous qualitative studies have identified stigma as a major barrier to accessing and staying on PrEP [8, 15‐19].

To best understand PrEP stigma and the ways in which it impacts PrEP decision-making, standardized and validated measurement tools are needed. Past quantitative measurements of PrEP stigma have predominantly consisted of individual belief statements, with each belief statement then assessed for association with PrEP-related outcomes such as willingness to take PrEP [14, 20, 21]. Such items are diverse in content and wording, as well as response format, with some using semantic differential options (unpleasant ….. pleasant) [22] and others using Likert scale options (strongly disagree, disagree, neutral, agree, strongly agree). With known differences in participant preferences, time and cognitive burden, and impact on psychometric properties, [23‐31] response option types used in items and scales assessing PrEP stigma are also important to consider.

Given that stigma may be a substantial barrier to the expanded use of PrEP, there is a need for a validated stigma scale. Based on a stigma theory [32] and with attention to an optimal response option strategy, we developed and evaluated a brief measure of PrEP Stigma- the HIV PrEP Stigma Scale (HPSS).

From June 8th 2018 to June 14th 2018, 279 participants were enrolled in the study, and their demographic data are presented in Table 1, stratified by the randomized order in which they received the two scale versions. Participants were from 40 different states across the United States, with median 4 participants per state and range 1 to 21 (Supplement 3). Participants were predominantly white (85%), with fewer identifying as Latino (8%), Black (2%), Asian (2%), or multiracial/other (4%). The majority identified as gay (80%). Participants were 13% aged 18–24, 15% aged 25–34, 18% aged 35–49, and 53% aged 50 and above. Over half had college education or higher (61%), and earned greater than $50,000 annually (57%). A minority of participants (9%) were currently taking PrEP.

Factor analysis of each scale indicated unidimensionality, based on scree plots of Eigenvalues (Supplement 4). Each scale had a single item with a factor loading below 0.4, indicating poor item fit; these two items were removed from all subsequent analyses (Table 2). All remaining items had factor loadings and item-rest correlations above the commonly used threshold of 0.4 [47]. Cronbach’s alpha values indicated high internal consistency, with Semantic Differential 0.88 and Likert 0.82.

Overall, the sample reported low- to moderate-levels of PrEP stigma (Table 2). Scales were coded such that higher values equate to higher levels of stigma. Most participants completed at least half of Semantic Differential items (200/279, 72%), but a greater proportion completed at least half of Likert items (264/279, 95%). Randomization order did not significantly influence rates of item completion; for full detail regarding missingness by item, see Supplements 5 and 6. The overall mean Semantic Differential score was 2.97 on a 7-point scale, approximately 15% (1-point) below the midpoint of the range. The overall Likert scale mean was 2.59 on a 5-point scale, equating to a value slightly closer to neutral (3) than to disagree (2), approximately 8% (0.41/5.0 scale points) below the midpoint.

Figure 1 shows the percent distribution of participants’ responses to Likert scale items. Notably, for the majority of items, less than 15% of individuals had responses categorized as stigmatizing. Conversely, many agreed that individuals on PrEP receive praise (65%), are taking care of their health (89%), and that their friends would be supportive (69%). In fact, half would feel proud to take PrEP. Yet substantial minorities held stigmatizing views: being ashamed to take PrEP in front of others (13%) or anticipating problems with sex partners (17%), being seen by others as slutty (21%) or negative judgment (31%). The mixed views regarding PrEP stigma is perhaps best demonstrated by neutral being the most commonly-selected scale category for stigmatizing statements.

The Semantic Differential HPSS was correlated with 2 out of 6 hypothesized variables: willingness to be on PrEP and perceived PrEP effectiveness (Table 3). Both relationships were correlated in the expected direction. The strength of these relationships, based on Cohen’s guidance for interpreting correlation coefficients, was small [48]. The Likert HPSS was correlated with 5 out of 6 hypothesized variables, each in the expected direction. Mistrust in the health care system was correlated with Likert HPSS responses in the expected direction, but was not statistically significant. The strength of the statistically significant correlations with hypothesized variables was small for 1, moderate for 3, and strong for 1. The scale accounted for 25% of the overall variance for the strongly correlated construct, willingness to be on PrEP. A sensitivity analysis in Supplement 7 varied missing data criteria assumptions for Table 3, but identified no substantial differences.

We assessed whether the Likert HPSS correlated with demographic or other sexual behavior variables and found few associations (Table 4). In unadjusted models, sexual orientation, PrEP use (current or past), and recent STI diagnosis were associated with PrEP stigma. In adjusted models, the same set of variables and income were associated with PrEP stigma. Participants who reported PrEP use had lower PrEP stigma scores by 0.52 and 0.53 points, respectively, compared to those who did not.

We developed and validated a PrEP stigma scale, the HPSS. In an assessment of Semantic Differential and Likert HPSS versions, both demonstrated face validity by covering key domains in the literature, indicated internal consistency through high Cronbach’s alpha values, and had unidimensionality. The Likert HPSS substantially outperformed the Semantic Differential scale in terms of construct validity, correlating with 5 out of 6 variables hypothesized to be associated with PrEP stigma. It also performed better in terms of completion rates, likely indicating higher acceptability. Of particular relevance to public health programs, the scale accounted for 25% of the variance in participants’ willingness to be on PrEP and 17% of the variance in participants’ perceptions of community attitudes towards PrEP. The strength of these associations indicates that further research regarding PrEP stigma is merited, particularly to better understand the relation between PrEP stigma and willingness to initiate PrEP. This call for further research to develop stigma reduction interventions has been echoed in several recent reviews of PrEP stigma [11‐13].

Overall, PrEP stigma was low-moderate in the sample, with the average median HPSS item response slightly closer to neutral than to disagree for the Likert responses, in which higher values indicated more stigma. A slight majority of participants indicated they would be proud to take PrEP, and strong majorities anticipated support from their friends and families for taking PrEP. Yet a minority of participants anticipated negative consequences, such as being perceived negatively by doctors, experiencing negative judgements, or having problems with sex partners. Despite the relatively moderate levels of overt PrEP stigma, having higher stigma score was strongly associated with lower willingness to take PrEP. Similarly, in a recent study, believing that ‘PrEP is for promiscuous people’ was found to be associated with lower interest in PrEP among black and white MSM in the Southeastern US [14].

The majority of previously published work identified no difference between performance of Semantic Differential and Likert scales, [25‐30] yet some studies found Semantic labeling increased performance [23, 24, 30] There are several possible reasons that the Likert scale in our study items outperformed the Semantic Differential scale. First, it was difficult to translate more complex logic of Likert items into the simplified Semantic Differential format we were using. Second, we selected a transformation of items from Likert to Semantic that required greater changes to item wording than some prior studies. These prior studies used nearly identical wording for both scales, changing only the response format. We opted for a more condensed Semantic scale, using a single stem for all items (People taking PrEP are …). Higher completion rates for Likert items relative to Semantic Differential items may indicate that such items are more acceptable or interpretable, which conforms to previous findings that participants prefer Likert scale formats [31]. Last, it is possible that the performance across different response formats depends on the specific domains of assessment. Regardless of the reason for the difference, the substantial performance difference identified in this study argues for more use of head-to-head comparisons of scale response formatting.

HPSS was designed based on a stigma framework with three domains (internal, anticipated, and experienced stigma) and on three attributes abstracted from PrEP stigma literature (shame, character judgments, social support), yet our factor analysis revealed a unidimensional scale. The three dimension stigma framework has been validated for HIV stigma for people living with HIV [49, 50] and for substance abuse stigma for people who have substance abuse histories [51]. There are several notable differences between this study and past research. PrEP is a protective behavior, rather than a disease or disorder, potentially influencing scale dimensionality. The HPSS was designed to accommodate both those using and not using PrEP, so responses may be formulated based on various sources: personal experience on PrEP, observed experiences of others on PrEP, discussion among friends/community, online information/discussions, or even supposition. In a sample solely consisting of individuals possessing the stigmatizing trait (PrEP users), responses would likely be primarily based on personal experience and might be multidimensional. Our sample did not include a sufficient number of PrEP users (n = 25) to perform a separate validation with that group. Future work with PrEP users should be conducted to determine if unique subscales emerge and relate to other important outcomes. Regardless of latent constructs, using the stigma framework to structure development of the scale held substantial value by facilitating content validity: ensuring that our items covered diverse topics from different vantages. We found utility for each scale item, with item-rest correlations well above a threshold indicating irrelevance (0.3) and well below a threshold indicating excessive overlap with other items (0.9).

A recently published study among MSM in Chicago developed a PrEP stigma scale and found that Black participants and participants in geographic areas with higher concentrations of HIV incidence had higher levels of PrEP stigma [52]. The scale developed for the Chicago study was published after data collection finished for the present study. There are, however, some key benefits of the scale presented in our study. Most importantly, our study thoroughly assessed construct validity and determined the scale to perform well based on that assessment; the prior scale had no stated assessment of validity. The current study also assessed scale performance across two versions, with findings leading to identification of a scale that substantially outperformed its competitor. It is important to note that there was considerable overlap in item topics across the two scales, including promiscuity, responsibility, and daily use, indicating face validity for each instrument. Future studies may be useful to further assess the relative performance of the two scales.

This study has a number of limitations. First, the sample was majority White and older, and only included MSM; the scale may not perform similarly in other settings. This concern is partially mitigated by the diverse source of studies that provided items that comprise the scale. These studies totaled over 1500 participants, and represented substantial diversity across age, gender, sexual orientation, race, and nationality. We anticipate that the scale will likely perform well across diverse settings, however, future research is needed to provide additional validation across other groups and settings. We are currently investigating how the scale will perform in a study that targets recruitment of over 190 Black MSM in urban areas, [53] and also in a study that targets recruitment of over 120 Latino MSM or Black MSM in rural areas [54]. It is important to note that for the present study, although we did not observe significant differences in mean scale scores by race/ethnicity, we did not have sufficient sample size to adequately assess potential areas of difference. Last, the sample size did not allow for conduct of confirmatory factor analysis: a future assessment should consider confirmatory factor analysis of the scale.

We developed and validated a PrEP stigma scale that is strongly correlated with willingness to be on PrEP. Having a uniform and validated measurement tool, such as the one presented here, can facilitate improved assessment of the impact of PrEP stigma on PrEP initiation and maintenance in care. Moreover, such measurement can help track trends across populations. Goffman notes that, “an attribute that stigmatizes one type of possessor can confirm the usualness of another … stigma then is a relationship between attribute and stereotype.” [10] PrEP is an excellent example of this: PrEP use (attribute) is stigmatized in some communities (negative stereotype), but considered beneficial (positive stereotype) in others [15]. Ongoing assessment of PrEP stigma may help us better understand its influence on PrEP adoption, a potentially vital step in bringing PrEP to scale to dramatically reduce new HIV infections.