Our scale development process was comprised of a literature review and qualitative research, delineation of conceptual domains and definitions, item development, quantitative field testing, and psychometric analysis (Wilson,
2006). Building on prior contraceptive research (Raine-Bennett & Rocca,
2015), the work was guided by the Necessity-Concerns Framework from medication adherence research (Horne et al.,
2013). This framework postulates that, when determining whether to initiate or continue using a medication, individuals balance their perceived need for the medication (necessity) with their concerns and the perceived benefits of using it (concerns). Applied to prescription contraception, we view individuals’ willingness to use contraception as being driven not only by desire to avoid pregnancy (necessity), but also consideration of pertinent concerns and benefits about contraception, including its acceptability and worry about safety and side effects (concerns). Our work aims to develop a measure capturing the construct of contraceptive concerns and beliefs.
Qualitative Research
Our first step was to conduct a rigorous literature review and original qualitative research to inform the development and wording of scale items (Muñoz et al.,
2020). Prior to data collection, the key content areas and topic guides were reviewed by the community advisory board of the University of California, San Francisco (UCSF) Preterm Birth Initiative. We then held seven focus group discussions (FGDs, N = 42) and 13 in-depth interviews (IDIs) among 55 sexually active adolescents and adults aged 15–29 years recruited from three reproductive health clinics in Northern California. Participants had to be assigned female at birth due to our interest in beliefs and concerns about prescription contraceptives, which are only available for these individuals. FGDs were scheduled for particular times, and those who could attend were asked to, while IDIs were scheduled for other participants. Trained focus group moderators and interviewers asked participants about their lived experiences with contraception, their likes and dislikes about contraceptive methods, and the impact contraception has had on their bodies and lives. FGDs were age-stratified (adolescents aged 15–19 years, adults aged 20–29); all participants identified as cisgender. Childcare was provided, and participants were remunerated $100 for FGD participation (about 75 min) and $50 for interview participation (about 45 min). We continued data collection until we reached thematic saturation.
One-quarter of qualitative research participants identified as Latinx; 26% as White; 19% as Black; 15% as Asian, Pacific Islander, Hawaiian Native; and 15% as Multiracial or other races. Participants voiced both negative and positive attitudes about contraception from their own experiences and those of their peers (Muñoz et al.,
2020). Of primacy were frustrations with side effects, including moodiness and irregular bleeding, and feelings that contraception is “invasive” or “not natural.” Participants expressed skepticism about long-term safety and fears about future fertility. Views on contraception reached beyond their own bodies and were situated within a social context of family member and community judgement (e.g., “looked down upon”). Participants also verbalized benefits of using contraception across similar domains, including favorable side effects (e.g., “controls the cramping”), the benefits of preventing pregnancy (“peace of mind”), and positive social perceptions (“being responsible”).
Domains and Item Development
We originally adopted a broad working definition of the construct we aimed to measure, conceptualizing “contraceptive acceptability” as a person’s feelings and opinions that they accept to be true about prescription contraceptive methods. Based on the qualitative work and existing literature, we identified seven interrelated draft domains, each comprised of the negative aspects or concerns and positive aspects or benefits. The primary domains referenced safety, health and side effects from contraceptive use, as well as skepticism and trust around the promotion of contraception. Other domains included concerns with the process of obtaining and using contraception and stigma, and benefits including pregnancy prevention and positive connotations of use (Appendix). We did not know at the outset if the construct was unidimensional, if the draft domains themselves would comprise unique psychometric dimensions, or if the negative and positive aspects would fall into separate dimensions.
Based on the conceptual domains, we developed a library of candidate items for the measure, with each item borne directly from our qualitative work or published peer-reviewed qualitative literature. Items were translated into Spanish, and we honed items and their translations based on feedback from ten cognitive interviews with additional individuals from the same patient populations from whom we drew the focus groups and interviews. The final 55 candidate items covered the seven domains; each item was a statement about contraception, to which respondents indicated if they agreed, somewhat agreed, neither agreed nor disagreed, somewhat disagreed, or disagreed. Items were coded so that higher levels of concerns were higher on a 0–4 scale.
Initial psychometric analyses based on the large field test (see below) indicated that our original conceptualization was too broad a construct to measure with a single measurement instrument. We thus refined our construct to focus more narrowly on side effects, health and safety, skepticism and trust and named the scale the Concerns Scale for short. Related aspects of contraceptive acceptability, including concerns with obtaining and using contraception and stigma, were not included.
Quantitative Field Test
We tested the 55 items among patients seeking reproductive care from nine reproductive and primary care health facilities in the San Francisco Bay area between June 13, 2019 and February 26, 2020 (Harper et al.,
2022). Study facilities were primarily Department of Health and non-profit community clinics, including Federally Qualified Health Centers, School-based Health Centers, reproductive health clinics, and an outpatient public hospital obstetrics and gynecology clinic. Trained bilingual research assistants (RAs) approached all individuals in the waiting room and described the study; the RA was English/Spanish bilingual 80% of recruitment time. To participate, patients had to be aged 15–34 years, assigned female at birth, sexually active in the prior six months, and able to read and speak English or Spanish. The RA obtained verbal consent to participate from eligible and interested patients using a tablet. Participants then completed the 30-min anonymous electronic survey on the same device in the waiting room. Contraceptive Concerns items were early in the survey to facilitate their completion prior to the clinical appointment, but the few participants who were called into their appointment could complete the survey afterwards. Participants received $20 cash or gift card upon survey completion. The study was approved by UCSF’s Institutional Review Board and was performed in accordance with the ethical guidelines of the 1964 Declaration of Helsinki.
Psychometric Analysis
We used item response theory (IRT) for analyses (De Boeck & Wilson,
2004; Hays et al.,
2000), supplemented with exploratory factor analysis and classical methods to determine dimensionality and internal consistency (Cronbach,
1990). Our aim was to iteratively reduce the 55 items to a set of 4–6 items, given it is rarely feasible to include long scales in contraceptive research surveys. Our approach balanced creating a scale with high internal consistency (i.e., reliability) while also capturing the full scope of contraceptive concerns (i.e., validity).
To select scale items, we first assessed item completion, removing those with > 5% missing. We examined the distribution of responses to each item to ensure items served to differentiate respondents’ attitudes and removed those with highly skewed responses, as they did little to differentiate respondents’ concerns levels (Edelen & Reeve,
2007).
Using ACER ConQuest 4.5 (Camberwell, Australia), we iteratively fit item responses to unidimensional partial credit item response models and examined item fit, internal structure validity, and differential item functioning, removing less optimally performing items until we arrived at six final items. We considered a weighted mean-squared index of 0.75–1.33 as indicative of good fit to the model (Wright & Masters,
1982). For internal structure, we ensured that for each item, respondents endorsing higher response categories (reflecting greater concerns) had correspondingly higher overall Concerns scores. We also plotted item thresholds relative to attitudes levels (e.g., Wright Maps) to ensure items served to differentiate respondents along the spectrum of attitudes and confirm correct ordering of each item’s category locations.
Once the six items were selected, we fit a final series of models to establish item parameters and the scale’s psychometric properties. We repeated the steps outlined above and assessed internal consistency with the separation reliability coefficient. We fit four new partial credit differential item functioning (DIF) models, each which incorporated item-by-characteristic interaction terms (De Boeck & Wilson,
2004). The characteristics included age, sexual orientation, race and ethnicity, and maternal education level as an indication of socioeconomic status (SES). We used maternal education as a socioeconomic indicator rather than the participants’ educational level because almost half of the sample was adolescent and still pursuing additional education, and many were unlikely to know their household incomes. We considered item-by-characteristic parameter effect sizes of ≥ 0.6 logits as evidence of DIF (Paek,
2002; Steinberg & Thissen,
2006).
Supplementing the IRT analyses, we used exploratory factor analysis to ensure the scale’s items loaded onto a single factor with eigenvalue > 1 (Kline,
1986). We averaged summed raw scores across items, calculating item-total correlations and examining internal consistency (Cronbach’s α).
Finally, as a test of external validity, we investigated the relationship between Concerns scores and current use of a prescription contraceptive method, fitting a Poisson regression model and calculating predicted probabilities of use. We also fit a multivariable model, using multiple imputation with chained equations to account for missing covariable data. We hypothesized that higher Concerns scores would be associated with lower contraceptive use.
Using the final Concerns Scale, we investigated variations by sociodemographic characteristics with bivariable linear regression and fit a multivariable model using multiple imputation for missing data. We used Stata 15 for classical and regression analyses (College Station, TX).