Psychometric Validation of the Perception of Injection (PIN) Questionnaire Using Data From Two Phase III, Open-Label, Active-Controlled, Non-Inferiority Studies in People Living With HIV
verfasst von:
Vasiliki Chounta, Hilary F. Byrnes, Mickael Henry-Szatkowski, Dominy Browning, Christina Donatti, Jeremy Lambert
Currently, there are no patient-reported outcome tools specifically validated for use in people living with human immunodeficiency virus (PLHIV) to measure treatment injection acceptance and experience. The Perception of Injection (PIN) questionnaire was modified with consent from the Vaccinees’ Perception of Injection (VAPI), a validated instrument developed by Sanofi Pasteur. The objective of developing the PIN was to provide information on participant experience with injectable therapies, including acceptance of pain, injection-site reactions, and tolerability following injections in PLHIV.
Methods
This post hoc analysis used data from participants who received the long-acting intramuscular cabotegravir plus rilpivirine combination treatment every 4 weeks, as part of the ATLAS (NCT02951052) and FLAIR (NCT02938520) studies, to evaluate the psychometric properties of the PIN questionnaire.
Results
These findings support the reliability, validity, and responsiveness to change for the PIN questionnaire in PLHIV.
Conclusion
As a clinical trial endpoint, the PIN questionnaire could provide valuable evidence around the acceptance and experience of injections in PLHIV which could have implications for treatment adherence in this population.
Vasiliki Chounta, Mickael Henry-Szatkowski and Jeremy Lambert: affiliation at the time of the study.
Key Summary Points
Why carry out this study?
Currently, there are no tools available to measure the acceptance and experience of injectable therapies for people living with human immunodeficiency virus (PLHIV)
The Perception of Injection (PIN) questionnaire was developed to assess acceptance of pain, injection-site reactions, and tolerability following injections in PLHIV
This post hoc analysis aimed to evaluate the psychometric properties of the PIN questionnaire in PLHIV receiving long-acting intramuscular antiretroviral therapy, using data from two phase III studies
What was learned from the study?
The PIN questionnaire has been validated as an appropriate and reliable tool to provide evidence of experience of injection in PLHIV
These findings support use of the PIN questionnaire as a key endpoint in clinical trials
Introduction
Treatment with combination antiretroviral therapies provides durable viral suppression among people living with human immunodeficiency virus (PLHIV) type 1 [1]. However, lifelong daily oral therapy can be burdensome, which can impact adherence and increase the risk of treatment failure [2]. A two-drug combination regimen comprising intramuscular (IM) injections of cabotegravir (CAB) and rilpivirine (RPV), administered every month or every 2 months, is approved for the treatment of HIV-1 infection in adults [3]. This may offer improved acceptability, adherence, and treatment satisfaction compared with the current standard of care with daily administration of an oral regimen (current antiretroviral therapy) [4]. PLHIV experience with HIV treatment regimens in terms of treatment satisfaction, acceptance, and health-related quality of life (HRQoL) is an important consideration given the need for lifelong treatment [4, 5]. Currently, the available patient-reported outcome (PRO) tools capture HRQoL [HIV/acquired immune deficiency syndrome (AIDS)-Targeted Quality of Life (HAT-QoL)] and treatment satisfaction [HIV Treatment Satisfaction Questionnaire (HIV-TSQ)] among PLHIV; however, PROs that specifically measure acceptance and experience with injectables are lacking for this population. Such PRO tools will be of increasing importance as HIV treatment modalities continue to change.
Anzeige
To fill this gap, the Perception of Injection (PIN) questionnaire was derived from the Vaccinees’ Perception of Injection (VAPI) questionnaire [6] and adapted for gluteal IM injection [7] in PLHIV. The PIN questionnaire was developed to provide crucial information on experience with injectable therapies, including acceptance of pain, injection-site reactions (ISRs), and tolerability following injections in PLHIV. As a next step, it is important to evaluate the PIN’s psychometric properties in a population of PLHIV receiving long-acting IM antiretroviral therapy.
This post hoc analysis aimed to evaluate the psychometric properties of the PIN questionnaire in PLHIV using data from two phase III studies [First Long-Acting Injectable Regimen (FLAIR; NCT02938520) and Antiretroviral Therapy as Long Acting Suppression [ATLAS; NCT02951052)] both of which have previously demonstrated the non-inferiority of a long-acting IM CAB plus RPV combination every 4 weeks compared with a standard of care, daily, oral three-drug regimen in PLHIV [8, 9]. As part of both studies, participants completed multiple questionnaires, including the PIN questionnaire, to assess their acceptability, tolerability, acceptance of pain, and ISRs following injections every 4 weeks [7].
Methods
Study Design and Participants
This was a post hoc statistical analysis conducted using data from participants who received the monthly long-acting injectable combination treatment (CAB + RPV) as part of the FLAIR and ATLAS studies. The study design and eligibility criteria of the FLAIR and ATLAS studies have been previously described [8, 9]. Briefly, FLAIR was a randomized (1:1), multicenter, open-label, non-inferiority study, wherein PLHIV could either continue their current daily oral antiretroviral therapy or switch to long-acting CAB + RPV therapy administered every 4 weeks for 100 weeks. PLHIV were enrolled if they had not previously received antiretroviral therapy and if they had achieved virologic suppression with daily oral antiretroviral therapy during the induction phase of the study [8]. ATLAS was a randomized (1:1), multicenter, parallel-group, open-label study, wherein participants could either continue their current daily oral antiretroviral therapy or switch to long-acting CAB + RPV therapy administered IM into the gluteal muscle every 4 weeks for 52 weeks. Enrolled participants were PLHIV who had been receiving antiretroviral drugs in an uninterrupted regimen without virologic failure and without a change in medication for at least 6 months prior to screening [9].
PRO Measures
As part of the two studies, PLHIV completed several PRO questionnaires at various time points, as described in the Supplementary Material (Fig. S1). The PIN questionnaire was completed at weeks 5, 41, and 48. Respondents were asked to consider the prior week (post-injection) at weeks 5 and 41, and the prior 4 weeks at week 48. Because the PIN questionnaire is injection-specific, and participants had no experience with the injection under investigation at study entry, the questionnaire was not administered at baseline. Pooled data from both the ATLAS and FLAIR trials were used for this post hoc psychometric validation of the PIN questionnaire.
Anzeige
The PIN questionnaire included items that assess participant experience with injections. The PIN measure was derived from the VAPI questionnaire and adapted for PLHIV receiving long-acting CAB + RPV. VAPI was chosen as the model as it had previously been validated and found to be a reliable tool for the assessment of vaccine injections [6]. Additionally, VAPI was designed to evaluate aspects of the participant experience similar to those intended to be explored in PLHIV, meaning that only minor changes were required in the development of the PIN questionnaire. These include replacement of the term ‘vaccination’ with ‘injection’ throughout the questionnaire, a change in item 3 from ‘pain in your arm’ to ‘pain in your buttock’, and changes in items 11 and 15 where ‘lifting your arm’ is replaced by ‘walking’. The PIN questionnaire contains 21 items, grouped into four multi-item domains: ‘Acceptance of ISR’ scale score (2 items); ‘Bother from ISR’ scale score (6 items); ‘Leg movement’ scale score (4 items); and ‘Sleep’ scale score (4 items); and five individual items: pain during injection; anxiety before injection; anxiety after injection; willingness to be injected in the future; and overall satisfaction with mode of administration. Participant responses were scored on a 5-point Likert scale with 1 representing the least favorable perception of injection and 5 the most favorable; domain scores were calculated as a mean of all items in the domain. Other PRO instruments used in analyses are briefly described below.
The 12-item Short Form Health Survey (SF-12) is a measure derived from the Medical Outcomes Study 36-Item Short Form Health Survey, containing the same eight domains that assess general health status and mental health distress, with responses collected on a graded scale or as Yes/No answers [10].
Overall function and well-being were evaluated with the shorter 14-item, 3-dimension version of the HAT-QoL questionnaire, which covers life satisfaction, feelings about medication, and disclosure worries, with participant responses collected on a 5-point scale ranging from ‘All of the time’ to ‘None of the time’ [11].
Treatment satisfaction was evaluated using the 12-item HIV-TSQ status and change versions [HIV-TSQ(s, c)] [12, 13], while treatment acceptance was measured using the 3-item ‘General acceptance’ dimension of the ACCEPT questionnaire, which assesses how participants weigh advantages and disadvantages of long-term medication [14].
Post-injection pain was assessed using a numeric rating scale (NRS), whereby a participant selects a whole number between 0, ‘no pain’ and 10, ‘extreme pain’, that best reflects the intensity of pain at injection.
Assessment of Psychometric Properties
Following the relevant PRO Food and Drug Administration Guidance for Industry [15], the psychometric properties of the PIN were investigated to determine the instrument’s adequacy in terms of reliability, validity, and responsiveness. This included the quality of questionnaire completion (including the extent of missing data), the extent to which the possible range of responses for each item is selected (item-level analysis), the intercorrelation of items that contribute to a score (internal consistency reliability), score stability over time when change is not expected (test–retest reliability), assessment of whether relationships among items, domains and concepts conform to a priori hypotheses (construct validity), and whether changes over time in individuals or groups are reflected in a difference in score (responsiveness/sensitivity to change).
Quality of Completion and Item-Level Analysis
The frequency and percentage of response choices and missing data were determined for each PIN questionnaire item at weeks 5, 41, and 48 of the study. Floor and ceiling effects were explored for each of the PIN questionnaire items by calculating the proportion of participants with the minimum possible score (floor) and the proportion of participants with the maximum possible score (ceiling) at weeks 5, 41, and 48.
Internal Consistency Reliability
Internal consistency reliability reflects the extent to which individual items are consistent with each other in the same dimension and reflect a single underlying concept. Item–total correlations (correlations between each PIN questionnaire item and the total score after omitting the item) were calculated. A minimum coefficient of 0.30 was used as the benchmark to denote a moderate correlation [16]. Internal consistency was investigated by calculating Cronbach’s alpha coefficient for each PIN questionnaire domain at week 5, with a coefficient ≥ 0.80 considered as evidence for strong internal consistency [16], but a coefficient ≥ 0.70 was considered acceptable [17]. Coefficients ≥ 0.90 were flagged as indicating potential redundancy.
Test–Retest Reliability
Test–retest reliability (i.e., the extent to which the questionnaire yields the same scores each time it is administered, all other variables being stable) was evaluated for all PIN items by calculating the intra-class correlation (ICC), using the ICC (3, 1) coefficient as per the definition of Shrout and Fleiss [18]. Test–retest reliability was assessed between weeks 41 and 48 among stable participants, defined as participants who indicated stability over time on both the SF-12 and the HAT-QoL between weeks 24 and 48. Test–retest reliability was interpreted as follows: ICC < 0.50 poor reliability, ICC 0.50–0.74 moderate reliability [19], ICC 0.75–0.89 good reliability [16, 20, 21], and ICC 0.90–1.00 excellent reliability [22].
Construct Validity
Convergent validity (i.e., the extent to which the scores correlate with scores from other PRO instruments measuring similar concepts) was assessed between the PIN questionnaire domain scores and the domain scores of HAT-QoL, SF-12, and ACCEPT at week 48, as well as between the PIN questionnaire domain scores and the score from the post-injection pain NRS at weeks 5 and 41 on an individual participant basis. Correlations were evaluated based on pre-defined ranges: 0.10–0.29 (weak correlation), 0.30–0.49 (moderate correlation), and 0.50–1.00 (strong correlation) [23, 24].
Confirmatory factor analysis (CFA) was performed using week 5 data to summarize how well the variables reflected the hypothesized structure. Model fit was assessed using comparative fit index (CFI; values ≥ 0.90 indicated acceptable fit) [25], standardized root mean residual (SRMR; values < 0.10 were considered acceptable; values < 0.08 were preferable) [26], and root mean square error of approximation (RMSEA) models (values < 0.06 were considered acceptable) [27].
Known-groups validity (i.e., the extent to which the instrument is able to distinguish clinically different groups) was performed to determine the degree to which the PIN questionnaire was able to discriminate between participant severity groups hypothesized a priori to be different. For this, a comparison was made between the PIN item and the domain scores in participants grouped according to pain severity based on the 10-point post-injection pain NRS at weeks 5 and 41. The statistical significance of differences in scores between groups was calculated using one-way analysis of variance.
Responsiveness
Responsiveness (i.e., sensitivity of the questionnaire to detect change over time) was determined by assessing change between weeks 5 and 48 in PIN questionnaire items and domain scores for participants showing change on the HIV-TSQs between baseline and week 44 and for participants showing change on the HIV-TSQc between weeks 5 and 48. Effect sizes were calculated to determine the sensitivity to change over time of the PIN domains. Based on Cohen, effect sizes (d) of ≥ 0.80 were considered large, ≥ 0.50 medium, and ≥ 0.20 small [28]. Responsiveness of the PIN was also assessed with correlation coefficients where values of 0.10–0.29 were classified as weak correlations, 0.30–0.49 as moderate, and 0.50–1.00 as strong correlations according to widely accepted conventions [23, 28].
Data Analysis
The psychometric analysis sample included all participants who provided at least 1 valid response to a PIN questionnaire item at any time point. No imputation or replacement of missing data was performed for the psychometric analyses. Data processing and analysis were conducted by ICON plc using SAS software (SAS Institute, Cary, NC, USA, v.9.4).
Compliance with Ethics Guidelines
This post hoc psychometric analysis was conducted on anonymous data and no direct participant contact or primary collection of individual human participant data occurred. Study results are in tabular form and aggregate analyses that omits participant identification, therefore informed consent or ethics committee or IRB approval were not required.
The PRO data used in this analysis were a secondary endpoint in two clinical trials: ATLAS and FLAIR, both of which were conducted in accordance with the principles founded in the Declaration of Helsinki and with Good Clinical Practice. The ATLAS and FLAIR protocols were approved by an institutional review board or ethics committee of each study site.
Anzeige
All participants provided written informed consent to participate in the clinical trials and any subsequent analysis of the data as it was derived from those clinical trials.
Results
Population
Overall, 283 participants from FLAIR and 308 participants from ATLAS who received treatment were included in the analysis. Full details of the baseline demographics, and clinical characteristics data for the FLAIR and ATLAS populations have been published in detail elsewhere [8, 9]. Briefly, most participants were male (FLAIR, 78%; ATLAS, 68%) and white (FLAIR, 76%; ATLAS, 69%), with a median (range) age of 34 (19–68) years for FLAIR and 40 (21–74) years for ATLAS [8, 9].
Psychometric Analysis
Quality of Completion and Item-Level Analysis
While all four PIN dimensions and the five individual PIN items were examined, a specific focus on the PIN ‘Acceptance of ISR’ subscale is described in detail below. The quality of completion, frequency, and percentage of missing data per item, and the response choice of PIN, were assessed at week 5 (first assessment) followed by weeks 41 and 48 (Supplementary Materials; Table S1). A large proportion of participants completed the PIN questionnaire domains at weeks 5, 41, and 48, with no missing data at week 48 and very low rates of missing data (from 0 to less than 1%) at weeks 5 and 41. Item level analysis examined the distribution of item responses selected by the participants, as well as evaluation of floor and ceiling effects for each item. Floor effects (≥ 20% of participants reporting the best/minimal possible score) were found across all PIN items at weeks 5, 41, and 48, except for the week 5 ‘bothered by pain in buttock’ score. For this item only, 14.5% of participants responded ‘not at all’ and 8.3% responded ‘extremely’, and this was further reduced to 0.2% and 0.4% by weeks 41 and 48, respectively. No ceiling effects were found. The ‘Bother from ISR’ domain score was the lowest mean [standard deviation (SD)] domain score at weeks 5 [1.60 (0.56)], 41, and 48 [1.40 (0.42) and 1.41 (0.45), respectively], with lower scores indicating minimal/no bother by injection (Table 1). In the ‘Acceptance of ISR’ domain of the PIN, the majority of participants reported that pain and ISRs were ‘very acceptable’ or ‘totally acceptable’ after their first CAB + RPV long-acting injection at week 5, with a mean (SD) score of 2.10 (1.04).
Table 1
Mean PIN questionnaire domain scores at weeks 5, 41, and 48
PIN questionnaire domain score
Week 5 (n = 503)a
Week 41 (n = 439)a
Week 48 (n = 532)a
Bother from injection-site reaction score, mean (SD)
1.60 (0.56)
1.40 (0.42)
1.41 (0.45)
Sleep scale score, mean (SD)
2.08 (1.02)
1.41 (0.60)
1.45 (0.65)
Acceptance scale score, mean (SD)
2.10 (1.04)
1.57 (0.74)b
1.56 (0.71)
Leg movement scale score, mean (SD)
2.08 (1.05)
1.42 (0.60)
1.41 (0.61)
aReported n represents the number of participants with PIN questionnaire data available at the given time point
bn = 438
The score of a domain is calculated as the mean of all items of the domain. Domain scores range from 1 to 5, with higher scores representing worse perception of injection
PIN Perception of Injection, SD standard deviation
Internal Consistency Reliability
The assessment of the PIN’s internal consistency reliability and homogeneity found generally strong correlations between single PIN questionnaire items and the item–total correlations within each domain, and exceeded the pre-defined acceptable threshold of 0.30 for all items. Cronbach’s alpha coefficient for the overall PIN was 0.92, with values for the four domain scores ranging from 0.80 to 0.92, indicating strong internal consistency reliability; however, values above 0.90 may indicate an overly homogenous measure with likely item redundancy. The acceptance and bother of ISR scale demonstrated good internal consistency, with Cronbach’s alpha coefficient of 0.85 and 0.80, respectively. Both the ‘Leg movement’ and ‘Sleep’ domains had a Cronbach’s alpha coefficient > 0.90, which may indicate redundancy. Overall, and for each domain, omission of each item had little effect on Cronbach’s alpha coefficients, suggesting that all items contribute similarly to the domain score.
Test–Retest Reliability
The number of participants meeting the criteria for stability was low, and expanding the definition of stability did not substantially increase the number of ‘stable’ participants. Therefore, the test–retest reliability analysis was conducted for the total population between weeks 41 and 48 (n = 428 for items 1–16 and 21; n = 427 for items 17–20). All ICCs for individual PIN questionnaire items were relatively low (range 0.35–0.55) indicating poor to moderate reliability, with item-level ICC considered poor if < 0.50, moderate if 0.50–0.74, and excellent if 0.90–1.00.
Validity
Convergent Validity
Correlations between all PIN questionnaire domains and the post-injection pain NRS were strong (> 0.50) at week 5 and moderate to strong (0.43–0.61) at week 41 (Table 2). However, correlations between the PIN domains ‘Bother from ISR’ and ‘Acceptance of ISRs’ with the ACCEPT score were weak at week 48 (< 0.30). Correlations were moderate (> 0.30) between all PIN questionnaire domains and the HAT-QoL ‘HIV Medication’ domain at week 48, except for the ‘Sleep’ domain, for which the correlation was weak (– 0.27, p < 0.0001) (Table 2). These results provide evidence for convergent validity of the PIN.
Table 2
Correlation between PIN questionnaire domains and HAT-QoL, SF-12, and ACCEPT scores at week 48 (n = 522), and post-injection pain NRS at weeks 5 (n = 527) and 41 (n = 226)
PIN questionnaire domain score
HAT-QoL
SF-12
ACCEPT
Post-injection pain NRS (week 5)
Post-injection pain NRS (week 41)
Disclosure Worries domain
Life Satisfaction domain
HIV Medication domain
PCS
MCS
Total score
Bother from injection-site reaction score, correlation, p value
− 0.20
p < 0.0001
− 0.22
p < 0.0001
− 0.32
p < 0.0001
− 0.14
p = 0.0014
− 0.33
p < 0.0001
− 0.12
p = 0.0074
0.54
p < 0.0001
0.61
p < 0.0001
Sleep scale score, correlation, p value
− 0.11
p = 0.0151
− 0.23
p < 0.0001
− 0.27
p < 0.0001
− 0.15
p = 0.0007
− 0.30
p < 0.0001
− 0.10
p = 0.0267
0.64
p < 0.0001
0.43
p < 0.0001
Acceptance scale score, correlation, p value
− 0.19
p < 0.0001
− 0.32
p < 0.0001
− 0.33
p < 0.0001
− 0.16
p = 0.0004
− 0.38
p < 0.0001
− 0.20
p < 0.0001
0.60
p < 0.0001
0.55
p < 0.0001
Leg movement scale score, correlation, p value
− 0.19
p < 0.0001
− 0.29
p < 0.0001
− 0.31
p < 0.0001
− 0.16
p = 0.0004
− 0.34
p < 0.0001
− 0.12
p = 0.0091
0.61
p < 0.0001
0.48
p < 0.0001
Reported n represent the number of participants with PRO data available at the given time point enabling the correlations to be calculated
Correlation coefficients ranging from 0.10 to 0.29 were classed as weak correlations, from 0.30 to 0.49 as moderate correlations and from 0.50 to 1.00 as strong correlation
AIDS acquired immune deficiency syndrome, HAT-QoL HIV/AIDS targeted quality of life, HIV human immunodeficiency virus, MCS Mental Component Summary, NRS numeric rating scale, PCS Physical Component Summary, PIN Perception of Injection, PRO patient-reported outcome, SF-12 12-item Short Form Health Survey
Similarly, moderate correlations were seen between all PIN questionnaire domains and the SF-12 Mental Component Summary at week 48 (Table 2). Correlations were weak between the PIN questionnaire domain scores and the HAT-QoL ‘Disclosure Worries’ domain, the HAT-QoL ‘Life Satisfaction’ domain (except for PIN questionnaire Acceptance scale score, which showed a moderate correlation) and the SF-12 Physical Component Summary domain at week 48 (Table 2), indicating evidence for discriminant validity.
Confirmatory Factor Analysis
CFA using week 5 data found that the final model fit the data marginally well: CFI: 0.94, SRMR: 0.04, RMSEA: 0.09 [90% confidence interval (CI) 0.09–0.10] p < 0.0001, chi-square: 466.13, p < 0.0001 (Fig. 1).
Fig. 1
Model fit based on the hypothesized structure. AGFI adjusted goodness of fit index, CFI comparative fit index, DF degrees of freedom, faccept factor for the acceptance of ISR scale, fbother factor for the bother from ISR scale, fleg factor for the leg movement scale, fsleep factor for the sleep scale, ISR injection-site reaction, LL lower limit, PIN Perception of Injection, Pr probability, RMSEA root mean squared error of approximation, SRMR standardized root mean square residual, UL upper limit
×
Exploratory factor analysis found that PIN questionnaire item 3 (How bothered were you by pain at your buttock?) may load more strongly on the ‘Acceptance of ISRs’ domain than on the ‘Bother from ISR’ domain. However, item 3 did not load strongly on either factor, so two alternative models were examined: a 4-factor model in which item 3 loaded on the ‘Acceptance of ISRs’ domain [CFI: 0.94; SRMR: 0.05; RMSEA: 0.09 (90% CI 0.08–0.10); chi-square: 449.92]; 4-factor model with item 3 removed [CFI: 0.94; SRMR: 0.04; RMSEA: 0.10 (90% CI 0.09–0.11); chi-square: 428.56]. Overall, there was no clear advantage of any alternative model over the original.
Known-Groups Validity
Known-groups validity assessment demonstrated generally acceptable validity of the PIN questionnaire domains at weeks 5 and 41 based on post-injection pain (Fig. 2). Indeed, at both time points, higher (worse) PIN scores were reported for participants reporting greater levels of post-injection pain on the NRS. Similarly, acceptable validity of the PIN questionnaire items was demonstrated at weeks 5 and 41 based on post-injection pain, with higher items scores reported for participants reporting higher post-injection pain.
Fig. 2
PIN questionnaire score distributions by NRS pain severity group at weeks 5 (a) and 41 (b). Pain severity categories were updated based on low numbers in the ‘severe to extreme pain group’. *p < 0.0001 for between group comparisons (Kruskal–Wallis). Known-groups validity: PIN items and domain scores per pain severity groups defined using the 10-point post-injection pain NRS at weeks 5 and 41: 0–1 no/very mild pain; 2–4 mild to moderate pain; 5–7 moderate to severe pain; 8–10 severe to extreme pain. NRS numeric rating scale, PIN Perception of Injection, SD standard deviation
×
Responsiveness
In the ability to detect change analysis, effect sizes (0.50–0.79) were classified as medium, except for the PIN ‘Bother from ISR’ domain, which had a small effect size (0.38), indicating an overall moderate responsiveness of the PIN questionnaire (Table 3). Weak correlations (− 0.10) were observed between change in PIN acceptance and change in HIV-TSQs total score between baseline and week 44. Few statistically significant correlations were seen between changes in PIN domain scores and change in HIV-TSQs (i.e., ‘Leg movement’ domain score from weeks 5 to 48 and change in HIV-TSQs total score between baseline and week 44 (p = 0.0353), and between change in ‘Bother from ISR’ domain from weeks 5 to 48 and change in HIV-TSQc total score at week 48 (p = 0.0048; Table 3). All the remaining correlations between change in the PIN domain score and change in the HIV-TSQs were classified as weak (− 0.06 to − 0.14). Overall, effect sizes were medium (with exception of ‘Bother from ISR’, small effect size) despite correlations between change in the PIN domain scores and change in both the HIV-TSQs and the HIV-TSQc being weak.
Table 3
Correlation between the change in PIN questionnaire domain scores and the change in HIV-TSQs total scores and HIV-TSQc total scores
Change in PIN questionnaire domain score between weeks 5 and 48
Change in HIV-TSQs total score between baseline and week 44
HIV-TSQc total score at week 48
Effect size
Acceptance scale score, correlation, p value
− 0.10
p = 0.1123
0.12
p = 0.0730
0.55
Bother from injection-site reaction scale score, correlation, p value
− 0.12
p = 0.0653
0.18
p = 0.0048
0.38
Leg movement scale score, correlation, p value
− 0.14
p = 0.0353
0.08
p = 0.2464
0.75
Sleep scale score, correlation, p value
− 0.06
p = 0.3728
0.11
p = 0.0897
0.71
Correlation coefficient values ranging from 0.10 to 0.29 were classed as weak correlations, from 0.30 to 0.49 as moderate, and from 0.50 to 1.00 as strong correlations. An effect size of 0.80 or more is considered large, 0.50 medium and 0.20 as small
As summarized in Table 4, this post hoc analysis provides support for the psychometric properties of the PIN questionnaire in a population of PLHIV treated with an IM injected two-drug combination therapy (long-acting CAB + RPV) in two phase III trials [8, 9]. Item-level analysis indicated floor effects for most PIN items, where participants reported little to no bother and few, if any, symptoms associated with the injections, hence the quality of completion can be considered moderate. In assessing the PIN’s internal consistency reliability, item–total correlations were generally strong and indicated high item homogeneity, with an overall Cronbach’s alpha coefficient of 0.92 and values for domain scores ranging from 0.80 to 0.92. These values indicate strong internal consistency; however, the overall values above 0.90 may indicate high item homogeneity [29], suggesting that some items can be removed to reduce redundancy without impacting validity of the questionnaire. In particular, the Leg movement and Sleep scale showed a strong internal consistency with Cronbach’s alpha of 0.91 and 0.92, respectively, potentially indicating item redundancy. Similarly, Cronbach’s alpha with each item omitted ranged between 0.88 and 0.89 for Leg movement scale and 0.89–0.90 for Sleep scale, suggesting that all items contribute similarly to the scale. The strong internal consistency reliability and homogeneity found in this study, and therefore likely item redundancy of some PIN items, could be further explored to reduce redundancy and/or respondent burden in future analysis.
Table 4
Summary of psychometric evidence for the PIN questionnaire
In assessing construct validity for the PIN in terms of the correlations between the domain scores of PIN and the domain scores of other PRO instruments, the PIN demonstrated moderate correlations with other relevant clinical outcome assessment (COA) measures defined a priori. In addition, construct validity was also assessed for the PIN in terms of its ability to differentiate between different populations when a group difference is expected (known-groups validity). Convergent/discriminant validity were generally as expected; correlations between all PIN questionnaire domains and the post-injection pain NRS were strong or moderate. There was strong support for the instrument’s known-groups validity: at both time points (weeks 5 and 41), the PIN was able to differentiate between severity groups as defined by levels of post-injection pain on the NRS. Correlations between the PIN domains ‘Bother from ISR’ and ‘Acceptance of ISRs’ and the ACCEPT score were found to be weak, contrary to expectations. It is generally assumed that individuals who perceive injections negatively are less likely to be satisfied with the treatment, compared with those who perceive injections more positively. However, since PROs measuring general acceptance and satisfaction with therapy are multifactorial and explore several parameters of treatment (such as convenience, flexibility, and lifestyle fit), there are probably confounding factors influencing correlations between PIN and those instruments, requiring further work to draw firm conclusions. Correlations were also weak between the PIN questionnaire domain scores and the SF-12 Physical Component Summary, the HAT-QoL ‘Disclosure Worries’ domain and the HAT-QoL ‘Life Satisfaction’ domain as expected, indicating discriminant validity. Overall, both convergent/discriminant validity and known-groups analyses provide moderate to strong validity evidence for the PIN.
Anzeige
Responsiveness of the PIN was moderately demonstrated, and effect sizes for the PIN domains were mainly medium.
In terms of limitations, the design of the phase III studies was not optimal for use in a psychometric validation for several reasons. Specifically, the 7-week period between assessments was much longer than the typical 4-week period to allow assessment of stability over time, and hence test–retest reliability was not applicable. Additionally, the length of time between medication administrations, and the fact that PRO instruments were not administered at the same time as the PIN questionnaire, were not optimal for evaluating responsiveness. As a result, CFA showed moderate support for the instrument’s domain structure, and further investigation may be needed to explore alternative scale structures and scoring. Also, test–retest reliability analysis demonstrated relatively low ICCs for individual PIN questionnaire items, indicating moderate to poor reliability, and the responsiveness could only be partially explored. While our study provides a preliminary indication on the responsiveness and test–retest reliability of the PIN questionnaire, future studies should be conducted to further confirm the PIN performance. Generalizability of the results to PLHIV with virologic failure may also be limited. However, the participant experience of injections, particularly pain, may not be influenced by virologic status, and this participant group (with virological suppression) may be more sensitive to dimensions captured by the PIN given that the switch to long-acting CAB + RPV was a choice, which would not be the case for participants with virological failure, for whom virologic suppression, safety, and efficacy would remain of greatest importance.
The strengths of this study were in the use of the merged ATLAS and FLAIR trial datasets, which provided a relatively large sample size based on a diverse population in terms of age, education, and years since HIV onset, allowing for planned psychometric tests and robust results. Additionally, a wide range of COA instruments included in the dataset could be used as reference measures for assessing various psychometric properties, and, since the data were drawn from interventional clinical trials, sensitivity to change over time could be tested over the duration of the study.
Conclusion
This analysis points towards the reliability, validity, and responsiveness to change for the PIN questionnaire in PLHIV. As such, the PIN questionnaire may provide valuable evidence as a key endpoint in pivotal trials, capable of assessing acceptance of pain, ISRs, and tolerability following injections in PLHIV, which could have important implications for treatment adherence in this population.
Anzeige
Acknowledgements
The authors would like to thank Nicolas Van de Velde, a former employee of ViiV, for his contributions to the study. We also thank the participants of the ATLAS and FLAIR studies.
Medical Writing/Editorial Assistance
Medical writing support was provided by Tatjana Lalic, PhD, and Anna Dawe, MSc, of Fishawack Indicia Ltd, UK, part of Fishawack Health, and was funded by ViiV.
Declarations
Conflict of Interest
The PIN Questionnaire is a modified version of the VAPI Sanofi Pasteur, 2009 All Rights Reserved. VAPI contact information and permission to use: Mapi Research Trust, Lyon, France. Email: PROinformation@mapi-trust.org—internet: www.mapi-trust.org. Dominy Browning and Christina Donatti are employees of ViiV Healthcare and may own stocks/shares in GSK. Vasiliki Chounta (current affiliation: none) was an employee of ViiV Healthcare at the time of the study and may own stocks/shares in GSK. Jeremy Lambert (current affiliation: UCB Pharma, Colombes, France), Mickael Henry-Szatkowski (current affiliation: IQVIA, Patient Centered Solution, Paris, France), and Hilary F. Byrnes were employees at the time the study was conducted of ICON plc which was paid to conduct the project.
Ethical Approval
This post hoc psychometric analysis was conducted on anonymous data and no direct participant contact or primary collection of individual human participant data occurred. Study results are in tabular form and aggregate analyses that omit participant identification, therefore informed consent or ethics committee or IRB approval were not required. The PRO data used in this analysis were a secondary endpoint in two clinical trials: ATLAS and FLAIR, both of which were conducted in accordance with the principles founded in the Declaration of Helsinki and with Good Clinical Practice. The ATLAS and FLAIR protocols were approved by an institutional review board or ethics committee of each study site. All participants provided written informed consent to participate in the clinical trials and any subsequent analysis of the data as it was derived from those clinical trials.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
Mit e.Med Innere Medizin erhalten Sie Zugang zu CME-Fortbildungen des Fachgebietes Innere Medizin, den Premium-Inhalten der internistischen Fachzeitschriften, inklusive einer gedruckten internistischen Zeitschrift Ihrer Wahl.
Mit e.Med Allgemeinmedizin erhalten Sie Zugang zu allen CME-Fortbildungen und Premium-Inhalten der allgemeinmedizinischen Zeitschriften, inklusive einer gedruckten Allgemeinmedizin-Zeitschrift Ihrer Wahl.
Psychometric Validation of the Perception of Injection (PIN) Questionnaire Using Data From Two Phase III, Open-Label, Active-Controlled, Non-Inferiority Studies in People Living With HIV
verfasst von
Vasiliki Chounta Hilary F. Byrnes Mickael Henry-Szatkowski Dominy Browning Christina Donatti Jeremy Lambert
Die Datenlage zur Wirksamkeit von Verbänden oder topischen Mitteln zur Prävention von Druckgeschwüren sei schlecht, so die Verfasser einer aktuellen Cochrane-Studie. Letztlich bleibe es unsicher, ob solche Maßnahmen den Betroffenen nutzen oder schaden.
Die Milch machts – sie bietet Frauen nach Daten einer großen Ernährungsanalyse den besten Darmkrebsschutz aller Lebensmittel, was am hohen Kalziumgehalt liegen dürfte. Am anderen Ende des Spektrums steht der Alkoholkonsum: Das Glas Wein am Abend ist eher ungünstig.
Auch niedrig dosierte Glukokortikoide zur Behandlung einer primären Glomerulopathie lassen offenbar die Infektionsgefahr steigen. In einer US-Studie hing das Risiko vor allem mit der kombinierten Anwendung von Immunsuppressiva zusammen.
Mit dem Einsatz künstlicher Intelligenz lässt sich die Detektionsrate im Mammografiescreening offenbar deutlich steigern. Mehr unnötige Zusatzuntersuchungen sind laut der Studie aus Deutschland nicht zu befürchten.
In diesem CME-Kurs können Sie Ihr Wissen zur EKG-Befundung anhand von zwölf Video-Tutorials auffrischen und 10 CME-Punkte sammeln. Praxisnah, relevant und mit vielen Tipps & Tricks vom Profi.
