Rationale and design of the validation of bladder health instrument for evaluation in women (VIEW) protocol

The Validation of Bladder Health Instrument for Evaluation in Women (VIEW) study is an IRB approved (ADVARRA #Pro00032238), multicenter study funded by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (NCT04016298), designed to test a questionnaire that can measure the degree of bladder health in epidemiologic research conducted across different populations and recruitment strategies as well as for use in local community intervention studies. The VIEW study builds upon the internal validation of the item pool for the instrument by evaluating dimensional validity and internal reliability and further establishes external validity. The goal is to create a Bladder Health Scale (PLUS-BHS) to establish basic inferential validity and to draw valid inferences about the distribution of bladder health in women in the general population and also in clinical research. Therefore, it is essential that the populations represented in the validation samples have some equivalence to the target populations the PLUS-BHI is intended to be used in [11]. The VIEW study protocol includes three population samples: a sample from a nationally representative general population and two locally recruited clinical center samples—women with a targeted range of symptom severity and type, and a postpartum group. Postpartum women are a focal population identified for future research due to the known high risk of developing LUTS in the peripartum period and evidence for successful LUTS prevention strategies [12, 13]. Recalling peripartum LUTS is inaccurate when assessed remote from the delivery, resulting in high recall bias and measurement error [14]. Adequate representation of this important population is critical to establish confidence that the PLUS-BHS will allow valid inference to be made by future studies of this population.

Effect of mode administration is a well-documented phenomenon in survey research [15, 16]. In addition to context-based effects associated with mode of survey administration [17‐20], other potential biases or sources of error associated with mode of administration are unknown. While a computer mode may increase efficiency of questionnaire completion, it also may produce measurement error. Thus, use of a computer may alter the distribution of PLUS-BHS [21, 22]. We intentionally include both paper and electronic modes in VIEW in order for the PLUS-BHS construction resulting from this validation process to be mode agnostic.

Measuring degrees of bladder health in a variety of settings requires a valid instrument, ideally self-administered, that can capture a spectrum from very healthy to very unhealthy. The PLUS Bladder Health Instrument (PLUS-BHI) item pool was generated using expert opinion, review and adaptation of existing instruments, and drafting of novel items to assess the elements of our conceptual model of bladder health. The items and language were refined by cognitive evaluation with community dwelling women and review of focus group data [10].

Inclusion criteria for the general and clinical center populations are outlined in Table 1. In general, non-pregnant, ambulatory women 18 yrs of age or older are targeted for enrollment.

The intended use of the validated PLUS-BHS is a self-administered questionnaire for women in general population research as well as in clinical research. Given the differing sampling approaches typically used for these different types of research, our goal is to ensure the PLUS-BHS is valid for use across two likely modes of administration: paper and pencil instrument (PAPI) and computer assisted instrument (CASI).

Demographics items include both standard demographics as well as items to assess gender identification and sexual orientation. Items to assess general medical history and history of pelvic floor disorders are included. The survey packet also includes items to assess current LUTS and validated measures that serve as external criterion in establishing external validity. Details of the criterion instruments are further described below (Table 4). Dependent on branching, the number of items a respondent is presented ranges from 63 to 90 items. The total PAPI survey length, inclusive of all self-report measures, is 49 pages.

The intended use of a measure dictates the level of evidence required for inferential validity of a measure [28‐30]. For this work, we are employing Campbell and Fiske’s multitrait-multimethod (MTMM) matrix to evaluate hierarchical and cumulative levels of evidence for convergent and discriminant validation of PLUS-BHS score [31]. External variables that rely on multiple and independent sources and methods of data collection in comparison to dimensions of the emergent PLUS-BHS will provide a high level of evidence to support inference of PLUS-BHS score as valid for research in the general population as well as clinical research. The external measures employed by this study were identified to satisfy the requirements of multiple sources and multiple methods: the referent method is self-report survey and the referent source is study participant.

Item distributions will be evaluated for floor and ceiling effects. Items that exhibit greater than 85% response in the tail of the distribution will be removed. Item non-response will be evaluated with items demonstrating greater than 20% non-response considered for removal. Test–retest reliability will be evaluated for all items comparing item response on initial PLUS-BHI completion to item response on retest version of PLUS-BHI using Bland–Altman with 95% interval agreement threshold will be used for comparison of continuous variables (including categorical variables with greater than 6-point response options), and chi-square used for comparison of categorical variables, with Fisher’s exact used as needed.

A comparison of responses by mode (PAPI vs. CASI) will be made both between and within responses from the general population sample and the clinical samples separately. Items will be evaluated using Tukey’s HSD or chi-square (Fishers as needed). Items demonstrating significant differences across mode of administration or across sample will be considered for removal.

Internal dimensional validity will be identified and evaluated using factor analytic approaches. Two investigators utilizing multiple methods principal component analysis and factor analysis and rotation methods (orthogonal/oblique) will work independently and periodically compare and discuss results. Internal consistency of item groupings will be evaluated with Cronbach’s alpha. Item groupings with alpha > 0.9 will be further evaluated to determine whether items within grouping are, in reality, subtle variations of the same question. Item groupings with alpha < 0.4 will be rejected. Several criteria will be applied in evaluation of dimensions and factor retention: the Kaiser-Guttman rule (Eigenvalues > 1.0), factor loading thresholds of 0.60/0.40, scree plots, Kaiser–Meyer–Olkin (KMO) residuals off-diagonal partial correlations measure of sampling adequacy > 0.70 [28, 39, 40]. This analysis will be iterative. Emergent factor structures across mode and sample will be compared with the aim of identifying factor structures that are consistent across mode and across sample. The postpartum sample data will be evaluated similarly, with comparison of the factor structure made to final factor structure from general population sample and general clinical sample.

Our evaluation of levels of evidence is based on the work of Campbell and Fiske’s Multitrait-Multimethod (MTMM) matrix relative to the use of multiple-operationalism (multiple items to assess the range of a concept) to evaluate and establish levels of validity, although we use the term “dimension” in lieu of “trait.” [31]. Correlation values across multiple data sources and methods within each emergent dimension (factor) populating the MTMM matrix will allow evaluation of convergent validity and divergent validity.

Sample sizes are based on ensuring sufficient numbers of item response to conduct factor analysis. We aim to enroll the number of women necessary to achieve 10 participants per PLUS-BHI item included in the dimensional analysis [41], although less conservative estimates of 5–10 participants per item have been suggested [42]. Based on a maximum of 90 items a participant may respond to, our target number of completed BHIs in the general population sample (n = 694) and the clinical sample (n = 354) provides 8 and up to 12 participants per item evaluated, dependent on merging samples. This estimate is also conservative in that it accounts for all branching and not all participants will indeed branch based on their responses.

Many health care providers have spent their clinical and research careers studying and treating women with urinary symptoms as evidenced by the greater than 50 validated instruments to assess symptoms and quality of life impact in women with LUTS [43]. Previous efforts have focused on defining and treating the disease and not on prevention. The long-term goal of the PLUS Consortium is to decrease the incidence and prevalence of LUTS in women by identifying and then modifying risk and protective factors. Central to this goal is the ability to measure bladder health across a woman’s life course. The PLUS-BHI development and validation described in this manuscript is considered by the PLUS Consortium to be the cornerstone of future efforts in LUTS prevention research.

Critical to the development of the PLUS-BHS for research purposes is having a definition for bladder health [9, 44]. One of the greatest challenges of developing the PLUS-BHS is that we are measuring an abstract theoretical construct such as bladder sensation (a component of bladder health) by asking specific questions that are intended to capture the range of meaning of normal and abnormal “bladder sensation” [45]. We recognize that the questions about bladder function are likely to be interpreted in numerous ways by respondents due to the inability to discriminate between the responses. This is compounded by the reality that the presence of bladder symptoms in women are often considered a “normal” part of aging or secondary to childbirth. As a result, a woman might choose a response that describes her urinary leakage accurately, but attribute minimal disruption in her quality of lifestyle or bother to the symptom because she has been conditioned to believe it is normal. In addition, some LUTS are episodic such as stress incontinence or urinary tract infection symptoms yet are contributors or detractors from bladder health. Additional challenges to measuring overall bladder health relate to the fact that responses to questions can vary based on whether or not a woman is experiencing symptoms at the time of the instrument completion or has ever experienced any symptoms across her life course. Finally, it is challenging to design a measurement of health that is generalizable and interpretable across populations, the life course and settings. To address these issues, we use multiple samples across population and bladder health states and include a clinical interview and review of specific bladder testing by a variety of judges to improve use of the instrument across contexts.

One of the primary intended uses of the validated PLUS-BHS is to establish a U.S. general population estimate of women’s bladder health, across the life course. To ensure the PLUS-BHS is valid for this inference it is essential to validate using a national general population sample of women. Some of the challenges in conducting national population research is the identification of a representative sample frame, minimizing the well-known problem of coverage error. With survey methods sampling, the DSF with household level enumeration of every known mailing address within the U.S. is the address base sampling frame with the lowest coverage error that exists. The use of random assignment of both general population participants as well as, general clinical participants to the multiple modes (PAPI, CASI and within CASI Web primary as well as push to web) allows a valid and unconfounded comparison of differences in both response rate, item-response as well as distributional differences across mode. The inclusion of clinical samples using community based recruitment allows a clinical evaluation of women across the spectrum of potential bladder health. The inclusion of objective clinical test data provides the “hard ratings” required as a standard of evidence for valid inference with the intended use of the PLUS-BHS. A major strength of our validation work is the inclusion and evaluation of women during the postpartum period. While we would not expect women’s bladder health to be considered stable during pregnancy, we expect the postpartum period to be a factor in women’s bladder health over their lifetime.

While the goal of the PLUS-BHS is to be used across the female lifespan including adolescent girls and adult women, this initial validation was limited to English speaking women ≥ 18 years of age. The development of a similar instrument for adolescents and Spanish speaking females is underway and will be facilitated by the outcomes of the VIEW study. Since the instrument is specifically designed for women, who have different lower urinary tract anatomy and symptoms when compared to men or transgender women, this instrument is not intended for use with males or adolescent females. Lastly, the PLUS-BHI is only tested for clinical research in ambulatory women; thus, women who have functional and neurogenic LUTS are not included in the spectrum of bladder health assessed by the PLUS-BHI. The questions and answer stems might not be appropriate in this population and would need further testing to determine their ability to discriminate bladder health from bladder disease in certain populations.

The VIEW study includes women from the general population and from those recruited around clinical centers, those who are recently postpartum and who attend clinical practices for health care. While this range of recruitment can be a strength of the study to assure a wide range of contexts, it is also possible that the distribution of bladder health may not be comparable between the populations. While clinical center recruitment was intentionally designed in order to capture the full spectrum of very healthy to very unhealthy, the screening instrument we used (the PPBC) may not accurately discriminate across the true severity of disease and does not address impact of symptoms. Similarly, validated questionnaires were not used to stratify women according to type of LUTS for inclusion in the clinical examination. It is also possible that women recruited from clinical practices, specifically specialty practices (e.g. urology, urogynecology) and with self-reported severe symptoms may be fundamentally different than women who may have symptoms but who do not seek out care for LUTS. As a result, the PLUS-BHS may discriminate between participants with similar LUTS who have different expectations for their bladder health. Modifications to the scoring of the PLUS-BHS may need to be made for general, clinical and postpartum populations and the PLUS-BHS would not be as generalizable as desired.

The VIEW study participants are randomized to the use of paper or computer instruments in order to evaluate potential mode effect on items and subsequently develop a scale that is mode agnostic. While we aim to have a mode agnostic instrument, we acknowledge that it may not be feasible to conduct population based research using CASI mode given systemic biases associated with women’s access to technology. Additionally, while the use of smartphones and technology has increased in many parts of the U.S., we anticipate differences in response rates by mode of administration across age groups and by socioeconomic status.

In conclusion, we present the rationale and approach to development and validation of a novel instrument for the measurement of bladder health in epidemiologic research. The scored bladder health scale will provide the foundation to assess the distribution of bladder health in women and girls within the United States and allow for future study of a variety of factors associated with bladder health. With this foundational instrument developed, future iterations and adaptations will be possible to include adolescent and Spanish-speaking females and pregnant populations. Additional efforts in the future will focus on assessing sensitivity to change and minimum important differences for use in intervention studies. These data will inform future intervention trials in the promotion of bladder health and prevention of lower urinary tract symptoms in women across the life course.