Satisfaction and experience with colorectal cancer screening: a systematic review of validated patient reported outcome measures

We conducted an exhaustive search in MEDLINE (PubMed), EMBASE (Ovid), PsycINFO (Ovid) and CINAHL (EBSCOHost) without language or date restrictions. We kept the search updated while we conducted the review and performed the last search in October 2020. The detailed search strategies and dates are available in Annex A. We also searched in BiblioPRO and checked the references listed in included studies. We designed a search strategy combining controlled vocabulary from each database and text words related to the topics review (e.g., satisfaction and colorectal screening). Although a proposal to find studies on PROMs measurement properties exists [27], we defined and used a more specific list of terms to filter the search results.

We included validation studies which reported the development and/or the evaluation of one or more measurement properties of questionnaires measuring patient experience or satisfaction with colorectal cancer screening, irrespective of the screening test used. To avoid being too restrictive, we also included studies on the development or validation of questionnaires that measure patient experience or satisfaction with colonoscopy (irrespective of it was performed in the context of a screening program) and with the notification process of a screening result. We also considered studies (irrespective of their design) that assessed patient experience or satisfaction with colorectal screening as an outcome. From these studies, we tried to obtain information on the questionnaire used to measure the outcome and tried to locate the validation study to consider its inclusion.

We limited the inclusion to studies published in English, Spanish, French and Italian. We excluded studies that assessed satisfaction with the decision to uptake screening and studies that used alternative methods to questionnaires to measure experience or satisfaction, such as interviews or diaries.

Two authors independently assessed the results of the search for eligibility, and then made a final decision based on the full text of the references deemed eligible. Disagreements were resolved with the help of a third reviewer.

We developed and pilot-tested a case report form (CRF) using Google Forms. The CRF is available from the authors on request. Two authors independently extracted data from included studies and disagreements were resolved with the help of a third reviewer. When full questionnaires were not reported in the paper, we tried to contact the corresponding authors in order to obtain them.

We extracted the following data from eligible documents following the recommendations from the COSMIN user manual [23, 28]: 1. General characteristics of the study (country, year of publication, study design, objective, main outcomes); 2. Characteristics from the questionnaire targeted population or those that participated in the validation; 3. Main characteristics of the questionnaire (name, original language and available translations, administration characteristics, domains measured, number of items, evidence for validity); 4. Information on questionnaires psychometric properties; 5. Information on interpretability of questionnaires (the degree to which a quantitative score or a change in score of a questionnaire can have a qualitative meaning) and feasibility (the ease of application of the questionnaire in a setting). Interpretability and feasibility are not considered measurement properties, but are important aspects for selecting a questionnaire to use in practice [23].

We assessed the methodological quality of each measurement property study using the COSMIN Risk of Bias checklist [24]. According to this checklist, methodological quality of studies was rated as either “very good”, “adequate”, “doubtful” or “inadequate” for each measurement property assessed. We used the COSMIN taxonomy to determine which measurement property were assessed in each study.

The result of each measurement property study was rated against the updated criteria for good measurement properties based on Terwee et al. [29] and Prinsen et al. [30] (Annex B). Each result was rated as either sufficient (+), insufficient (−), or indeterminate (?).

Following the COSMIN manual, we graded the quality of the evidence for the rating of each measurement property of each questionnaire using the GRADE approach [31], which specifies four levels of quality of evidence (high, moderate, low or very low) depending on the presence of four factors (risk of bias, indirectness, inconsistency and imprecision). If the overall rating for a measurement property is indeterminate (?), the quality of the PROM cannot be judged and there will be no grading of the quality of the evidence [23].

The process of assessing methodological quality of studies, rating measurement properties and grading the evidence was done by two authors independently and differences were resolved by consensus. Attempts were made to contact the authors of included PROMs for information on all measurement properties of questionnaires.

According to ratings on methodological quality and the results of measurement properties, included PROMs were classified providing a recommendation on the most suitable questionnaire to be used [23] (Table 1).

We used descriptive statistics to synthesize findings, calculating absolute frequencies and proportions as appropriate. We planned to quantitatively pool the results reported by different studies on measurement properties of each questionnaire. However, we were not able to do so as we only found one study for each questionnaire. A statistical analysis was performed using SPSS, version 25.0 (SPSS Inc., Chicago, IL, USA). We report the findings of the review as a narrative synthesis of the characteristics and measurement properties from each included questionnaire.

No ethical approval was required as this study is a systematic review.

We describe the eligibility process in a PRISMA flowchart [26] (Fig. 1). We screened the titles and abstracts of 3749 references obtained from the searches, selected 158 records for full-text assessment and finally included 80 studies. Reasons for exclusions are detailed in Fig. 1.

We identified 80 studies published from 1992 to 2020 that used questionnaires to measure patient experience or satisfaction with colorectal cancer screening or with the conduction of non-screening colonoscopy, sigmoidoscopy or with bowel preparation. Most of them were published in North America (38, 47.5%) and Europe (26, 32.5%) from 2010 on (Table 2). Most studies were experimental or observational studies in which patient experience or satisfaction were measured as outcomes (75, 93.7%) and only five studies (6.2%) described the development of a questionnaire or its validation. These 80 studies used 75 different questionnaires, being most of them self-administered (70, 93.3%) and written in English (61, 81.3%). Most were created de novo (51, 68.0%) or developed from existing questionnaires (12, 16.0%), with very few studies using an existing questionnaire (8, 10.7%).

From the 75 questionnaires identified, only seven (9.3%) were reported as validated tools [32‐38]. There were two studies that reported using validated questionnaires, but we were unable to locate the development or validation studies or to obtain a copy of them (although an effort was made to contact authors) [37, 38]. For that reason, we could only analyze five validated questionnaires (6.6%) [32‐36].

All five validated questionnaires measured patient satisfaction. Four questionnaires assessed satisfaction with endoscopic procedures (Table 3). Two were developed in the context of colorectal cancer screening (the Colonoscopy Satisfaction and Safety Questionnaire (CSSQP) for colonoscopy conducted after a positive stool test [32] and the Screening Flexible Sigmoidoscopy Assessment Questionnaire for screening sigmoidoscopy [36]). The other two measured satisfaction in the context of elective upper or lower endoscopy (the Spanish modified Group Health Association of America- 9 Questionnaire (SmGHAA-9 m) [34] and the Post Procedure Questionnaire [35]). The remaining questionnaire assessed satisfaction with bowel preparation for colonoscopy [33].

Three questionnaires used the English language and were developed and validated in the USA [33, 35, 36], while two were developed and validated in Spain and used the Spanish language although an English translation is available [32, 34]. All questionnaires were self-administrated with the exception of the SmGHAA-9 m [34] which was administered by telephone.

The questionnaires were validated in samples of women and men between 50 and 69 years old, with the exception of the Patient Satisfaction Scale with Bowel Preparation and the Post-Procedure endoscopic questionnaire [33, 35], which were evaluated in adults up to 80 years old. Table 4 describes the characteristics of included populations.

Methodological quality of studies on each measurement property was evaluated according to the COSMIN Risk of Bias checklist [24] (Table 5). Ratings are provided for only those measurement properties assessed in each study.

According to the COSMIN guidelines, content validity is the most important measurement property [23] and it arises from the assessment of the relevance, comprehensiveness and comprehensibility of the PROM. Evidence on these parameters comes from development and validation studies. A detailed evaluation of the quality of questionnaires’ development studies is available in Annex C. Methodological quality of studies for content validity was rated as doubtful for all questionnaires because it was not clear if patients and experts were asked about relevance, comprehensiveness, and comprehensibility in the validation studies.

Structural validity and internal consistency address the internal structure of a questionnaire, and are the next most important measurement properties [23]. Three studies were of adequate quality for structural validity and of very good quality for internal consistency [32, 35, 36]. The remaining two were of doubtful quality for internal consistency [33, 34]. Methodological quality of studies on remaining measurement properties is summarized in Table 5.

In Annex D we provide an example on how the methodological quality evaluation and the rating of measurement properties were conducted for one questionnaire.

Detailed information on interpretability and feasibility of questionnaires is summarized in Annex E and F. Overall, studies provided scarce information about interpretability: most showed a low percentage of missing total scores (from 1.4 to 6.2%) [32‐34, 36] but only two provided information on floor and ceiling effects [32, 33] and none on the minimal important change or minimal important difference. Regarding feasibility aspects, none of the studies provided information on the completion time, the cost of the questionnaire or the existence of copyright. The CSSQP [32] and the SmGHAA-9 m [34] are available in Spanish and in English. However, none of these questionnaires were culturally adapted nor validated in a setting different to that in which they were created (cross-cultural validation).

This systematic review identified many studies that measured patient satisfaction or experience with colorectal cancer screening or procedures and tests included in these preventive programs. These findings mean that patient reported measurements are increasingly being considered in this setting and other research fields as well [20]. However, the majority of these studies used non-validated questionnaires, which is consistent with findings of another systematic review of PROMs on patient satisfaction in breast cancer screening [39]. This shows clear room for improvement since the use of non-validated PROMs may result in a limited trustworthiness in measurements obtained through their use.

Only five questionnaires have been validated for measuring patient satisfaction in relation to screening colonoscopy and sigmoidoscopy [32, 36], non-screening endoscopy (both upper and lower) [34, 35] and with bowel preparation [33]. Although two additional studies reported the use of validated questionnaires [37, 38], we were unable to obtain the required information to appraise them. We did not identify any validated questionnaire for assessing satisfaction or experience with the use of stool tests for colorectal cancer screening, which is the most used screening test in European screening programs [5, 10], or with other aspects of the screening process such as the communication of screening results.

The decision to use one PROM over another will depend on different factors, but it is important to ascertain both the methodological quality of studies in which the PROMs were validated and the questionnaire measurement properties themselves. We used the COSMIN methodology to classify each PROM into three possible categories that have different implications regarding the potential to recommend one PROM over another. From the five validated questionnaires included, only two (CSSQP [32] and Post-procedure questionnaire [35]) showed sufficient content validity and internal consistency to be recommended for their use in practice.

There are additional important factors to consider when choosing a PROM. For example, the population for which the PROM is intended, the availability of cross- cultural validation of the questionnaire and aspects related to its feasibility. The CSSQP [32] and the post-procedure questionnaire [35] measure satisfaction with different procedures and are targeted at different populations. The CSSQP [32] assesses the safety and satisfaction with colonoscopy conducted after having a positive stool test in the context of colorectal cancer screening. On the other hand, the post-procedure questionnaire [35] measures satisfaction with both upper and lower non-screening endoscopy. These differences in population are important as screening program attendees have peculiarities with respect to those who attend colonoscopy for other factors (e.g., evaluation of symptoms, surveillance of polyps, etc.). They are healthy people with no symptoms that have not sought health care and may experience high levels of anxiety [40], so their expectations (and therefore their satisfaction) may differ from the rest of patients [32]. Another important aspect is that the CSSQP [32] was developed and validated in the Spanish population. Although this questionnaire was translated to English following a translation back-translation process, it still has not been culturally adapted nor validated in other populations. In the same way, the post-procedure questionnaire [35] was developed and validated in a population from the USA and is only available in English. If questionnaires are used in countries other than those in which they have been developed and validated, it is necessary to translate them (with a translation-back translation process), conduct a culturally adaptation and finally study their cross-cultural validity before their use [28, 41‐44]. Cross-cultural validity is evaluated assessing whether the scale is measurement invariant or whether differential item functioning occurs between at least two culturally different groups of people [23]. None of the included questionnaires were culturally adapted and neither conducted this sort of validation.

The remaining three questionnaires [33, 34, 36] did not report data in enough detail to ascertain their validity. It does not mean that these questionnaires cannot be recommended, but further validation studies will be needed [23].

To our knowledge, this is the first systematic review to identify and assess PROMs for measuring patient satisfaction and experience with colorectal cancer screening. However, there are hundreds of systematic reviews using the COSMIN methodology. Some have focused on screening [45‐48], and one specifically assessed PROMs for measuring patient satisfaction or experience in relation to breast cancer screening [39]. These systematic reviews also found variability in the methodological quality of included studies [19, 39, 45, 46, 48]. In our work, methodological quality of studies ranged from inadequate to very good, depending on the measurement property assessed.

We were able to assess limited information about some (but not many) psychometric properties of questionnaires, which is consistent with results of similar literature reviews [19, 39]. However, unlike other settings in which there are some studies assessing each PROM, we only found one study for each questionnaire reporting data related to its development and validation. We did not find further validation studies, which limits the available evidence on the questionnaires’ validity, as each new study provides further confirmation of the ability of a questionnaire to measure the construct of interest [19].

Another important aspect is that patient satisfaction and patient experience, despite being related, are not exactly the same [19, 20, 49]. Some authors advocate that measuring patient experience is preferred because it is a more descriptive and objective measure (rather than evaluative) and is less affected by gratitude bias and other factors [19, 20, 49]. As these concepts have been used interchangeably many times in the literature [50], we included both so that important information was not left out. However, all validated questionnaires included measured patient satisfaction.

We conducted an exhaustive search including sources that index questionnaires and measurement instruments for the identification of all available PROMs, but the possibility of selection bias still exists as we limited the inclusion to studies published in English, Spanish, French and Italian, and we did not look at grey literature to check the use of questionnaires in technical reports assessing the results from local or national screening programs. Despite this drawback, we could expect that, at least for validated PROMs, the researchers made the effort to report the process in a journal article. It is also possible that we did not evaluate all measurement properties of questionnaires because this information was not reported in published papers. In order to minimize this bias, we attempted to contact questionnaire developers for further information and complete scoring. However, we did not always receive an answer. It is possible that included questionnaires have been harshly criticized in their methodological quality as the COSMIN methodology considers applying the lowest rating of any standard in the box to the overall rating of each study (“the lowest score counts” principle) [24].

This study has several strengths. To our knowledge, this is the first paper that reviews and appraises available PROMs to measure patient experience or satisfaction with colorectal cancer screening. We conducted systematic searches in four different databases and the selection and data abstraction processes were conducted in duplicate in order to minimize selection bias and errors. We used an internationally agreed and explicit methodology (COSMIN [23]) for assessing the methodological quality of studies and questionnaires’ measurement properties. We must highlight, though, that the use of this guidance is limited to trained and skilled researchers and that its use is burdensome at some stages. There is clear room for improvement for reporting in this field. Recently, guidelines for reporting primary studies on measurement properties have been published [51] and their use should improve transparency and facilitate the appraisal of these studies. On the other hand, reporting guidelines for reviews of PROMs would be of great interest to ease the preparation of manuscripts in this field and improve the quality of such evidence syntheses.

This systematic review will help clinicians, managers, policy makers and researchers to select the most suitable PROM taking into consideration their context of use. This will, in turn, facilitate the systematic use of these validated questionnaires to identify areas for enhancement from the patients’ perspective and drive improvements in the quality of colorectal cancer screening programs.

Some validated questionnaires with good measurement properties for measuring patient satisfaction with screening and non-screening colonoscopy are already available, so it is not necessary to develop new questionnaires on this aspect [32, 35]. Efforts should be made in conducting further validations of existing questionnaires (assessing all psychometric properties), in translating them into different languages and validating them in different populations, so they could be used in different settings. However, for measuring satisfaction with bowel preparation or screening sigmoidoscopy, we could identify only one validated questionnaire for each procedure, with not sufficient psychometric properties for it to be recommended for use. In these cases, it would be necessary to conduct further validation studies or even develop new PROMs. Likewise, we did not identify any validated PROM for assessing patient experience or satisfaction with the use of stool tests for colorectal cancer screening or with the communication of screening results, so it would be necessary to develop and validate PROMS to measure these aspects. Any additional effort to develop new PROMs in this field should ensure the involvement of the public in their development and validation [52].