Evaluation of a clinical risk index for advanced colorectal neoplasia among a North American population of screening age

Colorectal cancer (CRC) is one of the most common cancers in women and men worldwide [1, 2]. It is estimated that 136,830 persons were diagnosed with CRC in the United States in 2014 [3]. CRC screening is recommended by the United States Preventive Services Task Force for persons at average risk with annual fecal occult blood test (FOBT), periodic flexible sigmoidoscopy (FS), or colonoscopy [4]. Utilizing easy-to-collect information on clinical risk factors may provide a useful approach in identifying asymptomatic persons who should be referred for colonoscopy rather than a stool test, based on their risk of harboring advanced neoplasia (AN) [5].

Several risk indices or prediction tools for AN or advanced proximal neoplasia (APN) of the colon have been developed [6‐20]. These indices have encompassed multiple risk factors for CRC including age, sex, body mass index (BMI), smoking, alcohol, dietary history (red meat consumption), physical activity and in some indices, distal colorectal findings. Intended use for risk indices not employing distal findings include identifying individuals who might be recommended for screening colonoscopy instead of FOBT or fecal immunochemical test (FIT). An example is the work of Kaminski et al. [16], who recently developed a risk index designed to estimate the likelihood of detecting AN at colonoscopy using age, sex, family history of CRC, smoking history and BMI.

We performed an evaluation of the risk index developed by Kaminski et al. [16] by externally validating its performance using a well-characterized cohort of asymptomatic individuals undergoing screening colonoscopy.

A total of 5137 participants comprised our external validation cohort. Table 2 outlines the distribution of clinical risk factors and colonoscopy findings among the study participants. The mean age (SD) of the cohort was 58.3 (6.2) years and 54.9 % were women (n = 2821). More than half (55.5 %) of the participants reported not ever having smoked and approximately a quarter (26.2 %) had an adenoma or cancer (n = 1344) when categorized by most advanced finding at colonoscopy. The prevalence of AN in the study cohort was 6.8 %.

Figure 1 outlines the proportion of AN across overall risk scores. The prevalence of AN increased as the risk scores increased from 1 to 6. Our sample had very few (n = 2) participants with a score of 7 or 8 who had AN. The likelihood of detecting AN increased from 3.6 to 13.1 % for those with a risk score of 1 to 6 respectively.

The multivariable-adjusted model (Table 3) shows the adjusted odds ratios (ORs) along with 95 % confidence intervals (CI) and p-values for associations between risk index factors and AN. Older age (≥55 years), smoking ≥20 pack-years and a BMI ≥30 were significantly associated with AN. The c-statistic for the multivariable logistic model in our cohort was 0.64 (95 % CI = 0.61–067) indicating modest overlap in probability of predicting AN across risk scores. When those older than 66 years of age were excluded from the cohort (n = 644), the performance of the risk index did not materially change (c-statistic 0.64; 95 % CI = 0.60–0.67). The prevalence of AN in the age-restricted sample ranged from 3.6 to 11.1 % for those with risk scores of 1 and 6 respectively (Fig. 1).

Table 4 outlines the performance characteristics of the risk score in our external validation cohort and includes sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). The PPV (probability of AN given a certain score) for individuals with a risk score of 6, and 7 and 8 (combined) was 12.4 and 5.9 % respectively in our external validation cohort.

We report here that the risk index developed by Kaminski et al. [16], when externally validated in a large cohort of 5137 asymptomatic individuals aged 50 to 74 years, was found to be less predictive of AN compared to the Kaminski et al. [16] cohort. Excluding those older than 66 years of age did not materially change this finding. We found that the overall prevalence of AN in our cohort ranged from 3.6 to 13.1 % compared to 4.3 and 13.7 % in the cohort of Kaminski et al. [16] for those with risk scores of 1 and 6 respectively. The prevalence of AN further increased for those with risk scores of 7 or 8 in the Kaminski et al. [16] cohort (19.1 %), while decreasing substantially to 5.9 % in our cohort. The c-statistic in our sample was similar to that of Kaminski et al. [16] for the validation set (0.64 and 0.62 respectively). Associations between smoking, BMI and AN as previously reported in Kaminski et al [16] were also confirmed in our cohort. However, unlike in the test sample of Kaminski et al. [16] we did not observe an association between family history and AN, nor did we find evidence of an interaction between BMI and sex. A possible barrier to applying the risk index developed by Kaminski et al. [16] in clinical practice is the ability to differentiate between the bulk of the patients who fall in the middle of the risk score continuum as a threshold for classifying individuals into low and high risk categories was not specified [22].

Previous studies have found that the performance of risk indices has been limited when validated in separate independent cohorts. For example, Imperiale et al. [6] developed a risk index for APN using information on age, sex and distal colorectal findings. Approximately 67 % of persons with APN were classified as high risk in the derivation subgroup and the c-statistic was 0.81, indicating good to excellent discrimination [6]. Levitzky et al. [23] externally evaluated the risk index of Imperiale at al [6] in a sample of 1481 white, 1329 black and 689 Hispanic participants. The authors reported that the likelihood of having APN was only moderately discriminated by the risk index (c-statistic of 0.63 for blacks, 0.68 for Hispanics, and 0.62 for whites) [23]. We have also recently reported on an evaluation of the risk index of Imperiale et al. [6] using the same cohort described in this study [24]. Similarly to Levitzky et al. [23], we found that the Imperiale et al. [6] risk index did not perform as well in our cohort (c-statistic 0.62; 95 % CI = 0.58–0.66) [24]. If all persons classified in the intermediate and high risk categories were recommended for colonoscopy, only 73.7 % of those with APN would have been identified [24] compared to approximately 92 % in the Imperiale et al. cohort [6].

A review of risk prediction models for colorectal cancer found that no model sufficiently covers all known risk factors for CRC [25]. The authors suggested that a new comprehensive model is needed that is suitable for assessment of people across the full range of risk. However, concerns remain about the addition of risk factors, which may in turn compromise the use of the index in routine clinical practice because of the additional time required to collect the information. Further refinement of these indices may provide more consistent discriminating abilities and the capability to help physicians and their patients make informed decisions regarding screening. Refinement of these indices could include the use of additional risk predictors found to be useful in epidemiological studies, such as waist circumference (WC) instead of or in conjunction with BMI [26]. Recently, a study from the Netherlands reports on a risk prediction model for AN by utilizing quantitative results from the FIT in conjunction with risk factors for AN [27]. This strategy improved discrimination ability significantly compared to a model utilizing FIT results only [27]. Although this strategy must be further validated in a similar study like our own in other independent cohorts, it has the possibility to improve the effectiveness of FIT-based CRC screening, rendering it a prospectively useful tool in the prioritization of colonoscopy resources.

Our findings must be considered in light of the study strengths and limitations. We included information on those older than 66 years. Thus, our cohort of 50 to 74 year olds is representative of an average risk screening population. Excluding those older than 66 years of age did not make a difference to our results and thus strengthens our conclusion. We had a larger proportion of men in our cohort with a more equal distribution of men and women compared to Kaminski et al. [16]. On the other hand, participants in our cohort were recruited from two different sites from outpatient colonoscopy clinics. There is the possibility of selection bias as participants who provided consent may be different from those who decided not to provide consent. The questionnaire used to collect information on risk factors was originally designed in the Veterans Affairs Cooperative Study 380 and then adapted for the Canadian population [21]. As such, there is the possibility that the questions or items were not identical between the two cohorts. In addition, we only had 34 persons with a risk score of 7 or 8, only two of whom had AN, potentially limiting our ability to evaluate the performance of this index for higher risk individuals.

In conclusion, the risk index for AN using age, sex, family history of CRC, smoking history and BMI, as derived by Kaminski et al. [16] was found to be less predictive of AN in our North American population of screening age. The index requires further evaluation and refinement to better predict AN in average risk persons. However, the index may be useful in very low-resource settings where access to screening colonoscopy is very limited.