Development of models for cervical cancer screening: construction in a cross-sectional population and validation in two screening cohorts in China

Cervical cancer is the fourth most frequently diagnosed cancer and the fourth leading cause of cancer death in women, with an estimated 570,000 new cases and 311,000 deaths in 2018 worldwide [1]. Cancer morbidity and mortality have decreased in developed countries due to the implementation of routine cervical cancer screening [2], and testing for high-risk human papillomavirus (hrHPV) has improved cervical cancer prevention efforts [3].

The decision for modern cervical cancer screening programs is often made based on age, cytology, and hrHPV testing results. For example, the USA has different cervical cancer screening guidelines for women in different age groups. For women aged 30 to 65 years, guidelines recommend the use of cytology and hrHPV co-testing due to its high sensitivity. HPV testing has not been recommended in women aged 21–29 without atypical squamous cells of undetermined significance (ASC-US) due to its low specificity [4]. However, the cytology and hrHPV co-testing would increase the number of referrals, unnecessary diagnostic procedures, and costs of the health care system [5, 6].

Cervical intraepithelial neoplasia grade 2/3 (CIN2/3) can progress to cervical cancer if left untreated, and therefore identifying women who would benefit from further monitoring and/or treatment is important [7]. However, we need to ensure that unnecessary referrals are avoided. To retain the high sensitivity of our current primary screening tests and improve the specificity, additional screening biomarkers such as HPV genotyping [8, 9], E6 oncoprotein [10], or p16/Ki-67dual staining [11] have been developed and shown to have high specificity as triage tests for HPV positive women, respectively. However, decision-making in routine cancer screening may become more complicated as additional biomarkers are added and screening algorithms become increasingly complex. These complex screening algorithms may have limited practical applications.

In recent years, machine learning methods play an important role in selecting an appropriate combination of multiple biomarkers. With the increasing availability of large national databases and computing power, the use of machine learning methods in medical science and health care has been rapidly growing [12, 13]. Studies have shown that machine learning methods such as logistic regression and support vector machine (SVM) can enhance prediction performances by providing clinicians with valuable evidence-based prognostic information. By using the machine learning methods, we may substantially improve the sensitivity and specificity of cervical cancer screening, avoiding unnecessary colposcopy referral, and simplifying decision-making in clinical practice.

The aim of this study was to develop models that have better prediction of CIN2+ by using age, cytology, and hrHPV testing, with or without other biomarkers. Models were constructed and evaluated in a cross-sectional population enriched with CIN and cervical cancer. External validation in two screening cohorts was further presented to demonstrate the usefulness of our methods.

In this study, we developed and evaluated machine learning-based models to predict CIN2+ or CIN3+ for cervical cancer screening. A logistic regression model using hrHPV, cytology, and age was set as the base model due to its superior performances in prediction and colposcopy referral rates reduction. Improved clinical performance of the base model can be gained by incorporating E6 oncoprotein and/or HPV genotyping information. External validation in two screening cohorts further demonstrated that our models had better clinical performances than routine screening methods. The 3-year risks of CIN2+ for the predicted-negative women depended on the thresholds of the model, but the improvement of clinical performance at baseline can be obtained whichever threshold was chosen.

Different models were used for cervical cancer screening in previous studies. Karakitsos et al. used the learning vector quantizer neural network classifier on cytological diagnosis, HPV DNA test, E6/E7 HPV mRNA test, and p16 immunostaining to build an algorithm to facilitate the classification of CIN2+. This model improved the AUC (0.916) significantly compared to cytology diagnosis alone (0.866) [16]. In the study conducted by Branca et al., comprehensive multivariate models were constructed by a panel of 13 biomarkers to predict CIN2+, giving the AUC of 0.897 [17]. A Korean study developed a web-based tool on age, cytology and presence of 15 hrHPV genotypes in a SVM model to identify the patient features that maximally contributed to progression to cervical lesions, which obtained an accuracy of 74.41%. However, this model was not developed for cancer screening and their result was highly dependent on the proportion of positive and negative individuals they selected [18]. Several studies used logistic regression to establish predictors for histologic grade or risk stratification based on the epidemiologic risk factors and the molecular markers. In a large study of around 100,000 women using race, smoking status, insurance, marital status, median income, and previous HPV test result as predictors, their model only obtained an AUC of 0.81 for CIN2+ [19]. Another study of 1,477 women reported that the most predictive factors were mRNA level, DNA index, parity, and age, and the AUC was 0.99 for HSIL and 0.81 for LSIL [20]. However, findings from previous studies may not be replicable across studies due to differences in adjustment factors, sample size, and degrees of diagnoses.

The clinical performances of our extended models included HPV genotyping and/or E6 oncoprotein showed to be better than the base model in each of the study population assessed, but resulted in a slight increase in cost. HPV genotyping can be a byproduct of HPV testing that has little additional costs but more additional information. E6 oncoprotein is pivotal in initiation and maintenance of oncogenic transformation by HPV [21] and associated with viral persistence [22, 23]. The protein testing is a lateral flow immunoassay designed for low- and middle-income countries [15]. When conducting study in SC-I, we assumed that only people positive with HPV mRNA result could express E6 oncoprotein. Therefore, our protein testing was performed only in the HPV16/18/45 mRNA-positive participants (N = 126). These results showed that additionally testing for the E6 oncoprotein in a limited group of people could yield better screening performance than the base model. In addition, the models recommended fewer women to receive immediate colposcopy compared to HPV and cytology testing alone or co-testing but had the same cost with co-testing in the real-world setting to collect HPV testing and cytology information, hence could reduce unnecessary diagnostic procedures and costs.

Cytology diagnosis is subjective in nature, and its reproducibility and accuracy are affected by the cytologist’s skill [24]. In our study, the cytology results diagnosed by senior cytologists were performed in a high-quality laboratory in Beijing, which may not be generalizable to all the cytology laboratories [25]. For example, the cytology diagnosis in SC-II was conducted by the best cytologist in China, whose sensitivity (0.961) was higher than HPV testing (0.902). In order to better extrapolate, models based on the cytology results from junior cytologists were also evaluated. Although we found that model performances were affected by the skill level of the cytologists, the clinical performances of our models were still increased, compared to hrHPV testing and cytology co-testing within the same cytologist’s skill level.

Our study demonstrated that machine learning could incorporate multiple screening methods into one algorithm and develop models by the current cervical cancer screening indicators, which has the potential to be a reliable screening method considering its better clinical performance and lower referral rate.