Development and validation of image quality scoring criteria (IQSC) for pediatric CT: a preliminary study

Computed tomography (CT) plays a vital role in pediatric patients. For any action on making CT safer for children, the principle of optimization stipulated by the International Commission on Radiological Protection (ICRP) requires balancing radiation dose with the image quality so that necessary diagnostic information is not compromised [1, 2]. ICRP has recommended that image quality in CT should be not higher than needed for confident diagnosis [3]. Pediatric CT protocols must be tailored based on clinical indications to ensure that CT radiation dose to pediatric patients is appropriate [4‐8]. Several studies during the last half of a century have tried to assess image quality objectively [9‐13]. Parameters like quantitative image noise, signal-to-noise ratio, contrast-to-noise ratio, modulation transfer function (MTF), normalized noise power spectrum (NNPS), detective quantum efficiency (DQE), contrast details, and forced-choice just noticeable difference (JND) have been studied. Most objective scoring criteria are useful for academic studies rather than for applying in day-to-day practice in clinical situations globally. The results show that a particular image quality parameter is sensitive in detecting a specific aspect of information in the image but inadequate as an overall measure of image quality in terms of clinical usefulness. Subjective scoring systems for image quality and some using image noise, contrast, sharpness, and artifacts have been applied to CT image quality [13‐16]. Such studies focus on the visual aspect of quality. The subjective judgment of images by a radiologist has been used routinely in day-to-day life, and it does provide a powerful tool to judge many aspects of image quality and information content simultaneously as needed for diagnosis. Some papers have compared the subjective scoring of images with objective indices and have confirmed the usefulness of subjective scoring [9, 10]. Generally, individual preferences of images’ quality acceptability lead to variability in image quality and radiation dose. The interobserver variability is considered a substantial source of the problem that leads to inconsistent and inadequate subjective image quality assessments. If the criteria developed are such that they have a lower interobserver variability, it may become widely acceptable.

A recent publication has introduced a new concept of acceptable quality dose (AQD) and emphasized the need for developing criteria for satisfactory image quality to integrate subjective image quality aspects with radiation dose for optimization studies [17]. A method of image quality criteria that can be widely accepted in the context of non-academic centers and have less interobserver variability is needed [18]. Such an approach would assist in CT radiation dose optimization as well as dose survey studies of image quality [17]. Combining complex issues of service delivery into a single score of assessment like the option of a five-star rating system in most service providers provides an easy and uncomplicated way to assess user satisfaction.

In the backdrop of the above scenario, we developed the image quality scoring criteria for CT images of children with the following objectives

For this study, we propose image quality scoring criteria (IQSC) for the assessment of pediatric CT image quality for routine and clinical indication-based situations. Further, the goal of our study was to assess the diagnostic acceptability and interobserver variability of pediatric CT with this clinical indication-based image quality scoring criteria (IQSC).

This study was performed as a quality improvement project and thus exempted from the need for approval from the institutional review board (IRB). MKK received research grants from Siemens Healthineers and Riverian Inc. for unrelated projects. Other co-authors have no pertinent financial disclosure.

Table 2 summarizes mean ± standard deviation (SD) age, gender, weight, and BMI, CTDIvol and DLP for different CT protocols of included patients. The median subjective image quality scores based on IQSC are summarized in Table 3. Frequency distribution of IQSC for routine and various clinical indication based CT protocols is summarized in Table 4.

The reasonable degree of agreement in this preliminary study among different observers is encouraging for further work in this area and has the promise of a simple tool for image quality scoring that can be made practicable.

In the 1990s, there were efforts in developing image quality criteria notably through the work of European Commission projects [15, 16]. Initially, quality criteria were developed for radiographic images including mammography, and later, also for CT [15, 16, 19]. These criteria provide an educational tool so that users could become aware of the essential features in the radiological image. However, despite two decades of existence, there is paucity of information on these criteria in dose survey studies and in achieving optimization of dose and image quality in clinical practice. Most publications focus on image quality criteria based on body regions rather than specific clinical indications in each body region [13, 14, 20‐25]. Our study simplified image quality assessment based on subjective judgments rather than measurable values while making it more specific and relevant to clinical indications.

The driving force for the development of the scoring system was the broader global feasibility and applicability, relying upon subjective assessment of imaging specialist rather than downplaying it. Unfortunately, there is a general tendency of rejecting the variability in human perception, whereas the same is not the case with machines. The radiation dose for the same radiological examination varies widely even using the same type of equipment from the same manufacturer and with patients of similar body size [25] (see Table 4). Variability is part of research studies. We need to assess the variability and find ways to reduce it rather than getting deterred by it. The fact that there is substantial variability among interpreters of images is well known. We evaluated variability in image quality scoring.

The IQSC is an attempt to fill the current gap of a need to integrate image quality in terms of information content with radiation dose. Adequate visualization of relevant structures in the image formed the basis for scoring to depict information content for specific clinical indications. For example, in the case of craniostenosis CT examinations, sutures and skull shape are important while brain parenchyma is not. On the other hand, for VP shunt evaluation CT, demonstration of ventricular systems is the key. In a study of children with suspicion of craniosynostosis, the image quality was assessed using parameters’ image noise, image sharpness, overall diagnostic acceptability, and artifacts [26]. In another study, a subjective three-point scale ranging from very good to non-diagnostic image quality rating of perfusion disturbance, intracranial peripheral vessel depiction, and motion or streak artifacts was used [27].

In our study, for all five radiologists, the lesion detection was unaffected for all CT exams. For five readers and when we take the full picture of all examinations combined, two-third (67%) assessments were leading to a score of 3, 29% to score of 4, and only 4% to score of 2 (Table 4, Fig. 7). When scores 3 or 4 are combined to represent acceptable image quality scores, they cover 96% of situations. The median image quality scores for all pediatric CT exams for readers were the score of 3. For all pediatric CT exams, the interobserver agreement among five readers (acceptable image quality [scores 3 or 4] vs sub-optimal image quality ([scores 1 and 2]) was moderate to very good (kappa 0.4–1). The exception for the score of 2 (in 4%) cases was seen due to the presence of motion artifacts and non-inclusion of the anatomy in one case. The same radiologist also rated three cases of craniostenosis CT as suboptimal or limited due to the inability of assessing third and fourth ventricles, which do not represent the target regions with this protocol. Therefore, the radiation doses with VP shunt patency CT (for assessment of ventricular system) are higher than for the craniostenosis CT protocol.

It must be pointed out that the radiologists were provided the scoring criteria but did not receive prior training based on the scoring system to avoid any effect on interobserver variation. The latter was important since we wanted to assess the baseline variability.

Differences between subjective image quality scores 3 or 4 indicate that some radiologists are more radiation dose conscious than others and this creates the need for training and culture development on acceptance of low dose CT images (score 3) which does not have the image appearance of higher dose images (score 4). It has been shown that training leads to acceptance of low dose CT images and radiation dose optimization [23]. This paper showed that post-training image quality assessed subjectively in terms of artifacts, gray-white matter differentiation, and visualization of posterior fossa structures, and the need for repeat CT examination was not inferior to pre-training levels for most items. Our study adds an element of quantification of image quality based on the described criteria.

The balance among CT image quality and radiation dose is important; hence, image quality scoring criteria needs to be simple and practical for image quality considerations in dose surveys. Therefore, we propose IQSC for better practicability and the improvement of subjective assessments. It must be emphasized that objective assessments have several limitations over subjective assessments in terms of a limited aspect of image quality that each objective parameter assesses. On the other hand, the subjective evaluation can simultaneously compare several features. Indeed, modern medicine is largely dominated by subjective assessments and individual judgments. Clinical interpretation of radiological images is predominantly subjective. While it is vital to make progress in objective models, it is equally important to keep on making advancement towards enhancing the utility of observer models or scoring criteria to fully utilize human potential [28‐30]. We believe that IQSC can aid in radiation dose optimization in children.

The scoring system is not meant to look at which aspect contributed to the overall score. It is the same when the noise index is measured and used as a criterion for image quality. Noise index does not convey any idea about diagnostic value or diagnostic quality of the image. Our study focuses on overall image quality rather than defining specific characteristics.

Assessment of image quality in our study was limited to one institution and pediatric radiologists only. This calls for extensive and multi-centric study to establish the usefulness of these criteria.

The image quality scoring criteria (IQSC) for routine and also clinical indication-based imaging scenarios for pediatric CT protocols provide a simple and practical tool for assessing image quality with a reasonable degree of interobserver agreement. The score can trigger the need for cause analysis. The study calls for an extensive and multi-centric study to establish the usefulness of these criteria.