Structured Reporting versus Free-Text Reporting for Appendiceal Computed Tomography in Adolescents and Young Adults: Preference Survey of 594 Referring Physicians, Surgeons, and Radiologists from 20 Hospitals

Acute appendicitis is the most common indication for emergent abdominal surgery. Computed tomography (CT) plays a vital role in the diagnosis of appendicitis owing to its availability and excellent diagnostic performance. Care providers are often dependent on CT results in determining patient management, ranging from hospital discharge to appendectomy; because clinical and laboratory findings are known to be insufficient for accurate diagnosis (1). Preoperative CT utilization rates in patients undergoing appendectomy now reaches 90% in the United States (2) and South Korea (3).

CT results regarding the likelihood of appendicitis need to be delivered accurately and efficiently by radiologists to referring physicians or surgeons. This is particularly important when CT findings are not judged as “definitely positive” or “definitely negative” by radiologists. Equivocal results are reported in approximately 5–13% of appendiceal CT examinations and 31–52% of the equivocal cases are subsequently confirmed as appendicitis (4, 5). In such uncertain cases, accurate diagnosis and appropriate patient management are determined through further in-hospital clinical observations (1) and/or additional imaging tests (6, 7). The failure to communicate a diagnostic uncertainty (or certainty) may lead to unnecessary additional observations or imaging tests, potentially resulting in inappropriate or delayed patient management. Miscommunication may be compounded by involvement of many care providers with varying experience levels including rotating trainee doctors (8, 9), who use different or ambiguous terms to deliver diagnostic certainty. Further confusion may result when non-English speaking care providers attempt to communicate the diagnostic certainty in English, which is not a rare situation throughout the globe.

The diagnostic certainty needs to be clearly communicated, ideally in a highly-standardized manner. A Likert scale-based structured reporting (SR) form may be an effective measure for successful standardization. The need for SR has long been recognized in the field of radiology (10, 11, 12). However, recent surveys (13, 14) showed that the SR is rarely used in practice except for breast (15) or cardiac imaging (16). It has therefore been suggested that SR should be developed for clinically important but less complex examinations, where communication issues have been observed (17). In this context, several researchers have advocated the use of SR for appendiceal CT (18, 19) or ultrasonography (20, 21, 22). To date, the proposed SR forms have not gained much traction.

We recently had the opportunity to distribute SR form for appendiceal CT across 20 hospitals through the course of a large clinical trial (Low-dOse CT for Appendicitis Trial, LOCAT; http://www.locat.org) (7, 23). The trial aimed to compare low-dose (2 mSv) and standard-dose (8 mSv or less) CT in the diagnosis of appendicitis, among patients aged 15–44 years. We anticipated that standardized communication using the purpose-designed SR form may lead to better diagnostic and clinical outcomes. An investigation to demonstrate the advantages of SR over free-text reporting (FTR) in association with patient outcomes would require an unrealistically large sample. This is because regardless of the reporting style, the diagnostic performance of CT for appendicitis is already excellent, which leaves only a small margin of improvement by using SR over FTR. It is more practical to first survey the willingness of care providers to adopt the SR. This study aimed to survey care providers' preference for the use of SR vs. FTR for appendiceal CT, in adolescents and young adults.

This prospective study was approved by an ethical committee. A waiver of consent was granted. In October 2012, the LOCAT lead investigator invited 121 hospitals to participate in LOCAT through the Korean Society of Abdominal Radiology network. Twenty teaching hospitals with a median annual appendectomy caseload of 269 (interquartile range, 158–428) participated in LOCAT (7, 23). LOCAT finally enrolled 3074 patients aged 15–44 years and more than 500 care providers participated in LOCAT from December 2013 to February 2017. We disseminated the SR form for appendiceal CT across the 20 hospitals as a part of the LOCAT protocol. The SR form has been used in research studies (24, 25) and also as part of usual care in the LOCAT lead center. The form aimed to determine the likelihood of appendicitis on a 5-point Likert scale (Supplementary Table 1 in the online-only Data Supplement). During the study period, the 20 LOCAT sites used the SR form for all trial patients and also as a part of usual care.

Our survey was conducted from February 2016 through November 2016, during the last phase of LOCAT. Three radiologists jointly created the online survey after reviewing the literatures for key features of a high-quality radiology report (10, 11, 17, 18, 26). The survey included five preference questions (question items Q3–Q7) including usefulness in patient management, communicating the likelihood of appendicitis, convenience, style and format, and overall preference (Table 1). Responses were recorded using a 5-point Likert scale ranging from “definitely SR” to “definitely FTR.” Each participant was also questioned regarding his/her prior clinical experience with using an SR for appendiceal CT (Q1), and for any other CT applications (Q2), as well as his or her commitment in answering all the questions (Q8). An invitation with a link to the survey was emailed to 706 care providers, who were reported by the site principal investigators to be involved in the care of trial patients in 2016. Invitees included 231 radiologists, 254 emergency physicians, and 221 surgeons. The median number of the invitees per site was 31 (interquartile range, 23–40) (Fig. 1, Table 2). The initial rollout involved three hospitals, and then gradually expanded to other hospitals. Weekly reminder emails were sent to non-responding invitees to solicit their participation, until the response rate for each hospital reached 80%.

Fig. 1
Flow diagram for selection of study participants.

• Click for larger image
• Download as PowerPoint slide

Table 1
Survey Questionnaire

• Click for larger image
• Click for full table
• Download as Excel file

Table 2
Number of Participants

• Click for larger image
• Click for full table
• Download as Excel file

In May 2017, three months after the completion of LOCAT, we conducted another online interview with 20 radiologists who respectively led the LOCAT study at each of the 20 hospitals. We asked each of the radiologists which of SR or FTR the hospital was using for appendiceal CT in their usual care of patients aged 15–44 years. For hospitals not consistently using SR, the reason/s for not using SR were sought. We also asked if the hospital was shifting to SR in the near future.

We analyzed the data after excluding participants with insufficient self-reported commitment (defined as a response of “never committed,” “hardly committed,” or “unsure” to Q8) in the response. Descriptive statistics, including percentages and medians, were used to summarize the results. Stacked bar graphs were generated to illustrate the results for subgroups, categorized by specialty department and job position, and for individual hospitals.

For the overall preference (Q7), a multivariate logistic regression analysis was performed after binary collapse of the 5-point scale responses, such that “definitely SR” or “probably SR” were included as preference for SR. The tested covariates included participant characteristics (department, job position, and prior experience in SR) and hospital characteristics (annual number of appendectomies, and the number of enrolled patients in LOCAT). A p value of < 0.05 was considered to indicate statistical significance. Stata 14.0 (StataCorp LLC, College Station, TX, USA) was used for all statistical analyses.

Of the 706 invitees, 608 (86.1%) participated in the survey. The participation rate was 99.1% (229/231), 82.7% (210/254), and 76.5% (169/221), for the radiologists, emergency physicians, and surgeons, respectively (Table 2). Four radiologists, three emergency physicians, and seven surgeons were excluded from the analysis due to self-reported insufficient commitment in the response (Supplementary Table 2 in the online-only Data Supplement). Final analyses included a total of 594 participants (median 25 [interquartile range, 20–35] per hospital; including 225 radiologists, 207 emergency physicians, and 162 surgeons). Participants had variable prior experience in SR for appendiceal CT: 143 (24.1%), 155 (26.1%), 158 (26.6%), and 138 (23.2%) of them had experience of 10 or less, 11–30, 31–100, and more than 100 cases, respectively. For other CT applications, 320 (53.9%) participants had prior experience in SR (Table 3).

Table 3
Participant Characteristics

• Click for larger image
• Click for full table
• Download as Excel file

The numbers of participants who responded “definitely SR” or “probably SR” to Q3–Q7 were 319 (53.7% [95% confidence interval, 46.9–57.8%]), 385 (64.8% [60.8–68.7%]), 291 (49.0% [44.9–53.1%]), 281 (47.3% [43.2–51.4%]), and 306 (51.5% [47.4–55.6%]), respectively. In contrast, the numbers of participants who responded “definitely FTR” or “probably FTR” to Q3–Q7 were 88 (14.8% [95% confidence interval, 12.1–17.9%]), 78 (13.1% [10.5–16.1%]), 159 (26.8% [23.2–30.5%]), 194 (32.7% [28.9–36.6%]), and 163 (27.4% [23.9–31.2%]), respectively. The remaining 187, 131, 144, 119, and 125 participants responded with “indeterminate” to Q3, 4, 5, 6, and 7, respectively. For the overall preference (Q7), 87 (14.6%), 219 (36.9%), 125 (21.0%), 132 (22.2%), and 31 (5.2%) participants responded with “definitely SR,” “probably SR,” “indeterminate,” “definitely FTR,” and “probably FTR,” respectively. In terms of communicating the likelihood of appendicitis (Q4), 121 (20.4%), 264 (44.4%), 131 (22.1%), 60 (10.1%), and 18 (3.0%) participants responded with “definitely SR,” “probably SR,” “indeterminate,” “definitely FTR,” and “probably FTR,” respectively.

For each of Q3–Q7, there were only minor variations in the response pattern across the participants' departments or job positions (Fig. 2). The overall preference (Q7) for SR over FTR was slightly higher among surgeons and emergency physicians than among radiologists, and among attendings than among trainees. In terms of communicating the likelihood of appendicitis (Q4), the preference for SR was slightly higher among attendings than among trainees.

Fig. 2
Stacked bar graph showing survey responses for questions Q3–Q7 regarding preference between SR and FTR according to participants' departments and job positions.
Length of each stack indicates percentage of participants in that category with respect to entire study group. Each number is number of participants included in stack. FTR = free-text reporting, SR = structured reporting

• Click for larger image
• Download as PowerPoint slide

There were considerable inter-hospital variations in the response pattern (Fig. 3). The per-hospital percentage of the participants who responded with “definitely SR” or “probably SR” to Q7 ranged from 30% to 74% across the 20 hospitals. The percentage was above 50% in 11 hospitals, and below 50% in nine. On the contrary, the per-hospital percentage of participants who responded with “definitely FTR” or “probably FTR” was below 30% in 12 hospitals, 30–50% in seven, and 60% in one.

Fig. 3
Stacked bar graph showing per-hospital survey responses for overall preference (Q7) between SR and FTR.
Length of each stack indicates percentage of participants in particular hospital with respect to entire study population. Each number is number of participants included in stack. As of May 2017, five hospitals (denoted by ^*) were completely using SR, eight hospitals (denoted by ^†) were partially using SR, and seven hospitals (denoted by ^‡) were rarely using SR in usual care.

• Click for larger image
• Download as PowerPoint slide

The logistic regression analysis showed that the overall preference for SR over FTR for future practice (i.e., responses of “definitely SR” or “probably SR” to Q7) was significantly associated with attendings (p = 0.007), prior experience with using SR for appendiceal CT in more than 100 cases (p = 0.003), hospitals performing less than 400 appendectomies annually (p = 0.048 or less), and hospitals with 170 or more patients enrolled in LOCAT (p = 0.030). More surgeons preferred SR to FTR than radiologists, although the difference did not reach statistical significance (Table 4).

Table 4
Multivariable Logistic Regression Analysis for Overall Preference (Q7)

• Click for larger image
• Click for full table
• Download as Excel file

As of May 2017, five hospitals were using the SR form in nearly all relevant patients. Eight hospitals were using the SR form in some cases. Seven hospitals rarely used the SR in usual care. The reasons for not using SR in the 15 partially-using or rarely-using hospitals included: care providers' unfamiliarity to SR, and the lack of SR education or promotion to the care providers. Two of the 15 hospitals were planning to completely shift to SR in the near future, ten hospitals chose to keep using FTR partially, and the remaining three hospitals decided to abandon SR (Supplementary Table 3 in the online-only Data Supplement).

FTR for radiologic studies has been criticized due to non-standardized, ambiguous descriptors for diagnostic certainty (10). Considerable discrepancies between radiologists' intended nuances in their reports and the referring physicians' understanding of the reports exist (27, 28, 29). SR is often advocated for improving communication (10, 11, 17, 27, 30). However, it is well known that SR does not always increase referring physicians' satisfaction, or reporting accuracy (31). SR has also been criticized for the inefficiency in making reports due to frequent disruptions of established routines in image interpretation, and the rigidity of the form, which limits descriptions for complex cases or unexpected finding (31, 32, 33, 34). Due to these limitations, current usages of SR have not met expectations (13, 14, 26).

However, appendiceal CT in adolescents and young adults is uniquely suitable for SR. First, the diagnostic task is simple (i.e., the likelihood of appendicitis), and the report conclusion can be efficiently summarized in SR using a Likert scale. Second, unexpectedly complex cases are rare, since most alternative diagnoses are limited to a small number of non-serious diseases in the target patient population (5, 23). Third, standardized communication is essential because interdisciplinary collaborative decisions need to be made urgently and often involve less experienced care providers including rotating trainee doctors, who may use different or ambiguous diagnostic descriptors.

Our survey involved 594 care providers from 20 hospitals. In comparison, recent studies advocating for the use of SR in appendiceal ultrasonography (20, 21, 22) or CT (18, 19) involved only smaller numbers of participants, from one or two hospitals. Our result showed the participants' overall preference for SR over FTR, although not unanimously. Responses to the preference questions (Q3–Q7 regarding the usefulness in patient management, communicating the likelihood of appendicitis, convenience, style and format, and overall preference) indicated that 47.3–64.8% of the participants preferred SR (responding with either “definitely” or “probably”), while 13.1–32.7% preferred FTR (responding with either “definitely” or “probably”). In total, 51.5% of the participants responded as preferring SR, while 27.4% responded as preferring FTR in the overall preference (Q7), which we considered as the most comprehensive and important question item.

Importantly, the overall preference between SR and FTR varied considerably across the hospitals, while there were only minor variations across the participants' departments or job positions. This reflects that SR implementation is a matter of hospital system rather than of an individual department or job position. Therefore, a successful SR implementation requires hospital-wide systematic collaborative efforts to make small but substantial changes in the workflow. In our experience, a motivated radiologist is key to successful SR implementation in a hospital. The lead radiologist must guide the implementation through persistent and collaborative efforts, e.g. incorporation of the SR form into the radiology information system, education of colleagues and care providers, regular compliance audit for radiologists, and constructive feedback to reluctant care providers.

Our subgroup analyses revealed several care-provider or hospital characteristics that were associated with the overall preference for SR over FTR (Q7). These results conversely indicate challenges in the SR implementation. For successful SR implementation, reluctant care providers and hospitals must be the focus of further education and encouragement. First, the preference for SR over FTR was more pronounced for a participant with greater experience with the use of SR for appendiceal CT (more than 100 cases), and for a hospital that enrolled more patients (170 or more) in LOCAT, in which the same SR form was mandatorily used as a trial procedure. These results suggest that there are learning curves to overcome until the care providers become proficient in, and thereby comfortable, with SR. Second, the preference for SR was more pronounced among attending care providers than among trainees, and among surgeons than among radiologists, although this difference did not reach statistical significance. These results are interesting because the attendings and surgeons, rather than the trainees and radiologists, were typically responsible for the final clinical decision regarding patient management, particularly whether to operate or not, and therefore, they would see the benefit of clearer communication of the CT results. Third, the preference for SR was associated with a hospital with smaller appendectomy volume (less than 400 annually). The higher-volume hospitals had less need to shift to SR, probably because they were able to effectively communicate through conventional FTR.

In our follow-up survey three months after the completion of LOCAT, only five hospitals (25%) kept using SR for nearly all the relevant patients, while the remaining 15 hospitals partially or rarely used SR in usual care. These disappointing results clearly highlighted the difficulty in the sustained use of SR, despite our efforts in embedding SR into the rigid framework of a clinical trial. Further study is needed to develop a measure for the sustained use of SR.

Our study had limitations. First, despite the large scale of the study, it is unclear if our survey results can be generalized. All our 20 hospitals were teaching hospitals participating in LOCAT, and therefore, they may have been more motivated or better resourced toward the report standardization. Second, our survey relied on the voluntary participation of care providers involved in LOCAT. The survey participants may have answered more positively than non-participants (35). Third, we did not measure the diagnostic or clinical advantages of SR over FTR. As we stated earlier, such a comparative investigation would require a huge sample. Despite the considerable inter-site variations in the preference for SR over FTR, LOCAT data showed little difference in clinical outcome, such as negative appendectomy rates and appendiceal perforation rates, across the 20 trial sites (23). We are not aware of any prior study directly comparing SR and FTR for appendiceal CT with respect to care providers' preference, diagnostic performance, or clinical outcomes. In terms of appendiceal ultrasonography, recent studies (21, 36, 37, 38) of a before-and-after design advocated that the use of SR instead of FTR can lead to lowering second-line CT utilization and marginal diagnostic improvement.

In conclusion, we disseminated the Likert-based SR form for the likelihood of appendicitis across 20 hospitals through the course of a large clinical trial. Our survey including 594 care providers from the 20 hospitals showed their overall preference for SR over FTR. The subgroup analysis results suggest that successful SR implementation requires hospital-wide systematic efforts and involves some learning curves. However, further study is needed to investigate the sustained use of SR.

The online-only Data Supplement is available with this article at https://doi.org/10.3348/kjr.2018.0109.

Supplementary Table 1

Evaluated Findings and Scoring Criteria in SR Form

Supplementary Table 2

Self-Reported Commitment Level in Response

Supplementary Table 3

Use of SR in 20 Hospitals