Feasibility of a standardized ultrasound examination in patients with rheumatoid arthritis: a quality improvement among rheumatologists cohort

In order to be enrolled in the training program it was required that the participating rheumatologist had passed two standard courses of US diagnostics arranged by the Danish Society of Diagnostic Ultrasound. All rheumatological centres in Denmark employing a rheumatologist, who has passed the required US course, were invited to participate in the project. The participating centres borrowed identical ultrasound machines with a fixed preset. Ten machines were available for the study. Ten centres agreed to participate in the study.

Table 1 shows the scanning experience and degree of supervision received by the participating rheumatologists. The scanning experience was measured in months and number of examinations and the degree of supervision was defined as 1 = none, 2 = moderate, 3 = extensive.

The training program consisted of two parts. In the first part all the participating rheumatologists had an individual, two days, hands-on, training course. This part of the program took place before inclusion of patients. The rheumatologists were trained in a standardized US examination of the wrist and MCP II-V joints which has demonstrated an excellent reliability [16]. The rheumatologist performed US examinations of the wrist and MCP joints in 10 patients with RA under supervision by one of two tutors (KE and STP). The tutors had extensive parallel working experience in performing the standardized US examination (KE five years and STP more than 10 years). In the second part the tutors evaluated all the performed US examinations, which were submitted from each centre for immediate feedback by e-mail. In case of failure to reach 80% correct scores, the centre was asked to recall the patient for a repeated examination.

The patients were enrolled consecutively in the rheumatological centres. Inclusion criteria were: RA diagnosed according to the American College of Rheumatology's criteria [17], Doppler activity in the wrist and/or MCP joints, ongoing therapy with methotrexate and scheduled for treatment with etanercept (only first time treatments). Exclusion criteria were treatments with other biologics within two month or lack of Doppler activity. The patients were scanned five times during the first year of etanercept therapy, or as long as they remained on this therapy. The US examinations were performed at baseline, after 2, 12, 24 and 52 weeks. The study was approved by the local ethics committee (KF 01 31 8007) and informed consent was obtained from each patient before study entry. At baseline, the patients accepted a possible delay of treatment up to one week in the case of an unacceptable baseline ultrasound examination.

At all centres, the US examinations were performed with a GE Logiq^® 9 (Milwaukee, Wisconsin, USA) using a 14 MHz centre frequency linear array matrix transducer. A preset for the ultrasound examinations was installed on all machines and remained unchanged throughout the study period. The participants were allowed to make any changes to the grey scale settings in the scanning situation. The participants were not allowed to adjust the Doppler parameters except for box size and position and Doppler focus.

The Doppler preset was adjusted for maximum sensitivity for low flow (pulse repetition frequency of 0.4 kHz, lowest wall filter on 45 Hz, and 7.5 MHz Doppler frequency) with Doppler gain just below noise level [18]. The patient was placed sitting opposite the investigator with the hand placed on a cushion in order to keep the hand relaxed. The wrist was scanned in four positions, three dorsal and one volar position. The MCP joints were scanned in three positions, two dorsal and one volar. All scans were performed in the longitudinal plane. All participants were instructed to use generous amounts of scanning gel to avoid compression on the tissues. In all positions specific anatomic landmarks had to be present in the image to ensure the same scanning positions in all examinations (see Table 2). Once the anatomic landmarks were identified in each position, the colour Doppler was activated. While keeping the landmarks in the image, the transducer position was adjusted until the scan plane containing the most colour Doppler was found. The transducer was held in this position for a couple of heart cycles, and the image was frozen. With the cine-loop function, the frames with maximum and minimum colour Doppler activity in the synovial tissue were selected and stored (Figure 1). Subsequently, the colour Doppler was deactivated and the corresponding grey scale image containing all landmarks was obtained and stored.

All US examinations were sent to the tutors for evaluation. The examinations were evaluated according to a standardized scoring system assessing the quality of all stored images. The participants received an evaluation of each US examination by e-mail. The first 50 examinations were evaluated by both tutors in order to obtain consensus. A total of 16 positions were investigated at each US examination. The total number of outcome items scored in each of the 16 positions was nine, thus a total of 144 scores was given. The scoring system was dichotomous as each outcome was assessed as accepted/not accepted. The US evaluation contained the following items: storing of all relevant images, correct annotation and image orientation (left-right), well-defined landmarks, correct focus in both grey-scale and Doppler images, presence of air or Doppler artefacts and finally correct position of the Doppler box.

In order to get the US examination approved, 80% of the items should be accepted. The cut-off point of 80% was chosen in accordance with other studies investigating learning experience in musculoskeletal US [12, 13]. At each US evaluation, the duration of the examination was noted in order to evaluate the changes in the time spent.

The data structure of the study design included clustered data (i.e, patients within clinics with repeated measures), thus the hierarchical model with continuous data was applied. Statistical analyses were based on a linear mixed model: Patients and Rheumatologists were applied as random factors when assessing associations over time. This was modelled using the Restricted Maximum Likelihood (REML) default option in SAS PROC MIXED: based on a random coefficient model in order to assess the different possible linear associations across rheumatology clinics simultaneously [19]. Random coefficient models emerge as natural mixed model extensions of simple linear regression models in a hierarchical (nested) data setup. To combine the individual study results we performed the hierarchical modelling using SAS software (version 9.2).

There were 13 rheumatologists participating in the study. Eight of the rheumatologists (62%) succeeded in following one or more patients for one year (Figure 2). In total, 104 patients were enrolled in the study, of these 60 (58%) completed the 1 year follow-up examination (Figure 2). The mean number of patients enrolled by each rheumatologist was 8 (1-16). As presented in Table 1 the mean patient age was 54.6 (SD 14.2) years and mean disease duration was 8.8 (SD 7.7) years. Mean Disease Activity Score of 28 joints (DAS28crp) was 5.1 points (SD 1.2) with a mean swollen and tender joint count of 8 (SD 5) and 11 (SD 9), respectively and mean CRP of 20.2 mg/L (SD 21.7). As presented in Table 1 the mean scanning experience among the rheumatologists was 54 months (SD 50) and the mean number of US examinations performed before entering the study was 807 (SD 1475). Within 3 months of onset, three of the participating rheumatologists withdrew from the study; two of them due to new appointments and one because of lack of patients. The three withdrawing rheumatologists were replaced by three others, who fulfilled the inclusion criteria. Only three of the participating rheumatologists achieved a score below 80% in their initial examinations while later in the program, all scans were above the cut-off level (Figure 3).

Nine of the 13 participating rheumatologists (69%) had either a consistent or slightly improved learning curve throughout the project, whereas 4 (31%) demonstrated a decreasing learning curve (Figure 3). Of these, two dropped out of the project within the first three months.

As illustrated in Figure 4 most of the participating rheumatologists had above 80% correct scores from the very beginning of the study and the rheumatologists' scores increased in the study period (p = 0.0004; Figure 4A) corresponding to an expected improvement of 1% per hundred rheumatologist days in the study. In contrast, the time spent on an examination decreased approximately 10% per hundred rheumatologist days throughout the study period (p < 0.0001; Figure 4B). Relating the time spent on the US assessment with the scores achieved also showed a significant association (p < 0.0001). This though, was of a minor magnitude corresponding to less than 2% worsening in the US score for each of the 10 minutes saved.