Evaluation of radiological workstations and web-browser-based image distribution clients for a PACS project in hands-on workshops

The methodology and outcome of a hands-on workshop for the evaluation of PACS (picture archiving and communication system) software for a multihospital PACS project are described. The following radiological workstations and web-browser-based image distribution software clients were evaluated as part of a multistep evaluation of PACS vendors in March 2001: Impax DS 3000 V 4.1/Impax Web1000 (Agfa-Gevaert, Mortsel, Belgium); PathSpeed V 8.0/PathSpeed Web (GE Medical Systems, Milwaukee, Wis., USA); ID Report/ID Web (Image Devices, Idstein, Germany); EasyVision DX/EasyWeb (Philips Medical Systems, Eindhoven, Netherlands); and MagicView 1000 VB33a/MagicWeb (Siemens Medical Systems, Erlangen, Germany). A set of anonymized DICOM test data was provided to enable direct image comparison. Radiologists (n=44) evaluated the radiological workstations and nonradiologists (n=53) evaluated the image distribution software clients using different questionnaires. One vendor was not able to import the provided DICOM data set. Another vendor had problems in displaying imported cross-sectional studies in the correct stack order. Three vendors (Agfa-Gevaert, GE, Philips) presented server-client solutions with web access. Two (Siemens, Image Devices) presented stand-alone solutions. The highest scores in the class of radiological workstations were achieved by ID Report from Image Devices (p<0.005). In the class of image distribution clients, the differences were statistically not significant. Questionnaire-based evaluation was shown to be useful for guaranteeing systematic assessment. The workshop was a great success in raising interest in the PACS project in a large group of future clinical users. The methodology used in the present study may be useful for other hospitals evaluating PACS.