The accuracy of preliminary diagnoses made by paramedics – a cross-sectional comparative study

A cross-sectional comparative study design was conducted using a questionnaire to gather data from the paramedics. Hospital records were used to collect the physician’s diagnoses.

The participants of this study were all paramedics working in different levels of emergency practice. In Finland, where this study was conducted, the emergency medical service has three-tiers consisting of first response units, basic and advanced life support level ambulances and physician staffed units [17]. Paramedics assess their patients independently and evaluate their need for care. Physician units only participate the highest priority calls, however, paramedics have the opportunity to consult a physician if they need medical support. Prehospital electrocardiograms are systematically electronically transmitted to hospital for a later interpretation of a physician.

In Finland, the professional qualifications for ambulance crew members at the basic level are: at least one paramedic must be registered health professional with prehospital emergency care training and the other must be a health professional or a fireman. At the advanced level one paramedic must be a registered nurse with additional prehospital emergency care training or a specific registered prehospital emergency care nurse (bachelor-level degree) and the other must be a health professional or a fireman [18].

This study was conducted in one hospital district in Finland. The hospital district covers an area of 7820 km2 and has approximately 220,000 inhabitants. The local Emergency Medical Service consists of 17 advanced level ambulances staffed with a total of 158 paramedics. Annually there are over 29,000 emergency calls leading to an ambulance response. According to hospital records, an estimated 1000–1200 patients are transported by EMS to the central hospital monthly. Approximately 85% of the patients are transported without a referral but the exact numbers are not known due to the fact that they are not systematically documented by the central hospital.

In this study we used two datasets. The first dataset was a semi-structured questionnaire developed for this specific study. The questionnaire was based on a systematic literature review conducted for this study and on the national targeted learning outcomes of paramedic education. There was an expert panel with two emergency medical doctors, three paramedics and a healthcare teacher who commented on the questionnaire in two rounds and then reached a consensus by adding one more question and modifying two others. The expert panel also recommended the use of an ICPC-2 classification to state the preliminary diagnosis so that the answers would be less ambiguous. The questionnaire consisted of eleven questions in two parts (Table 1). The preliminary diagnosis was stated according to the ICPC-2 classification or with the paramedics own words if not found among the ICPC-2 classification codes.

Paramedics were asked to complete a questionnaire concerning patients transported to the emergency department of the central hospital between 30.3.2018 and 30.4.2018. Emergency calls and patients without a referral were included. A total of 415 questionnaires were returned of which 378 were included in the study. Questionnaires with an invalid social security number as a patient identification (n = 15) or a missing preliminary diagnosis (n = 1) were excluded. In addition, one questionnaire completed outside the data collection time (n = 1) was excluded.

The second dataset was formed from the hospital’s electronic patient records in which the physician’s diagnosis, according to ICD-10 classification, was collected at the end of the caring period in the emergency department. Furthermore, only the diagnosis concerning this specific hospitalization was included. In this phase, patient cases with a missing physician’s diagnosis (n = 17) or patient cases with a referral (n = 2) were excluded.

Patient identification details from the questionnaires were extracted to a database (Microsoft Excel©). The physician’s diagnosis was audited manually from the hospital’s patient records by the patient’s identification number and a date. To avoid bias the physician’s diagnosis was first extracted to a spreadsheet and the preliminary diagnoses from the questionnaires were extracted later and an interpretation of the accuracy was then made. After comparing diagnoses the data was de-identified. All diagnoses were divided into categories and subcategories which described the pathophysiology of the disease or injury.

The data was analyzed with IBM SPSS Statistics® (version 25). Descriptive statistics were described with frequencies and percentages. The accuracy of diagnoses in certain categories was analyzed by using a two-tailed binomial test in order to ascertain whether the paramedics were able to make preliminary diagnosis more accurately depending on the type of disease. Logistic regression analysis was used to test the statistical significance of the relationship between background variables and the accuracy of the preliminary diagnosis. The level of statistical significance was considered to be a p value of less than 0.05.

Ethical approval for this study was received from the Ethics Committee of Satakunta Hospital District (02.03.2018). Additionally, the study permission for the data gathering from the paramedics and from the hospital records was obtained from the central hospital (statement codes: ETMK 33/2018, 37/2018). For the paramedics, their participation in the study was completely on a volunteer basis with an assurance of anonymity. The data collection did not have any impact on the care of patients and informed consent of the patients was not required because of the de-identification of the data.

A total of 71 (45%, n = 158) paramedics participated in the study. Their level of practice was basic life support (4.8%) and advanced life support (95.2%). The median number of the work experience in years was 7.0 (IQR 7.1). The educational background of the paramedics varied (Table 2) and two paramedics did not complete the background section of the questionnaire. The number of questionnaires returned was 378; this equals the number of patient cases included in the study.

Most of the ambulance calls (87%) were coded with low priority and the remainder (13%) were high priority calls. The paramedics consulted a physician in 31% of the patient cases. Preliminary diagnoses concerning cardiovascular diseases (16%) and respiratory diseases (14%) were the most common in this study (Table 3).

The correct preliminary diagnosis was determined in 263 of the patient cases resulting in an overall accuracy of 70%. The accuracy of preliminary diagnosis was strongly related to the patient’s case type (Table 3). Paramedics determined the preliminary diagnoses with the highest accuracy in cases related to mental health and drugs, soft-tissue injuries and cerebral strokes. Among the most common type of emergencies were situations related to cardiovascular diseases, which the paramedics determined with an accuracy of 71%, though, cardiac arrhythmias were recognized with an accuracy of 93%. However, the level of accuracy was most deficient in respiratory emergencies and infectious diseases (Table 3).

In the infectious diseases category, the paramedics recognized influenzas and urinary infections with an accuracy of 13 and 20%, respectively. The typical preliminary diagnosis they determined in these cases was either fever or general weakness. In respiratory emergencies, the most common diagnosis made by the physician was pneumonia, which the paramedics recognized with an accuracy of 54%.

The educational background of the paramedic and the given self-assessment value of the preliminary diagnosis were statistically significant to the accuracy of the preliminary diagnosis (Table 4). Both a bachelor level degree in pre-hospital nursing and continuing education in advanced level pre-hospital care increased the accuracy of the preliminary diagnosis. If a paramedic gave a good self-assessment value to the preliminary diagnosis, this determined that the odds would be higher for the preliminary diagnosis to be accurate (Table 4).

The aim of this study was to evaluate the accuracy of the preliminary diagnoses made by paramedics and the variables related to the diagnosis. The overall accuracy of the preliminary diagnoses made by the paramedics was 70%, which has also been reported in previous studies [12]. This can be considered satisfactory allowing for the limited information, lack of diagnostics in the field and the challenging environment, even though the majority of the cases in our study were non urgent. Despite the lack of the time pressure, the paramedics are challenged by a large variety of complex cases including aged patients with multiple comorbidities and general weakness.

The highest level of accuracy, that of mental health and drug related emergencies and soft-tissue injuries, seems obvious due to the nature of these situations. Nevertheless, in one other study [12] the levels of accuracy (58, 75% respectively) in these categories were remarkably different. The high level of recognizing cerebral strokes (81%) is extremely important because of the survival and recovery of the brain and functioning of the patient [19] and the result is in agreement with other studies [12], although, in this present study, the categories were more specific including only ischemic and hemorrhagic strokes and transient ischemic attacks. The overall accuracy of the preliminary diagnoses in cardiovascular diseases was satisfactory, however it must be emphasized that the recognition of cardiac arrhythmias was on a very high level, suggesting that the paramedics can consistently interpret electrocardiogram in this area. The effect of prehospital delays in survival in acute illnesses e.g. ST-segment elevation myocardial infarction [14, 20] patients is unquestionable, which makes it crucial for paramedics to interpret electrocardiograms.

The level of accuracy was lowest in respiratory emergencies (57%) and infectious diseases (31%). The result in respiratory emergencies is even lower than reported in other studies in which the level was 65% [10, 12]. The unsatisfactory level of accuracy in preliminary diagnoses may be a consequence of the complicated pathophysiology of respiratory diseases [10‐12] or suggest an inadequate examining of the patient. Identifying respiratory diseases and understanding the anatomy and pathophysiology of respiratory system is an essential part of paramedic competencies due to the fact that emergencies related to respiratory diseases or shortness of breath are one of the most common emergency calls [10, 11].

The recognition of infectious diseases seemed lower compared to other preliminary diagnosis areas according to our results. The preliminary diagnosis of influenza proved to be especially difficult but that could partly be explained by the fact that there was no code for influenza in the ICPC-2 classification list, although the participants were instructed to state the preliminary diagnosis with their own words if not found in ICPC-2 classifications. Moreover, urinary infections were not recognized either on a satisfactory level. Although common in the geriatric population, the symptoms of urinary infections may remain very unclear, and it can even be argued, that not all diagnoses of urinary infection made in the ED are necessarily accurate.

The most commonly used preliminary diagnoses that were used instead of a specific infection code were fever and general weakness, which could be considered reasonable options in this context. Ultimately, the most important aspect concerning infections is undoubtedly the early recognition of severe infections, especially sepsis, because early recognition improves the patient’s outcome [13, 21]. In these situations an accurate preliminary diagnosis must be determined. However, in our study, there were only a few cases of severe infections and no conclusions can be made concerning them. In addition, a more precise preliminary diagnosis might be needed, for example, when considering transportation decisions or the patients need to be isolated. For more accurate diagnostics, adding the possibility of point of care testing for paramedics [4] might be one solution.

The relationship between the bachelor degree in pre-hospital emergency nursing and the accuracy of the preliminary diagnosis was clear. The relationship between specific emergency nursing education and paramedic competencies in general has been reported in previous studies [6, 10, 15, 22]. In Finland, the bachelor degree in pre-hospital emergency nursing takes 4 years and concentrates on pre-hospital emergencies, however, the students are also registered as nurses. A bachelor’s degree in nursing does not give readiness for working in a pre-hospital setting but, as reported in the study results, the specialization courses in pre-hospital nursing increases the capability of a paramedic’s clinical decision making and the accuracy of their preliminary diagnoses.

The paramedics were asked to provide a self-assessment value to the preliminary diagnosis they determined. Overall, the paramedics were confident with the preliminary diagnoses they made. The results suggest that in cases of a low self-assessment value the actual physician’s diagnosis was also likely to be undefined or related to a general weakening without a specific diagnosis. Furthermore, the paramedics with a work experience of less than 5 years were less confident about their preliminary diagnoses than their more experienced colleagues, nevertheless, the amount of work experience was not statistically significant as regards the accuracy of preliminary diagnoses. The higher self-assessment values may be explained by professional growth and increased self-confidence and it is important to give enough support to novice paramedics in order to reinforce their clinical decision-making.

The main limitation of this study and generalization of the results is the sample size and the possibility of bias. The number of patient cases included in this study was 378 and all the cases that meet the inclusion criteria were recruited. The size of the sample was sufficient for the patient cases to be divided in statistically representative categories by the type of the emergency or illness and the cases also represent the national distribution of EMS’s transportation codes. We were not, however, able to make prior power calculations since there were only estimations available for the numbers of patients transported by ambulances to the emergency department. However, based on the estimations we achieved a response rate of approximately 40%.

The data was collected in the area of one hospital district during 1 month in the spring, and this also reduces the generalization of the results as more of a certain type of illnesses e.g. influenzas may occur at this time of year. Furthermore, the sample covers well the variety of different cases, except pediatrics or incidents of labor which were not included in this study due to the fact that they are not treated in the emergency department.

Second, the questionnaire used in this study was self-developed since there was no suitable questionnaires available. The questionnaire was not pilot-tested but there was an expert panel and two rounds of assessment to ensure the validity and clarity of the instrument. The questionnaire was designed to be semi-structured to maintain the homogeneity of the answers.

The third limitation of this study is the use of the diagnosis made by physician’s as a reference standard to determine the accuracy of the paramedic’s preliminary diagnosis. A physician’s diagnosis may not always be completely accurate and there is a possibility that the time between the preliminary and actual diagnosis and treatment given in the prehospital care may affect the patient’s condition and symptoms, thus resulting in a different diagnosis. We wanted to use the diagnosis at the patient’s discharge from ED and not the admission diagnosis due to the fact that the discharge diagnosis differs from the admittance diagnosis in every ninth patient [23]. Furthermore, the preliminary diagnoses were made by using the ICPC-2 classification, whereas the ED discharge diagnoses were made by using the ICD-10 classification. This may have affected the agreement analysis of the preliminary diagnosis and physician’s diagnosis since the ICPC-2 classification codes included more general descriptions of symptoms.