Background
Methods
Setting
Instrument development
-
0 = Absent: item is incorrectly absent (it was necessary for the triagist to perform this action)
-
1 = Insufficient: item is performed insufficiently
-
2 = Moderate: item is not performed insufficiently, but also not sufficiently
-
3 = Sufficient: item is performed sufficiently, but there is room for improvement
-
4 = Good: item is performed optimally; there is no room for improvement
Score (%) | Inter-rater agreement (%) | ICC | |||||||
---|---|---|---|---|---|---|---|---|---|
0 | 1 | 2 | 3 | 4 | N/A | Complete | Reasonable | ||
N = 342 |
N = 114 | N = 114 | N = 114 | ||||||
PHASE 1 INTAKE (item 1–8) | 48.0 | 74.3 | .16 | ||||||
Collect personal and residence information | |||||||||
1. Gathers, at an appropriate time, the personal and residence information (c) | 1 | 3 | 10 | 18 | 64 | 3 | 48.2 | 78.9 | .20 |
ABCDE-check (vital signs)
| |||||||||
2. Asks to speak to the patient (c/m) | 17 | 3 | 5 | 2 | 12 | 61 | 78.1 | 86.0 | .70 |
3. Makes the right choice whether or not to perform the ABCDE-check (m) | 13 | 12 | 13 | 14 | 49 | – | 36.8 | 65.8 | -a
|
4. Checks the ABCDE-criteria adequately and draws the right conclusion (m) | 11 | 14 | 7 | 11 | 9 | 49 | 40.4 | 57.9 | .11 |
Open orientation | |||||||||
5. Gives caller sufficient time to describe the situation (c) | 0 | 2 | 12 | 23 | 63 | – | 45.6 | 84.2 | .16 |
Medical problem and help request | |||||||||
6. Asks for the medical problem and its development (m) | 0 | 4 | 13 | 30 | 53 | – | 36.8 | 86.0 | .24 |
7. Determines and explicitly states the help request (c) | 26 | 8 | 10 | 16 | 40 | – | 47.4 | 71.9 | .06 |
Complaint (in triage system) | |||||||||
8. Selects (in the triage system) the appropriate complaint which yields the highest urgency (m) | 5 | 8 | 9 | 11 | 67 | – | 50.9 | 64.0 | .15 |
PHASE 2 TRIAGE AN FOLLOW-UP (item 9–17) | 50.9 | 73.5 | .27 | ||||||
Questioning according to triage methodology | |||||||||
9. Asks at least the essential questions belonging to the specific complaint (m) | 4 | 9 | 17 | 30 | 40 | – | 36.0 | 82.5 | .27 |
10. Works according to the triage system of the GP cooperative (m) | 1 | 4 | 3 | 12 | 21 | 61 | 61.4 | 62.3 | .19 |
History and medication | |||||||||
11. Asks the relevant questions with regard to history and medication (m) | 19 | 6 | 5 | 12 | 53 | 5 | 52.6 | 81.6 | .57 |
Urgency estimation | |||||||||
12. Recognizes changes in the status of the patient and reacts adequately (m) | 2 | 3 | 1 | 2 | 2 | 91 | 85.1 | 86.8 | -a
|
13. Makes an appropriate urgency estimation (m) | 3 | 13 | 13 | 16 | 56 | – | 37.7 | 65.8 | .27 |
Follow-up action | |||||||||
14. Chooses the right follow-up action (m) | 1 | 9 | 10 | 14 | 66 | – | 48.2 | 73.7 | .16 |
15. Gives the right (selfcare-) advice (m) | 5 | 5 | 5 | 9 | 28 | 47 | 53.5 | 66.7 | -a
|
16. Gives the right safety net advice about how to act in case of a change in the situation (m) | 21 | 8 | 6 | 13 | 33 | 20 | 43.0 | 64.9 | .10 |
17. Provides concise information which can be clearly understood by the caller (c) | 0 | 3 | 11 | 23 | 64 | – | 40.4 | 77.2 | .30 |
PHASE 3 FINISHING-OFF (item 18–19) | 25.4 | 60.5 | .15 | ||||||
Check follow-up action | |||||||||
18. Asks if the follow-up action is understood and feasible (c) | 13 | 8 | 17 | 26 | 35 | – | 27.2 | 60.5 | .15 |
19. Checks whether the caller agrees with the follow-up action and shows an open attitude if the caller does not agree (c) | 10 | 9 | 18 | 25 | 37 | 23.7 | 60.5 | .14 | |
GENERAL (item 20–24) | 36.9 | 72.5 | .19 | ||||||
Structure | |||||||||
20. Structures the conversation (c) | 1 | 5 | 23 | 33 | 38 | – | 32.5 | 78.9 | .10 |
21. Makes use of open and closed questions adequately (c) | 1 | 8 | 18 | 32 | 42 | – | 31.6 | 79.8 | .19 |
Summary | |||||||||
22. Gives a summary at an appropriate time, verifies and adjusts the summary if necessary (c) | 31 | 12 | 11 | 20 | 26 | – | 33.3 | 71.1 | .27 |
Sympathize | |||||||||
23. Pays attention to the experience of the caller (c) | 12 | 10 | 15 | 21 | 30 | 12 | 24.6 | 52.6 | .20 |
Consultation general practitioner | |||||||||
24. Consults the general practitioner only if necessary (m) | 4 | 2 | 6 | 9 | 79 | – | 62.3 | 79.8 | .05 |
MEDICAL ITEMS | 51.6 | 73.1 | .19 | ||||||
COMMUNICATION ITEMS | 39.3 | 72.9 | .19 | ||||||
TOTAL | 44.9 | 72.5 | .19 |
Data collection
-
One highly urgent (U1-U2 from Dutch triage system), one moderately urgent (U3-U4) and one low urgent call (U5; telephone advice only)
-
Each call had a different reason for encounter
-
Exclusion of unintelligible calls, prematurely terminated calls, call back contacts, administrative contacts (e.g. for a prescription)
Statistical analyses
Results
Description of calls and triagists
Measurement characteristics
Distribution of responses
Inter-item correlations
Item 1 | Item 2 | Pearson r
|
---|---|---|
3. Makes the right choice whether or not to perform the ABCDE-check | 4. Checks the ABCDE-criteria adequately and draws the right conclusion | .80 (N = 175) |
18. Asks if the follow-up action is understood and feasible | 19. Checks whether the caller agrees with the follow-up action and shows an open attitude if the caller does not agree | .79 (N = 342) |
9. Asks at least the essential questions belonging to the specific complaint | 10. Works according to the triage system of the GP cooperative | .77 (N = 135) |
13. Makes an appropriate urgency estimation | 14. Chooses the right follow-up action | .67 (N = 342) |
8. Selects (in the triage system) the appropriate complaint which yields the highest urgency | 9. Asks at least the essential questions belonging to the specific complaint | .66 (N = 342) |
6. Asks for the medical problem and its development | 21. Makes use of open and closed questions adequately (c) | .62 (N = 342) |
10. Works according to the triage system of the GP cooperative | 8. Selects (in the triage system) the appropriate complaint which yields the highest urgency | .62 (N = 135) |
6. Asks for the medical problem and its development | 9. Asks at least the essential questions belonging to the specific complaint | .61 (N = 342) |
10. Works according to the triage system of the GP cooperative | 17. Provides concise information which can be clearly understood by the caller | .61 (N = 135) |
9. Asks at least the essential questions belonging to the specific complaint | 21. Makes use of open and closed questions adequately (c) | .60 (N = 342) |
Internal consistency
Reliability
Inter-rater reliability
Intra-rater reliability
Agreement (%) | ||
---|---|---|
Complete | Reasonable | |
Phase 1: Intake (N = 416) | 60.8 | 83.2 |
Phase 2: Triage and follow-up (N = 468) | 63.9 | 86.5 |
Phase 3: Finishing-off (N = 104) | 47.1 | 70.2 |
General (N = 260) | 52.7 | 86.2 |
Medical items (N = 728) | 64.6 | 84.3 |
Communication items (N = 572) | 59.1 | 84.0 |
Total (N = 1248) | 55.1 | 84.1 |
Internal versus external raters
Complete agreement (%) | Reasonable agreement (%) | |||
---|---|---|---|---|
Internal vs external |
External vs external
| Internal vs external |
External vs external
| |
Phase 1: Intake (N = 416) | 49.7 |
48.0
| 75.3 |
74.3
|
Phase 2: Triage and follow-up (N = 468) | 49.2 |
50.9
| 69.5 |
73.5
|
Phase 3: Finishing-off (N = 104) | 27.0 |
25.4
| 56.8 |
60.5
|
General (N = 260) | 38.6 |
36.8
| 71.9 |
72.5
|
Medical items (N = 728) | 51.3 |
51.6
| 72.3 |
73.1
|
Communication items (N = 572) | 40.6 |
39.3
| 70.6 |
72.9
|
Total (N = 1248) | 42.3 |
44.9
| 70.8 |
72.5
|
Differences between raters
Location | Rater | Total | Medical items | Communication items |
---|---|---|---|---|
Mean (SD) | Mean (SD) | Mean (SD) | ||
1 | 1 | 2.95 (0.69) | 2.93 (0.95) | 2.81 (0.72) |
2 | 2.63 (0.59) | 2.66 (0.75) | 2.56 (0.70) | |
1 and 2
|
2.79 (0.66)
|
2.79 (0.86)
|
2.68 (0.72)
| |
2 | 3 | 2.88 (0.71) | 2.87 (0.75) | 2.90 (0.70) |
4 | 2.42 (0.76) | 2.54 (0.95) | 2.23 (0.74) | |
3 and 4
|
2.65 (0.77)
|
2.71 (0.86)
|
2.56 (0.79)
| |
3 | 5 | 2.98 (0.68) | 3.23 (0.78) | 2.67 (0.72) |
6 | 3.30 (0.56) | 3.25 (0.66) | 3.29 (0.62) | |
5 and 6
|
3.14 (0.64)
|
3.24 (0.72)
|
2.98 (0.74)
| |
4 | 7 | 3.14 (0.66) | 2.96 (0.85) | 3.23 (0.61) |
8 | 3.13 (0.40) | 3.31 (0.44) | 2.91 (0.50) | |
7 and 8
|
3.14 (0.55)
|
3.13 (0.70)
|
3.07 (0.58)
|