Background
Variable | China | Institute for Health Metrics and Evaluation | Million Death Study | Agincourt | Matlab |
---|---|---|---|---|---|
Region | China | N/Aa
| India | South Africa | Bangladesh |
Sample size | 1,502 | 1,556 | 12,225 | 5,823 | 3,270 |
Ages | 15+ years | 15 to 105 years | 1 to 59 months | 15 to 64 years | 20 to 64 years |
Number of CODs | 31 | 32 | 15 | 17 | 17 |
Population | Hospital deaths | Hospital deaths | Community deaths | Community deaths | Community deaths |
Proportion ill-defined deathsb
| 0% | 0% | 3% | 12% | 2% |
Physician coding | Coding by a panel of three physicians assisted with medical records and diagnostic tests | Coding by one physician assisted with medical records and diagnostic tests | Dual, independent coding of VA records, disagreements resolved by reconciliation, and for remaining cases by adjudication by a third physician | Dual, independent coding of VA records, disagreements resolved by third physician. | Single physician re-coding of VA records after initial coding by another physician. |
Methods
Datasets
Computer-coded verbal autopsy methods
InterVA-4
Open-source random forest
Open-source tariff method
Testing
Dataset splits and resampling
Dataset | Training/testing cases | Number of diagnostic indicators | |||
---|---|---|---|---|---|
King-Lu | Open-source random forest | Open-source tariff method s | InterVA-4 | ||
China
| 1100 / 400 | 48 | 48 | 48 | N/A |
Institute for Health Metrics and Evaluation
| 1100 / 400 | 96 | 96 | 96 | N/A |
Million Death Study
| 1100 / 400 | 89 | 89 | 89 | N/A |
1100 / 1100 | 89 | 89 | 89 | N/A | |
6100 / 6100a
| 89 | 89 | 89 | 245 | |
Agincourt
| 1100 / 400 | 104 | 104 | 104 | 245b
|
1100 / 1100 | 104 | 104 | 104 | 245 | |
2900 / 2900 | 104 | 104 | 104 | 245 | |
Matlab
| 1100 / 400 | 224 | 224 | 224 | 245 |
1100 / 1100 | 224 | 224 | 224 | 245 | |
1600 / 1600 | 224 | 224 | 224 | 245 |
Performance metrics
Results
Individual-level agreement on cause of death
Test cases | Open-source random forest | Open-source tariff method | InterVA-4 | Average for top cause (%) | ||||
---|---|---|---|---|---|---|---|---|
Dataset | Top (%) | Top 3 (%) | Top (%) | Top 3 (%) | Top (%) | Top 3 (%) | ||
China
| 400 | 35 | 57 | 36 | 70 | N/A | N/A |
36
|
Institute for Health Metrics and Evaluation
| 400 | 33 | 55 | 34 | 53 | N/A | N/A |
34
|
Million Death Study
| 6100 | 58 | 82 | 52 | 76 | 42a
| 63a
|
51
|
Agincourt
| 2900 | 45 | 77 | 42 | 69 | 42 | 58 |
43
|
Matlab
| 1600 | 49 | 74 | 52 | 74 | 48 | 64 |
50
|
Average
|
44
|
69
|
43
|
68
|
44
|
62
|
Dataset | Test cases | Open-source random forest | Open-source tariff method | InterVA-4 | Average for top cause (%) | |||
---|---|---|---|---|---|---|---|---|
Top (%) | Top 3 (%) | Top (%) | Top 3 (%) | Top (%) | Top 3 (%) | |||
China
| 400 | 33 | 55 | 32 | 64 | N/A | N/A |
33
|
Institute for Health Metrics and Evaluation
| 400 | 31 | 54 | 32 | 48 | N/A | N/A |
32
|
Million Death Study
| 6100 | 55 | 81 | 48 | 70 | 38a
| 60a
|
47
|
Agincourt
| 2900 | 42 | 75 | 38 | 62 | 39 | 56 |
40
|
Matlab
| 1600 | 45 | 72 | 48 | 68 | 45 | 59 |
46
|
Average
|
41
|
67
|
40
|
62
|
41
|
58
|
Population-level agreement on cause-specific mortality fraction
Datasets | Test cases | King-Lu (%) | Open-source random forest (%) | Open-source tariff method (%) | InterVA-4 (%) | Average (%) |
---|---|---|---|---|---|---|
China
| 400 | 84 | 79 | 75 | N/A |
79
|
Institute for Health Metrics and Evaluation
| 400 | 88 | 73 | 63 | N/A |
75
|
Million Death Study
| 6100 | 96 | 64 | 33 | 70a
|
66
|
Agincourt
| 2900 | 94 | 72 | 38 | 75 |
70
|
Matlab
| 1600 | 95 | 69 | 59 | 72 |
74
|
Average
|
91
|
71
|
54
|
72
|