Background
On March 20, 2003, the United States invaded Iraq. In November 2002, four months prior to the invasion, MedAct published a report projecting that a US military offensive against Iraq could have enormous public health impacts in the short-, medium- and long-term [
1]. The report estimated that the immediate, direct conflict-related casualties could, within only months of the start of a conventional war, range from 48,716 to 261,100 for Iraqi civilians, combatants and coalition forces combined, excluding the resultant indirect casualties and the possibility of a civil war.
In April 2003, the coalition forces removed Hussein's regime from power and US President Bush announced that major combat operations had ended [
2]. As Hussein's Baath Party continued to disintegrate in the summer of 2003, the nature of the conflict began to shift. Although the U.S. declared major combat operations completed, insurgents continue to battle the coalition occupying forces using asymmetric warfare and guerilla tactics.
More than four years following the declaration that major combat operations had ended, violence in Iraq continues. Sectarian and insurgent attacks are responsible for the majority of current violence. Most attacks target civilians on the basis of their ethnicity or religion, or perceived affiliation with the new Iraqi government or occupying forces. The United Nations has stated that Iraq is on the brink of civil war [
3].
Determining the total number of deaths resulting directly and indirectly from armed conflict is challenging. The war has caused the destruction of much of the health infrastructure and health information systems in Iraq, systems already weakened by UN sanctions in the decade prior [
4]. As a result, attempts to determine the total number of Iraqi deaths have proved particularly problematic [
5,
6].
Iraqi government ministries, non-government organizations, academic institutions and media reporting agencies have attempted to estimate the total war-related deaths. The methods used to determine mortality have varied and, as a result, the mortality estimates are widely divergent. We critically examined the existing studies estimating the extent of Iraqi deaths and reviewed their methodological quality. We aimed to provide estimates of total mortality, average deaths per day, along with crude and cause-specific mortality rates attributable to violence. As a result, we chose a broad scope for the data abstraction.
Methods
Search strategy
We (CT, EJM, HA) conducted a systematic search of the following electronic databases from inception to January 2008: Allied and Complementary Medicine Database (AMED), Cumulative Index to Nursing & Allied Health Literature (CINAHL), Cochrane Library and Cochrane Central, Excerpta Medica (EMBASE), MedLine, Education Resources Information Center (ERIC), HealthStar, Agency for Healthcare Research and Quality – all through Ovid, Web of Science, and the United Nations websites (UNDP, UNHCR, UNICEF, WFP, WHO). We had no language restrictions and also specifically searched in Arabic. The key search term used was "Iraq". Where the volume of studies and reports produced was unmanageable, additional limits were set by adding the terms "casualties or deaths or mortality or civilian."
Additionally, we (CT, EJM, HA) conducted a non-structured review of ReliefWeb, Google Scholar and Google, recognizing the likelihood that investigators may have conducted research that they did not publish in health sciences information databases [
7]. Detailed review of the references listed in key studies and reports provided an additional source of studies. Where unpublished reports or data were referenced, key individuals provided further information and access to these studies.
Inclusion and Exclusion Criteria
We included studies and reports that assessed the total number of Iraqi deaths since March 20, 2003, the start of the war. Eligible studies incorporated Iraqi civilians and non-combatants in the total mortality estimates and were based on primary research done with the intent – either wholly or in part – of determining mortality estimates; studies that included combatants in addition to civilians and non-combatants were also included. We included unpublished primary research that was sufficiently reported via media sources and where key data could be extracted.
We excluded studies and reports that were limited to a specific sub-population (i.e. children under 5 years); however, studies were included if estimates on a specific sub-population were reported as a component of a larger total population number. Studies that reported only 'casualties' and did not separate out mortality estimates were excluded, as casualty reports may also include injuries. Reports of total mortality estimates which were included as part of a separate data source were not included, to reduce the possibility of duplicate reporting on the same data set. However, where methodologies differed and mortality estimates were not comparable, all studies were included.
Data Abstraction
Using a priori defined study characteristics, we (CT, EJM) abstracted the following data from all studies: 1) date of data collection; 2) time period for reported deaths; 3) total number of deaths reported; 4) each of total combatant deaths, total civilian/non-combatant deaths; and, 5) cause-specific deaths attributable to violence, where available [Additional file
1]. In our review, 'mortality' refers to death from any cause, whereas cause-specific mortality confers the reported reasons for death, generally deaths due to violence. Additionally, data abstracted for the population-based surveys included methods of sample selection, sample size and pre-war data comparisons. For other studies, both published and unpublished, we also extracted information on study type and data source.
Methodological Quality
There is a paucity of empirical evidence examining methodological quality in mortality estimates. We recently convened a methods workshop to determine recommended quality indicators in mortality surveys [
8]. These methodological quality recommendations should only be applied to population-based studies and address issues of coverage, bias, completeness and accuracy. For the purpose of this manuscript, we extracted data on the following: a) whether the sample was sufficiently representative of the underlying population affected by the conflict; b) whether the population sampling avoided bias (i.e. was the random sampling systematic or simple); c) whether the response rate was reported; d) whether efforts were made to confirm deaths; and, e) whether households were revisited to confirm findings. Appendix 1 explains the rationale behind these quality indicators.
Statistical Analysis
Given the heterogeneous methods and time periods applied to estimating mortality, we did not pool the included studies. Further, given the intense political interest in the mortality estimates, we provided details from each study. Where not already done by study authors, we (CT, EJM) calculated crude mortality rates (CMRs) and cause-specific mortality rates due to violence (VMRs) per 1,000 per year and average deaths per day based on the total mortality figures presented and time periods referenced in each study or report.
Mortality rates provide inferences on the average rate of death among a population over a specific time period. Where exact dates of birth and death events are not known, mortality rates are typically calculated as (Total number of deaths during period/Population alive at the end of the period + half of deaths during period – half livebirths during period) × (rate multiplier/analysis period in time units of choice). The rate multiplier (e.g. 1,000 or 10,000 people) and time unit (e.g. day, month, year) are chosen so as to express the rate in convenient units (in emergencies, typically per 10,000 people per day; in this paper, per 1,000 people per year). Where CMRs and VMRs were not provided by study authors, we calculated mortality rates for each study based on this formula, excluding livebirths as they are unknown. Our population estimates are based on the United Nations Development Program/Iraq Ministry of Planning and Development Cooperation population estimations for 2004. For all calculations we determined the total number of days in the study periods. Where the exact start and end date is either unknown or simply implied, we assumed either start, end or mid-month projections, indicated using ().
Discussion
Estimating mortality during conflicts, such as the Iraq war, presents important obstacles. Appropriate methods exist, such as population-based methods of retrospective surveys and community-based prospective surveillance, and are widely available on the internet and medical literature [
27,
33]. However, despite the availability of appropriate and accepted methodologies, studies we reviewed here used diverse methodologies and were of widely divergent quality – some with major weaknesses, and, presumably therefore resulting in differing estimates of mortality. Nevertheless, the media frequently reports on these estimates and they can have an important influence on the political process and subsequent foreign policy decisions [
26]. Given the importance of these reports, it is concerning that there is such a wide variance in their methods and estimates.
We have systematically reviewed the literature to identify all of the estimates of mortality and describe their methodology to bring more transparency to this critical area of epidemiology. Our review identified a total of 10 published studies that included 5 population-based studies [
16‐
20] and 5 passive reporting studies [
13,
23‐
25,
28]. Only 3 studies have been published in peer-reviewed sources [
16,
18,
20], although non-journal based publications may also hold to academic standards. We also identified 3 additional studies based on secondary reports with no identifiable source document [
29,
31,
32]. The studies provided mortality estimates varying from 48 deaths per day to 759 deaths per day. Although daily death rates and violent mortality rates have been calculated for each study, these figures should serve only as a general framework because they obscure the methodological differences in each study and surges of violence that may occur at specific times. For example, some studies limited mortality totals to civilians/non-combatants, while others included combatants; some studies included only violent deaths, others included deaths for all causes. Some also excluded Kurdish areas of Iraq, while others did not. In addition, the different methods for gathering data each inherently have strengths and limitations that affect the range, whether higher or lower, of mortality estimates produced (See Appendix 2). There are, as yet, no tools to critically appraise the quality of evidence from armed conflict studies and it is clear that there is a pressing need for the development of appraisal checklists, and further consideration of systematic reviews/meta-analyses of humanitarian emergency data [
34].
The two broad classes of data collection methods, population-based and passive reporting, partly explain the variance in the estimates (See Appendix 2) [
27]. The population-based methods are well established and a generally accepted method within the fields of epidemiology [
27,
33]. Studies using a population-based method are more sensitive for estimating mortality, by identifying non-reported deaths.
All of the 5 included studies [
16‐
20] that used population-based methods also used established methods to reduce investigator-driven bias, including random sampling,
a priori sample size estimation,
a priori specification of locations and PPS; four reported on sampling imprecision through presentation of findings with confidence intervals [
16‐
18,
20]; and, three attempted to acquire accuracy of deaths [
16‐
18]. Two of the population-based studies requested death certificates to verify causes of death and were successful in obtaining them in the vast majority of reported deaths [
16,
18]. While such methods may yield accurate estimates of national rates by sampling only a small proportion of the population, this accuracy is critically dependent on the representativeness of the sample. However, such methodologies are less specific in identifying causes of death and may be susceptible to reporting and sampling biases. This is most appropriately addressed through the requests for death certificates. Compilation from primary sources or passive reporting methods, that rely upon media and/or official sources for mortality information are likely to be more specific, however, would be expected to considerably underestimate true mortality by not capturing unreported deaths and indirect deaths, from non-violent effects of war, for example, that are not often attributed to the ongoing conflict [
35].
Of the population-based studies, the Roberts and Burnham studies provided the most rigorous methodology as their primary outcome was mortality [
16,
18]. Their methodology is similar to the consensus methods of the SMART initiative, a series of methodological recommendations for conducting research in humanitarian emergencies [
33]. Another population-based study, the Iraq Living Conditions Survey, reported lower death estimates that we assume is due to the survey being conducted barely a year into the conflict, a higher baseline mortality expectation, and differing responses to mortality when houses were revisited [
21]. However, not surprisingly their studies have been roundly criticized given the political consequences of their findings and the inherent security and political problems of conducting this type of research [
36,
37]. Some of these criticisms refer to the type of sampling, duration of interviews, the potential for reporting bias, the reliability of its pre-war estimates, and a lack of reproducibility. The study authors have acknowledged their study limitations and responded to these criticisms in detail elsewhere [
38]. They now also provide their data for reanalysis to qualified groups for further review, if requested.
Recently, a study published in the New England Journal of Medicine estimates that there were approximately 151,000 violence-related deaths from March 2003 to June 2006. The authors estimate that the completeness in reporting of deaths was 62% and that the underreporting for violent deaths may be as much as 50%. In addition, the authors did not indicate whether verification was sought by requesting death certificates and there may be a resistance of some household members to disclose cause of death. In a previous study conducted by members of the research group, household re-visits determined that there was under-reporting in the under-5 mortality rate.
Of the passive surveillance studies the IBC study was, until recently, the most frequently cited by media sources and coalition force politicians [
26]. The IBC was largely established as an activist response to US refusals to conduct mortality counts. This account, however, is problematic as it relies solely on news reports that would likely considerably underestimate the total mortality. This method does not count indirect deaths, such as increased chronic illness due to the war, or deaths that are not publicly reported. More recently, the media has relied on Iraq Ministry of Health reports that estimate 75–100 deaths per day, and a cause-specific mortality rate due to violence of 1.01 to 1.34 per 1,000 per year [
32]. This report, based on total daily deaths/body counts from hospitals across Iraq and the Baghdad morgue, is likely to provide a more accurate estimate than the IBC, but would similarly not identify indirect deaths or deaths not reported to the health facility [
36]. There is the potential that this report may overestimate the early war death toll because it extrapolates current daily death rates to produce a death total and mortality rates have steadily increased during the most recent months. However, like other passive reporting studies it would again underestimate the total mortality by not capturing unreported and indirect deaths. The utility of such collection data serves strictly as a 'sentinel case' alert that should prompt further population-based cluster sampling before such findings are widely disseminated or quoted as fact.
There have been ongoing discussions and disagreements regarding the most reliable figures derived from these studies, including extensive deconstruction of the methodological merits of each [
39,
40]. It is well known that collecting mortality data during times of protracted violence faces inherent challenges of investigator, participant, and data security. The People's Kifah study exemplifies the dangers involved in data collection related to conflict settings, as they reported that the study was terminated due to the kidnapping of a data collector.
There are several strengths and limitations to consider in interpreting this review. Strengths include the extensive searching and identification of studies that met our inclusion criteria. Other reviews and websites summarizing Iraq war deaths have also reported on estimates produced by the NGO Coordinating Committee [
27]. This estimate has, however, never been made public and sources citing it have had difficulty confirming its existence. We were successful in communicating with the Coordinating Committee and they have verified that this data does not exist (Cedric Turlan, personal communication, Dec 2006). We attempted to communicate with the authors of all published population-based studies.
There are also several limitations to consider in this review. As this topic is highly politicized, it is possible that mortality studies exist but have not been made publicly available. Indeed, as we found with the Iraqiyun and People's Kifah studies – both whose study results have been reported in the media but where the primary research is unpublished – there is reason to believe that further mortality estimates may exist. Ascertaining the reason for death is a somewhat specious endeavourer as participant recall and interpretation inevitably influences their reported reasons for death. While we would expect violent mortality to be more clearly understood than infectious diseases, for example, we recognize that the same victims affected by violent injuries may also die from diseases that are pre-existing or acquired.
Our denominator for calculating mortality rates is based on the United Nations Development Program/Iraq Ministry of Planning and Development Cooperation estimations for 2004. It is possible that this estimate is slightly misleading as we would expect some further population displacement since 2004, through either immigration or internal displacement. Finally, we aimed to review the methodological quality of the included studies. Although these quality indicators are based on consensus, there is a general lack of empirical data to guide recommendations of minimum reporting of methods and results [
41]. It is possible that other methodological items may be more important and we plan to further address these issues in upcoming work.
It has yet to be determined how long the sectarian violence in Iraq will continue and what the impact will be in terms of total mortality. The results of this systematic review demonstrate that there are immensely different mortality estimates. However, the volume of studies produced also indicates that the international community is attentively watching events unfolding in Iraq. The studies and reports show that, despite varying estimates, large numbers citizens in Iraq are still dying as a result of the war. With the growing body of data, politicians and policy makers are remiss to ignore the evidence and make policy decisions that do not consider the health, safety, security and rights of the citizens they claim to protect.
Appendix 1: Quality considerations for retrospective surveys
Coverage: Was the population sampled sufficiently representative of the underlying population affected by the conflict?
This question determines whether the households or settings visited appropriately represent populations affected by the conflict.
Bias: Was the population sampled to avoid bias?
As some populations will be more, or less, affected by the conflict, study investigators should make efforts to reduce bias in sampling the population. The most common manner to reduce bias is random selection of participating households.
COMPLETENESS: Is the response rate reported?
The response rate provides inferences on how likely the population sampled may represent the overall population. Is there reason to believe that those that did not respond are systematically different than those that did respond?
ACCURACY: Were efforts made to confirm deaths?
Did the study investigators aim to confirm deaths through corresponding evidence, eg. Death certificates.
Was a sample of households revisited to confirm findings?
Did the study investigators revisit a random sample of study households or settings to re-inquire about deaths and did they find the same results?
Competing interests
Edward Mills has previously received funding from the Canadian Government to conduct methodological research on mortality estimates. Frederick Burkle was the interim Minister of Health for Iraq during the crisis period in 2003. No funds were provided for this study.
Authors' contributions
FB, CT, KW, TT, GHG, HA, and EJM contributed to manuscript development and to the development of the quality indicators. CT and EJM conducted the database search and data abstraction for all studies. HA conducted the database search in Arabic. All authors read and approved the final manuscript.