Background
Methods
Overview
Identification of the research question
Formulation of the theoretical framework
Search for primary studies
Disaster |
AND quality OR monitoring OR evaluation OR “after action report” OR “after-action-report” OR “after action review” OR “after-action-review” OR “after event report” OR “after-event-report” OR “after event review” OR “after-event-review” OR drills |
AND response OR recovery OR preparedness OR management |
AND Health |
Selection and quality appraisal of studies
Inclusion | Exclusion |
---|---|
Papers which examined at least one stage of the theoretical framework (document and assess; disseminate; or implement) | Papers which focused on instruments used for the evaluation or adoption of performance standards, which are highly specific to the hospital setting |
Papers which examined the evaluation of quality improvement efforts at the hospital level were included if their content was deemed to be applicable to health departments in general | Papers which focused on evaluating elements of preparedness (i.e. facilities or training of health personnel) or were reporting the health outcomes after a disaster rather than the response itself |
Papers which focused on research methods and metrics for the evaluation of disaster management and did not explore programme or policy aspects of the evaluation | |
Papers which reported the results of an evaluation but did not provide a description of how the evaluation was conducted, or how the lessons learned were disseminated or incorporated into future plans |
Extraction, analysis and synthesis of relevant data
Refinement of the theoretical framework
Results & findings
Description of primary studies included in the analysis
Stage of M&E | Context | Mechanism | Outcome |
---|---|---|---|
Document and assess | Assurance that staff will not be penalised for mistakes. | A culture of learning is instilled in the health department | More accurate and high quality documentation and assessment of disaster response |
Training in the procedures and skills for documenting and assessing a disaster response are embedded into regular work | Provides staff with an opportunity to practice and become proficient in the analytical skills required to successfully document and assess their response. | Better prepared staff | |
Using external partners to document and assess the disaster response | External partners have the time, knowledge and skills to thoroughly document and assess a response | Improved depth and quality of documentation and assessment | |
Disseminate | Disseminating standardised reports | Allows all health departments to perceive the experiences of others as relevant to their own work | Promotes inter-agency learning |
Implement | Positive leadership attributes | Establishes a culture of quality improvement within the health department which increases commitment | Promotes successful implementation of changes to improve performance |
Shared decision making structures | Fosters a sense of ownership and responsibility amongst personnel which increases motivation and commitment | Promotes successful implementation of changes to improve performance |
Demi-regularities identified in the ‘document and assess’ stage of the monitoring and evaluation cycle
Demi-regularities identified in the ‘disseminate’ stage of the monitoring and evaluation cycle
Demi-regularities identified in the ‘implement’ stage of the monitoring and evaluation cycle
Discussion
Summary of findings
Strengths and limitations
Stage of theoretical framework | Document reference | Country setting | Type of disaster | Level of health system |
---|---|---|---|---|
Document and Assess | ||||
Green et al. 2003 [20] | Guatemala | Disaster scenario - propane tank explosion at an open-air food vendor | Hospital | |
Klein et al. 2005 [34] | USA | Bioterrorism | Hospital | |
Macrae 2014 | UK | Healthcare disasters | Hospital | |
Seale 2010 [28] | USA | Hurricanes (Rita 2005, Ike 2008) | Post-acute rehabilitation facility | |
Biddinger et al. 2008 [32] | USA | All public health emergencies | State and local public health departments | |
Nelson, Lurie & Wasserman 2007 [23] | USA | All disasters | Public health system | |
Piltch-Loeb 2014a [26] | USA | All disasters | Public health system | |
Piltch-Loeb 2014b [33] | USA | All disasters | Public health system | |
Savoia, Agboola & Biddinger 2012 [14] | USA | H1N1 pandemic and hurricanes (Ike 2008, Gustav 2008, Katrina 2005) | Public health system | |
Seid et al. 2007 [22] | USA | All disasters | Public health system | |
Stebbins & Vukotich 2010 [29] | USA | Hepatitis A outbreak | Public health department | |
Adini et al. 2013 [3] | Israel | Mass casualty incidents | National health system | |
aBirkland 2009 [30] | USA | All disasters | Not health specific | |
aDonohue & Tuohy 2006 [13] | USA | All disasters | Not health specific | |
aDufty 2013 [31] | Australia | All emergencies | Not health specific | |
aSpillsbury et al. 2007 [27] | Not country specific | Not disaster specific | Not health specific | |
Disseminate | ||||
Mears et al. 2010 [35] | USA | All public health emergencies | State public health system | |
Nelson, Lurie & Wasserman 2007 [22] | USA | All disasters | Public health system | |
Piltch-Loeb 2014a [26] | USA | All disasters | Public health system | |
Savoia, Preston & Biddinger 2013 [36] | USA | All public health emergencies | Public health system | |
aDonohue & Tuohy 2006 [13] | USA | All disasters | Not health specific | |
Implement | ||||
Adini et al. 2012 [21] | Israel | Mass casualty events | Hospital | |
Klein et al. 2005 [34] | USA | Bioterrorism | Hospital | |
Bevc et al. 2012 [37] | USA | All disasters | Local health department | |
Chan et al. 2010 [38] | Indonesia | Tsunami | Local health clinics | |
Seale 2010 | USA | Hurricanes (Rita 2005, Ike 2008) | Post-acute rehabilitation facility | |
Seid et al. 2007 [22] | USA | All disasters | Public health system | |
Adini et al. 2013 [3] | Israel | Mass casualty incidents | National health system | |
aBatalden & Davidoff 2007 | Not country specific | Not disaster specific | Healthcare systems | |
aDonohue & Tuohy 2006 [13] | USA | All disasters | Not health specific |