Background
Methods
Large extent | Some extent | Unclear | Not at all | ||
---|---|---|---|---|---|
A Population: Representativeness of target population, setting & reach of intervention
| |||||
1 | Are data presented on variations in participation rate in improved housing interventions by a) setting b) delivery staff/organisations c) residents (for intervention among general target population not study area) | 0 | 0 | 0 | 39 |
2 | Is the intended target audience for adoption clearly described | 11 | 18 | 8 | 2 |
3 | Is the intended target setting for adoption clearly described? | 4 | 27 | 5 | 3 |
4 | Is there analysis of the baseline socio-demographic and ‘condition tested’ (health status) of evaluation participants versus non-participants? (relating to evaluation population only) | 0 | 0 | 2 | 37 |
B Intervention: Implementation & adaptation
| |||||
5 | Are data presented on consistency of implementation of intervention & its different components? | 0 | 2 | 2 | 35 |
6 | Are data presented on the level of training of experience required to deliver the programme or quality of implementation by different types of staff? | 0 | 1 | 1 | 37 |
7 | Is information reported on whether/how the intervention is modified to individuals/households within the study? | 5 | 6 | 0 | 11 |
8 | Are data presented on mediating factors or processes (mechanisms) through which the intervention had an impact? | 2 | 12 | 4 | 21 |
C Outcomes for decision making
| |||||
9 | Are the reported health (even if only one measure of health is comparable) outcomes comparable to wider policy/other studies? | 23 | 14 | 0 | 2 |
10 | Have additional outcomes of potential adverse impacts been reported? e.g. socio-economic impacts | 4 | 21 | 1 | 13 |
11 | Have authors demonstrated consideration of variation in reported health outcomes (key outcome of interest) by population sub-groups, or intervention setting/delivery staff? | 2 | 4 | 1 | 32 |
12 | Is there sensitivity analysis of dose–response/threshold level required to observe health effect (effect on key outcome of interest not proxies)? | 3 | 4 | 1 | 31 |
13 | Are data on costs presented? Are standard economic/accounting methods used? | 2 | 19 | 0 | 18 |
D Maintenance and institutionalisation of intervention
| |||||
14 | Are long term effects reported? (12 months or longer since exposure to the intervention) | 10 | 13 | 4 | 11 |
15 | Are data reported on the sustainability (or reinvention or evolution) of programme implementation and intervention, at least 12 months after the formal evaluation? | 0 | 0 | 0 | 29 |
16 a | Is the drop-out rate/attrition reported? | 19 (Yes) | 10 (N/A) | ||
16 b | Are data on attrition by baseline health status of dropouts reported and are analyses conducted of the representativeness of remaining sample at time of final follow-up (or main follow-up time point- as appropriate)? | 0 | 0 | 0 | 29 (10 N/A) |
Results
Application of Green & Glasgow tool
Reporting of external validity in housing improvement studies
Author year | Study design | Internal validity grade | External validity domains (maximum possible score) | ||||
---|---|---|---|---|---|---|---|
Reach & representat’n (4) | Implementat’n & adaptation (4) | Outcomes (5) | Maintenance & institutionalisat’n (4) | Total (17) | |||
Intervention: Warmth & Energy Efficiency improvements (post 1980) (n = 19)
| |||||||
Heyman et al. 2010 | RCT | A | 1 | 2 | 3 | 2 | 8 |
Braubach et al. 2008 | CBA | A | 2 | 1 | 1 | 1 | 5 |
Howden-Chapman et al. 2008 | RCT | A | 2 | 1 | 1 | 1 | 5 |
Barton et al. 2007 | RCT | A | 2 | 1 | 3 | 2 | 8 |
Howden-Chapman et al. 2007 | RCT | A | 2 | 0 | 3 | 1 | 6 |
Platt et al. 2007 | CBA | A | 1 | 1 | 4 | 2 | 8 |
Lloyd et al. 2008 | CBA | B | 1 | 0 | 3 | 2 | 6 |
Shortt et al. 2007 | CBA | B | 2 | 2 | 2 | 2 | 8 |
Somerville et al. 2000 | UBA | B | 2 | 1 | 3 | 1 | 7 |
Hopton et al. 1996 | CBA | B | 2 | 0 | 2 | 1 | 5 |
Warm Front Study Group 2006 | RC | C | 1 | 1 | 5 | 0 | 7 |
Allen 2005 a | UBA | C | 1 | 1 | 2 | 2 | 6 |
Allen 2005 b | UBA | C | 1 | 2 | 1 | 1 | 5 |
Health Action Kirklees 2005 | R | C | 1 | 0 | 1 | 0 | 2 |
Eick et al. 2004 | RCT | C | 2 | 1 | 4 | 2 | 9 |
Winder et al. 2003 | UBA | C | 1 | 0 | 0 | 1 | 2 |
Caldwell et al. 2001 | CBA | C | 2 | 2 | 3 | 2 | 9 |
Green et al. 1999 | RC | C | 1 | 0 | 3 | 0 | 4 |
Iversen et al. 1986 | CBA | C | 1 | 0 | 1 | 1 | 3 |
Mean (range)
| 1.47 (1–2) | 0.84 (0–2) | 2.37 (0–4) | 1.26 (0–2) | 5.95 (2–9) | ||
Intervention: Rehousing/retrofitting +/− neighbourhood renewal (post 1995) (n = 10)
| |||||||
Kearns et al. 2008 | CBA | A | 1 | 1 | 3 | 2 | 7 |
Thomson et al. 2007 | CBA | A | 2 | 0 | 2 | 2 | 6 |
Critchley et al. 2004 | CBA | A | 0 | 1 | 4 | 1 | 6 |
Thomas et al. 2005 | CBA | B | 2 | 1 | 3 | 1 | 7 |
Barnes et al. 2003 | CBA | B | 2 | 0 | 2 | 2 | 6 |
Evans et al. 2002 | CBA | B | 1 | 0 | 2 | 0 | 3 |
Blackman et al. 2001 | UBA | C | 2 | 0 | 2 | 2 | 6 |
Wells 2000 | UBA | C | 2 | 0 | 1 | 2 | 5 |
Ambrose 1999 | UBA | C | 2 | 1 | 3 | 2 | 8 |
Halpern 1995 | XUBA | C | 2 | 1 | 2 | 1 | 6 |
Mean (range)
| 1.6 (0–2) | 0.50 (0–1) | 2.40 (1–4) | 1.50 (0–2) | 6 (3–8) | ||
Intervention: Provision of basic housing needs/developing country intervention (n = 6)
| |||||||
Cattaneo et al. 2006 | RC | B | 2 | 1 | 4 | 1 | 8 |
Choudhary et al. 2002 | RC | B | 1 | 1 | 3 | 1 | 6 |
Aga Khan Health Service 2001 | XCBA | B | 1 | 1 | 2 | 1 | 5 |
Spiegel et al. 2003 | XCBA | C | 2 | 0 | 1 | 1 | 4 |
Aiga et al. 2002 | XCBA | C | 1 | 1 | 3 | 0 | 5 |
Wolff et al. 2001 | XCBA | C | 2 | 0 | 2 | 1 | 5 |
Mean (range)
| 1.50 (1–2) | 0.67 (0–1) | 2.67 (1–4) | 0.83 (0–1) | 5.5 (4–8) | ||
Intervention: Rehousing from slums (pre 1965) (n = 4)
| |||||||
Wilner et al. 1960 | CBA | A | 2 | 1 | 3 | 2 | 8 |
McGonigle et al. 1936 | XCBA | B | 2 | 1 | 4 | 1 | 8 |
Ferguson 1954 | RC | C | 1 | 0 | 1 | 1 | 3 |
Chapin 1938 | UBA | C | 2 | 1 | 4 | 2 | 9 |
Mean (range)
| 1.75 (1–2) | 0.75 (0–1) | 3.00 (2–4) | 1.50 (1–2) | 7.00 (3–9) | ||
TOTAL (n = 39 studies) MEAN (range)
| 1.54 (0–2) | 0.72 (0–2) | 2.49 (0–4) | 1.28 (0–2) | 6.00 (2–9) |