Background
There is ample evidence of differences in health between deprived and non-deprived areas. Residents of deprived areas report worse health than those in non-deprived areas [
1‐
4]. Several studies suggest that these health inequalities can partly be explained by different levels of area disorder [
5‐
9]. Residents of deprived areas perceive more physical disorder (e.g. litter, graffiti) and social disorder (e.g. nuisance from neighbours or youth) than residents of non-deprived areas [
5,
7,
8]. Area disorder may cause people to feel unsafe, which may negatively affect their health by increasing stress, anxiety, physical inactivity, or social exclusion [
10].
A review has shown that, numerous area-based initiatives have been implemented in deprived areas across Western-Europe in the past decade [
11]. These initiatives consist of multiple interventions that aim to tackle the various socio-economic and environmental problems in deprived areas, including physical and social disorder. Two of these area-based initiatives have been evaluated for their impact on area disorder and subsequent safety concerns. Results have been mixed. After 6 years, target areas of the English New Deal for Communities had significantly larger reductions in perceived lawlessness and dereliction than other deprived areas [
12]. There were no differences in reductions of fear of crime and feeling unsafe after dark. After 5 years, target areas of the English Single Regeneration Budget saw larger reductions in the number of residents feeling very unsafe than the rest of England [
13]. However, differences were small and not tested for significance. There were no differences in reductions of disorder such as vandalism, loose dogs, and litter.
These quantitative evaluation studies have been criticized for their lack of attention to mechanisms of change and programme theory [
14‐
16]. To improve future initiatives, research needs to extend its attention from outcomes towards the processes leading to these outcomes. Pawson and Tilley’s [
17] realist evaluation methodology offers a useful approach to understand the inner workings of complex initiatives. The realist methodology aims to uncover how an intervention works, for whom, and under what conditions. More specifically, it tries to identify the so-called mechanisms of change. Mechanisms refer to individuals’ responses triggered by the intervention that lead to change. These mechanisms will only be activated under certain conditions. Interventions are often based on assumptions about possible mechanisms and conditions, but these so called programme theories are rarely made explicit. A key purpose of the realist methodology is to identify these programme theories, and to try and refine it using evidence on how the interventions worked in practice.
Only few studies have applied the realist methodology to understand how area-based interventions have influenced area disorder and safety concerns. A good example is that of Nanninga and Glebbeek [
18]. They explored whether and how two new sports fields had an impact on nuisance from youth and related crime in the Netherlands. Anticipated mechanisms of change included ‘boredom reduction’ and ‘role modelling’. Police records showed that since the arrival of the sports fields, there were less reports of nuisance, but more reports of crime. The drop in nuisance could be explained by the anticipated ‘boredom reduction’ mechanism and the newfound ‘confrontation reduction’ mechanism.
Little is known about how area disorder and safety are affected by area-based interventions that are oriented at households. A pioneering intervention in this field is the Dundee Families Project of 1996 [
19]. This project aimed to help families that were evicted or at risk of eviction because of neighbour nuisance. A mix of counselling, family support, surveillance, and regulations was offered to families, either by reaching out to families in their own homes or by admitting families to a temporary home and supporting them there. In the latter case, families could either be admitted to a core residential unit with up to four other families that were part of the project, or to one of the dispersed flats run by the project. An evaluation study revealed that the intervention was able to reduce nuisance caused by these families [
19]. Unfortunately, the study did not explore how, for whom or under what conditions this intervention was able to reduce neighbour nuisance and safety concerns.
An opportunity to acquire such knowledge arose with the introduction of the Meeting for Care and Nuisance (MCN) (Overleg Zorg en Overlast) in the four most deprived districts of Arnhem, a mid-sized city in the east of the Netherlands. This intervention is part of a large area-based initiative that was implemented in 2008 in the forty most deprived districts of the Netherlands, including the four deprived districts in Arnhem. MCN aimed to reduce neighbour nuisance (e.g. loud music, fights, neglected property), which was regarded to be the main cause of perceived unsafety in these areas. The current study aimed to explore how, for whom, and under what conditions MCN had an impact on neighbour nuisance and area-level safety in the four target districts. Following the realist methodology, we first identified the programme theory on how MCN was anticipated to work. Then, we used this programme theory as a guide to assess how MCN actually worked.
Methods
Design
An embedded case study design was applied [
20]. Each of the four deprived target districts in Arnhem represented a separate case. Within each case, three levels of analysis were distinguished: 1) the organisations that were involved in the implementation of MCN, 2) the households that were causing nuisance, 3) the four deprived districts at large.
The intervention
Neighbour nuisance is addressed by MCN in different steps. To start with, each district holds monthly meetings with the police, local housing corporations, the care coordinator, and the district manager. Under the leadership of the district manager, organisations exchange information about new and existing nuisance households during the meetings. New households are included based on signals of nuisance reported by the participating organisations, neighbours, care professionals, or others. For each new nuisance household, the care coordinator develops a plan of action that specifies what needs to be done and which organisation is responsible for what action. Plans of action are discussed with the organisations during the monthly meetings. When plans are agreed upon, the households are paid a visit by the care coordinator and a representative of one of the other organisations. The care coordinator discusses the plan of action with the household. The household’s vision on the plan is incorporated in a guidance agreement that specifies what the household should do to reduce the nuisance (e.g. turning down their music after 10 p.m). Usually, the guidance agreement is accompanied with assistance from care professionals, as many of the households deal with unresolved underlying problems (e.g. debt, addiction, psychiatric illness, unemployment, or neglect). For households who have received assistance in the past, such assistance can be conditional (e.g. households first have to clean their front yard in order to receive debt assistance). If households are unwilling to cooperate, their case is forwarded to the justice department, who decides on whether households will be threatened with sanctions such as eviction or benefit reduction. Cases are closed when no new nuisance signals are reported. During the entire process, the care coordinator monitors the progress of the households and organisations.
The cases
MCN was first implemented in 2006 in district 1. This district is a working-class district characterized by low levels of social cohesion and trust in the municipality. It has a long history of social problems and neighbour conflicts. Many repressive and restructuring interventions have been implemented in the past, but without success. In 2010, MCN was extended to districts 2 to 4. District 2 is a centrally located and densely populated district that is characterized by high levels of creative enterprises and students. Problems related to drug users, criminal activities, and deterioration of public spaces have prompted past interventions that have been somewhat successful. Districts 3 and 4 are both post-World War II districts that are characterised by uniform low-quality housing occupied by people of low socio-economic status and starters at the housing market. Various restructuring and social interventions have been implemented in the past, but with limited success. More information about the characteristics of the four districts can be found in Table
1 [
21]. As district 3 covers a relatively large and diverse area, characteristics and results are described separately for three different parts of this district.
Table 1
Characteristics of the districts and its residents [
21]
Target district 1 | 5825 | 510 | 1340 | 28 % | 16500 | 33 % |
Target district 2 | 7250 | 69 | 10581 | 28 % | 14200 | 25 % |
Target district 3a | 4505 | 427 | 1456 | 36 % | 17000 | 38 % |
Target district 3b | 5570 | 172 | 3705 | 44 % | 18200 | 26 % |
Target district 3c | 7180 | 135 | 5327 | 18 % | 13900 | 42 % |
Target district 4 | 8175 | 143 | 5727 | 20 % | 18000 | 45 % |
City of Arnhem | 148070 | 10154 | 1511 | 43 % | 21000 | 18 % |
Data collection
Based on the principles of the realist evaluation, data collection was iterative and included a wide range of sources (Table
2). Two types of qualitative evidence were obtained to assess how MCN was expected to work, i.e. the programme theory underlying MCN. First, we searched the web for action plans that set out how policy makers expected MCN to work. Five action plans were included. Second, we searched the literature for scientific theories that complemented the expectations set out in the action plans.
Documents | | | | | | | |
Action plans | | | | | | | |
Description of interventions planned for district 1–4 as part of the larger area-based initiative (incl. MCN) | Municipality of Arnhem | DOC1 | 2007 | | X | | |
DOC2 | 2007 | | | | X |
DOC3 | 2007 | | | X | |
| | DOC4 | 2007 | X | | | |
| | DOC5 | 2007 | X | | | |
Progress reports | | | | | | | |
Progress report of interventions in district 1–4 as part of the larger area-based initiative (incl. MCN) | Municipality of Arnhem | DOC6 | 2009 | X | X | X | X |
DOC7 | 2009 | X | X | X | X |
DOC8 | 2010 | X | X | X | X |
DOC9 | 2010 | X | X | X | X |
DOC10 | 2010 | X | X | X | X |
| | DOC11 | 2011 | X | X | X | X |
| | DOC12 | 2011 | X | X | X | X |
| | DOC13 | 2011 | X | X | X | X |
| | DOC14 | 2012 | X | X | X | X |
Progress report of household interventions (incl. MCN) | Government | DOC15 | 2010 | X | | | |
Progress report of MCN | Welfare organisation | DOC16 | 2011 | X | X | X | X |
| DOC17 | 2011 | X | X | X | X |
| | DOC18 | 2012 | X | X | X | X |
Media reports | | | | | | | |
Online news report about a guided tour to district 1–4 (incl. MCN) | Journalist | DOC19 | 2009 | X | X | | |
| DOC20 | 2010 | X | X | X | X |
Online news report about MCN | Journalist | DOC21 | 2009 | X | | | |
Newspaper section about safety in Arnhem (incl. MCN) | Journalist | DOC22 | 2010 | X | X | X | X |
Magazine about Dutch social enterprises (incl. MCN) | Journalist | DOC23 | 2011 | X | | | |
Short video of an interview with an MCN care coordinator | Journalist | DOC24 | 2011 | | | X | |
Survey | | | | | | | |
Survey about liveability, safety, neighbourhood problems, victimization, and municipal functioning as perceived by residents of Arnhem (incl. district 1–4) | Municipality of Arnhem | SURV1 | 2005 | X | X | X | X |
| 2007 | X | X | X | X |
SURV2 | 2009 | X | X | X | X |
SURV3 | 2011 | X | X | X | X |
Interviews | | | | | | | |
Interviews with the district managers of district 1–3 | n.a. | INT1 | 2012 | X | | | |
INT2 | 2013 | X | | | |
INT3 | 2012 | X | X | X | |
| INT4 | 2013 | X | X | X | |
| | INT5 | 2012 | | X | | |
| | INT6 | 2013 | | X | | |
Three types of qualitative and quantitative evidence were obtained to assess how MCN actually worked. First, we searched the web for documents that reported on experiences with MCN. Search terms included “OZO”, “Overleg Zorg en Overlast”, “Zorgcoordinator”, and “Arnhem”. Seventeen progress reports and media reports were included that contained information on mechanisms, outcomes and/or contexts specific to MCN in practice.
Second, we obtained existing quantitative data on district-level perceptions of neighbour nuisance and area safety from the Survey for Liveability and Safety (2005, 2007) and the Integral Safety Monitor (2009, 2011). Both surveys aimed to assess residents’ perceptions of safety and liveability and included questions regarding overall safety perception, criminal victimization, perceived neighbourhood problems, and judgement of police and municipal government. For the Survey for Liveability and Safety, individuals of 18 years and older were asked to fill in a paper-and-pencil or web-based questionnaire. In 2005, a total of 6941 adults were approached in the four target districts, of which 1661 adults completed the survey. This resulted in a response rate of 24 %, ranging from 22 to 27 % between the four target districts. The response rate was lower than in the city of Arnhem as a whole, where the response rate was 29 %. In 2007, a total of 4433 adults (12 % of the population) were approached in the four target districts, of which 1210 adults completed the survey. This resulted in a response rate of 27 %, ranging from 24 to 29 % between the four target districts. The response rate was lower than in the city of Arnhem as a whole, where the response rate was 37 %. For the Integrated Safety Monitor, individuals of 15 years and older were asked to fill in a paper-and-pencil or web-based questionnaire, or were interviewed by telephone. In 2011, a total of 4175 individuals (11 % of the population) were approached in the four target districts, of which 1463 adults completed the survey. This resulted in a response rate of 35 % in all four target districts. The response rate was lower than in Arnhem as a whole, where the response rate was 42 %. For 2009, information about sample sizes and response rates per district were unknown.
Third, we performed two rounds of individual semi-structured interviews with the district managers of three target districts. The district manager of district 4 was unwilling to participate. The first round of interviews took place after we constructed the programme theory in September 2012. District managers were asked to confirm, falsify or refine our programme theory based on their experiences with MCN in practice. The second round of interviews took place in November 2013, after we assembled and integrated the available evidence on how MCN actually worked. District managers were asked to confirm, falsify, or refine our integration of the evidence, taking into account their own experiences with MCN. During the interviews, the district managers provided us with two extra progress reports not available on the web.
Data analysis
Data analysis was guided by the framework approach of Ritchie and Spencer [
22]. First, we articulated the programme theory. We extracted information from the action plans about the outcomes and mechanisms that were anticipated to result from MCN. These expectations were complemented with information from the scientific theories. Second, we indexed the evidence on how MCN actually worked, using the programme theory as a guide. We flagged information about the mechanisms and outcomes set out in the programme theory, but were also alert for unanticipated mechanisms, outcomes, or conditions. For each extracted piece of information, we specified the data source, year, and district (if applicable). Third, we charted the extracted information. The information was separated into three datasets: one on the mechanisms, one on the conditions, and one on the outcomes. Within each dataset, information was sorted by level of analysis: organisations, nuisance households, and districts. Finally, we mapped and interpreted the information. Recurrent patterns of information were grouped and labeled. When new patterns emerged, data sources were checked again for possible additional information. Patterns were regularly discussed with the members of the research team. Where possible, patterns were compared across cases.
The Medical Ethics Committee of the Academic Medical Centre in Amsterdam, the Netherlands, has confirmed that ethics approval is not necessary, as the Medical Research Involving Human Subjects Act (WMO) does not apply to our study.
Discussion
This realist evaluation study aimed to explore the inner workings of MCN, an area-based intervention to reduce neighbour nuisance in the four most deprived districts of Arnhem, the Netherlands. Results indicate that interventions like MCN are able to reduce neighbour nuisance in deprived areas. By exploring conditions for change, we uncovered why the reduction in neighbour nuisance led to improved perceptions of area safety in some districts but not in others.
Limitations
This study had some potential limitations that should be taken into account when interpreting the results. According to the realist evaluation method, the impact of the mechanisms that are activated by a programme largely depends on conditions in which they are activated [
17]. One of the central aims of a realist evaluation is to identify such conditions. At the district level, we were able to identify several conditions that enabled or constrained the success of MCN, such as population density and the extent to which MCN is integrated into a wider safety approach. However, we had no detailed information on the implementation of MCN in the different districts. As a result, we were limited in our ability to explain between-district differences in effects of MCN on safety perceptions.
We had only limited information available about the intensity of the intervention, including budget and number of organisations involved. Some information about budget was available from a social cost-benefit analysis, which showed that 10 % of the costs for MCN were spent on staff costs for the care coordinators and 90 % was spent on new care trajectories for the nuisance households [
26]. However, more detailed information on budget and staffing is needed.
Information on mechanisms and outcomes at the level of nuisance households originated mostly from interviews with district managers and documents from the municipality and other participating organisations. Neighbourhood residents’ views on mechanisms were unknown and their views on the outcome measure were only indirectly included in the form of the number of nuisance signals reported to the authorities. Information about nuisance households’ views on mechanisms and outcomes were absent. We acknowledge that neighbourhood residents and nuisance households might have disclosed alternative views on the mechanisms and outcomes of MCN. More specifically, the participating organisations may have been overly positive about the success of MCN and a more sober picture may have arisen from interviews with household members or their neighbours.
Quantitative data about district level outcomes were obtained from four repeated cross-sectional surveys. Over the years, response rates in the four target districts varied between 24 and 35 %. This is lower than response rates in the city of Arnhem as a whole, and the average response rate for web-based surveys [
27]. The low response rate in our districts may have biased our results if non-response was selective in ways related to our study outcome, and if the selectivity of the non-response changed over the years. Unfortunately, we were unable to perform non-response analyses. To reduce some of this bias, survey data was weighted for age and gender and for ethnicity in 2009 and 2011.
Refining the programme theory
Among participating organisations, the coordinated partnership strategy applied by MCN led to an increase in the efficiency with which households were approached and helped. As suggested by Winer and Karen [
23], this outcome may be the result of improved coordination among organisations. Coordination was improved by aligning the roles of the organisations, and by shortening the communication lines. MCN was able to increase efficiency not only because of improved coordination, but also because of improved leadership. This mechanism may explain why power issues have not been a problem with MCN, while Winer and Karen anticipated power issues to be a problem with this strategy [
23]. Our results further suggest that two conditions should be met in order for this type of partnership to engender the abovementioned mechanisms. First, privacy issues need to be settled, for example by means of protocols, in order to ensure free exchange of confidential information. Second, it is important to involve only a small group of organisations that are all oriented at the same goal, in our case nuisance reduction.
Among the majority of the nuisance households, the joint assistance and enforcement strategy applied by MCN led to a reduction in nuisance. As suggested by Clarke’s situational crime prevention theory [
24], this outcome may be the result of a reduction in the underlying reasons for nuisance, which limited the opportunities for nuisance. Reasons for nuisance were successfully tackled by stimulating conscience, setting rules, facilitating compliance, and controlling disinhibitors. While Clarke suggests using physical strategies (e.g. signs, facilities) to activate these mechanisms, our results show that more socially oriented strategies may be effective as well. Conscience was stimulated by means of the house visit, which made households aware of their behaviour and its impact on others. Rules were set by means of the guidance agreement, which urged households to take responsibility for their behaviour. Compliance was facilitated and disinhibitors were controlled by means of assistance, which helped households to solve problems like debt and addiction. MCN was able to reduce nuisance among most households not only because it reduced the underlying reasons for nuisance but also because it increased feelings of formal surveillance. This mechanism relates to one of Clarke’s other opportunity-reducing measures: increasing risk. In a few households, MCN was unable to reduce nuisance by tackling reasons or increasing perceived surveillance. For some of these families, pressure proved to be a successful mechanism to reduce nuisance. We should note that this mechanism operated in only a small minority of households.
Among residents of one of the four districts, the reduction in neighbor nuisance that was accomplished by MCN led to improved district-wide perceptions of area safety. As suggested by Taylor’s incivilities thesis [
25], this outcome may have been the result of improved district-wide perceptions of informal and formal social control. The fact that results were only visible in one of the four districts, suggests that the incivilities thesis only holds under certain conditions. A first condition appears to be that the intervention needs to be part of a wider safety approach. Perceptions of area safety are not only affected by neighbour nuisance, but also by problems like litter, crime, or decay, and by processes like gentrification. A second condition appears to be that the target area needs to be relatively small and centrally located within the city. This seemed to facilitate the success of the wider safety approach. Moreover, we suggest that this type of area may also be more densely populated, as was the case with district 2 (see Table
1), which increases exposure to the nuisance and subsequent actions taken by organisations.
Methodological considerations
This study illustrates how a realist evaluation can help strengthen the evidence base for complex area-based interventions that aim to improve social determinants of health like area safety. Despite the limitations of our study, we were able to show how and under what conditions a multi-component area-based intervention was able to reduce neighbour nuisance and improve perceptions of area safety. Conventional quantitative evaluation studies would have concluded that MCN has failed, since it was not followed by improved safety perceptions in most target districts. This study shows that these interventions are too complex to judge based on a simple pass or fail verdict, and that more complex evaluation methods like the realist evaluation are needed to understand its complexity. Moreover, this study illustrates how information on mechanisms of change and conditions for success may help refine both the intervention’s programme theory and the theoretical frameworks on which the programme theory is built. However, more realist evaluations are needed to establish a solid evidence base for the health impact of complex area-based interventions, as only a limited amount of information can be retrieved in a single study like ours [
28]. Moreover, due to the inherent subjectivity of the realist approach, this type of study needs replication as well as clear standards for analyses and reporting [
29].
Competing interests
The authors declare that they have no competing interests. None of the authors are in any way—financially or personally—related to the leadership of the intervention and the subjects of the qualitative interviews.
Authors’ contributions
DK and AEK designed the study. DK and AEK performed interviews with district managers. DK collected all other relevant data. With the help of AEK and JH, DK analyzed the data and wrote the manuscript. All authors have read and approved the final version of the manuscript.