Insights for future programmes
The review found evidence on the impact and implementation of various strategies with an individual orientation within three national UK policy programmes: the ONE Advisory Service, New Deal for Disabled People and Pathways to Work. What lessons are there for planning future policies and improving existing initiatives in this area?
One key finding for future programmes is the need for more effective and long-term evaluation of any programmes initiated by central government. No randomised controlled trials (RCTs) were identified in our searches, and whilst those observational studies using comparison groups or areas provided reasonably robust evidence, most did not adequately control for or discuss the problems of selection bias in both the intervention and comparison groups. This limits the conclusions that can be drawn about the employment effects reported in these studies. Overall the qualitative studies were well-designed although, given the space available in these extensive reports, relatively few provided adequate details of how the findings and conclusions were derived from the data. The multi-method comprehensive programmes of evaluation, such as those for the New Deal and Pathways provide good models of how to evaluate both the effects and the processes of complex social policy interventions. One notable absence, however, is that of well-designed longitudinal studies. Such studies may address a number of questions raised by the studies reported here, including whether the employment outcomes reported were sustainable
There was evidence from several studies that personal advisors and individual case management in these schemes helped some participants back to work, but there was widespread selection into these programmes of more work-ready claimants, which biased the results of the evaluations. This also resulted in inequalities in access to the support on offer. Claimants 'job readiness', age, type of disability or health condition or family situation, as well as local labour market conditions and welfare benefits context influenced whether an incapacity benefit claimant was selected or not. This included the self-selection deriving from the voluntary nature of work-focused interviews under New Deal and from the fact that whilst these were mandatory under the Pathways to Work pilots, they were only open to new claimants.
In addition, Personal Advisors across the initiatives were more willing to work with claimants who were more work-ready, often in order to fulfil job outcome targets or other organisational needs. From an evaluation perspective, selection into programmes of those claimants closer to the labour market makes it difficult to assess how effective these programmes can be with those more distant from the labour market, and how programmes might be changed to increase their effectiveness for these groups.
The problem of selection also hindered the interpretation of evidence of differential impact. There was an intriguing finding of differential impact from one
New Deal study, for example, with a greater likelihood of employment following participation in the programme for participants who had been claiming incapacity benefit for three years or more, those furthest from the labour market and those living in areas with the highest rates of economic inactivity. The voluntary nature of
New Deal and the selection activities of advisors, however, coloured the interpretation of these findings so that we cannot draw conclusions about whether the differential effect was robust. Examining the employment impact of
Pathways, one report [
33] found no significant difference by gender, whereas the follow up [
34] found a larger positive effect for women and for those with dependent children. Neither the quantitative nor qualitative studies of the
Pathways pilots provide any clear explanation of what aspect of the programme might be producing these differential effects.
A recent assessment of the effectiveness of
Pathways to Work [
3] noted that the positive employment outcomes found in the pilot studies have not been replicated since the programme was rolled out nationally. While new claimants moved off benefits sooner than without the programme, around 80% of this was due to claimants failing the medical assessment at an earlier stage in
Pathways areas rather than their participation in the programme. The assessment also notes that recent evaluations indicate that once on incapacity-related benefits, new claimants are just as likely to move into employment without
Pathways support as with it. The report concluded that the obligatory work focused interviews and early medical assessment in
Pathways were the key aspects of the programme that move people off benefits and into work, whilst the voluntary components, such as the
Condition Management Programme and
Return to Work Credit, apparently had no additional employment impact. The assessment, however, did not provide data on differential effects, nor did it take into account the problems of selection identified in our review. As
Pathways has largely been applied to new and repeat claimants, who are much more likely to return to the labour market unaided, it is perhaps not surprising that its voluntary components show no additional employment effect.
Another issue for these programmes, somewhat mitigated in
Pathways to Work by its mandatory nature, was that awareness of the schemes and reach among the target population of incapacity benefit claimants was generally low. The overall take-up rate for the
New Deal for Disabled People, for example, amounted to just over 3% of the eligible population between 2001 and 2006 [
48]. Uptake of return to work and in-work benefits was also low. From a policy perspective, it is important to build into national programmes improved arrangements for promoting awareness of the services and incentives on offer, together with careful monitoring of differential access and outcome of initiatives.
The studies reviewed were mostly conducted between 2000 and 2006, a period of general employment expansion in the UK, but this was not associated with a reduction in the stock of incapacity benefit claimants [
49]. Whether the positive (if limited) employment outcomes indicated for some programmes would be maintained in less favourable employment conditions remains to be tested.
Each of the three main programmes under review here employed work-focused interviews as the main means of assessing claimants' work-readiness and designing an individualised package of assistance aimed at assisting their movement into the labour market. The qualitative studies provided valuable insights into the difficulties of implementing and operating these individualised active labour market programmes. A common theme in the qualitative studies with Personal Advisors was that helping those furthest from the labour market (i.e. the majority of existing claimants) requires more time than is often acknowledged by programme managers or within the job outcome targets. Linked to this is that the achievement of positive outcomes relied on the building of trust through the creation of understanding, positive and supportive relationships between the staff and claimants, usually on a one-to-one basis. Personal Advisors and claimants saw potential barriers to the establishment of such trusting relationships in the limited time available for personal contact, in claimants concerns about potential income reduction through loss of benefits and in that part of the personal advisor role which enforced benefit sanctions for failure to attend work-focused interviews. Finally, the targets that advisors were set were focussed largely on how many disabled people were moved off benefits and into work, and ignored important improvements in moving people closer to getting a job. There was some suggestion that such targets encouraged 'cream-skimming', particularly among private providers of Job Broker services.
The studies reviewed provide little evidence on the comparative effectiveness of public, private and voluntary sector delivery models. The
ONE studies reported no differences in employment outcomes between the Private and Voluntary Sector (PVS) model and the other models. The poor performance of the PVS model resulted in it being omitted from the subsequent
Jobcentre Plus Pathfinders programme [
50]. Furthermore, the 2010 National Audit Office assessment of
Pathways to Work concluded that Jobcentre Plus
Pathways had performed better than the voluntary/private sector provider-led
Pathways in terms of getting mandatory participants into work [
3]. Despite this evidence, proposed reforms in the UK still favour the private and voluntary sector model on the (unsupported) assumption that it is more effective.
The findings suggest that the financial incentives for disabled people built into the national programmes, such as the Return to Work Credit and Permitted Work, can support the process of transition into work. They also highlighted obstacles to the effectiveness of the schemes. The financial incentives were set too low or did not last long enough to assist with transitions into employment, particularly where this meant that other benefits (e.g. housing benefit, council tax benefit) were reduced when returning to work. These incentives also had low take up due to low levels of awareness among claimants or restricted entitlement to the incentives (the latter is particularly the case with Permitted Work).
A distinct shift in thinking in the UK on what strategies might help disabled people into work was signalled with the introduction the Condition Management Programme as part of
Pathways to Work. Whilst the studies reviewed here did not provide evidence of the programme's employment impact, they provided useful insights for policy. The studies generally supported the themes noted in the Personal Advisor studies, that reintegrating claimants furthest from the labour market requires longer-term and personal engagement with individuals. This seems to be particularly critical in relation to people with mental health conditions and the intensive support needed to help them into work [
51].