“Who needs secure services for personality disorder?” Results of an expert Delphi study with professional staff

Offenders with PD are still considered one of the most difficult groups to treat, with many studies reporting engagement difficulties, high rates of attrition [19], and poor psychosocial and clinical outcomes [20]. Individuals that commence treatment but disengage prematurely have been shown to have a greater risk of recidivism compared to those who never started treatment in the first place [21]. Therefore, it is important that offenders with PD receive treatment best suited to their needs, risk and presentation.

Prior to 2015, most services for offenders with PD were concentrated in high and medium secure hospitals under the Dangerous and Severe Personality Disorder (DSPD) programme [22]. However, in 2015 those resources were reorganised in accordance with the new Offender Personality Disorder (OPD) pathway strategy. The OPD strategy was commissioned to improve identification and assessment of offenders with PD, providing psychologically informed services for the most high-risk and complex cases [23]. It aims to offer a holistic approach, considering an offender’s journey, from conviction and sentencing through to community re-settlement, and proposing a coherent and consistent transition between the specialist criminal justice and health services. One of the main aims of the OPD strategy was ensure many more offenders with PD were treated in prison settings, with transfer to hospital reserved only for those unable to benefit from this.

Estimates suggest that 20,000 offenders in England and Wales meet the OPD criteria [24]. However, the associated costs for treating offenders with PD are high [25], and treating these individuals in a specialised forensic mental health service is much more expensive than a standard prison programme. For example, it is estimated that it costs £175,000 (approx. USD $222,000) to detain a patient in a Medium Secure Unit per annum [26]; in contrast, detaining an offender in a Male Category B prison costs an estimated £25,007 (approx. USD £31,700) per patient per annum [27].

Therefore, a key rationale of the move to the OPD Pathway was the very low cost-effectiveness of the hospital services contained within the DSPD Programme, which offered a very small number of beds with a very high monetary investment to prevent few future offences [28]. Establishing specialist prison-based services enables many more high-risk PDOs to receive interventions within a more efficient setting; this also implies that decisions to transfer an individual to a specialist forensic mental health service need to be based upon sound evidence and professional consensus. However, the question of what constitutes appropriate transfer from prison to hospital has never been clarified. The OPD pathway sets out very broad, non-exclusive criteria such as co-morbid severe mental health problems, added neurological difficulties, uncertain or disputed diagnosis and high risk [29]. Some of these criteria also lack a clear rationale, with many individuals meeting these criteria serving their sentence in a prison environment. Existing evidence suggests that currently, decision-making within teams working with offenders with a PD diagnosis is inconsistent, inequitably applied – e.g. less often to ethnic minority cases – and can even tend to undermine the institutional purpose of these hospital services [30]. This is likely because, in the UK the legislation covering hospital transfer was.

Multi-disciplinary working and specialist training are considered critical for success in working with offenders with PD [31]. The OPD strategy highlights the importance of joint operations and collaboration between prison, probation and secure mental health services [32]. Thus, within OPD pathways services, support and advice are provided by clinicians working alongside probation officers, and that a formulation and pathway plan are completed jointly [23]. This means clinicians and practitioners who play an integral role in the assessment and treatment of offenders with PD are often responsible for the decision to make a referral to a health pathway. To date, the perspectives of professionals with expertise in the field have been absent from empirical research regarding this matter, despite a recent literature review identifying the potential negative impacts of working with this offender group (Freestone et al., 2015). Due to the fundamental role they play, capturing the perspectives of these individuals is fundamental to developing coherent and consistent practice.

A systematic review of the literature [33] indicated several common characteristics of individuals with PD transferred to health pathways, as well as identifying predictors, mediators and moderators of successful treatment. Whilst the systematic review yielded 38 relevant papers, no article clearly identified the characteristics associated with appropriate and successful transfer from prison to a hospital setting. Therefore, greater clarity is needed. This study aims to address this gap in the literature by consulting directly with senior professionals practising within the OPD pathway to obtain consensus regarding the factors associated with a need for medium and high secure psychiatric services for male offenders with PD.

Participants were professional experts or published researchers in the field who were affiliated to one of three professional domains: forensic mental healthcare; criminal justice and specialist commissioning. Researchers were identified through a systematic review of the literature with corresponding authors contacted. Clinicians were identified through a snowballing process beginning with established working relationships with the study team, as well as searching clinical service directories. Electronic snowballing then ensued, with those responding asked to nominate any other individuals they thought would meet the eligibility criteria.

To be eligible in the study individuals needed to be:

Delphi surveys typically yield response rates of approximately 66–82% [36], meaning a larger number of professional experts were contacted than the required sample. We aimed to recruit equal numbers of professional experts from each domain.

A 37-point Likert scale Delphi survey (Additional file 1) was developed using the SurveyMonkey web package. The question content was derived from a systematic review of the literature [30] and items were classified into four themes: diagnosis; clinical characteristics; offence history; and ‘other’. The survey was piloted by two independent professionals. Jairath and Weinsten [37] emphasise the importance of pilot testing to identify any errors or administration difficulties.

An initial email was sent to professional experts in the field outlining the study, detailing the inclusion criteria, gauging interest and requesting agreement to participate. Individuals who agreed to participate were emailed a link to access the questionnaire via the SurveyMonkey website (www.​surveymonkey.​com). A copy of this initial questionnaire is available as Additional file 1. On activating the link, participants had a total of 14 days to complete the survey. A reminder email was sent to participants if they had not completed the survey within 7 days.

In round one, participants were asked to rate the importance of characteristics of offenders with PD across three outcome dimensions:

Participants could respond via a 5-point Likert rating scale (highly likely, somewhat likely, unsure, somewhat unlikely, highly unlikely), and were able to insert free-text comments or qualifying remarks. Participants were also asked to comment on the following additional factors: level of risk, the impact of the therapeutic relationship, important staff qualities and the optimum length of stay.

The results from the first round were analysed and used to create the second-round survey, and participants who had consented to be contacted again were emailed a link to the second-round survey. In the second round, participants were presented with feedback on the responses of the panel as a whole - in this case the percentage of panellists who suggested a specific item would be suitable for hospital admission– and were asked to re-rate the item using the same Likert scale as in previous rounds.

Ethical approval for this study was obtained by the North Central London Research Consortium, NoClor), the NHS Local Ethics service, the NOMS Research Approval Committee and the UK Health Research Authority (HRA). The reference for the Ethical Approval is 16/WS/0004.

For both rounds, written consent was obtained electronically, and participants were informed of their right to withdraw at any stage. To ensure anonymity, participants were administered a unique, eight-digit identification number which allowed the researcher to identify their data should the individual decide to withdraw from the study.

First-round responses were analysed to provide an initial breakdown of the frequency of endorsed items (n, %). The five-point Likert scale attached to each item was condensed into two categories: ‘Likely’ combining the ‘highly likely’ and ‘somewhat likely’ responses; and Unlikely/Unsure combining the ‘highly unlikely’, ‘somewhat unlikely’ and ‘unsure’ responses. The ‘Unsure’ responses were pooled with the ‘Unlikely’ responses because this was interpreted by researchers as a failure to positively endorse an item. Consensus of an item was defined as equal to or greater than 50% agreement of the ‘Likely’ responses. Free-text responses were analysed using thematic analysis and any additional items absent from the initial survey were highlighted. Items mentioned repeatedly under any item in the survey and endorsed by more than 20% of individuals were also included in the second-round survey.

For the second round, items from the original questionnaire with > 50% endorsement, or new items with > 20% endorsement were included in a revised questionnaire, and respondents were asked to re-rate the new items in the list on the same Likert scale. Those items exceeding 50% consensus were included in the final list of factors, and then used as the basis of a draft screening questionnaire for admission to PD services.

Consensus (50% endorsement or better) was reached for 23 (66%) of the items regarding suitability of hospital admission: 5 (14%) from diagnosis, 8 (22%) from characteristics, 8 (22%) from offence history and 6 (16%) from other. Two items from offence history were merged (previous convictions; early onset of offending) due to their similar content, meaning a total of 22 items were included in the second-round survey. The highest consensus item was co-morbid PD & Severe and enduring Mental Illness (SMI) with 100% of experts indicating suitability for hospital admission. Ambiguity arose from one of the items (IQ), meaning further clarification was required during the second-round survey. Eleven items (31%) failed to meet the 50% threshold, meaning these items were discarded and omitted from the second-round survey.

During round-two consensus was reached for 22 (81.48%) of the 27 items. Table 2 displays the number and percentage of endorsed items in ranked order. PD & SMI remained the most frequently endorsed item with 34 (94.44%) individuals indicating suitability for hospital admission. Despite the high number of items endorsed by participants throughout, panel members consistently emphasised the importance of the individual’s presenting need, overall clinical picture and formulation in their free text responses.

Five items (18.52%) failed to reach consensus: having had ten or more convictions prior to the age of 18 (45.71%); individuals with learning difficulties (45.71%; age (44.44%; sexual index offence (40%) individuals deemed medium risk of violence (37.14%). The iterative process of developing the final checklist is detailed in Fig. 1.

During round two, all five of the included diagnostic categories reached consensus among participants (PD & SMI, 94.44%; co-morbid PDs, 86.11%; borderline D, 83.33%; PD & SMI not severe and enduring, 75%; paranoid PD, 69.44%). Co-morbid presentations were endorsed more frequently than diagnosis alone, with participants accentuating that hospital admission should be reserved for those individuals with complex presentations:

“Related to complexity and severity [and therefore] worthy of higher resourced hospital unit.”

“Again, [co-morbid presentations] likely give [the] perception that such clients are more complex and risky, therefore more likely to need hospitalisation”

“[T]he more complex presentations may require admission simply for better assessment and understanding; however I would still expect the vast majority of people could [then] be managed / treated in prison as well as they could be in hospital.”

PD and co-morbid SMI was the most frequently endorsed category with 94.44% of respondents reporting suitability for hospital admission. Panel members emphasised the occurrence of psychotic symptoms as the important feature, rather than the PD diagnosis itself. Of the diagnoses alone, BPD reached the highest consensus 83.33%, with participants highlighting the difficulty of assessing and managing risk, as well as the increased resource demands in cases of repeated self-harm and the impact this had on staff working with this diagnosis:

“[It] depends with how client is presenting. I think the anxiety generated by working with this client group generates more anxiety in staff teams that [elicit] feelings of powerlessness that possibly lead to compensate with hospitalization”

“I think it would be wrong to assume that all service users would need to be presenting with BPS traits to the detriment of others who do not. They are definitely a key client group requiring PD services who also create the highest rate of staff burn out.”

During round two, seven (87.5%) of the eight included characteristic items reached consensus (psychiatric history, 91.67%; motivation, 83.33%; suicidal ideation, 77.78%; poor social functioning, 69.44%; interpersonal aggression, 66.67%; impulsivity, 55.56%; hostility, 50%). Previous psychiatric history including a past diagnosis of SMI or a previous inpatient psychiatric admission (91.67%) was the most frequently endorsed item, with experts suggesting the outcome of previous admissions needed consideration.

Suicidal ideation (77.78%) was the second most endorsed item with panellists reinforcing the need to assess risk in their free-text responses. Respondents linked hospital admission to acute risk and short-term placements aimed at reducing the immediate risk of harm:

“It is easier to observe someone in hospital and the environment tends to be safer, in addition short term pharmacological interventions can be helpful. I think these admissions should be focussed and shorter term as part of an overall pathway though.”

“Maybe appropriate where there is acute risk.”

An offender’s age was the least frequently endorsed characteristic, with only 44.44% of panel members indicating this was an important characteristic when considering hospital admission.

For round two, two of the offending items were merged – early onset offending and having ten or more convictions prior to the age of 18 – because of their similar content. This meant three items linked to offence history were included. Previous offence history was the only item to reach consensus (51.43%), with panel members rather divided in their responses. Experts indicated in their free-text responses that offence history alone would not necessarily be sufficient to determine hospital admission, highlighting the need to consider the severity and nature of the offending history, the diagnosis and presentation, taking into consideration other clinical factors and the formulation:

Relevant but not a determining factor. May bring secondary information around impact of previous interventions which would be relevant

Each time around needs to be assessed case by case to remain curious as to what's changed or what the problem is.

During round one, panellists were asked what level of risk they thought was most suited for treatment in a hospital setting: low, medium, high or very high-risk individuals. The results suggest experts feel hospital admission should be reserved for those individuals who pose a significant risk to others; endorsement rates for this factor were medium risk (60.71%), high risk (86.25%) and very high risk (72.55%), meaning all three of these items were re-ranked in the second-round survey. In the second round, survey participants reached consensus on both the high and very high risk categories but failed to do so for medium risk individuals. This confirms that professional experts believe that hospital admission should be reserved for the most high-risk cases.

During round two participants again reached consensus on mental illness managed by medication (68.57%); previous psychological treatment received in prison (60%), attendance at any other psychological programme (51.43%) and mental illness managed by medication (66.67%). Free-text responses relating to engagement with previous psychological programmes emphasised the importance of considering the individual’s engagement and outcome of treatment with experts split on what this meant for hospital admission:

Depends on progress / gain rather than ‘completion’ although not dropping out would be a positive marker

It may be indicative of motivation, progress but admission to hospital services should be made based on current presentation in my view.

Additionally, panellists were asked to comment on the importance of the therapeutic relationship in treatment success, 100% of respondents indicated the significance of this when offenders received care in the hospital setting. Respondents emphasised the significance of the therapeutic relationship, suggesting this underpinned the process, considered to be more important than treatment modality:

My understanding of the literature is that regardless of intervention methods the therapeutic alliance is one of the most important factors. My clinical experience also supports this

Therapeutic relationship has the biggest impact on treatment outcomes irrespective of treatment modality.

Experts were given the opportunity to identify any additional characteristics associated with suitability for hospital admission not included in the first-round survey. Experts generated a total of 32 additional items, and items with two or more endorsements are displayed in Table 3. The responses were reviewed and evaluated by the researchers to prevent duplication and to ensure they were in line with the research question.

All four newly generated items reached consensus in round two: self-harm (82.86%); lack of progress in prison (77.14%); need for further assessment (71.43%); and previous trauma (62.86%). Self-harm was the highest endorsed item: panel members expressed the view that hospital admission might be applicable to help stabilise patients and reduce risk. Interestingly, participants utilising the free text box highlighted that this alone should not warrant a hospital admission:

Should not be a reason for admission to hospital: forensic or not! Should be trigger for active treatment though

May indicate need for secure environment. Other factors determine more if FMH [forensic mental health] is appropriate.

The item relating to individuals with LD did not reach consensus. (45.71%). There was acknowledgment from participants that this population are likely to require specialist services, however, forensic mental health services might not be the most beneficial:

Depends on services: most MSUs can't manage them and the staff are not used to them; it causes all sorts of problems.

Yes, to a specialist LD service. This way the staff have additional training and can be responsive to the service users’ needs. It also provides the service users with a better opportunity for progression, given they cannot access mainstream offending behaviour treatment, placing them at a disadvantage to other service users.

The final step of this study was to combine the identified and endorsed items from Round Two of the Delphi into a checklist to screen for suitability for potential transfer to forensic mental health services. Those items which received more than 50% endorsement by experts were included into the checklist, which was divided into three key areas: diagnosis, clinical characteristics; and prior history and engagement (see Table 4). Internal Reliability scores for the three areas were acceptable, with Cronbach α = .863 for the diagnosis scale, α = .836 for the clinical scale and α = .826 for the progression scale.

The checklist requires external validation and calibration (i.e., what is the threshold for admission) but could serve in the meantime as a guide to factors that clinicians should consider when an offender who may potentially benefit from transfer to forensic services is first identified. This questionnaire could be readily validated against existing data on offenders transferred into Forensic Mental Health services and calibrated against successful onward transfers from these services.

Co-morbidity items were endorsed more frequently than PD diagnoses alone, reflecting professional experts’ view that hospital settings should be reserved for the most complex cases where co-morbidity is a significant issue. Additionally, participants’ responses highlighted that the type of PD was an important consideration, with BPD and PPD deemed more suitable for transfer to the hospital setting. Likewise, acute risk of suicide and repeated self-harm were also clinical characteristics requiring consideration, emphasising the importance of assessing the severity and risks posed. Individuals deemed to pose a ‘low-risk’ of violence was the least frequently endorsed item, which is in line with the OPD pathway criteria as it excludes men who are low-risk of harm, despite at times having equally severe personality difficulties [22]. What is also clear is that the consensus around diagnostic suitability is relatively weak, implying that this issue is not considered to be of prime importance. Each case requires careful thought and consideration, with professionals emphasising the importance of the overall clinical picture and formulation.

Prior treatment history and engagement were thought to be highly significant when making decisions about an individual’s pathway. Factors such as lack of progress in prison, and the individual being motivated, were considered relevant by panellists when considering suitability for hospital admission and successful treatment outcome but were not considered risk factors in future re-offending. The universality of offenders’ motivation as a driver of efficacy highlights the importance of focussing on it throughout their custodial sentence. Interventions that target engagement, commitment and PD symptomatology, such as the ‘goals-based approach’ [40] have shown promise and are already utilised in some clinical settings. Similar models could be employed across an offender’s journey from sentencing to community re-settlement.

Decisions regarding hospital admission are also complex because they typically involve several professionals from a variety of different agencies in the UK: the NHS, the prison and probation services, commissioners and the Ministry of Justice [22] A considerable degree of heterogeneity was observed between professional experts’ responses across the different items, with many items not reaching the 50% consensus threshold. If professionals have such varied and diverse opinions regarding this subject matter, it is legitimate to question how consistently the broad guidance currently given in the Strategy translates in practice. This study has been able to extract those factors about which there is some consensus, and which can provide a structured, informed approach to making referrals to forensic mental health services. This study therefore not only reinforces the importance of clear decision-making criteria, but also of collaborative working across the distinct parts of the system [23, 32].

The Delphi method is an anonymous, iterative process that aims to obtain expert consensus, over a complex problem through a process of convergence. This method is advantageous as it gathers expert opinion where lack of clarity exists, across dispersed geographical areas [41]. Similarly, it protects participant anonymity, which encourages the free expression of views. The construction of the Delphi was derived from a systematic review of the literature, ensuring its content was grounded in previous research and adding validity to the results. Since many of the critical decisions about care pathways and suitable services for PDOs within the CJS are made by the very group of professionals sampled, the study offers a unique insight into the emerging consensus within this population. Where such a population is available, Delphi and similar methods are often used to develop clinical guidelines under conditions of uncertainty [42] including NICE guidance [43].

Surowiecki [44] suggested the following are required to ensure the robustness of group findings: diversity of expertise; independence; decentralisation; and aggregation. The current study adhered to these principles, ensuring a heterogeneous sample of experts from health, criminal justice and commissioning, encompassing diversity of job role and title. Many Health Service and CJS decisions rely on professional opinion born of years of training and experience. Experts completed the survey independently, reducing the effects of bias due to group interactions. Finally, the methods of aggregation were decided in advance, thus reducing the influence of researcher bias. All panel members had a wealth of experience and professional training, providing relevant and thoughtful free-text responses which further add to the strength of the findings.

Professional consensus is subjective in nature, based entirely on opinion which has the potential for error. Critics of the Delphi method suggest it lacks in scientific rigour [45], and is, or should be, categorised at the bottom of the hierarchy of evidence [46]. The invitation to participate in the survey was sent to over 100 experts, with only a proportion expressing interest and completing the survey. It has been suggested that those who participate are likely to be impacted upon by the outcome of the process [39], which in turn suggests that commitment to the process could well reflect an individual’s own interest in the subject matter. This could, in theory, indicate a biased sample.

The Delphi study had a modest sample size, with a high number of individuals from health and the criminal justice system, with a small, albeit proportional, number of commissioners (8%). Similarly, panel members were predominately professionals working and practising in England, meaning the generalisation of results to, for example, Wales should be taken with caution.

Additionally, the process of aggregation in the Delphi study is open to arbitrary judgement [40] with studies defining consensus in a variety of ways. While this degree of subjectivity allows for freedom within a research project, it does impact on what is reported. For this study, 50% endorsement was required, however other studies have set this figure higher e.g. Hart et al. [47] set theirs at 90% and Berk et al. [37] at 80%. However, in an area where consensus is lacking and likely to be controversial, this figure can be lower and in some cases below 50% (e.g. [48]). In this case, a lower endorsement rate of 50% was preferred for two reasons: i) due to the lack of an established evidence base for the effectiveness of services for personality disorder, determined from the literature; and ii) to ensure that a breadth of opinion was captured by the survey and no potentially important factors omitted. This was on the premise that items could subsequently be tested empirically and retained or discarded.