A shared meaning of key concepts in the assessment of mental health care delivery was lacking. For example, in full text papers reviewed, a number of papers were excluded where the concept of service availability had been variously interpreted as service utilisation, service workforce and service capacity. Thirty three papers related to service availability were excluded because they provided no data, 17 papers were excluded because they provided data only on workforce capacity, and seven papers were excluded because availability was conceptualized as either service utilisation or as availability of interventions.
Characteristics of included studies
Of the 46 eligible studies, 36 (78.3%) [
19‐
54] were descriptive, and 10 (21.7%) were analytical [
55‐
64]. Thirty six papers (80.4%) presented service availability data from a single country, of which 19 [
20,
28,
32‐
34,
37,
40‐
42,
50,
51,
53‐
57,
59,
61,
64] took a regional or local approach, while 17 [
19,
21‐
27,
30,
36,
38,
39,
43‐
46,
49] looked at availability from a national level. Ten papers presented service data from more than one country, of which seven [
29,
35,
47,
48,
52,
58,
60] took a regional or local approach, and three [
31,
62,
63] were at the national level. Overall, excluding two papers which included over 40 Lower Income Countries and Lower-Middle Income Countries (LIC/LMIC), not all of which were identified, 22 papers (48%) used data from Europe, most notably Spain and Italy, nine papers (20%) were from Africa, seven (15%) from Asia, four (9%) from the Middle East, two (4%) from the Americas (one from USA and one from Chile), and one (2%) from Australasia. Of the LIC/LMIC countries studied, eight were from Africa, and three were from Asia. However, in 25 studies (54.3%) the precise boundaries of the study area were not formally defined.
Twenty eight studies (60%) provided socio-demographic context [
21,
24,
25,
27,
29,
31,
34,
37‐
40,
42‐
45,
47‐
50,
52,
53,
55‐
57,
59‐
62]. Two papers [
34,
53] which presented data from atlases of mental health care included comprehensive local area data. Of the 16 studies which linked one or more socio-demographic indicators with mental health, only four provided supporting evidence with validated indicators using a standardised instrument (e.g. European Social Demographic Schedule -ESDS) [
34,
48,
59,
60]. These four papers all used the European Service Mapping Schedule (ESMS) for service availability data. Papers based on WHO-AIMS and MHCP instruments also included legislative and policy context at a national level.
Where target populations were formally defined, 11 studies included children and/or adolescents [
19,
21,
25,
30,
31,
36,
40,
50,
51,
53,
63]; three studies included people with alcohol and other drug dependence (AOD) [
36,
61,
63]; two studies included people with intellectual disability (ID) [
21,
36]; three were specific to serious mental illness or psychosis [
57,
62,
63]; two included people over 65 years [
21,
51]; and one study each included the following subpopulations: maternal/perinatal mental health [
36]; people requiring long term rehabilitation [
54]; survivors of suicide attempts [
57];and socially marginalized groups [
47]. A further 21 studies did not specify a particular mental health population.
The main characteristics of included studies are detailed in Table
2.
Table 2
Characteristics of included studies
ESMS/DESDE | 12(26.1%) | 6 | 6 | 4 | 6 | 10 | 9 | 7 | 0 | 8 | 9 | 6 | 3 | 0 | 3 |
WHO-AIMS | 11(23.9%) | 9 | 2 | 3 | 9 | 9 | 7 | 4 | 7 | 6 | 2 | 0 | 4 | 3 | 4 |
MHCP | 3(6.5%) | 3 | 0 | 0 | 3 | 0 | 3 | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 3 |
PRIME study | 1(2.2%) | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 |
PROMO study | 1(2.2%) | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 0 |
Adult Service Mapping Exercise | 3(6.5%) | 2 | 1 | 0 | 3 | 2 | 3 | 3 | 0 | 1 | 2 | 1 | 2 | 0 | 0 |
Method described | 10(21.7%) | 9 | 1 | 1 | 10 | 2 | 1 | 5 | 0 | 4 | 1 | 1 | 3 | 0 | 6 |
No method provided | 5(10.9%) | 5 | 0 | 0 | 5 | 4 | 0 | 0 | 0 | 3 | 0 | 0 | 1 | 0 | 4 |
| 46 (100%) | 36 (78%) | 10 (21.7%) | 10 (22%) | 36 (78%) | 22 (48%) | 25 (54%) | 21 (46%) | 11 (24%) | 27 (58.7%) | 15 (32.6%) | 8 (17%) | 14 (30%) | 3 (7%) | 21 (46%) |
We then analysed the methods used in included studies (Table
3). Six instruments providing data on service availability were identified in the included studies, and these were used in a total of 31 papers. Three of these were psychometrically validated instruments: ESMS/DESDE (Description and Evaluation of Services and Directories for Long Term Care-an evolution of the ESMS and thus described together) (used in 12 papers: [
20,
28,
34,
48,
52,
53,
56,
58‐
61,
64]); WHO-AIMS: (used in 11 papers [
21,
22,
24,
25,
27,
31,
33,
50,
51,
62,
63]); and MHCP (used in three papers [
43‐
45]). ESMS/DESDE and WHO-AIMS are based on taxonomies of care (ESMS/DESDE on a hierarchical tree taxonomy), and DESDE has undergone formal ontological analysis [
65]. The MHCP is structured into four domains relevant to policy, including context, resources, provision and outcomes. However, while the MHCP provided a taxonomy for mental health systems generally, it should be noted that the domains for health service delivery did not include any classification of service types. Two other instruments- those of the Best Practice In Promoting Mental Health In Socially Marginalized People In Europe study (PROMO) in 14 European capital cities [
47] and the Programme for Improving Mental Health Care in five LMICs study (PRIME) [
29] were designed specifically for those studies, and were included in one paper each. The ASME, used in three papers [
23,
54,
55], was designed specifically for the English context. WHO-AIMS, MHCP, and the instruments from the PRIME and PROMO studies are instruments designed specifically for mental health services, while ESMS/DESDE and ASME have a broader health service application. ESMS/DESDE was developed for all long term care services. Fifteen studies did not use a structured framework [
19,
26,
30,
32,
35‐
42,
46,
49,
57], of which five did not provide any method [
37,
40‐
42,
46]. Four of these [
37,
40,
41,
42] formed part of a group of seven papers in a special supplement related to a conference on mental health care in capital cities: however three of this seven papers were excluded from this study as they did not include any data on service availability.
Table 3
Characteristics of methods used by included studies
Ontology based | Yes | No | No | No | No | No | 0 |
Taxonomy based | Yes | Yes | No | No | No | No | 0 |
Psychometrically validated | Yes | Yes | Yes | No | No | No | 0 |
Unit of analysis | Macro (Organ-isations) | No | Yes | Yes | Yes | Yes | No | 14 |
Meso (Services) | Yes | Yes | Yes | Yes | Yes | Yes | 13 |
Micro (Teams) | Yes | No | No | Yes | No | No | 5 |
Number of comparison studies | Regional comparisons within a single country | 4 | 1 | 0 | 2 | 0 | 0 | 0 |
International comparisons at regional level | 4 | 0 | 0 | 0 | 1 | 1 | 0 |
International comparisons at national level | 0 | 3 | 0 | 0 | 0 | 0 | 2 |
Longitudinal comparisons | 0 | 0 | 0 | 0 | 0 | 0 | 4 |
Glossary included | Yes | Yes | No | No | No | No | 0 |
Data sources | Service providers | National level data from ministries, organ-isations etc; aggregated regional data where national data not available | Govt and other national level data sources | Local Implementation Teams | Govt and non govt reports, triangulated with local key co-ordinators | Service providers | X |
Sectorsa included | H,S,E,Ed,Ho,O | H,S,E,Ed,Ho,O | H,S | H,S | H,S,Ho | H,S,E,Ho | X |
Mental health specific or generic | Generic health | MH specific | MH specific | Generic health | MH specific | MH specific | X |
Accessibility | Open Access but requires training | Open Access | Instrument itself unable to be accessed online | Unable to access online | Accessible online but specific to PRIME study | Study specific-Unable to access instrument online | X |
Study design | Survey/interviews | Survey/interviews | Survey/interviews | Survey | Survey | Survey/interviews | X |
In the case of ESMS/DESDE papers, the unit of analysis was care teams provided by individual services, aggregated at local level (2A in the mTT matrix), while in WHO-AIMS, ASME and MHCP papers, services data was aggregated at national level (1A in the mTT matrix). Of the 23 papers not using taxonomy based instruments (i.e all those papers not using ESMS/DESDE or WHO-AIMS), eight, including all three papers using the MHCP, counted services provided at a higher organisational level of care, such as psychiatric hospitals in a local area, along with individual services, such as day centres or mental health departments in larger organisations, thus conflating these different levels of care [
30,
36,
37,
39,
40,
43‐
45]. In a further seven papers [
23,
29,
35,
41,
42,
55,
57], including two of the three papers using the ASME, individual services were conflated in the same way with individual care teams (section 4A of the mTT matrix) such as crisis resolution teams, or assertive outreach teams.
Of the 15 papers which did not use a specific instrument to frame their analysis of service availability data, three [
30,
36,
39] used internationally based frameworks, five [
19,
26,
32,
49,
57] used a framework relevant specifically to the region in which the study took place, four [
37,
40‐
42] categorised their data around service types but did not justify their categorisation or their choice of units of analysis, and three [
35,
38,
46] did not specify any framework for their data on service availability. Of those studies using international frameworks, two [
30,
36] were based on the Mental and Social Health Atlas of Saudi Arabia, which used the framework provided by the WHO Mental Health Atlas, while the third drew broadly on the WHO Mental Health Atlas, as well as recommendations from the 2001 WHO World Health Report to structure their findings [
39]. Three studies described service availability according to the specific structure of the national system under study [
19,
26,
49], while one described service availability based on a regionally prescribed framework of services required for the prevention of recurrent suicidal behaviour [
57].
Terminology used to identify units of analysis varied widely, but only ESMS/DESDE and WHO-AIMS provide glossaries of terms used. MHCP studies included detailed qualitative data at the local level in order to ameliorate the effect of terminological variability on data interpretation. Terms used in papers for residential care included “psychiatric hospitals”, “supportive homes”, “crisis homes”, “safe homes”, “social rehabilitation centres”, “group homes”, “short and long term residential units”, “community based psychiatric inpatient units, respite, and community residential facilities” and those for non-residential care including “day hospitals” “psychiatric clinics”, “outpatient clinics”, “day centres”, “mental health dispensaries”, “mental health departments in social diseases prevention centres”, “day treatment facilities”, “fixed clinics”, “outpatient department”, “community mental health centres”, “sheltered workshops”, “day activity services”; “crisis resolution teams”, “assertive outreach teams”, “early intervention in psychosis team”, “home care nursing services”, and “mobile crisis teams”.
Data was obtained from sources at different levels of the health system. Studies using the ESMS and DESDE and the PROMO instrument take a bottom up approach, gathering data from providers at individual service level. WHO-AIMS takes a top down approach, the papers using this instrument collecting national data at a high level from sources such as heads of departments, universities, and professional boards. Where the instrument was used at a regional level, data was collected from similar sources at that level. In these studies however, the data is still interpreted through a national prism. Papers using the MHCP instrument and that of the PRIME study used both national and local sources, both methods combining national level data with qualitative data from the local level gathered from sources including professionals, clients, families and other stakeholders. The PRIME study is undertaken at district level, but uses a top-down approach, with data from administrative databases, key officials and service heads. Data for the ASME was gathered at a national level from Local Implementation Teams, although one paper [
54] first identified relevant Trusts providing rehabilitation services using the ASME, and then went to the individual units to obtain data. In the 15 papers using other, non- framework based methods, existing administrative databases or literature were sourced, with four also using surveys sent to senior health or government officials [
36,
38,
39,
57].
Seven studies included the health sector only [
30,
32,
38,
39,
46,
62,
63]. Eighteen studies included the health and social sectors [
19,
20,
22,
23,
26,
29,
31,
33,
45,
49,
50,
52,
54‐
56,
59,
61,
64]. This included papers using MHCP and ASME. At least one other sector, such as employment, education, justice, or housing was included in almost half of included studies (21 papers) [
21,
24,
25,
27,
28,
34‐
37,
40‐
44,
47,
48,
51,
53,
57,
58,
60]. This included papers using ESMS/DESDE, WHO-AIMS, and those from the PRIME and PROMO study. The instrument of the PROMO study included several sectors, but for a limited target population (marginalised populations).
Of the 36 studies undertaken within a single country, seven [
28,
51,
53‐
55,
59,
61] included comparison at regional or local level, and four included a comparison over time [
19,
30,
32,
38]. All of the ten cross country studies included comparison of service availability: seven at regional or local level [
29,
35,
47,
48,
52,
58,
60], and three at national level [
31,
62,
63].
Forty-one papers (89%) identified themselves, or were assessed by us, as being situational and/or gap analyses. The remaining five papers comprised the following: efficiency analyses [
58,
64] territorial planning [
59], ecological analysis [
57] and standard description for comparison [
60]. Thirty-two studies (70%) included recommendations for policy makers related to service provision based on the findings. Visual tools were used in 12 papers (25%), four of which incorporated graphics issued by Geographical Information Systems. In three of these the visual tool presented data on service availability.
The methodological characteristics of included papers are summarised in Table
3.
In those papers using instruments to provide data on service availability, this was WHO-AIMS in 11 papers (24%) [
21,
22,
24,
25,
27,
31,
33,
50,
51,
62,
63], ESMS/DESDE in 12 papers (26%) [
20,
28,
34,
48,
52,
53,
56,
58‐
61,
64], MHCP in three papers [
43‐
45] (7%), ASME in three papers (7%) [
23,
54,
55] and the PRIME [
29] and PROMO [
47] project instruments in one paper each (2%).