A preliminary taxonomy and a standard knowledge base for mental-health system indicators in Spain

Health indicators are tools designed to measure the health status of people and the functioning of health services through the various factors that influence them (demographic, economic, social) [1, 2]. These provide the basic information for system analysis and decision-making in policies, planning and health management. The area of mental health presents added difficulties for the development of a useful list of health indicators for a variety of reasons. Firstly, this is a complex area in which health, social, educational and criminal and justice services coexist, where the care teams are multidisciplinary, and in which an integral care focus should be adopted [3]. Secondly, there are no reliable biological indicators for either the disorders assessed or the results, which complicates epidemiological and outcome research. Thirdly, mental health has been included late into the general health system (in Spain from 1986), it presents problems of under-financing and the lack of national data bases which exists in other disciplines (e.g. Oncology or AIDS) [4].

Although the available international instruments do provide a useful source of indicators (e.g. WHO-AIMS [5] or the Mental Health Country Profile [6]), these listings have not been developed as knowledge bases and their taxonomy and hierarchy has not been formalised in an explicit way. An indicator base may allow to select indicators from this base for specific uses in studies, projects and plan monitoring, as well as in specific services, programmes or target populations.

In 2008, the Clinical Management Working Group of the Spanish Society of Psychiatry (known by its Spanish acronym GClin-SEP) started the development of a preliminary taxonomy and a related knowledge-base of mental health indicators which would facilitate a future standard indicator set which could be used for inter-regional comparison in Spain, related to the National Health System Mental Health Strategy [7], taking into account the challenges and problems previously described [8, 9].

As a first step a preliminary taxonomy and a related knowledge-base for mental health indicators in Spain was planned. A taxonomy may be defined as a particular classification arranged in a hierarchical structure providing supra and subtype relationships. Within the health care technology field a health knowledge-base is 'a system of storage, classification and presentation of relevant health information which includes databases, glossaries, articles, presentations and other documents regarding a specific health area or subject' [10]. This should assist the development of a list of basic indicators which would, in turn, facilitate informed evidence in mental health planning.

This project is aimed to developing a conceptual map and a knowledge-base of mental health indicators suitable for mental-health planning which permits inter-regional comparisons, follow-up and evaluation of the health systems that currently exist in Spain. For this a mixed qualitative method was followed using frame analysis and nominal groups.

Frame analysis is a broadly defined method of enumerating and defining ideas and themes within a larger topic that is particularly useful for formalising concepts [11]. Of the four components of frame analysis, we focused on "frame bridging," which manifested as collaborating with experts who are interested in topic but do not commonly interact due to different training backgrounds or other reasons, and on "frame amplification," or the clarifying and elaborating of a framework from which to think about the issue of discussion [12]. Frame analysis has previously been applied to a wide range of social and health-related topics, such as consensus-building in online special-interest advocacy groups and understanding of the culture of nurse mangers [13]. Two members of the core group with a background in mental health system research (LS-C), and mental health geography and data management (JAS), searched the relevant literature in PubMed and Google Scholar using the key words: 1) "Mental Health", 2) "Care", "System", "Policy", "Planning" and 3) "Indicator(s)"; as well as a review of other technical documents available such as lists of general indicators of health relevant to mental health, and mental health lists from international, European and national organisations. Also considered were various plans and health reports from the Autonomous Communities or regions in Spain and lists developed by scientific associations. As the aim of this project was to develop a taxonomy usable in Spain within the European context, indicator lists from the US were not included in the analysis. The two researchers arranged this content according to key topics and prepared a framing document and a list of key areas and questions to be debated by the nominal groups.

The nominal group technique helps to deal with ill-structured domains while it allows a more structured approach than focus groups, as well as the use of prior information and knowledge. Once ideas and related questions are listed, its relevance to the central problem can be discussed following a question made by the facilitator, ideas can be re-formulated and clustered into coherent groups. All members are encouraged to participate in the discussion following a sequential order and every round is followed by a final debate [14, 15]. In the health sector nominal groups have been previously used to develop the preliminary taxonomy of health related habits and lifestyle [16] and its integration into primary care [17].

An iterative process was followed to develop the preliminary taxonomy and the related knowledge-base. In all 14 experts in mental health service research and indicator analysis with very different background participated in two nominal groups: a core working group and an external group. The core working group was comprised of seven members: four psychiatrists with experience in the evaluation and management of services, one expert in data-analysis (Knowledge Discovery from Data -KDD) [3], a health geographer, and an expert in health and social management in the field of mental health. The core group hold three face-to-face meetings in 2009 and 2010, combined with three conference calls and periodic contact by e-mail.

Additionally, a panel of experts from the Scientific Association PSICOST provided external support to this core group. This external panel had seven members, a coordinator (LS) and a moderator (JAS). The panel also followed a nominal group methodology and it was comprised of two psychiatrists (LS and JCG), one psychologist (CR) with experience in services evaluation, a public health expert in epidemiology (JA), an expert in health-indicator data analysis (CG), and a public administration manager with experience in mental health and disability (FA).

For the development of this taxonomy the model and terminology used at the International Classification of Functioning (ICF) [18] was adopted for defining health constructs, domains and dimensions. For the definition of entities, their hierarchy and type, we used a basic formal terminology: <it is a>, <it is comprised of>, <it is part of>. A conceptual map was drafted using a tree structure for coding and organising the indicators. This approach had been used previously for the description of resource indicators and the use of mental health services in Spain and in other European countries [19]. This diagram allows the organisation of indicators into classes (domains), subclasses (subdomains) and additional types. This structure allows the addition of new indicators or the subdivision of previously defined indicators where necessary, without altering the hierarchical structure of the taxonomic system.

Subsequently, the two reviewers developed a list of relevant databases, a wide-ranging list of mental health indicators, and a glossary. With respect to the database, and bearing in mind all the information available, the following question was formulated for the nominal expert panel: "Is this a relevant indicator for the evaluation of the mental health system in the various Autonomous Communities?". 'Relevant' was defined here as 'closely connected with the subject and valuable and useful to mental health planners and stakeholders' based on the definition provided by the Oxford Advanced Learner's Dictionary http://​www.​oxfordadvancedle​arnersdictionary​.​com. The responses were organised into a 4-level Likert scale according to their relevance (none, doubtful, moderate and high).

The results were reviewed by the members of the working group and the information gathered was used to develop a definitive list which was added to the preliminary knowledge base and which can be seen at the Spanish Society of Psychiatry website SEP [20]. The external nominal panel provided an evaluation of the relevance of the various indicators which was reviewed by the core working group.

The present work is framed in the context of informed evidence for health policy and planning [21]. The concept of informed evidence is replacing that of evidence-based care and highlights the need for quality registers and the greatest possible number of information sources available for decision-making in health policy, including local provision and organisation at different levels (micro, meso and macro) [22].

To our knowledge this is the first preliminary taxonomy of indicators of the mental health system and its related knowledge-base. Other preliminary taxonomies have been recently developed to formally organise other areas of knowledge such as health indicators [23], patient safety and medical errors [24], or health related habits [16]. This preliminary taxonomy does not pretend to develop a completely different conceptual map to what is currently used in the field, but to formally organise the available information and provide a hierarchical order using common terminology as much as possible. The definitions selected were also those more commonly accepted. The extent of the area of health indicators is such that it hinders a complete review of the material; especially for a restricted group with a limited budget. This knowledge base has an incomplete character and several limitations.

First, this knowledge-base is country-specific and its generalisability and transferability outside Spain is limited. In any case it is important to note that country-level information is very relevant for international health system research [22]. Furthermore, the heterogeneity of the Spanish mental health system makes it a unique case for studying different care models under quasi- or universal health care coverage. The existence of 17 different publicly funded mental health systems, with their own policy and practices may provide useful insight for many countries. They range from a practically do-nothing approach until very recently in some regions, to the transformation of the old psychiatric hospitals, complete separation of funding and provision, with market competition and high participation of the private sector working under agreements set by the public health system (e.g. Catalonia). They may have one single public system (comprising both funding and provision) without closure of psychiatric hospitals (e.g. Basque Country) or a public system with full deinstitutionalisation and closure of psychiatric hospitals (e.g. Andalusia) [25]. In addition the conceptual map included in this preliminary taxonomy has been designed to facilitate the incorporation of new domains and sub-domains as the system is expanded, so it may be refined when information from other sources is incorporated (e.g. user-oriented mental health report cards), or when it is used in other countries in Europe.

Second, this knowledge base is expert-oriented and it has excluded international indicator lists not developed or used in Spain, such as the Mental Health Country Profile [6]; the NF-10 and its related instruments in the US [26] or the 'State Report Cards' by the National Alliance on Mental Illness (NAMI) [27]. As said, this knowledge-base should be complemented by user-oriented indicator lists based on concerns reported by consumers which are not currently available in Spain.

Third, there are great differences regarding the degree to which this information can be accessed. The majority of indicators are available on a national and regional scale but these are limited in small health areas. The limitations of scale, periodicity and sources mean that some indicators cannot be selected despite their potential relevance. The sources of information for the calculation of the indicators are highly heterogeneous with the institutes of statistics, and information from health administrations and social welfare being the principle sources of data. Furthermore, the reference year of these sources varies across the 17 regions or Autonomous Communities in Spain, and even within the same Autonomous Community. This is related to the fact that, after the devolution process started in 1986, the Spanish Health System actually comprises 17 different health systems with wide variation in mental health care organisation and policy [25].

Fourth, the extended list included important indicators that have not been incorporated to the 200 indicator list due to usability problems in the Spanish case. These comprise patient reported outcomes, stigma and sensitation, suicide prevention, prevention of depression, training and human rights. To date human rights have been specifically assessed in a single region (Asturias), and results have not been published yet. In any case a list of 200 indicators is too large to be practical for decision making, even though other main lists and instruments contain a similar number of indicators (e.g. WHO-AIMS [5]). The OECD list comprises 12 indicators which are included in the expanded GClin-SEP list. Unfortunately just one is currently collected in Spain [28]. GClin-SEP is conducting a Delphi panel on the relevance and usability of these indicators to produce a brief list of 50 indicators usable for comparing mental health systems across the 17 regions, and for the standard monitoring of the Spanish National Mental Health Strategy. This Delphi study will provide data on the feasibility and face validity of the indicators registered in this listing.

In addition, there is scant information on the psychometric properties of the indicators in the care system [2]. The development of a preliminary taxonomy is complementary to the psychometric analysis of the indicator set. Health system indicators are very basic health technology tools, and hence, their feasibility, consistency, validity, reliability, redundancy, sensitivity to change, level of generalisability, and impact analysis should be evaluated following standard procedures [29]. The existing gap between the literature on the psychometric properties of indicators and its broad use in health service and health system research may be partly related to a lack of awareness by researchers, planners and funding agencies of the relevance of this topic and the need for additional funds in this field.

This preliminary taxonomy and its related knowledge-base should serve those embarking on a study of the Spanish Mental Health System, and it may be also valuable to researchers looking for selected indicator lists in specific areas within mental health system research in Spain. It may be also relevant as a contextual case to those analysing indicator lists in other countries, particularly in Europe. On the other hand the preliminary taxonomy and its related conceptual map and hierarchy would require comparison with other related international initiatives and further analysis following a formal ontology approach [23]. These results should be challenged in other European countries to improve the indicators on Mental Health Systems in this world region.