Towards Structural Quality Indicators for Intensive Community-Based Care Programmes for Substance Abusers

The results show that intensive community-based care is not a simple intervention or methodology pur sang. It requires a highly complex organisation, which is reflected by the diversity of the clusters. The emphasis on cooperation with other institutes is significant, and the participants said that this should ideally be characterised as a chain of care (Van de Lindt 2000). This means that single services provided by separate institutes need to be strongly linked. This also means that intensive community-based care is characterised by interorganisational and interdisciplinary service and that it addresses more than one living area or problem at a time. This is a logical conclusion, since the target group of intensive community-based care consists of clients with multiple problems.

The results also show that intensive community-based care is a new healthcare service that needs to find its place in the existing healthcare system. Comments about the role and responsibility of staff members show that the methodology itself and the staff providing it are not yet completely embedded and supported in the current system. There need to be formal agreements that all parts of intensive community-based care, such as outreach activities or social work, be implemented. The staff members providing these activities also need formal support, both material and non-material, as well as recognition from their own organisation.

Intensive community-based care is innovative, and this is perceptible, less directly, in other statements: e.g., the need for clear financial arrangements, or a clear policy regarding client privacy. The need for clear financial arrangements stems from the current lack of funding for intensive community-based care activities; these activities do not always meet the present financial criteria, measured in terms of productivity. Since outreach activities are not always considered directly productive, they are not always reimbursed (GGD Nederland and GGZ Nederland 2005). The need for a clear policy regarding client privacy follows from the pro-active character of intensive community-based care. This differs from traditional healthcare, where service providers wait until the client comes to their offices on their own initiative; in intensive community-based care, service providers are outreaching and pro-active, doing fieldwork, approaching clients unasked, and inquiring about the client. This approach raises ethical questions about client privacy (Henskens 2004; Priebe et al. 2005; Young et al. 1998).

It is interesting that the map not only shows that the service of intensive community-based care needs to be managed, but also that the staff needs to be managed. The individuals who provide the services are viewed as important contributors to the quality of intensive community-based care. The mentioned requirements were content-related (e.g., specifically trained staff, familiarity with rehabilitation), as well as personal (e.g., being open minded, having a pioneering spirit). There is an increasing belief that the character of the service provider accounts for a large part of the quality of intensive community-based care (Henskens 2004; Priebe et al. 2005; Young et al. 1998), and the results of this study support this.

These findings were compared with the previous concept map (Roeg et al. 2005). The current concept map was performed in line with the previous one, using the same sampling strategy (though with different participants from different programmes), the same procedures, and the same software and analyses. Only the themes differed: the previous concept map considered all components that contribute to the quality of intensive community-based care, whereas the current one focussed exclusively on the structural components. Components that had been mentioned in the previous brainstorming session were divided into three groups of indicators: structure, process, and outcome indicators (Table 2). This division was reached inductively, and is in line with Donabedian's theory on types of quality indicators (1980). The current concept map focused specifically on structure, as it was believed that this had not been fully explored in the previous concept map.

Because the current concept map focused on a subsection of the previous one, it was not difficult to link the two, with the results of the current map classed under the analogous part of the first one.

As expected, the current concept map provides additional information. Five of the seven clusters are supplementary to the previous concept map, addressing other elements of structure. These five clusters appear in the regions ‘finance’, ‘orientation of the staff’, and ‘professional quality.’ The two remaining clusters of the current map, on the other hand, correspond to the previous map: ‘interorganisational cooperation and coordination’, and ‘professionalisation’ were both addressed by the previous concept map. ‘Interorganisational cooperation and coordination’ includes statements that also fit in the cluster ‘preconditions for care’ in the first map. Similarly, ‘professionalisation’ includes comparable statements to those in the cluster ‘relation to regular care’ in the first map. The two concept maps can be combined by adding the five additional clusters to the first map, and by combining the statements of the two equivalent clusters of the two maps.

This study was carefully prepared and conducted to ensure its quality. Table 4 presents an overview of all the techniques employed to improve the quality of this study. According to Yin (1993), general criteria for inductive research are construct validity, internal validity, external validity, and reliability. Construct validity deals with the use of appropriate instruments and measures to operationalise the construct being investigated. In this study, a validated and well-defined method was used to ensure construct validity. Concept mapping was developed by Trochim (1989) in the 1980s, and since then the method has increasingly been applied in healthcare research (e.g. Johnsen et al. 1999; Nabitz et al. 2005; Van Weeghel et al. 2005). Because of its popularity, a number of statistical packages, such as Concept Mapping (Trochim 1989) and Ariadne (Nederlands Centrum Geestelijke Volksgezondheid and Talcott 1995) have been developed. Internal validity involves measuring the intended construct, dimensions, or mechanisms (for causal studies). In the current study, a number of techniques were employed to ensure internal validity. Multiple observers were present during the concept mapping session (chair, secretary taking minutes, and researcher). All paid attention to the quality of the statements during the brainstorming session and checked that they were sound, valid, clear, and logical. They also supported the participants in rating and sorting the statements. External validity refers to whether the findings can be generalised. In this study, the question is whether the dimensions that were found apply to all types of intensive community-based care programs in the Netherlands. A number of techniques were employed to ensure generalisability: purposive sampling and maximum variation sampling strategies both increase the analytical generalisability of the topic (Bowling 2000; Yin 1993). Reliability refers to whether the study can be replicated with the same results. Careful documentation of procedures is one pre-requisite, and in this study we used a formalised protocol of the concept map technique, keeping notes throughout the process.

Finally, consensus increases the internal validity of research findings (Johnson 1997). In concept mapping, mean importance rates can be regarded as a measure of consensus. To further enlarge validity, one might consider removing statements with a low mean rate (e.g., <2.50) from Table 1, though this has not been done here.

Although several techniques were employed to improve the study’s quality, two methodological limitations should be mentioned. These concern saturation and external generalisability. In the tradition of theory building, it is normal to stop the process when theoretical saturation is reached (Strauss and Corbin 1998; Zomerdijk 2005). Theoretical sampling is commonly used to achieve saturation; this is an ongoing purposive sampling procedure, in which cases are selected until incremental learning becomes minimal because the same phenomena are being observed (Zomerdijk 2005). In the current study, theoretical sampling was not used, so we cannot be positive that the point of saturation was reached. Additional data would be needed to test the taxonomy presented here for saturation. External generalisability is supported by the techniques mentioned in Table 4. However, since data were only collected in the Netherlands, further samples are needed to show whether this taxonomy applies in other countries in Europe, or beyond.

As a preliminary check for saturation and external validity, we compared these findings with the literature. As was mentioned, the structural components of intensive community-based care have hardly been studied; instead, we used a selection of important reviews (i.e. Kroon 1996; Mueser et al. 1998; Phillips et al. 2001; Rapp 1998). From these, we selected the structural components that had some theoretical meaning, either as part of a model that has been studied and proven effective, or which were identified as critical features based on literature study or expert consultation.

Eleven distinctive components were found (see Table 3). We then compared these components with the structural components in the previous and the current concept map. For the previous concept map, we used the original table (in Roeg et al. 2005), as Table 2 does not show all the statements.

Four of the components mentioned in the literature appear directly in the concept maps: shared caseloads, 24-h coverage, definition of target population, and required training for service providers. Six components mentioned in the literature also appear in the maps, but less directly. For instance, the component ‘multidisciplinary versus monodisciplinary teams’ appears in the maps in various forms. The cluster ‘interorganisational cooperation and coordination’ in the second concept map contains statements that refer to this issue. The statements ‘care chain approach (comprehensiveness and continuity)’ and ‘inter-sectorial and institutional exchange of expertise’ both refer to multidisciplinarity; similarly, ‘construction of a separate and new organisational unit’ refers to a multidisciplinary team or department. In the cluster ‘profile of staff’, the statement ‘service providers are generalists’ refers to monodisciplinarity.

On the other hand, one component mentioned in literature is explicitly different from the concept map findings: namely, direct service provision versus care coordination. This distinction is not explicitly made in the concept maps; instead a lot of attention is paid to service provision in interorganisational cooperation, with two structures specifically mentioned: the new organizational unit and the care chain approach. A new organizational unit would consist of a formed team of service providers working at different institutions and providing services together, whereas a care chain involves close collaboration between institutes. A care chain, though, provides more than just care coordination (there is very close collaboration), but less than direct services (as the services are provided by persons working in different institutions).

Comparison with the literature also shows that many components have remained untouched so far and that the findings presented here add to the literature in a number of ways. First, they add several domains to the study of intensive community-based care: e.g., the forms and consequences of cooperation, means and preconditions, registration and evaluation, and the personal characteristics of staff. Second, this is the first attempt since Intagliata (1982) to organize the critical components of intensive community-based care in a theoretical manner rather than in a practical one (for example, presenting a model programme).

This study has stressed the importance of looking at the separate components of intensive community-based care rather than looking at entire packages, and a major step was taken towards developing a taxonomy of intensive community-based care. Such a classification system is a necessary tool to improve the quality of future studies, since it allows us to describe programmes in detail, and thereby makes it easier to compare them. Only with improved descriptions of experimental and control services can the findings of randomised controlled trials be properly compared and the contribution of individual programme components be disentangled.

As explained above, saturation and external generalisability still need to be checked. Saturation can be reached by performing additional semi-structured interviews with participants selected using theoretical sampling strategies. External generalisability can be tested by performing additional interviews with experts from other countries to see whether the taxonomy also covers their views on intensive community-based care or whether new elements are found. We are currently carrying out a follow-up study in which the taxonomy is being saturated, translated into a questionnaire, and is being used to describe the various programmes in the Netherlands (Roeg et al. 2008).

This taxonomy can also be useful to practitioners. It shows which components need to be considered when (re)designing a programme of intensive community-based care, and what to focus on when comparing programmes. A taxonomy is also a useful tool for internal and external communication. Being able to explain the specifics of one’s programme internally is essential to ensure that all staff have the same goals and agree on procedures. And being able to clearly explain the programme to third parties can make the difference between getting funding or not.