The development and evaluation of a computerized decision aid for the treatment of psychotic disorders

Almost 1 % of the population in the western world will eventually fulfil the criteria of schizophrenia or a related severe mental illness [1]. Core symptoms of many people suffering from psychotic disorders are hallucinations, delusions, incoherent thoughts, memory problems, loss of initiative, flat affect, poverty of speech and social withdrawal [2]. Moreover, patients frequently experience problems with psychosocial functioning, such as a lack of daytime activities, social contacts, intimate relationships and a reduced quality of life [3, 4]. They often have poor physical health and experience medication side effects that contribute to an early onset of cardiovascular diseases. Different studies have shown a reduced life expectancy ranging from 10 up to 28 years [5, 6]. Some patients manage to recover both in terms of their symptoms, as well as in reaching personal and social goals. However the majority only partially recovers, with recurrence of symptoms and enduring personal and social problems often for the rest of their life. Especially patients with the most severe symptoms (fulfilling criteria for schizophrenia or schizoaffective disorders) often need lifetime medical, psychiatric and social care. Recommended treatment options are described in national treatment guidelines; in the Netherlands the Multidisciplinary Guideline for Schizophrenia is used (which is largely in line with the NICE guideline) [7]. The Optimal Treatment Project revealed that 2 years of optimal, evidence-based treatment led towards a clear trend in recovery from clinical impairment and social disability of patients with psychotic disorders [8]. Despite increasing evidence that pharmacological and psychosocial interventions are effective in improving clinical symptoms and patients’ functioning, the availability of treatment interventions and integration in psychiatric care is often suboptimal [9]. Also, many patients with psychotic disorders find it difficult to express their needs, show a decreased awareness of their symptoms and only partially understand the different possible treatment options. Therefore, psychological, medical and social problems often go undetected or untreated [10]. There is a challenge to monitor symptoms and unmet care needs of these patients in order to offer optimal care, especially in realizing their varying needs in different domains for many years.

Routine outcome monitoring (ROM) is one such way to monitor symptoms and care needs. ROM can be described as the use of standardized instruments to systematically and repeatedly measure different aspects of patients’ symptoms, health, social functioning and wellbeing in order to improve their treatment [11, 12]. For patients with schizophrenia and related mental health problems, regular participation in ROM contributes to systematic evaluation of their varying needs in (mental) health care over many years. This could offer these patients relevant treatment options adjusted to actual needs. However, only few clinicians use ROM data in their day-to-day work [13, 14]. In the Northern Netherlands, an extensive ROM screening called the Pharmacotherapy Monitoring and Outcome Survey (ROM-Phamous), consisting of a large array of instruments [15], has been implemented since 2007. Its main target is to identify needs of care in psychiatric, medical and social domains in order to optimize the treatment of patients with psychotic disorders. The data obtained also allows for scientific research. Although it has been shown that ROM-Phamous successfully helps to identify unmet needs, it is still not optimally used in clinical decision making and in offering recommended evidence-based treatment options to people with psychotic disorders [16‐18].

There is often not just a single best option when making treatment decisions in healthcare. Different treatment options may have varying risks and benefits, making it challenging to offer the optimal option when decisions are sensitive to personal preference. Clinical decision aids (CDAs) are evidence-based tools to support decision making in healthcare and have been gaining popularity in various medical disciplines [19]. A recent meta-analysis shows CDAs improve patients’ knowledge about available treatment options, facilitate accurate risk perception and increase their active involvement in the decision making process [20]. While knowledge about effective mental health care keeps growing, translation to daily clinical practice is lagging [21, 22]. CDAs can serve as a guideline implementation strategy by transferring evidence-based knowledge to day-to-day patient care. Despite the potential benefits of CDAs, their use in mental healthcare is very limited. In one study, a computerized CDA linking patient specific data to guidelines led to a decrease of symptoms and lowered re-hospitalization rates among people with psychotic disorders [23]. However, all diagnostic measurements for this CDA had to be collected by clinicians themselves, making the process time consuming. In the current study we describe Treatment E-Assist (TREAT). This is a recently developed computerized CDA that combines diagnostic patient data, collected using ROM-Phamous, with guidelines. TREAT facilitates the use of ROM-Phamous in daily clinical practice by summarising patients’ unmet needs. As a second step, evidence-based treatment recommendations based on the Multidisciplinary Guideline for Schizophrenia are generated to assist the clinician and patient to make shared decisions about these unmet needs.

This article describes the development of TREAT and the results of the pilot study, evaluating TREAT, as a computerized CDA designed for the treatment of patients with psychotic disorders. The pilot study tests the feasibility of TREAT in daily clinical practice by evaluating clinicians’ experiences when working with the application.

All clinicians had worked with ROM-Phamous for 7 years on average (ranging from 3 years to 9 years).

They were generally positive about ROM-Phamous, grading it with an average of 7 (SD 0.89) on a scale from 1 (poor) to 10 (excellent). They have positive feelings towards ROM-Phamous (subscale ‘emotion’), see it as a useful addition to their job (subscale ‘usefulness’), but don’t find it very easy to use (subscale ‘ease of use’). Results are depicted in Table 1. Clinicians were also positive about TREAT with an average rating of 7.5 (SD 0.84) and the integration of TREAT and ROM with an average rating of 7.3 (SD 1.37) on a scale from 1 (poor) to 10 (excellent). Most clinicians had used TREAT at least three times during the pilot study, with the exception of one clinician who only worked with TREAT a single time. The time TREAT was used differed per user: 5 to 15 min (1 psychiatrist), 15 to 30 min (1 nurse-practitioner and 1 physician) and more than 30 min per session (2 psychiatrist and 1 nurse-practitioner). Most clinicians find TREAT useful and expect it will help them improve their work. They state that it helps to interpret the (diagnostical) ROM-Phamous results and offers support in drafting the treatment plan. It also enhances awareness of the existing treatment options (subscale ‘usefulness’). TREAT fits with good clinical care, clinicians are proud of its development (subscale ‘emotion’) and expect to use TREAT in the future (subscale ‘usage behaviour’). TREAT is thought to be easy to use and requires little mental effort. Opinions about the lay-out of TREAT are mixed (subscale ‘ease of use’), for example some clinicians preferred the graphs displaying data vertically whilst others preferred it horizontally. Most clinicians state that they have enough time to use TREAT in daily clinical practice and that it helps them to work more efficiently (subscale ‘facilitating conditions’). They do not think TREAT will have impact on their professional autonomy (subscale ‘power’), on their job or on patient care in general (subscale ‘issue-impact’). Results are depicted in Table 2. Some clinicians preferred to use TREAT on the computer while others preferred to print the information. The topics that were mentioned in the open interview are depicted in Table 3.

Both clinicians and patients completed the CDRC questionnaires. All clinicians filled out the staff version of the CDRC but some of the patients experienced difficulties with their version of the questionnaire. They remembered most of the topics discussed in the feedback sessions but some had trouble to be specific or to categorize the different topics or mention specific treatment options that were suggested.

In general clinicians were positive about working with TREAT. Most of them found the system easy to use without requiring extra mental effort or time. This is important as usability and limited time investment are the two most important factors affecting successful implementation of CDAs [26]. One clinician did prefer the printed TREAT report for the feedback sessions because this person felt that the computer could disturb communication with patients. However, concerns from a previous study that computer use would be distracting, decrease eye contact or depersonalize the interaction [27], were not replicated. All clinicians indicated they would like to continue working with TREAT as they found it fitting with good clinical practice. Some concerns were expressed about the recommendations being or becoming too familiar over time. Lack of novelty could potentially stop clinicians from working with the application. In contrast, some clinicians became more aware of guidelines and discovered new treatment options. There is growing evidence that treatment in accordance with guidelines within mental healthcare, can positively affect patient care [8, 28, 29]. One clinician suggested that the ROM-Phamous results of previous years should be incorporated in order to evaluate changes over time. This exemplifies that the TREAT algorithms can be improved and tailored even more to individual patients. CDAs sometimes fail to take contextual information into account or have algorithms that insufficiently fit complex patient scenarios [30]. When asking clinicians about their opinions regarding ROM-Phamous, they indicated that the outcomes are not always easy to use for clinical decision making. This is in line with previous studies showing that the outcomes of ROM-Phamous are not used to its full potential in daily clinical practice [17, 18, 31]. TREAT simplifies the interpretation of these outcomes and facilitates a basis for more explicit decision making. Future research should focus on the effects of TREAT in the clinical decision making process. Lessons learned can improve the TREAT application, but may also help to develop other CDA’s.

Adequately informing patients about their health and available treatment options is the future of healthcare, in which CDAs should play a pivotal role. While CDAs are commonly used in medical fields such as oncology, orthopedics and cardiology, its use in mental healthcare is still very rare. With TREAT a high quality and easy to use CDA is now available in this field. Some adjustments will be made to the system based on the results of this pilot study. For instance the algorithms will be updated to ensure that TREAT reports can still be generated when part of the data is missing. ROM-Phamous consists of multiple instruments so patients are not always able to complete all questionnaires. Furthermore, two printable versions of the TREAT report will be added, namely a summary and the complete report. These adjustments will enable a large follow-up study aiming to investigate the effects of TREAT on clinical decision making. Because there are hardly any validated methodologies available to assess the content of treatment sessions in mental healthcare, modified and translated versions of the CDRC questionnaires were tested. Clinicians were able to fill out their version. However, patients experienced difficulties in categorizing the different topics that were discussed during the treatment sessions. These difficulties probably reflect cognitive difficulties in line with their psychiatric problems. Therefore, we will use only the staff version of the CDRC questionnaire for future research. Clinicians reported that patients appreciated the TREAT report and that it could improve shared decision making. Previous research in somatic medicine supports the notion that CDAs are effective in improving shared decision-making [20]. Recently there have been calls for widespread access to CDAs to improve the level of shared decision-making within mental healthcare [32]. Patients prefer an active role in the decision-making process and are more likely to adhere to their treatment plans if they actively participated in the draft [33]. The follow-up study is designed to assess whether CDA’s, in this case TREAT, can indeed increase shared decision making within a mental healthcare setting. So far research on computerized CDAs shows that they may improve disease management and diagnostics, however the effects on patients’ functioning and final clinical outcome remains unclear [34]. Assessing the effects of TREAT on patients’ symptoms, physical health and psychosocial functioning, will be another important goal of the follow-up study. If a CDA like TREAT is beneficial in treatment of psychotic disorders, it might be worthwhile to develop similar systems for different patient groups.

To the best of our knowledge, this study presents the first computerized CDA combining ROM and treatment guidelines in an electronic patient record within mental health care. TREAT was developed in close collaboration with healthcare professionals. This is a strength of TREAT, as sufficient knowledge of developers about the target group of CDAs is important for successful implementation [26]. ROM is sometimes experienced as behavioural control by some users who feel obliged by political or financial motives [18]. This was not replicated in previous or the current study [17, 18], as all clinicians had positive opinions about ROM-Phamous. A limitation of the current study is the small sample of only six clinicians from the same psychiatric institute. Although large enough to test the applicability of TREAT in daily clinical practice, a larger sample of clinicians from multiple centres are preferred when testing the effects of the application on the clinical decision making process and patient outcomes.