Insights of Patients and Clinicians on the Promise of the Experience Sampling Method for Psychiatric Care

The experience sampling method (ESM) receives increasing attention in psychiatry and holds the promise to greatly advance personalized health care (1). ESM involves the repeated sampling of people’s moods, symptoms, experiences, behaviors, and contexts (2). Research has thus far applied ESM to elucidate the daily life dynamics of myriad psychiatric disorders (3). Because ESM entails intensive self-monitoring and the resulting data can reveal individual models of associations among daily life experiences, ESM is assumed to also have relevance for psychiatric practice (4).

Although monitoring in itself may already benefit emotional self-awareness (5, 6), supplementing ESM monitoring with personalized feedback might provide feelings of empowerment (7) and even improve symptoms (8), suggesting potential usefulness for both patients and clinicians. Indeed, researchers believe that ESM may provide microlevel information that is difficult to catch from clinical impressions and that this information can add to processes of diagnostics, treatment choice, and relapse prevention (9, 10). However, the general assumption that ESM can be of value to psychiatric care lacks a solid evidence base, and how and when to apply ESM remains unclear. Thus far, ESM in research has often been short in duration (e.g., five to 14 days) with intensive sampling (three to 10 times a day) and without personalized feedback (11). In clinical practice, ESM can be expected to require a different form (8).

For health care innovations to be effectively introduced in clinical practice, premier stakeholders need to be included and barriers to implementation addressed beforehand (12). This requires a currently unavailable in-depth qualitative study into the views of patients and clinicians on the opportunities of ESM for psychiatry. Only one qualitative study reported that patients with psychosis recognized the advantages of ESM, but it did not include clinician views (13). Although patients will be the primary users of ESM, clinicians might be important stakeholders in introducing ESM and might use ESM themselves to inform treatment decisions (9, 14). This study is the first to map the relevance of ESM for psychiatric practice and examine barriers to and facilitators of implementation through focus groups and interviews with patients and clinicians.

Methods

Participants

Reporting of this study is done according to the Standards for Reporting Qualitative Research (15). Participants were clinicians (i.e., psychiatrists, psychologists, psychiatric nurses, or job coaches) and psychiatric patients who received mental health care during the study or in the recent past. Participants were selected with the aim of achieving maximum variation on age, gender, diagnosis (if a patient), experience with ESM or mobile technology, and affinity with research. Participants were recruited through posters and contacts at mental health institutions until no new information was heard during the interviews and focus groups (data saturation). After they signed up, participants were e-mailed more information on the study. They were also invited (but not required) to participate in an open-source ESM study (16) to try out ESM before participation.

Of the 31 patients who signed up for participation, 22 showed up and provided informed consent. The remaining nine did not provide reasons for the no-show. Of the 23 approached clinicians, 21 showed up and provided informed consent. The remaining two clinicians were unable to participate because of time constraints. The institutional review board of the University Medical Center Groningen approved of the study.

Interviewers

Interviews were conducted by authors FB (M.Sc., female, Ph.D. candidate) and LK (Ph.D., female, postdoctoral researcher and psychologist), who were trained in qualitative interviewing and analysis. Focus groups were conducted by FB as moderator, with assistance of ES (Ph.D., female, postdoctoral researcher) or LK. There was no contact between researchers and patients before the study. LK and FB knew some of the clinicians.

Interviews and Focus Groups

Interviews (on average 57 minutes) and focus groups (92 minutes) were conducted in several mental health care institutions and private practices in the northern Netherlands between June 2016 and February 2017. One focus group participant was later individually interviewed to elaborate on a potential downside of ESM she was reluctant to share in the focus group. Interviewers explained the study rationale and what ESM entails (see the online supplement to this article). ESM was explained as a method by which individuals can record their moods, experiences, behaviors, contexts, and thoughts multiple times per day on their smart phones (2). Example items (e.g., “I feel relaxed”) and possible ESM-derived feedback were shown, such as mood variation, mood during activities, and associations between mood and behavior (16).

A semistructured interview guide was used to ask open questions (see online supplements), covering the usefulness of ESM in general and specific phases of care, possible consequences of using ESM, implementation in care, and ESM protocol design. Example questions include, “What do you think of ESM?,” “How do you view the implementation of ESM in clinical care?,” and “Do you see possible risks or downsides to ESM?” All interviews and focus groups were audio-recorded and field notes were made.

Data Analysis

The digital audio recording of each interview and focus group was transcribed verbatim. Thematic analysis was applied by FB and LK according to the Qualitative Analysis Guide of Leuven (17). This approach involves the identification of central themes in the transcripts, which are iteratively verified against the data.

First, all transcripts were summarized in conceptual interview schemes and narrative reports to gain a holistic understanding of the participants’ experiences. Next, a concept code list was constructed based on subthemes identified in the data (e.g., time investment). FB and LK used this code list to independently code the transcripts in ATLAS.ti (version 8). Throughout this first round of coding, new codes were created when previously unidentified themes were encountered, and existing codes were more clearly defined through consensus. Hereafter, the code list was finalized and used in a second round of coding.

The codes were grouped in four overarching coding categories or central themes. These central themes were verified against all transcripts and discussed with ES and MW. Participants were invited to provide feedback on a summary of the central themes.

Results

Four themes were identified (see Figure 1). Participant characteristics are described in Table 1. For illustrative quotes related to the themes, see Tables 2 and 3.

Theme 1: Applications

Most patients and clinicians believed ESM could be applied flexibly in every phase of care, from diagnosis to relapse prevention, depending on the patients’ care needs. First, by monitoring symptoms, experiences, and contexts multiple times a day, many patients and clinicians suggested that ESM could be used to help patients focus on the present and increase real-time awareness of what influences their symptoms.

Second, all patients and clinicians believed that ESM and ESM-derived feedback could offer relevant insights on the severity and variation of symptoms; short- and long-term associations between symptoms, experiences, behavior, context, medication, drugs, and life events; symptom reduction; and patterns building up to symptoms over shorter (e.g., panic attack) or longer (e.g., depressive episode) time periods (Figure 2). As such, most patients and clinicians believed that ESM could be applied to strengthen patients’ self-management by providing them with concrete insights on how to cope with their symptoms.

Patients and clinicians also discussed employing ESM to determine intervention effects, thereby guiding decisions regarding future course of treatment. The majority of patients and clinicians suggested that the personalized nature of ESM has the potential to convince patients to start or continue interventions or behaviors if their effectiveness could be demonstrated by ESM-derived personalized feedback.

Finally, multiple patients mentioned that ESM might be used to alert patients and their clinicians of a change in symptom levels (e.g., if their ESM scores reach below or above a certain cut-off). Several patients argued that such alerts could help them notice the beginning of a downward spiral and could easily update clinicians on how they are doing. Possibly, personalized therapeutic advice could be attached to these alerts to help patients directly alleviate symptoms and practice treatment strategies in daily life. However, several clinicians were hesitant of the possibility of receiving alerts, worrying about patient safety, responsibility, and time constraints.

Theme 2: Advantages

Patients and clinicians identified several advantages of ESM for clinical practice. First, ESM may benefit the clinician-patient relationship by providing a framework for shared decision making. Multiple patients indicated that ESM may help articulate their experiences, consequently making them feel more heard and understood. As such, ESM was believed to lead to better mutual understanding between patient and clinician and provide a larger role for the patient perspective.

Second, ESM was generally seen as resulting in data that is “personalized,” “neutral,” “objective,” and “nonjudgmental.” These characteristics of ESM were contrasted with receiving explicit advice or insights from clinicians, which patients do not always accept. Personalized and objective ESM data were perceived as convincing and seen as the key to gaining insight and changing behavior, especially if the interpretation of ESM-derived feedback is not imposed on patients by clinicians.

Third, the majority of patients and clinicians believed that using ESM provides a more reliable overview of a given period than directly asking the patient or administering a retrospective questionnaire. These patients indicated a difficulty in stating how they had felt since the previous session, which is often influenced by current mood. Some clinicians and patients with bipolar disorder mentioned that ESM also maps mood fluctuations more accurately than once-a-day mood questionnaires such as the Life Chart (18).

Fourth, many patients and clinicians expected ESM to result in novel information because ESM has more items than traditional registration strategies and focuses more on mood, experiences, behavior, and context rather than symptoms alone; illuminates the time between treatment sessions, which is otherwise difficult to capture; may lower the threshold to disclose sensitive information; and offers the possibility of automatically generated models of symptoms and contexts (e.g., network analysis) otherwise unavailable to patients and clinicians. This may also enhance efficiency according to some patients, because problem areas can be found faster with ESM than with current, mostly retrospective, methods.

Fifth, some clinicians mentioned that the visual nature of ESM-derived feedback may help explicate associations between, for example, mood and behaviors that are normally verbally discussed in therapy.

Sixth, clinicians and patients expected smartphone-based ESM assessments to be less burdensome than paper-and-pencil registration and less easily forgotten because patients are reminded through prompts. Some clinicians speculated that ESM may bring psychiatric care more “up to date,” thereby increasing resonance with patients’ everyday environments. Finally, some patients expected to enjoy the very process of monitoring, learning about themselves through ESM-derived feedback, and checking whether certain expectations are reflected in the data.

Theme 3: Undesirable Effects and Limitations

Patients and clinicians identified several potential undesirable effects and limitations of ESM monitoring or feedback. First, several patients and clinicians indicated that ESM could be burdensome when assessments are too frequent or too long in duration, assessments interfere with patients’ activities, patients already complete other questionnaires, patients have to type in entries, and ESM items are irrelevant to the patient. Burden was suggested to be reduced by clear delineation of the assessment period and letting the patient choose the timing and focus of the assessments.

Many patients and some clinicians feared that ESM monitoring would negatively influence patients’ well-being or worsen symptoms. Some patients mentioned that they might start dreading the assessments or feel guilty and incompetent if they miss assessments. Further, some participants mentioned that ESM may continually remind patients of their symptoms rather than what goes well, which may worsen symptomatology but could also help them acknowledge and handle their situation. Other plausible negative influences that were mentioned by one of the clinicians were ESM monitoring becoming a ritual, a constant focus on themselves rather than getting help, and too much emphasis on symptom scores instead of the meaning of symptoms. Negative reactivity was suggested to be partially resolved by asking more neutral or positive questions.

Most patients and clinicians did not believe that ESM-derived feedback would have negative consequences but mentioned that these may arise when patients do not recognize themselves in the results; ESM data does not reveal clear patterns, confirming patients’ belief that “it does not matter what I do”; or important associations are uncovered but impossible or difficult to change. Generally, clinicians believed it to be their task as professionals to help patients cope with these consequences, and indicated that this could also be a helpful learning process. Some clinicians warned that expectations of the relevance of ESM for clinical practice could be too high, emphasizing that it is only a tool and will not drastically change psychiatric care.

ESM was perceived to be applicable to all types of psychiatric syndromes, but some clinicians speculated it to be less suitable for patients with limited insight into their symptoms (e.g., young children, patients with autism), patients who prefer pills over psychological treatment, patients with lower intelligence, patients who are less comfortable with technology, patients with insufficient mastery of the assessment language, patients with neurocognitive deficits, patients who keep asking for reassurance, and patients with a psychotic disorder for whom phone use may increase paranoia. Clinicians disagreed on the risks of ESM for patients with personality disorders, suicidal ideation, alcohol or substance use disorders, somatic symptom disorder, and obsessive-compulsive disorder, wondering whether constantly focusing on their symptoms might make them worse.

Theme 4: Requirements for Implementation

Several requirements for smooth implementation of ESM in clinical practice were described. First, all patients and clinicians agreed that ESM should be a collaborative process, where patients and clinicians decide together on the relevance and feasibility of ESM, clinician access to the data, desirability of patient and clinician alerts, relevant items, the frequency and duration of assessments, and the interpretation of ESM-derived feedback. If not regularly evaluated, ESM might lose its advantages. Patients preferred ESM-derived feedback to be discussed by mental health professionals with whom they have a long-standing relationship, such as psychiatric nurses or experts by experience.

Further, it was generally viewed that both patients and clinicians should have access to the patient’s ESM data, and both should have a role in deciding when it is examined. Several patients assumed they would be the owner of their data and that they could decide whether or not to share those data with others. Ideally, patients wanted to be able to initiate ESM monitoring themselves but also recognized that without clinician involvement, ESM would be less effective in gaining insights and changing behavior. Some clinicians imagined that direct access to the data (not via the patient) is necessary to integrate ESM into treatment. However, some clinicians were concerned that continuous access to the patient’s ESM data may enhance the power imbalance between the two. They further underscored that they cannot be expected to constantly monitor the data and act on elevated scores.

Third, some patients and clinicians stressed that ESM should never replace face-to-face contact. Contact with clinicians should not solely depend on ESM scores, and patients should be encouraged to ask for help directly rather than through ESM.

Fourth, a number of clinicians discussed how patients could be kept motivated. This starts with a proper rationale and patient input on relevant constructs. Some clinicians believed that certain patients would be motivated to try ESM out of curiosity or because of extreme distress. Others argued that patients would need appropriate reward for their efforts (e.g., continuous ESM-derived feedback, a focus on positive experiences, and advice and compliments). Motivation was believed to disappear if clinicians did not discuss feedback or once the patient had gained sufficient insights from ESM.

Fifth, several clinicians wanted to receive training on potential threats to the validity of ESM-derived feedback and the selection of the proper ESM protocol. This includes research-guided information on item formulation, assessment frequency and duration, minimum number of assessments, and feedback interpretation.

Finally, many patients and clinicians highlighted the limited time of clinicians and indicated that user-friendly software and reimbursement from insurance companies might help clinicians to incorporate ESM into care.

Discussion

The present qualitative study aimed to gain an in-depth understanding of the relevance of ESM for psychiatric practice and barriers and facilitators for implementation. Importantly, clinicians and patients recognized many of the applications and advantages of ESM also highlighted in research, such as the monitoring of treatment effects (19), the beneficial effects on awareness (5) and empowerment (7), the potential for shared decision making (20), the increased reliability of the data compared with traditional assessment methods (21), and the possibility of real-time alerts on elevated scores (22). This study provides the first evidence that these applications and advantages of ESM are indeed desired in practice. Our findings contrast with those of a previous qualitative study (13) that reported that although patients recognized the benefits of ESM, they were unsure of its relevance for their own situation. However, the aforementioned study was limited to one specific six-day application of ESM (without feedback) for a specific patient group (psychosis) and did not include the perspective of clinicians, which may explain the differing results.

Patients and clinicians stressed that successful use of ESM will depend on the active involvement of patients in the selection of the ESM protocol, interpretation of ESM-derived feedback, and subsequent action taken based on ESM. They further emphasized that the specific application of ESM should vary across treatment phases according to the patient’s care needs. The need for clear agreements on data access became especially apparent when discussing real-time alerts. Although desired by patients, both patients and clinicians feared potentially adverse situations caused by not knowing whether the data were viewed and acted upon. Our findings are in line with research showing that tailored care and shared decision making may improve patient satisfaction, treatment adherence, and health status (23). They further highlight that patient involvement and flexible application are crucial factors for implementation of ESM.

Both patients and clinicians mentioned symptom worsening as a potential undesirable effect of ESM, because ESM may continuously make patients aware of their symptoms. However, studies among patients with substance abuse or pain disorder found little evidence of such negative reactivity in short-term ESM (24, 25); in fact, studies in psychiatric patients so far reported only favorable effects of self-monitoring (5, 8). Reactivity might vary according to specific patient characteristics, such as symptom severity, neuroticism, or readiness for change (26, 27). When ESM is implemented in practice, reactivity will need to be controlled, as is also common practice in research settings, through careful construction and ordering of the items (3). Nonetheless, some patients and clinicians worried that monitoring in itself might fixate patients on their illness, thereby hampering their autonomy. By providing constant reminders of their patient status, ESM walks a fine line between improving self-management and undermining it (28). This potential downside may occur regardless of item content, and although ESM may benefit patient empowerment, as has been suggested (7), future research will have to show for whom and under what circumstances that holds true.

The general consensus of participants was that most patients could benefit from ESM. However, clinicians expected that ESM might be less useful for patients with autism, paranoia, or substance use disorders. Interestingly, patients themselves believed ESM could be relevant for all psychiatric syndromes, as is supported by research (3, 29). This suggests that the potential of ESM is dependent not so much on psychiatric syndrome but rather on the willingness of the patient.

Finally, patients and clinicians highlighted that clinician training and research-guided advice are essential to guarantee the validity of ESM and minimize potential undesirable effects. These recommendations and our experiences with using ESM in practice have led us to believe that actual implementation of ESM can only be realized when researchers provide a framework that translates clinical hypotheses to ESM protocols, ensures that these protocols meet the strict rules also applied in research (27), and provides valid interpretation of ESM-derived feedback.

Strengths of this study include the in-depth nature of the interviews and focus groups and the large and diverse participant sample, varying on age, gender, occupation, diagnosis, discipline, and experience with mobile technology. Further, by exploring the views of two premier stakeholders (patients and clinicians), our qualitative approach allowed us to formulate key recommendations on the utility and implementation of ESM.

In contrast to quantitative research, the goal of qualitative research is not to generalize but to describe and understand phenomena that may be time and context specific. As such, generalizing the results to settings other than The Netherlands should be done with caution. Furthermore, most participants in our study were asked to envision the role of ESM in clinical care without having used the method; experiencing ESM might offer different results. Finally, most patients had mood disorders. Envisioned advantages and applications may differ for patients with other types of disorders.

Conclusions

This qualitative study provides the first evidence that the relevance of ESM for psychiatric care is recognized by both patients and clinicians. Based on the study’s findings, we suggest the following key recommendations for the optimal implementation of ESM. First, patients and clinicians should apply ESM flexibly (across care phases) and collaboratively. Second, clinicians should make clear agreements with patients on data access. Third, patients and clinicians should be aware that patient’ moods or symptoms may worsen because ESM assessments remind patients of their symptoms. Fourth, patients and clinicians should regularly evaluate whether ESM helps or hinders patient self-management. Fifth, ESM should be applied to all psychiatric syndromes, and no patient group should be excluded a priori. Finally, ESM needs to be implemented by an interdisciplinary team of patients, clinicians, researchers, and information technology specialists. If these recommendations are followed, ESM might very well deliver on its promise for psychiatric care.