Background
Several reasons can explain the growing interest of clinical research in ambulatory assessment methods. Patients are generally poor in providing accurate, global, retrospective reflections of real-life experiences and behaviors, which is especially problematic for depressed patients who suffer from cognitive impairment [1] and for psychotic patients who also experience a loss of reality testing [2]. Therefore, the symptomatic dynamics in patients, which are essential for improving the understanding of severe psychiatric disorders [3‐6], cannot be captured precisely through retrospective assessments [7]. Moreover, symptoms occur naturally in the context of daily life and as such, assessing patients in a controlled environment might neglect the influence of context on a patient’s state (e.g., [8, 9]). In this perspective, a growing body of research is now relying on ambulatory assessment methods such as the Experience Sampling Method (ESM, [10]) or Ecological Momentary Assessment (EMA, [11]). Based on repeated and momentary self-evaluations in the individual’s life context, this methodology allows capturing the evolution of affects, cognitions, or behaviors over time with a high ecological validity, as it limits potential artifacts from laboratory settings or retrospective recall [7, 12‐14].
Despite the rapid increase of the use of momentary assessment methods in clinical psychology and psychiatric research, methodological challenges related to the use of such methods remain. In particular, as this method repeatedly prompts individuals to rate their current affective and cognitive state in the context of their daily life, it is plausible that the repetition of self-evaluations induces reactivity (e.g., increased burden, fatigue, or awareness of cognitive and affective state). A number of studies have reported reactive effects to the repeated assessments in several populations including healthy participants [15, 16] and in individuals suffering from chronic pain [17], substance use disorders [18‐21], depression [22], and anxiety disorder or psychosis [19]. These reactive effects could involve a loss engagement in the participants, decreasing both the feasibility and the data quality in studies relying on this assessment method. Yet, the factors influencing this reactivity have not been clearly defined. In patients suffering from chronic pain, the systematic review of Morren et al. [23] reported several associations between reactivity and (i) methodological parameters related to the design of ESM studies such as the length of the ESM questionnaire or the use of a financial compensation and (ii) the characteristics of the sample such as the age of the participants. In this view, methodological parameters related to the design of ESM studies or to the characteristics of the sampled population are likely to undermine the feasibility and the quality of the data collected. However, the systematic review of Morren et al. [23] focused on momentary assessments related to chronic pain and these observations are likely to differ when targeting different clinical populations. Particularly, this important topic has not been investigated in severe psychiatric disorders while numerous ESM-based studies have been performed in these clinical populations. Considering the heterogeneity of designs in the related literature (e.g., with respect to the duration of the follow-up, the amount of self-evaluations, or the type of device used for data collection), there is a crucial need to answer these methodological concerns.
Anzeige
Our objective is to investigate protocol- and subject-related factors that could undermine the feasibility of ambulatory studies and the quality of the data collected through this methodological framework in psychotic disorder (PD), major depressive disorder (MDD), bipolar disorder (BD), and high-risk individuals for these disorders (HR).
Methods
This protocol is based on the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) guidelines [24]. For transparency and completeness, a completed PRISMA-P 2015 checklist is provided as an additional file [see Additional file 1]. The protocol has also been registered in the International Prospective Register of Systematic Reviews database (PROSPERO 2017:CRD42017060322).
Search strategy
A systematic literature search will be performed without publication time limit in PubMed and Web of Science. Observational as well as randomized controlled studies will be included. Case studies, case reports, protocols, study designs, and systematic reviews will not be considered. The search strategy was designed to include relevant terms regarding ambulatory and momentary assessments (e.g., “experience sampling method” and “ecological momentary assessment”) as well as terms related to the clinical diagnosis of the participants under study (e.g., “psychotic disorder”, “major depressive disorder”, “bipolar disorder”). The search strategy will use either MeSH or keyword heading. A concept plan was built with the identified keywords and descriptors to run the search [see Additional file 2]. Unpublished studies as well as studies published in another language than English will not be considered.
Only studies using ESM/EMA designs in PD, MDD, BD, and HR will be included in this systematic review. If available within the included studies, data from non-psychopathological/healthy control groups (HC) will be considered and will serve as a reference group. Ambulatory studies involving only a single daily assessment will be excluded as this time sampling is not representative of the repeated momentary assessment that defines ESM research. To determine the eligibility of the original studies, two researchers (HV and AR) will independently conduct the screening of the studies in the title/abstract and full-text phases based on the inclusion and exclusion criteria. The screening results will be compared in order to identify any discrepancies. In case of a disagreement, a third researcher (IMG) will be consulted. We will use the PRISMA flow diagram to report quantitative and qualitative information on every phase of the selection process.
Anzeige
Data extraction
When available, data will be extracted for the following items: (i) general study characteristics (i.e., authors, title, year, study design); (ii) sample characteristics (i.e., number of participants included in the analysis, age, gender, clinical status, ethnicity, educational status, employment status, marital status, cohabiting status, medication); (iii) ambulatory design characteristics (i.e., number of momentary assessments per day, number of ambulatory assessment days, number of ambulatory assessment periods [continuous or intermittent assessment], delay between ambulatory assessment periods, ambulatory sampling method [fixed, semi-random or random sampling], time intervals between the ambulatory assessments within a day, time intervals between the first and the last ambulatory assessment within a day, time of the start and the end of the ambulatory assessments within a day, number of items in the ambulatory questionnaire, approximate mean duration of the ambulatory questionnaire, type of scales used in the ambulatory questionnaire, type of device used to perform the ambulatory assessment, type of incentive, amount of the incentive); and (iv) the attrition rate (proportion of individuals excluded from the study compared to the number of individuals included at baseline) and the compliance rate (proportion of self-evaluations completed by the participants compared to the theoretical maximal number of self-evaluations allowed by the design) of the included studies. When needed, the corresponding authors of the original studies will be contacted for further information. Data from the included studies will be extracted and stored in a customized spreadsheet structured according to the items mentioned above and that will be provided as part of the supplementary materials in the upcoming systematic review.
Assessment of methodological quality and risk of bias
The current systematic review does not investigate the main outcomes of the studies that will be included. As such, the conventional assessment of methodological quality is not directly applicable. Instead, the quality of the reporting of the attrition and compliance rates in the included studies will be assessed independently by HV and AR. The scale and the scoring developed to perform this assessment will be derived from the four-point item of the Grades of Recommendation, Assessment, Development, and Evaluation Working Group [25] to fit the aims of this review. This scale will be defined clearly and provided as supplementary material in the upcoming systematic review. If needed, the corresponding authors of the original studies will be contacted for further information.
Data analysis
Studies providing sufficient data regarding the main outcomes (i.e., attrition rates and compliance rates) will be analyzed. If data are not amendable to a statistical analysis, a qualitative synthesis of the main outcomes under study will be conducted. If a sufficient amount of commensurable data is available, a quantitative synthesis (meta-analysis) will be performed.
For attrition, we will analyze the reported/calculated proportions. For compliance, there is in principle a proportion (of completed self-evaluations) per participant, but this information is unlikely to be available. Instead, we will analyze the mean proportions (i.e., \( {\overline{p}}_i=\left(\sum \limits_{j=1}^{n_i}{p}_{ij}\right)/{n}_i, \) where p
ij
denotes the proportion of completed evaluations for the jth participant in the ith study and n
i
the group size). We expect either \( {\overline{p}}_i \) to be reported directly (either in terms of a proportion or percentage) or the total number of self-evaluations collected, which is easily converted to \( {\overline{p}}_i \) (i.e., if \( {\overline{p}}_i={x}_i/\left({n}_i\times {m}_i\right) \), where x
i
denotes the total number of self-evaluations collected and m
i
the theoretical maximal number of self-evaluations allowed by the design).
Both outcomes will be analyzed using a multilevel random-effects model [26] with random effects/intercepts for studies and groups within studies. Also, we will include the group type (factor with five levels: PD, MDD, BD, HR, HC) and differences in protocols as fixed effects in the model. The a priori planned analyses consist of (1) a comparison of groups with different clinical diagnoses (PD vs. MDD vs. BD vs. HR vs. HC), (2) a comparison of population groups (i.e., based on socio-demographic factors), and (3) the influence of protocol characteristics on the outcomes (e.g., low- vs. high-frequency ambulatory sampling in PD). Results will be reported in terms of 95% confidence intervals and (two-sided) p values for each outcome. Heterogeneity will be assessed using the Q test and the I-squared statistic. A qualitative synthesis summarizing the quality of the report of the main outcomes in the database will be provided.
Discussion and conclusion
To date, little to no information is available regarding the methodological factors that could undermine the data quality and feasibility of ambulatory studies related to severe psychiatric disorders despite a large body of research relying on heterogeneous protocols. Assessing this question within an individual study would involve large samples following numerous protocols with varying parameters and as such would be particularly challenging. Accordingly, there is a need to conduct a meta-analysis to provide an exhaustive assessment of the available evidence using clear and reproducible methods. The objective of this systematic review is to synthesize knowledge surrounding the data quality of ambulatory designs in severe psychiatric disorders with respect to various methodological parameters and as a function of the participants’ clinical status. This systematic review aims to provide evidence-based information to researchers and clinicians on the design of ambulatory protocols and, if applicable, to propose guidelines in order to improve the quality of the data collected in this context.
Acknowledgements
Not applicable.
Funding
This research was supported by an ERC consolidator grant to IMG (ERC-2012-StG, project 309767—INTERACT), by an FWO Odysseus fellowship to IMG (G0F8416N), and by the Innovative Medicines Initiative 2 Joint Undertaking (grant number: 115902, RADAR-CNS).
Anzeige
Availability of data and materials
All data generated or analyzed during this study will be included in the upcoming article [and its supplementary information files].
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Anzeige
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.