Hostname: page-component-848d4c4894-m9kch Total loading time: 0 Render date: 2024-05-12T16:26:52.427Z Has data issue: false hasContentIssue false
  • English
  • Français

Impact of data extraction errors in meta-analyses on the association between depression and peripheral inflammatory biomarkers: an umbrella review

Published online by Cambridge University Press:  09 November 2021

San Lee Open the ORCID record for San Lee [Opens in a new window] ,
Keum Hwa Lee Open the ORCID record for Keum Hwa Lee [Opens in a new window] ,
Kyung Mee Park ,
Sung Jong Park ,
Won Jae Kim ,
Jinhee Lee ,
Andreas Kronbichler ,
Lee Smith ,
Marco Solmi  and
Brendon Stubbs
...Show all authors
San Lee*
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea Department of Psychiatry, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea
Keum Hwa Lee
Affiliation:
Department of Pediatrics, Yonsei University College of Medicine, Seoul, Republic of Korea
Kyung Mee Park
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea Department of Psychiatry, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Republic of Korea
Sung Jong Park
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
Won Jae Kim
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
Jinhee Lee*
Affiliation:
Department of Psychiatry, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
Andreas Kronbichler
Affiliation:
Department of Medicine, University of Cambridge, Cambridge, UK
Lee Smith
Affiliation:
The Cambridge Centre for Sport and Exercise Sciences, Anglia Ruskin University, Cambridge CB1 1PT, UK
Marco Solmi
Affiliation:
Department of Psychiatry, University of Ottawa, Ontario, Canada Department of Mental Health, The Ottawa Hospital, Ontario, Canada Ottawa Hospital Research Institute (OHRI) Clinical Epidemiology Program, University of Ottawa, Ontario, Canada Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Institute of Psychiatry, Psychology and Neuroscience, Department of Psychosis Studies, King's College London, London, UK
Brendon Stubbs
Affiliation:
Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London SE5 8AF, UK Physiotherapy Department, South London and Maudsley NHS Foundation Trust, Denmark Hill, London SE5 8AZ, UK
Ai Koyanagi
Affiliation:
Parc Sanitari Sant Joan de Déu/CIBERSAM, Universitat de Barcelona, Fundació Sant Joan de Déu, Sant Boi de Llobregat, Barcelona, Spain ICREA, Pg. Lluis Companys 23, Barcelona, Spain
Louis Jacob
Affiliation:
Parc Sanitari Sant Joan de Déu/CIBERSAM, Universitat de Barcelona, Fundació Sant Joan de Déu, Sant Boi de Llobregat, Barcelona, Spain Faculty of Medicine, University of Versailles Saint-Quentin-en-Yvelines, Montigny-le-Bretonneux, France
Andrew Stickley
Affiliation:
Stockholm Center for Health and Social Change (SCOHOST), Södertörn University, Huddinge 141 89, Sweden
Trevor Thompson
Affiliation:
Department of Psychology, University of Greenwich, London SE109LS, UK
Elena Dragioti
Affiliation:
Pain and Rehabilitation Centre, and Department of Medical and Health Sciences, Linköping University, SE-581 85 Linköping, Sweden
Hans Oh
Affiliation:
School of Social Work, University of Southern California, CA, USA
Andre R. Brunoni
Affiliation:
Department of Psychiatry and Psychotherapy, University Hospital, LMU Munich, Munich, Germany Department of Psychiatry, Service of Interdisciplinary Neuromodulation, Laboratory of Neurosciences (LIM-27) and National Institute of Biomarkers in Neuropsychiatry (INBioN), Institute of Psychiatry, University of Sao Paulo, Sao Paulo, Brazil Departamento de Clínica Médica, Hospital Universitario, Faculdade de Medicina da USP, São Paulo, Brazil
Andre F. Carvalho
Affiliation:
Centre for Addiction and Mental Health (CAMH), Toronto, ON, Canada Department of Psychiatry, University of Toronto, Toronto, ON, Canada
Joaquim Radua
Affiliation:
Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Institute of Psychiatry, Psychology and Neuroscience, Department of Psychosis Studies, King's College London, London, UK Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institute, Stockholm, Sweden Imaging of Mood- and Anxiety-Related Disorders (IMARD) Group, Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), CIBERSAM, Barcelona, Spain
Suk Kyoon An
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
Kee Namkoong
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
Eun Lee*
Affiliation:
Department of Psychiatry, Yonsei University College of Medicine, Seoul, Republic of Korea Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea
Jae Il Shin*
Affiliation:
Department of Pediatrics, Yonsei University College of Medicine, Seoul, Republic of Korea
Paolo Fusar-Poli
Affiliation:
Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Institute of Psychiatry, Psychology and Neuroscience, Department of Psychosis Studies, King's College London, London, UK OASIS service, South London and Maudsley NHS Foundation Trust, London, UK Department of Brain and Behavioural Sciences, University of Pavia, Pavia, Italy
*
Author for correspondence: Eun Lee, E-mail: leeeun@yuhs.ac; Jae Il Shin, E-mail: shinji@yuhs.ac
Author for correspondence: Eun Lee, E-mail: leeeun@yuhs.ac; Jae Il Shin, E-mail: shinji@yuhs.ac
Author for correspondence: Eun Lee, E-mail: leeeun@yuhs.ac; Jae Il Shin, E-mail: shinji@yuhs.ac
Author for correspondence: Eun Lee, E-mail: leeeun@yuhs.ac; Jae Il Shin, E-mail: shinji@yuhs.ac
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Accumulating evidence suggests that alterations in inflammatory biomarkers are important in depression. However, previous meta-analyses disagree on these associations, and errors in data extraction may account for these discrepancies.

Methods

PubMed/MEDLINE, Embase, PsycINFO, and the Cochrane Library were searched from database inception to 14 January 2020. Meta-analyses of observational studies examining the association between depression and levels of tumor necrosis factor-α (TNF-α), interleukin 1-β (IL-1β), interleukin-6 (IL-6), and C-reactive protein (CRP) were eligible. Errors were classified as follows: incorrect sample sizes, incorrectly used standard deviation, incorrect participant inclusion, calculation error, or analysis with insufficient data. We determined their impact on the results after correction thereof.

Results

Errors were noted in 14 of the 15 meta-analyses included. Across 521 primary studies, 118 (22.6%) showed the following errors: incorrect sample sizes (20 studies, 16.9%), incorrect use of standard deviation (35 studies, 29.7%), incorrect participant inclusion (7 studies, 5.9%), calculation errors (33 studies, 28.0%), and analysis with insufficient data (23 studies, 19.5%). After correcting these errors, 11 (29.7%) out of 37 pooled effect sizes changed by a magnitude of more than 0.1, ranging from 0.11 to 1.15. The updated meta-analyses showed that elevated levels of TNF- α, IL-6, CRP, but not IL-1β, are associated with depression.

Conclusions

These findings show that data extraction errors in meta-analyses can impact findings. Efforts to reduce such errors are important in studies of the association between depression and peripheral inflammatory biomarkers, for which high heterogeneity and conflicting results have been continuously reported.

Keywords

Data extraction errordepressioninflammatory biomarkermeta-analysisumbrella review
Type
Original Article
Information
Psychological Medicine , Volume 53 , Issue 5 , April 2023 , pp. 2017 - 2030
Check for updates
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Albajes-Eizagirre, A., Solanes, A., & Radua, J. (2019). Meta-analysis of non-statistically significant unreported effects. Statistical Methods in Medical Research, 28(12), 3741–3754. doi: 10.1177/0962280218811349CrossRefGoogle ScholarPubMed
Belur, J., Tompson, L., Thornton, A., & Simon, M. (2018). Interrater reliability in systematic review methodology: Exploring variation in coder decision-making. Sociological Methods & Research, 50(2), 837–865. doi: 10.1177/0049124118799372CrossRefGoogle Scholar
Berlin, J. A., & Golub, R. M. (2014). Meta-analysis as evidence: Building a better pyramid. JAMA, 312(6), 603–606. doi: 10.1001/jama.2014.8167CrossRefGoogle ScholarPubMed
Beurel, E., Toups, M., & Nemeroff, C. B. (2020). The bidirectional relationship of depression and inflammation: Double trouble. Neuron, 107(2), 234–256. doi: https://doi.org/10.1016/j.neuron.2020.06.002.CrossRefGoogle ScholarPubMed
Bizik, G. (2010). Meta-analysis of plasma interleukine-6 levels in patients with depressive disorder. Activitas Nervosa Superior, 52(2), 76–80.CrossRefGoogle Scholar
Brunoni, A. R., Supasitthumrong, T., Teixeira, A. L., Vieira, E. L. M., Gattaz, W. F., Benseñor, I. M., … Maes, M. (2020). Differences in the immune-inflammatory profiles of unipolar and bipolar depression. Journal of Affective Disorders, 262, 8–15. doi: https://doi.org/10.1016/j.jad.2019.10.037.CrossRefGoogle ScholarPubMed
Buscemi, N., Hartling, L., Vandermeer, B., Tjosvold, L., & Klassen, T. P. (2006). Single data extraction generated more errors than double data extraction in systematic reviews. Journal of Clinical Epidemiology, 59(7), 697–703. doi: 10.1016/j.jclinepi.2005.11.010CrossRefGoogle ScholarPubMed
D'Acunto, G., Nageye, F., Zhang, J., Masi, G., & Cortese, S. (2019). Inflammatory cytokines in children and adolescents with depressive disorders: A systematic review and meta-analysis. Journal of Child and Adolescent Psychopharmacology, 29(5), 362–369. doi: 10.1089/cap.2019.0015CrossRefGoogle ScholarPubMed
Dargél, A. A., Godin, O., Kapczinski, F., Kupfer, D. J., & Leboyer, M. (2015). C-reactive protein alterations in bipolar disorder: A meta-analysis. Journal of Clinical Psychiatry, 76(2), 142–150. doi: http://dx.doi.org/10.4088/JCP.14r09007.CrossRefGoogle ScholarPubMed
Dowlati, Y., Herrmann, N., Swardfager, W., Liu, H., Sham, L., Reim, E. K., & Lanctot, K. L. (2010). A meta-analysis of cytokines in major depression. Biological Psychiatry, 67(5), 446–457. doi: 10.1016/j.biopsych.2009.09.033CrossRefGoogle ScholarPubMed
Ellul, P., Boyer, L., Groc, L., Leboyer, M., & Fond, G. (2016). Interleukin-1 beta-targeted treatment strategies in inflammatory depression: Toward personalized care. Acta Psychiatrica Scandinavica, 134(6), 469–484. doi: 10.1111/acps.12656CrossRefGoogle ScholarPubMed
Fernandes, B. S., Steiner, J., Molendijk, M. L., Dodd, S., Nardin, P., Goncalves, C. A., … Berk, M. (2016). C-reactive protein concentrations across the mood spectrum in bipolar disorder: A systematic review and meta-analysis. Lancet. Psychiatry, 3(12), 1147–1156. doi: 10.1016/s2215-0366(16)30370-4CrossRefGoogle ScholarPubMed
Ford, A. C., Guyatt, G. H., Talley, N. J., & Moayyedi, P. (2010). Errors in the conduct of systematic reviews of pharmacological interventions for irritable bowel syndrome. American Journal of Gastroenterology, 105(2), 280–288. doi: 10.1038/ajg.2009.658CrossRefGoogle ScholarPubMed
GetData Graph Digitizer. (2013). GetData Graph Digitizer. Retrieved 18 November 2019, from http://www.getdata-graph-digitizer.com.Google Scholar
Goldsmith, D. R., Rapaport, M. H., & Miller, B. J. (2016). A meta-analysis of blood cytokine network alterations in psychiatric patients: Comparisons between schizophrenia, bipolar disorder and depression. Molecular Psychiatry, 21(12), 1696–1709. doi: 10.1038/mp.2016.3CrossRefGoogle ScholarPubMed
Gotzsche, P. C., Hrobjartsson, A., Maric, K., & Tendal, B. (2007). Data extraction errors in meta-analyses that use standardized mean differences. JAMA, 298(4), 430–437. doi: 10.1001/jama.298.4.430Google ScholarPubMed
Goya-Maldonado, R., Brodmann, K., Keil, M., Trost, S., Dechent, P., & Gruber, O. (2016). Differentiating unipolar and bipolar depression by alterations in large-scale brain networks. Human Brain Mapping, 37(2), 808–818.CrossRefGoogle ScholarPubMed
Howren, M. B., Lamkin, D. M., & Suls, J. (2009). Associations of depression with C-reactive protein, IL-1, and IL-6: A meta-analysis. Psychosomatic Medicine, 71(2), 171–186. doi: 10.1097/PSY.0b013e3181907c1bCrossRefGoogle ScholarPubMed
Jones, A. P., Remmington, T., Williamson, P. R., Ashby, D., & Smyth, R. L. (2005). High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. Journal of Clinical Epidemiology, 58(7), 741–742. doi: 10.1016/j.jclinepi.2004.11.024CrossRefGoogle ScholarPubMed
Koenig, H. G., Cohen, H. J., George, L. K., Hays, J. C., Larson, D. B., & Blazer, D. G. (1997). Attendance at religious services, interleukin-6, and other biological parameters of immune function in older adults. International Journal of Psychiatry in Medicine, 27(3), 233–250. doi: 10.2190/40nf-q9y2-0gg7-4wh6CrossRefGoogle ScholarPubMed
Kohler, C. A., Freitas, T. H., Maes, M., de Andrade, N. Q., Liu, C. S., Fernandes, B. S., … Carvalho, A. F. (2017). Peripheral cytokine and chemokine alterations in depression: A meta-analysis of 82 studies. Acta Psychiatrica Scandinavica, 135(5), 373–387. doi: 10.1111/acps.12698CrossRefGoogle ScholarPubMed
Köhler-Forsberg, O., Buttenschøn, H. N., Tansey, K. E., Maier, W., Hauser, J., Dernovsek, M. Z., … Mors, O. (2017). Association between C-reactive protein (CRP) with depression symptom severity and specific depressive symptoms in major depression. Brain, Behavior, and Immunity, 62, 344–350. doi: https://doi.org/10.1016/j.bbi.2017.02.020.CrossRefGoogle ScholarPubMed
Komulainen, P., Lakka, T. A., Kivipelto, M., Hassinen, M., Penttila, I. M., Helkala, E. L., … Rauramaa, R. (2007). Serum high sensitivity C-reactive protein and cognitive function in elderly women. Age and Ageing, 36(4), 443–448. doi: 10.1093/ageing/afm051CrossRefGoogle ScholarPubMed
Liu, Y., Ho, R. C., & Mak, A. (2012). Interleukin (IL)-6, tumour necrosis factor alpha (TNF-alpha) and soluble interleukin-2 receptors (sIL-2R) are elevated in patients with major depressive disorder: A meta-analysis and meta-regression. Journal of Affective Disorders, 139(3), 230–239. doi: 10.1016/j.jad.2011.08.003CrossRefGoogle ScholarPubMed
Luo, D., Wan, X., Liu, J., & Tong, T. (2018). Optimally estimating the sample mean from the sample size, median, mid-range, and/or mid-quartile range. Statistical Methods in Medical Research, 27(6), 1785–1805. doi: 10.1177/0962280216669183CrossRefGoogle ScholarPubMed
Maes, M. (1995). Evidence for an immune response in major depression: A review and hypothesis. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 19(1), 11–38. doi: 10.1016/0278-5846(94)00101-mCrossRefGoogle ScholarPubMed
Messori, A., Scroccaro, G., & Martini, N. (1993). Calculation errors in meta-analysis. Annals of Internal Medicine, 118(1), 77–78. doi: 10.7326/0003-4819-118-1-199301010-00022CrossRefGoogle ScholarPubMed
Miller, A. H., & Raison, C. L. (2016). The role of inflammation in depression: From evolutionary imperative to modern treatment target. Nature Reviews. Immunology, 16(1), 22–34. doi: 10.1038/nri.2015.5CrossRefGoogle Scholar
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269.CrossRefGoogle ScholarPubMed
Munkholm, K., Vinberg, M., & Vedel Kessing, L. (2013). Cytokines in bipolar disorder: A systematic review and meta-analysis. Journal of Affective Disorders, 144(1–2), 16–27. doi: 10.1016/j.jad.2012.06.010CrossRefGoogle ScholarPubMed
Ng, A., Tam, W. W., Zhang, M. W., Ho, C. S., Husain, S. F., McIntyre, R. S., & Ho, R. C. (2018). IL-1beta, IL-6, TNF- alpha and CRP in elderly patients with depression or Alzheimer's disease: Systematic review and meta-analysis. Scientific Reports, 8(1), 12050. doi: 10.1038/s41598-018-30487-6CrossRefGoogle ScholarPubMed
Osimo, E. F., Baxter, L. J., Lewis, G., Jones, P. B., & Khandaker, G. M. (2019). Prevalence of low-grade inflammation in depression: A systematic review and meta-analysis of CRP levels. Psychological Medicine, 49(12), 1958–1970. doi: 10.1017/s0033291719001454CrossRefGoogle ScholarPubMed
Park, J. H., Eisenhut, M., van der Vliet, H. J., & Shin, J. I. (2017). Statistical controversies in clinical research: Overlap and errors in the meta-analyses of microRNA genetic association studies in cancers. Annals of Oncology, 28(6), 1169–1182. doi: 10.1093/annonc/mdx024CrossRefGoogle ScholarPubMed
Perrin, A. J., Horowitz, M. A., Roelofs, J., Zunszain, P. A., & Pariante, C. M. (2019). Glucocorticoid resistance: Is it a requisite for increased cytokine production in depression? A systematic review and meta-analysis. Frontiers in Psychiatry, 10, 423. doi: 10.3389/fpsyt.2019.00423CrossRefGoogle ScholarPubMed
Radua, J., Schmidt, A., Borgwardt, S., Heinz, A., Schlagenhauf, F., McGuire, P., & Fusar-Poli, P. (2015). Ventral striatal activation during reward processing in psychosis: A neurofunctional meta-analysis. JAMA Psychiatry, 72(12), 1243–1251. doi: 10.1001/jamapsychiatry.2015.2196CrossRefGoogle ScholarPubMed
Rowland, T., Perry, B. I., Upthegrove, R., Barnes, N., Chatterjee, J., Gallacher, D., & Marwaha, S. (2018). Neurotrophins, cytokines, oxidative stress mediators and mood state in bipolar disorder: Systematic review and meta-analyses. British Journal of Psychiatry, 213(3), 514–525. doi: 10.1192/bjp.2018.144CrossRefGoogle Scholar
Soneji, S. (2018). Errors in data input in meta-analysis on association between initial use of e-cigarettes and subsequent cigarette smoking among adolescents and young adults. JAMA Pediatrics, 172(1), 92–93. doi: 10.1001/jamapediatrics.2017.4200CrossRefGoogle ScholarPubMed
Steptoe, A., Kunz-Ebrecht, S. R., & Owen, N. (2003). Lack of association between depressive symptoms and markers of immune and vascular inflammation in middle-aged men and women. Psychological Medicine, 33(4), 667–674.CrossRefGoogle ScholarPubMed
Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., … Thacker, S. B. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA, 283(15), 2008–2012.CrossRefGoogle ScholarPubMed
Suarez, E. C. (2004). C-reactive protein is associated with psychological risk factors of cardiovascular disease in apparently healthy adults. Psychosomatic Medicine, 66(5), 684–691. doi: 10.1097/01.psy.0000138281.73634.67CrossRefGoogle ScholarPubMed
Tendal, B., Higgins, J. P., Juni, P., Hrobjartsson, A., Trelle, S., Nuesch, E., … Gotzsche, P. C. (2009). Disagreements in meta-analyses using outcomes measured on continuous or rating scales: Observer agreement study. BMJ (Clinical Research Ed.), 339, b3128. doi: 10.1136/bmj.b3128CrossRefGoogle ScholarPubMed
Wan, X., Wang, W., Liu, J., & Tong, T. (2014). Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Medical Research Methodology, 14(1), 135.CrossRefGoogle ScholarPubMed
Wium-Andersen, M. K., Orsted, D. D., Nielsen, S. F., & Nordestgaard, B. G. (2013). Elevated C-reactive protein levels, psychological distress, and depression in 73, 131 individuals. JAMA Psychiatry, 70(2), 176–184. doi: 10.1001/2013.jamapsychiatry.102CrossRefGoogle ScholarPubMed
Xiong, Z., Liu, T., Tse, G., Gong, M., Gladding, P. A., Smaill, B. H., … Zhao, J. (2018). A machine learning aided systematic review and meta-analysis of the relative risk of atrial fibrillation in patients with diabetes mellitus. Frontiers in Physiology, 9, 835. doi: 10.3389/fphys.2018.00835CrossRefGoogle ScholarPubMed
Supplementary material: File

Lee et al. supplementary material

Lee et al. supplementary material

Download Lee et al. supplementary material(File)
File 1.6 MB