Abstract
Objectives
Social and psychological interventions are often complex. Understanding randomised controlled trials (RCTs) of these complex interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of research, wastes resources, and fails to meet ethical obligations to research participants and consumers.
Methods
In this paper, we explain how reporting guidelines have improved the quality of reports in medicine, and describe the on-going development of a new reporting guideline for RCTs: CONSORT-SPI (an extension for social and psychological interventions).
Results and conclusions
We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed consensus on these guidelines (http://tinyurl.com/CONSORT-study).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abraham, C.. (2009). Wider recommendations to improve reporting of the content of behaviour change interventions. (For the Workgroup for Intervention Development and Evaluation Research). Retrieved November 14, 2012 from http://interventiondesign.co.uk/wp-content/uploads/2009/02/wider-recommendations.pdf
American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40.
American Psychological Association Journal Article Reporting Standards Group. (2008). Reporting standards for research in psychology: why do we need them? What might they be? American Psychologist, 63, 839–851.
Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., et al. (2008). Improving the reporting of public health intervention research: advancing TREND and CONSORT. Journal of Public Health, 30(1), 103–109.
Bonell, C. (2002). The utility of randomized controlled trials of social interventions: an examination of two trials of HIV prevention. Critical Public Health, 12(4), 321–334.
Bonell, C., Oakley, A., Hargreaves, J., Strange, V., & Rees, R. (2006). Assessment of generalisability in trials of health interventions: suggested framework and systematic review. BMJ, 333, 346–349.
Bonell, C. P., Hargreaves, J., Cousens, S., Ross, D., Hayes, R., Petticrew, M., et al. (2011). Alternatives to randomisation in the evaluation of public health interventions: design challenges and solutions. Journal of Epidemiological Community Health, 65, 582–587.
Bonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2012). Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Social Science & Medicine, 75(12), 2299–2306.
Boutron, I., Moher, D., Altman, D. G., Schulz, K., Ravaud, P., & for the CONSORT group. (2008a). Extending the CONSORT Statement to randomized trials of nonpharmacologic treatment: explanation and Elaboration. Annals of Internal Medicine, 148, 295–309.
Boutron, I., Moher, D., Altman, D. G., Schulz, K., Ravaud, P., & for the CONSORT group. (2008b). Methods and processes of the CONSORT group: Example of an extension for trials assessing nonpharmacologic treatments. Annals of Internal Medicine, 148(4), W60–W67.
Calvert, M., Blazeby, J., Revicki, D., Moher, D., & Brundage, M. (2011). Reporting quality of life in clinical trials: a CONSORT extension. The Lancet, 378, 1684–1685.
Campbell Collaboration. (2013). Retrieved 26 April 2013 from http://campbellcollaboration.org/.
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004). CONSORT statement: extension to cluster randomised trials. BMJ, 328, 702–708.
Cartwright, N., & Munro, E. (2010). The limitations of randomized controlled trials in predicting effectiveness. Journal of Evaluation in Clinical Practice, 16, 260–266.
Chalmers, I. (2003). Trying to do more good than harm in policy and practice: the role of rigorous, transparent, up-to-date evaluations. The ANNALS of the American Academy of Political and Social Science, 589(22), 22–40.
Cochrane Collaboration. (2013). Retrieved 26 April 2013 from http://www.cochrane.org/.
Craig, P., Dieppe, P., Macintyre, S., Mitchie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ, 337, 979–983.
Davidson, K. W., Goldstein, M., Kaplan, R. M., Kaufmann, P. G., Knatterud, G. L., Orleans, C. T., et al. (2003). Evidence-based behavioural medicine: what is it and how do we achieve it? Annals of Behavioral Medicine, 26, 161–171.
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. American Journal of Public Health, 94, 361–366.
Evans, T., & Brown, H. (2003). Road traffic crashes: operationalizing equity in the context of health sector reform. Injury Control and Safety Promotion, 10(1–2), 11–12.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: a synthesis of the literature. Tampa: University of South Florida.
Fraser, M. W., Galinsky, M. J., Richman, J. M., & Day, S. H. (2009). Intervention Research: Developing Social Programs. Oxford: Oxford University Press.
Gill, C. E. (2011). Missing links: how descriptive validity impacts the policy relevance of randomized controlled trials in criminology. Journal of Experimental Criminology, 7(3), 201–224.
Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008). What is missing from descriptions of treatment in trials and reviews? BMJ, 336, 1472–1474.
Goldbeck, L., & Vitiello, B. (2011). Reporting clinical trials of psychosocial interventions in child and adolescent psychiatry and mental health. Child and Adolescent Psychiatry and Mental Health, 5(4).
Grant, S., Montgomery, P., & Mayo-Wilson, E. (2012). Development of a CONSORT extension for interventions in public health and related disciplines. The Lancet, 380(Supp. 3), p. S14. Retrieved 26 April 2013 from http://www.thelancet.com/abstracts/public-health-science.
Hardeman, W., Michie, S., Fanshawe, T., Prevost, A. T., McLoughlin, K., & Kinmonth, A. L. (2008). Fidelity of delivery of a physical activity intervention: predictors and consequences. Psychology & Health, 23, 11–24.
Hawe, P., Shiell, A., & Riley, T. (2004). Complex interventions: how "out of control" can a randomised controlled trial be? BMJ, 328(7455), 1561–1563.
Henggeler, S. W., Schoenwald, S. K., Rowland, M. D., & Cunningham, P. B. (2002). Serious emotional disturbances in children and adolescents: Multisystemic therapy. New York: Guilford Press.
Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., et al. (2008). CONSORT for reporting randomised trials in journal and conference abstracts. Lancet, 371, 281–283.
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., et al. (2012). Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 6, CD000259.
Ladd, B. O., McCrady, B. S., Manuel, J. K., & Campbell, W. (2010). Improving the quality of reporting alcohol outcome studies: effects of the CONSORT statement. Addictive Behaviors, 35, 660–666.
Littell, J. H., Campbell, M., Green, S., & Toews, B. (2009). Multisystemic Therapy for social, emotional, and behavioral problems in youth aged 10-17. Cochrane Library of Systematic Reviews, 4, CD004797.
Marshall, M., Lockwood, A., Bradley, C., Adams, C., Joy, C., & Fenton, M. (2000). Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. The British Journal of Psychiatry, 176, 249–252.
Mayo-Wilson, E. (2007). Reporting implementation in randomized trials: proposed additions to the Consolidated Standards of Reporting Trials statement. American Journal of Public Health, 97, 630–633.
McGrath, P. J., Stinson, J., & Davidson, K. (2003). Commentary: the Journal of Pediatric Psychology should adopt the CONSORT statement as a way of improving the evidence base in pediatric psychology. Journal of Pediatric Psychology, 28, 169–171.
Medical Research Council. (2008). A framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council.
Medical Research Council. (2011). Using natural experiments to evaluate population health interventions: guidance for producers and users of evidence. London: Medical Research Council.
Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009). Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implementation Science, 4, 40.
Michie, S., Abraham, C., Eccles, M. P., Francis, J. J., Hardeman, W., & Johnston, M. (2011). Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implementation Science, 6, 10.
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., et al. (1999). Does quality of reports of randomized trials affect estimates of inter- vention efficacy reported in meta-analyses? Lancet, 352, 609–613.
Moher, D., Altman, D. G., Schulz, K. F., & Elbourne, D. R. (2004). Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond. Canadian Medical Association Journal, 171(4), 349–350.
Moher, D., Hopewell, S., Schultz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010a). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ, 340, c869.
Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010b). Guidance for developers of health research reporting guidelines. PLoS Medicine, 7(2), e1000217.
Moore, L. (2002). Research design for the rigorous evaluation of complex educational interventions: lessons from health services research. Building Research Capacity, 1, 4–5.
Moore, L., & Moore, G. F. (2011). Public health evaluation: which designs work, for whom and under what circumstances? Journal of Epidemiology and Community Health, 65, 596–597.
Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., Sanders, C. F. B., Askham, J., et al. (1998). Consensus development methods, and their use in clinical guideline development. Health Technology Assessment, 2(3).
Naleppa, M. J., & Cagle, J. G. (2010). Treatment fidelity in social work intervention research: a review of published studies. Research on Social Work Practice, 20, 674–681.
Newman, M., & Elbourne, D. (2004). Improving the usability of educational research: guidelines for the REPOrting of primary empirical research Studies in Education (The REPOSE Guidelines). Evaluation & Research in Education, 18(4), 201–212.
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist Synthesis: An Introduction. (University of Manchester: ESRC Research Methods Programme)
Perry, A. E., & Johnson, M. (2008). Applying the Consolidated Standards of Reporting Trials (CONSORT) to studies of mental health provision for juvenile offenders: a research note. Journal of Experimental Criminology, 4, 165–185.
Perry, A. E., Weisburd, D., & Hewitt, C. (2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6, 245–262.
Pildal, J., Hróbjartsson, A., Jørgensen, K. J., Hilden, J., Altman, D. G., & Gøtzsche, P. C. (2007). Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. International Journal of Epidemiology, 36(4), 847–857.
Prescott, R. J., Counsell, C. E., Gillespie, W. J., Grant, A. M., Russell, I. T., Kiauka, S., et al. (1999). Factors that limit the quality, number and progress of randomised controlled trials. Health Technology Assessment, 3(20).
Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995). Allocation concealment in randomised trials: defending against deciphering. Lancet, 359, 614–617.
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group. (2010). CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ, 340, 698–702.
Semaan, S., Kay, L., Strouse, D., Sogolow, E., Mullen, P. D., Neumann, M. S., et al. (2002). A profile of U.S.-based trials of behavioral and social interventions for HIV risk reduction. Journal of Acquired Immune Deficiency Syndromes, 30, S30–S50.
Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., et al. (2009). Can we systematically review studies that evaluate complex interventions? PLoS Medicine, 6(8), 31000086.
Stinson, J. N., McGrath, P. J., & Yamada, J. T. (2003). Clinical trials in the Journal of Pediatric Psychology: applying the CONSORT statement. Journal of Pediatric Psychology, 28, 159–167.
Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357.
Torgerson, C. J., Torgerson, D. J., Birks, Y. F., & Porthouse, J. (2005). A comparison of RCTs in health and education. British Educational Research Journal, 31(6), 761–785.
Tugwell, P., Petticrew, M., Kristjansson, E., Welch, V., Ueffing, E., et al. (2010). Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ, 341, c4739.
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., et al. (2012). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database of Systematic Reviews, 11, MR000030.
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Annals of Internal Medicine, 147(8), 573–577.
Wang, S., Moss, J. R., & Hiller, J. E. (2006). Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1), 76–83.
Welch, V., Petticrew, M., Tugwell, P., Moher, D., O'Neill, J., Waters, E., et al. (2012). PRISMA-Equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Medicine, 9(10), e1001333.
Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., et al. (2008). Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ, 337, a2390.
Acknowledgments
The CONSORT-SPI (Social and Psychological Interventions) International Advisory Group includes: J. Lawrence Aber, Distinguished Professor of Applied Psychology and Public Policy, Steinhardt School of Culture, Education, and Human Development, New York University; Chris Bonell, Professor of Sociology and Social Intervention, Centre for Evidence Based Intervention, University of Oxford; David M. Clark, Chair of Psychology, Department of Experimental Psychology, University of Oxford; Frances Gardner, Professor of Child and Family Psychology, Centre for Evidence Based Intervention, University of Oxford; Steven Hollon, American Psychological Association Guidelines Committee (Chair), Gertrude Conaway Professor of Psychology, Department of Psychology, Vanderbilt University; Jim McCambridge, Senior Lecturer in Behaviour Change, Department of Social and Environmental Health Research, London School of Hygiene and Tropical Medicine; Susan Michie, Professor of Health Psychology, Department of Clinical, Educational & Health Psychology, University College London; Laurence Moore, Professor of Public Health Improvement, Cardiff School of Social Sciences, Cardiff University; Mark Petticrew, Professor of Public Health Evaluation, Department Social and Environmental Health Research, London School of Hygiene and Tropical Medicine; Lawrence Sherman, Wolfson Professor of Criminology, Cambridge Institute of Criminology, Cambridge University; Steve Pilling, Director, Centre for Outcomes Research and Effectiveness, University College London; James Thomas, Associate Director EPPI-Centre, Reader in Social Policy, Institute of Education, University of London; Elizabeth Waters, Jack Brockhoff Chair of Child Public Health, McCaughey VicHealth Centre for Community Wellbeing, Melbourne School of Population & Global Health, University of Melbourne, Australia; David Weisburd, Director and Walter E. Meyer Professor of Law and Criminal Justice, Institute of Criminology, Hebrew University Faculty of Law, Jerusalem; Joanne Yaffe, Associate Professor, College of Social Work, University of Utah. This project is funded by the UK Economic and Social Research Council (ES/K00087X/1). We thank the Centre for Evidence-Based Intervention (Oxford University), the Centre for Outcomes Research and Effectiveness (University College London), and the National Collaborating Centre for Mental Health (NCCMH) for their support. S.G. is supported by a linked Clarendon Fund-Green Templeton College Annual Fund Scholarship to support his doctoral studies and research. D.M. is supported by a University Research Chair.
Conflict of interest
None.
Author statement
PM, EMW, and SG conceived of the idea for the project. All authors helped to draft the manuscript, and all have read and approved the final manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Grant, S., Mayo-Wilson, E., Hopewell, S. et al. Developing a reporting guideline for social and psychological intervention trials. J Exp Criminol 9, 355–367 (2013). https://doi.org/10.1007/s11292-013-9180-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11292-013-9180-5