Skip to main content
Erschienen in: Journal of General Internal Medicine 1/2014

01.01.2014 | Review Article

Crowdsourcing—Harnessing the Masses to Advance Health and Medicine, a Systematic Review

verfasst von: Benjamin L. Ranard, AB, Yoonhee P. Ha, MSc, Zachary F. Meisel, MD, MPH, MS, David A. Asch, MD, MBA, Shawndra S. Hill, PhD, Lance B. Becker, MD, Anne K. Seymour, MS, Raina M. Merchant, MD, MSHP

Erschienen in: Journal of General Internal Medicine | Ausgabe 1/2014

Einloggen, um Zugang zu erhalten

Abstract

Objective

Crowdsourcing research allows investigators to engage thousands of people to provide either data or data analysis. However, prior work has not documented the use of crowdsourcing in health and medical research. We sought to systematically review the literature to describe the scope of crowdsourcing in health research and to create a taxonomy to characterize past uses of this methodology for health and medical research.

Data sources

PubMed, Embase, and CINAHL through March 2013.

Study eligibility criteria

Primary peer-reviewed literature that used crowdsourcing for health research.

Study appraisal and synthesis methods

Two authors independently screened studies and abstracted data, including demographics of the crowd engaged and approaches to crowdsourcing.

Results

Twenty-one health-related studies utilizing crowdsourcing met eligibility criteria. Four distinct types of crowdsourcing tasks were identified: problem solving, data processing, surveillance/monitoring, and surveying. These studies collectively engaged a crowd of >136,395 people, yet few studies reported demographics of the crowd. Only one (5 %) reported age, sex, and race statistics, and seven (33 %) reported at least one of these descriptors. Most reports included data on crowdsourcing logistics such as the length of crowdsourcing (n = 18, 86 %) and time to complete crowdsourcing task (n = 15, 71 %). All articles (n = 21, 100 %) reported employing some method for validating or improving the quality of data reported from the crowd.

Limitations

Gray literature not searched and only a sample of online survey articles included.

Conclusions and implications of key findings

Utilizing crowdsourcing can improve the quality, cost, and speed of a research project while engaging large segments of the public and creating novel science. Standardized guidelines are needed on crowdsourcing metrics that should be collected and reported to provide clarity and comparability in methods.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Sobel D. Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time. New York: Walker Publishing Company; 1995. Sobel D. Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time. New York: Walker Publishing Company; 1995.
4.
Zurück zum Zitat Lintott C, Schawinski K, Bamford S, Slosar A, Land K, Thomas D, et al. Galaxy Zoo 1: data release of morphological classifications for nearly 900 000 galaxies. Monthly Notices of the Royal Astronomical Society. 2011;410(1):166–78. doi:10.1111/j.1365-2966.2010.17432.x.CrossRef Lintott C, Schawinski K, Bamford S, Slosar A, Land K, Thomas D, et al. Galaxy Zoo 1: data release of morphological classifications for nearly 900 000 galaxies. Monthly Notices of the Royal Astronomical Society. 2011;410(1):166–78. doi:10.​1111/​j.​1365-2966.​2010.​17432.​x.CrossRef
9.
Zurück zum Zitat Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June, 1946; signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948. [cited 2012 Jul 21]. Available from: http://www.who.int/about/definition/en/print.html Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June, 1946; signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948. [cited 2012 Jul 21]. Available from: http://​www.​who.​int/​about/​definition/​en/​print.​html
10.
Zurück zum Zitat Reips U-D, Birnbaum MH. Behavioral research and data collection via the Internet. In: Proctor RW, Vu K-PL, eds. Handbook of human factors in Web design. 2nd ed. Mahwah, New Jersey: Lawrence Erlbaum Associates; 2011:563–85.CrossRef Reips U-D, Birnbaum MH. Behavioral research and data collection via the Internet. In: Proctor RW, Vu K-PL, eds. Handbook of human factors in Web design. 2nd ed. Mahwah, New Jersey: Lawrence Erlbaum Associates; 2011:563–85.CrossRef
11.
Zurück zum Zitat Higgins JPT, Deeks JJ (editors). Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [Internet]. The Cochrane Collaboration; 2011 [updated 2011 Mar; cited 2012 Aug 1]. Available from www.cochrane-handbook.org Higgins JPT, Deeks JJ (editors). Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [Internet]. The Cochrane Collaboration; 2011 [updated 2011 Mar; cited 2012 Aug 1]. Available from www.​cochrane-handbook.​org
12.
Zurück zum Zitat Cooper S, Treuille A, Barbero J, Leaver-Fay A, Tuite K, Khatib F, et al. The challenge of designing scientific discovery games. California: Proceedings of the Fifth International Conference on the Foundations of Digital Games; Monterey; 2010:40–7. 1822354: ACM. Cooper S, Treuille A, Barbero J, Leaver-Fay A, Tuite K, Khatib F, et al. The challenge of designing scientific discovery games. California: Proceedings of the Fifth International Conference on the Foundations of Digital Games; Monterey; 2010:40–7. 1822354: ACM.
13.
16.
Zurück zum Zitat Bender J, O'Grady L, Desphande A, Cortinois A, Saffie L, Husereau D, et al. Collaborative authoring: a case study of the use of a wiki as a tool to keep systematic reviews up to date. 2011. Bender J, O'Grady L, Desphande A, Cortinois A, Saffie L, Husereau D, et al. Collaborative authoring: a case study of the use of a wiki as a tool to keep systematic reviews up to date. 2011.
17.
Zurück zum Zitat Cooper S, Khatib F, Makedon I, Lu H, Barbero J, Baker D, et al. Analysis of social gameplay macros in the Foldit cookbook. France: Proceedings of the 6th International Conference on Foundations of Digital Games; Bordeaux; 2011:9–14. 2159367: ACM. Cooper S, Khatib F, Makedon I, Lu H, Barbero J, Baker D, et al. Analysis of social gameplay macros in the Foldit cookbook. France: Proceedings of the 6th International Conference on Foundations of Digital Games; Bordeaux; 2011:9–14. 2159367: ACM.
18.
Zurück zum Zitat Khatib F, DiMaio F, Foldit Contenders G, Foldit Void Crushers G, Cooper S, Kazmierczyk M, et al. Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nature structural & molecular biology. 2011;18(10):1175–7. doi:10.1038/nsmb.2119. PubMed PMID: 21926992.CrossRef Khatib F, DiMaio F, Foldit Contenders G, Foldit Void Crushers G, Cooper S, Kazmierczyk M, et al. Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nature structural & molecular biology. 2011;18(10):1175–7. doi:10.​1038/​nsmb.​2119. PubMed PMID: 21926992.CrossRef
19.
Zurück zum Zitat Khatib F, Cooper S, Tyka MD, Xu K, Makedon I, Popovic Z, et al. Algorithm discovery by protein folding game players. Proceedings of the National Academy of Sciences of the United States of America. 2011;108(47):18949–53. doi:10.1073/pnas.1115898108. PubMed PMID: 22065763; PubMed Central PMCID: PMC3223433.PubMedCentralPubMedCrossRef Khatib F, Cooper S, Tyka MD, Xu K, Makedon I, Popovic Z, et al. Algorithm discovery by protein folding game players. Proceedings of the National Academy of Sciences of the United States of America. 2011;108(47):18949–53. doi:10.​1073/​pnas.​1115898108. PubMed PMID: 22065763; PubMed Central PMCID: PMC3223433.PubMedCentralPubMedCrossRef
20.
Zurück zum Zitat Chunara R, Chhaya V, Bane S, Mekaru SR, Chan EH, Freifeld CC, et al. Online reporting for malaria surveillance using micro-monetary incentives, in urban India 2010–2011. Malaria journal. 2012;11:43. doi:10.1186/1475-2875-11-43. PubMed PMID: 22330227; PubMed Central PMCID: PMC3305483.PubMedCentralPubMedCrossRef Chunara R, Chhaya V, Bane S, Mekaru SR, Chan EH, Freifeld CC, et al. Online reporting for malaria surveillance using micro-monetary incentives, in urban India 2010–2011. Malaria journal. 2012;11:43. doi:10.​1186/​1475-2875-11-43. PubMed PMID: 22330227; PubMed Central PMCID: PMC3305483.PubMedCentralPubMedCrossRef
21.
24.
Zurück zum Zitat Luengo-Oroz MA, Arranz A, Frean J. Crowdsourcing malaria parasite quantification: an online game for analyzing images of infected thick blood smears. Journal of medical Internet research. 2012;14(6):e167. doi:10.2196/jmir.2338. PubMed PMID: 23196001; PubMed Central PMCID: PMC3510720.PubMedCentralPubMedCrossRef Luengo-Oroz MA, Arranz A, Frean J. Crowdsourcing malaria parasite quantification: an online game for analyzing images of infected thick blood smears. Journal of medical Internet research. 2012;14(6):e167. doi:10.​2196/​jmir.​2338. PubMed PMID: 23196001; PubMed Central PMCID: PMC3510720.PubMedCentralPubMedCrossRef
26.
Zurück zum Zitat Mavandadi S, Dimitrov S, Feng S, Yu F, Yu R, Sikora U, et al. Crowd-sourced BioGames: managing the big data problem for next-generation lab-on-a-chip platforms. Lab on a chip. 2012;12(20):4102–6. doi:10.1039/c2lc40614d. PubMed PMID: 22918378; PubMed Central PMCID: PMC3477593.PubMedCrossRef Mavandadi S, Dimitrov S, Feng S, Yu F, Yu R, Sikora U, et al. Crowd-sourced BioGames: managing the big data problem for next-generation lab-on-a-chip platforms. Lab on a chip. 2012;12(20):4102–6. doi:10.​1039/​c2lc40614d. PubMed PMID: 22918378; PubMed Central PMCID: PMC3477593.PubMedCrossRef
27.
Zurück zum Zitat McKenna MT, Wang S, Nguyen TB, Burns JE, Petrick N, Summers RM. Strategies for improved interpretation of computer-aided detections for CT colonography utilizing distributed human intelligence. Medical image analysis. 2012;16(6):1280–92. doi:10.1016/j.media.2012.04.007. PubMed PMID: 22705287; PubMed Central PMCID: PMC3443285.PubMedCentralPubMedCrossRef McKenna MT, Wang S, Nguyen TB, Burns JE, Petrick N, Summers RM. Strategies for improved interpretation of computer-aided detections for CT colonography utilizing distributed human intelligence. Medical image analysis. 2012;16(6):1280–92. doi:10.​1016/​j.​media.​2012.​04.​007. PubMed PMID: 22705287; PubMed Central PMCID: PMC3443285.PubMedCentralPubMedCrossRef
28.
Zurück zum Zitat Nguyen TB, Wang S, Anugu V, Rose N, McKenna M, Petrick N, et al. Distributed Human Intelligence for Colonic Polyp Classification in Computer-aided Detection for CT Colonography. Radiology. 2012;262(3):824–33. doi:10.1148/radiol.11110938. PubMed PMID: 22274839; PubMed Central PMCID: PMC3285221.PubMedCrossRef Nguyen TB, Wang S, Anugu V, Rose N, McKenna M, Petrick N, et al. Distributed Human Intelligence for Colonic Polyp Classification in Computer-aided Detection for CT Colonography. Radiology. 2012;262(3):824–33. doi:10.​1148/​radiol.​11110938. PubMed PMID: 22274839; PubMed Central PMCID: PMC3285221.PubMedCrossRef
32.
Zurück zum Zitat Merchant RM, Asch DA, Hershey JC, Griffis HM, Hill S, Saynisch O, et al. A crowdsourcing innovation challenge to locate and map automated external defibrillators. Circulation Cardiovascular quality and outcomes. 2013;6(2):229–36. doi:10.1161/CIRCOUTCOMES.113.000140. PubMed PMID: 23481522.PubMedCrossRef Merchant RM, Asch DA, Hershey JC, Griffis HM, Hill S, Saynisch O, et al. A crowdsourcing innovation challenge to locate and map automated external defibrillators. Circulation Cardiovascular quality and outcomes. 2013;6(2):229–36. doi:10.​1161/​CIRCOUTCOMES.​113.​000140. PubMed PMID: 23481522.PubMedCrossRef
34.
Zurück zum Zitat Hussein A. Brokering knowledge in biosciences with InnoCentive. Interview by Semahat S. Demir. IEEE engineering in medicine and biology magazine: the quarterly magazine of the Engineering in Medicine & Biology Society. 2003;22(4):26–7. PubMed PMID: 14515687.CrossRef Hussein A. Brokering knowledge in biosciences with InnoCentive. Interview by Semahat S. Demir. IEEE engineering in medicine and biology magazine: the quarterly magazine of the Engineering in Medicine & Biology Society. 2003;22(4):26–7. PubMed PMID: 14515687.CrossRef
37.
Zurück zum Zitat Signorini A, Segre AM, Polgreen PM. The use of Twitter to track levels of disease activity and public concern in the US during the influenza A H1N1 pandemic. PLoS One. 2011;6(5):e19467.PubMedCentralPubMedCrossRef Signorini A, Segre AM, Polgreen PM. The use of Twitter to track levels of disease activity and public concern in the US during the influenza A H1N1 pandemic. PLoS One. 2011;6(5):e19467.PubMedCentralPubMedCrossRef
38.
Metadaten
Titel
Crowdsourcing—Harnessing the Masses to Advance Health and Medicine, a Systematic Review
verfasst von
Benjamin L. Ranard, AB
Yoonhee P. Ha, MSc
Zachary F. Meisel, MD, MPH, MS
David A. Asch, MD, MBA
Shawndra S. Hill, PhD
Lance B. Becker, MD
Anne K. Seymour, MS
Raina M. Merchant, MD, MSHP
Publikationsdatum
01.01.2014
Verlag
Springer US
Erschienen in
Journal of General Internal Medicine / Ausgabe 1/2014
Print ISSN: 0884-8734
Elektronische ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-013-2536-8

Weitere Artikel der Ausgabe 1/2014

Journal of General Internal Medicine 1/2014 Zur Ausgabe

Healing Arts

Visiting

OriginalPaper

A Leathery Lining

Leitlinien kompakt für die Innere Medizin

Mit medbee Pocketcards sicher entscheiden.

Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag

„Überwältigende“ Evidenz für Tripeltherapie beim metastasierten Prostata-Ca.

22.05.2024 Prostatakarzinom Nachrichten

Patienten mit metastasiertem hormonsensitivem Prostatakarzinom sollten nicht mehr mit einer alleinigen Androgendeprivationstherapie (ADT) behandelt werden, mahnt ein US-Team nach Sichtung der aktuellen Datenlage. Mit einer Tripeltherapie haben die Betroffenen offenbar die besten Überlebenschancen.

So sicher sind Tattoos: Neue Daten zur Risikobewertung

22.05.2024 Melanom Nachrichten

Das größte medizinische Problem bei Tattoos bleiben allergische Reaktionen. Melanome werden dadurch offensichtlich nicht gefördert, die Farbpigmente könnten aber andere Tumoren begünstigen.

CAR-M-Zellen: Warten auf das große Fressen

22.05.2024 Onkologische Immuntherapie Nachrichten

Auch myeloide Immunzellen lassen sich mit chimären Antigenrezeptoren gegen Tumoren ausstatten. Solche CAR-Fresszell-Therapien werden jetzt für solide Tumoren entwickelt. Künftig soll dieser Prozess nicht mehr ex vivo, sondern per mRNA im Körper der Betroffenen erfolgen.

Frühzeitige HbA1c-Kontrolle macht sich lebenslang bemerkbar

22.05.2024 Typ-2-Diabetes Nachrichten

Menschen mit Typ-2-Diabetes von Anfang an intensiv BZ-senkend zu behandeln, wirkt sich positiv auf Komplikationen und Mortalität aus – und das offenbar lebenslang, wie eine weitere Nachfolgeuntersuchung der UKPD-Studie nahelegt.

Update Innere Medizin

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.