Abstract
The aim of this study is to measure differences between three relevant altmetric providers: Altmetric.com, PlumX and Crossref Event Data (CED). More than 67,000 research papers, initially extracted from PlumX, were searched in Altmetric.com and CED to compare their counts. Differences between services were analyzed regarding the number of documents with an altmetric event and the counting differences in each metric. Results show that Altmetric.com is the provider with the best coverage of blog posts, news and tweets; while PlumX better collects Mendeley readers; and CED is the site that extracts more Wikipedia citations. The study concludes that there are important counting differences due to technical errors and extracting criteria. The article recommends the use of specific services for the analysis of particular metrics. While, it should be mandatory to employ the combination of several providers, if we want to perform an overall analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1), 11–17. https://doi.org/10.1087/20130103.
Altmetric Support (2017). How does Altmetric track mentions on Wikipedia? https://help.altmetric.com/support/solutions/articles/6000060980-how-does-altmetric-track-mentions-on-wikipedia (Visited: May 23, 2018).
Altmetric.com (2018). How it works. https://www.altmetric.com/about-our-data/how-it-works/.
Baessa, M., Lery, T., Grenz, D., & Vijayakumar, J. K. (2015). Connecting the pieces: Using ORCIDs to improve research impact and repositories. F1000Research, 4, 195. https://doi.org/10.12688/f1000research.6502.1.
Bornmann, L. (2014). Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics, 8(4), 935–950. https://doi.org/10.1016/j.joi.2014.09.007.
CED (2017). Twitter—event data user guide https://www.eventdata.crossref.org/guide/sources/twitter/ (Visited: May 23, 2018).
Champieux, R. (2015). PlumX. Journal of the Medical Library Association, 103(1), 63. https://doi.org/10.3163/1536-5050.103.1.019.
Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019. https://doi.org/10.1002/asi.23309.
Elsevier. (2017). Elsevier Acquires Leading ‘Altmetrics’ Provider Plum Analytics. https://www.elsevier.com/about/press-releases/corporate/elsevier-acquires-leading-altmetrics-provider-plum-analytics (Visited: May 23, 2018).
Erdt, M., Nagarajan, A., Sin, S. C. J., & Theng, Y. L. (2016). Altmetrics: An analysis of the state-of-the-art in measuring research impact on social media. Scientometrics, 109(2), 1117–1166. https://doi.org/10.1007/s11192-016-2077-0.
Fraumann, G., Zahedi, Z. & Costas, R. (2015). What do we know about Altmetric.com sources? A study of the top 200 blogs and news sites mentioning scholarly output. In Altmetrics workshop, 9 October 2015, Amsterdam Science Park, Amsterdam.
Gumpenberger, C., Glänzel, W., & Gorraiz, J. (2016). The ecstasy and the agony of the altmetric score. Scientometrics, 108(2), 977–982. https://doi.org/10.1007/s11192-016-1991-5.
Holmberg, K. J. (2015). Altmetrics for information professionals: Past, present and future. Chandos Publishing. ISBN: 9780081002773.
ImpactStory (2017). New widget and API. ImpactStory blog. http://blog.impactstory.org/new-widget-and-api/ (Visited: May 23, 2018).
Jobmann, A., Hoffmann, C.P., Künne, S., Peters, I., Schmitz, J. & Wollnik-Korn, G. (2014). Altmetrics for large, multidisciplinary research groups: Comparison of current tools. Bibliometrie – Praxis und Forschung, 3, http://www.bibliometrie-pf.de/article/viewFile/205/258.
King, G. (2015). Altmetric and Springer Launch Bookmetrix: A New Platform for Book Impact. Digital Science News Blog. https://www.digital-science.com/blog/news/altmetric-and-springer-launch-bookmetrix-a-new-platform-for-book-impact/ (Visited: May 23, 2018).
Kraker, P., Lex, E., Gorraiz, J., Gumpenberger, C. & Peters, I. (2015). Research data explored II: The anatomy and reception of figshare. arXiv:1503.01298.
Lindsay, J. M. (2016). PlumX from plum analytics: Not just altmetrics. Journal of Electronic Resources in Medical Libraries, 13(1), 8–17. https://doi.org/10.1080/15424065.2016.1142836.
Liu, J. (2013) New Feature: News Tracker. Altmetric: https://www.altmetric.com/blog/new-feature-news-tracker/ (Visited: May 23, 2018).
Meschede, C., & Siebenlist, T. (2018). Cross-metric compatability and inconsistencies of altmetrics. Scientometrics, 115(1), 283–297. https://doi.org/10.1007/s11192-018-2674-1.
Ortega, J. L. (2016). To be or not to be on Twitter, and its relationship with the tweeting and citation of research papers. Scientometrics, 109(2), 1353–1364. https://doi.org/10.1007/s11192-016-2113-0.
Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2015). Research data explored: Citations versus altmetrics. In A. A. Salah, Y. Tonta, A. A. Akdag Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of the 15th international conference on scientometrics and informetrics, Istanbul, Turkey (pp. 172–183).
Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2016). Research data explored: An extended analysis of citations and altmetrics. Scientometrics, 107(2), 723–744. https://doi.org/10.1007/s11192-016-1887-4.
Plum Analytics (2016). Wikipedia & altmetrics: Calculating mention metrics. https://plumanalytics.com/wikipedia-altmetrics-calculating-mention-metrics/.
Plum Analytics (2018). Coverage: Expanding the world of altmetrics. khttps://plumanalytics.com/learn/about-metrics/coverage/.
Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Altmetrics: A manifesto. http://altmetrics.org/manifesto/.
Robinson-García, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). Nuevos datos, nuevas posibilidades: Revelando el interior de Altmetric.com. El Profesional de la Information, 23(4), 359–366. https://doi.org/10.3145/epi.2014.jul.03.
Roemer, R. C. & Borchardt, R. (2015). Altmetrics. American Library Association, ISBN: 0838959652.
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062. https://doi.org/10.1002/asi.23833.
Tattersall, A. (Ed.). (2016). Altmetrics: A practical guide for librarians, researchers and academics. Facet Publishing. ISBN: 178330010.
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841. https://doi.org/10.1371/journal.pone.0064841.
Torres-Salinas, D., Gumpenberger, C., & Gorraiz, J. (2017). PlumX as a potential tool to assess the macroscopic multidimensional impact of books. Frontiers in Research Metrics and Analytics. https://doi.org/10.3389/frma.2017.00005.
Trueger, N. S., Thoma, B., Hsu, C. H., Sullivan, D., Peters, L., & Lin, M. (2015). The altmetric score: A new measure for article-level dissemination and impact. Annals of Emergency Medicine, 66(5), 549–553. https://doi.org/10.1016/j.annemergmed.2015.04.022.
Williams, C. (2015) Altmetric to provide expanded mainstream media tracking. Altmetric: https://www.altmetric.com/blog/moreover/ (Visited: May 23, 2018).
Williams, C. (2017) Help Altmetric further expand its coverage: Tell us about your favourite blogs. Altmetric: https://www.altmetric.com/blog/expandblogs/ (Visited: May 23, 2018).
Zahedi, Z., & Costas, R. (2018). General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators. PLoS ONE, 13(5), e0197326.
Zahedi, Z., Fenner, M. & Costas, R. (2015). Consistency among altmetrics data provider/aggregators: What are the challenges? In The 2015 altmetrics workshop, Amsterdam.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ortega, J.L. Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data. Scientometrics 116, 2123–2138 (2018). https://doi.org/10.1007/s11192-018-2838-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2838-z