Int. J. Morphol., 33(3):1183-1188, 2015.

 

Comparison between Impact Factor, Eigenfactor Score, and SCImago Journal Rank Indicator in Anatomy and Morphology Journals

 

Comparación entre Factor de Impacto, Eigenfactor Score e Indicador SCImago Journal Rank en Revistas de Anatomía y Morfología

 

Mario Cantín*,**; M. Muñoz***,**** & Ignacio Roa**,*****,******

* CIMA Research Group, Faculty of Dentistry, Universidad de La Frontera, Temuco, Chile.

** Doctoral Program in Morphological Sciences, CEMyQ, Faculty of Medicine, Universidad de La Frontera, Temuco, Chile.

*** Fellow Researcher, Universidad Científica del Sur, Lima, Perú.

**** Center of Research in Biomedical Sciences, Universidad Autónoma de Chile, Temuco, Chile.

***** Morfology Unit, Basic Biomedical Sciences Departament, Faculty of Health Sciences, Universidad de Talca, Talca, Chile.

****** Becario CONICYT-PCHA/Doctorado Nacional/2015-21150235.

Correspondence to:

---

SUMMARY: The Impact Factor (IF) is considered the best quality indicator for evaluation of scientific journals but has been criticized on many accounts, and its limitations have already been described extensively. New bibliometrics indicators, accepted by the scientific community, has been considered to evaluate quality ranking for journals using more complex algorithms and other databases. The aim of this study was to evaluated three indices of journal scientific impact: (IF), Eigenfactor Score (ES), and SCImago Journal Rank indicator (SJR), of mainstream Anatomy and Morphology journals in 2014. Specific anatomical and morphological journals were selected from Anatomy & Morphology category of Web of Science. The 2014 IFs and ESs were obtained from Journal Citation Report® and the SJR from the SCImago Journal and country rank website. We listed the journals and retrieved information by matching their international standard serial number. All journals were compared regarding their 2014 IF, ES and SJR and correlations between indices were evaluated using Pearson correlation. Twenty Anatomy and Morphology journals were identified, all indexed in both databases. The highest IF was 17 and lower 0.318. According to Eigenfactor score, the first place in the ranking was 0.01843 and the lower 0.00044, and to JRS the first position to 1.795 and the last position to 0.228. None of the journals had the same ranking to compare different indicators. Comparison between the IF and EF as well as the SJR yielded negative correlation (r= -0.012 and r= -0.037, respectively). All the analyzed journals have the highest standard of quality since they are indexed in the two most prestigious databases, WoS and Scopus. IF is the main index used by researchers for ranking the anatomy and morphology journals, but several shortcomings should be taken into account when they are using this index alone. SJR and ES can be more accurate quality indexes in certain conditions. It is recommended considering all these indices when judging the quality of the anatomy and morphology journals.

KEY WORDS: Impact Factor; Anatomy and Morphology; Eigenfactor Score; SCImago Journal Rank indicator.

---

RESUMEN: El factor de impacto (FI) es considerado el mejor indicador de calidad para evaluar las revistas científicas, pero ha sido criticado en muchos de sus cálculos y sus limitaciones se han descrito ampliamente. Nuevos indicadores bibliométricos aceptados por la comunidad científica han sido considerados para evaluar la calidad de las revistas utilizando algoritmos más complejos y otras bases de datos. El objetivo de este estudio fue evaluar tres índices de impacto: FI, Eigenfactor Score (ES) y SCImago Journal Rank (SJR), en revistas de anatomía y morfología de corriente principal para el año 2014. Las revistas específicas fueron seleccionados de la categoría Anatomía y Morfología de Web of Science (WoS). Los FI y ES del 2014 se obtuvieron desde el Journal Citation Report® y el SJR desde el sitio web SCImago journal and country rank. Se construyó la lista de revistas y se recupero su información, haciendo coincidir su número de serie estándar internacional en las bases consultadas. Se compararon todas las revistas en cuanto a su FI, ES y SJR del 2014, y las correlaciones entre los índices fueron evaluadas utilizando la correlación de Pearson. Se identificaron 20 revistas de anatomía y morfología para el periódo, todas indexadas en ambas bases. El FI más alto fue 17 y el menor 0,318. Según el ES, el primer lugar en el ranking fue del score 0,01843 y el último 0,00044, y para el SJR la primera posición para un impacto de 1,795 y la última un impacto de 0,228. Ninguna revista tuvo la misma posición en el ranking al comparar los diferentes indicadores. La comparación entre el FI y ES, así como el SJR produjo una correlación negativa (r= -0,012 y r= -0,037, respectivamente). Todas las revistas analizadas tienen un alto nivel de calidad, ya que están indexadas en las dos bases más prestigiosas, WoS y Scopus. EL FI es el índice más utilizado por los investigadores para evaluar a las revistas de anatomía y morfología, pero sus deficiencias y limitaciones deben ser tomadas en consideración cuando se utilice por sí solo. EL ES y SJR pueden ser índices más precisos para evaluar la calidad en ciertas condiciones. Se recomienda la consideración de todos estos índices al juzgar la calidad de las revistas de anatomía y morfología.

PALABRAS CLAVE: Factor de Impacto; Anatomía y Morfología; Eigenfactor Score; SCImago Journal Rank indicator.

---

 

INTRODUCTION

The mainstream scientific journals must meet quality criteria and are measured through scientometric tools. The quality of a scientific contribution is primarily estimated from the impact that it has in science, inferred from the citations in scientific articles that a contribution receives (Falagas et al., 2008a). This evaluation of research is important for various professional societies, individual scientists, scholarly institutions, and funding organizations, a process that is no stranger to anatomy and morphology journals (Cantín et al., 2015).

The Impact Factor (IF), first conceived in 1955 by Eugene Garfield, is known as a citation rate measures and is the major criteria for quality of scientific journals (Garfield, 2006; Krell, 2012). Journal IFs, which are published annually in the Journal Citation Reports, are widely regarded (the most popular) as a quality ranking for journals and used extensively by leading journals in their advertising (Elsaie & Kammer, 2009). The IF is generally defined as the recorded number of citations within a certain year (for example, 2014) to the items published in the journal during the two preceding years (2012 and 2013), divided by the number of such items (this would be the equivalent of the average citation rate of an item during the first and second calendar year after the year of publication). Only citations between journals indexed in the Thomson Reuter's Web of Science (WoS) are used.

SCImago research laboratory in 2007 developed other scientometric indicator, the SCImago Journal Rank (SJR); a journal quality indicator that uses Scopus indexed journals for quality assessment, applying the PageRank algorithm on the Scopus database, more complex than IF, that considers citations in Scopus database in a 3 years period (Ramin & Sarraf Shirazi, 2012).

A new bibliometric indicator, called Eigenfactor score (ES) was designed. Rank the journals by a similar algorithm as Google's Page Rank does, using the WoS-indexed journals for quality assessment. This score reflects not only the number of citations but also the prestige of citation source (Kianifar et al., 2014).

In the present study, we sought to identify and evaluate the main characteristics and differences between the widely used journal indicators. The aim was to compare the three mentioned quality metrics in the mainstream Anatomy and Morphology journals, based in the journals indexed in Web of Science and JCR in 2014, and discuss what should be considered when deciding the publication and scientific reading in the morphology field.

MATERIAL AND METHOD

Specific anatomical and morphological journals were selected from Anatomy & Morphology category (Life Sciences Biomedicine research area) of Web of Science Core CollectionTM website.

The 2014 IFs and ESs were obtained from Journal Citation Report® (JCR) through WoS. Relevant information was extracted from their source databases including influence of self-citations, citable documents, citations to original and review articles and their influence on 2014 were assessed. The 2014 SJR indicator, provided by the SCImago Journal and country rank, and developed by the SCImago research group were retrieved from its official website.

We listed the journals with IFs and ESs and retrieved information regarding their ranking in the SJR indicator list by matching their international standard serial number (ISSN). We also listed the journals with the SJR indicators and found their ranking in the list of journal IFs.

Journal ranking according to IF, ES and SJR were compared. Correlations between indices were evaluated using Pearson correlation. Analyzes were performed using GraphPad Prism version 5.0 (GraphPad Software, Inc., San Diego, USA) for Mac OSX. No institutional review board approval was obtained as this is not a human research study.

RESULTS

Twenty journals were identified with Anatomy and Morphology as the specific scope. All were indexed in WoS and Scopus databases. The most cited Journal was Brain Structure & Function, following by Developmental Dynamics and Anatomical Record - Advances in integrative anatomy and evolutionary biology. In contrast, the lower citation were obtained by Folia Morphologica, Advances in Anatomy Embryology and Cell Biology and Anatomical Science International. All the relevant information about journal quality measures for 2014 IFs is summarized in Table I.

When will rank journals according to IF, Advances in Anatomy Embryology and Cell Biology showed the highest impact (IF 17) and the first position. The last position was obtained by the International Journal of Morphology (IF 0.318). According to Eigenfactor score the journal Developmental Dynamics obtains the first place in the ranking (ES 0.01843) while Advances in Anatomy and Cell Biology Embryology the last position (ES 0.00044). Finally, according to SJR Brain Structure & Function could get the first position (SJR 1.795) while the journal Folia Morphologica the last position (SJR 0.228). Detailed information for each journal is summarized in Table II.

None of the journals had the same ranking to compare different indicators. The obtained Spearman correlation coefficients between studied indices IF and ES was -0.012, IF and SJR -0.037 and ES and SJR -0.090 (Fig. 1).

Table I. Information of the Anatomy & Morphology journals (WoS Category)
including 2014 Impact Factor, and other important citation analysis information.

Table II. Rankings of the Anatomy & Morphology journals in 2014
according to Impact Factor, Eigenfactor score and SCImago rank.

Fig. 1. Scatter plots of Impact Factor, Eigenfactor and SCImago journal rank indicator
indices evaluated in 2014 in correlation to each other for Anatomy and Morphology journals.

DISCUSSION

All the analyzed journals have the highest standard of quality since they are indexed in the two most prestigious databases, WoS and Scopus. Although the journal IF has been widely regarded as the best instrument for the evaluation of the quality of scientific journals, it has not been spared from criticism (Oosthuizen & Fenton, 2014). In an Editorial of Clinical Anatomy in 2012, the Editor Stephen W. Carmichael says that IF is the main metric used by academics such as the readers of this journal, even if this is not always appropriate (McKerahan & Carmichael, 2012). Main points of consideration regarding methodological aspects in the calculation of this index include the lack of assessment of the quality of citations, the inclusion of self-citations, the poor comparability between different scientific fields, and the analysis of mainly English-language publications (Falagas et al., 2008a). In addition, disadvantages of using IF as the sole method of assessment in anatomy and morphology is that database is dominated by American publications (and many articles published in journals of other countries are not taken into account, although they are indexed in mainstream) and has an english language bias (and language knowledge affects the number of articles cited for a publication) (Elsaie & Kammer).

The availability of research material to scientists and researchers determine their pattern of citation; some references are not available for many scientists due to limitation access and the indiscriminate use of PubMed (Manterola et al., 2014), allow lacks integration with all the journals indexed in WoS or Scopus; or the SciELO Network, of particular interest to many developed and developing countries, actually monitored by Thomson Reuters (Cantín, 2014). Journals indexed in PubMed usually have high visibility (Ramin & Sarraf Shirazi), and among Anatomy and Morphology journals indexed in WoS, Advances in Anatomy Embryology and Cell Biology, Acta Zoologica and International Journal of Morphology are not being indexed in PubMed. It can be predicted that by indexing in PubMed, this journal can get more citations and improve its IF, especially the last journal mentioned.

Besides, IF is not statistically representative of individual journal articles and correlate poorly with actual citations of individual articles. Review articles have the higher probability of getting cited than original articles (Weale et al., 2004; Kianifar et al.) and inflate the impact factor of journals. In our study, the Journal Advances in Anatomy Embryology and Cell Biology only publishes 3 review articles (between 2012 and 2013), and they also were indexed as Book Chapter, because this journal has an ISSN number (0301-5556) and at the same time an ISBN number (Print: 978-3-642-33310-1, Online: 978-3-642-33311-8). Thus, this journal with only 3 citable items has 51 cites in 2014 (specifically of their two reviews published in 2012), obtaining a high IF and first location in IF ranking; but the low number of citable items despite ranking 20th by ES. By another hand, IF depends on dynamics (expansion or contraction) of the research field, and small research fields tend to lack journals with high impact, as in anatomy and morphology (Galdames, 2013), where his interest is primary to anatomists, or some clinicians, but not for extensive research fields or other basic sciences, which have large numbers of researchers and many journals where they can be cited the contributions of morphological sciences. However, it is very contradictory to know that is an entirely different group of the most downloaded and read articles, which tend to be clinical reports and technique articles that do not get cited as often (Rosenstiel, 2015).

In a similar study, focused in Pediatric Neurology journals, Kianifar et al. reported a low correlation between metrics (IF with SJR and ES), but their journal ranks were similar to each other with small differences, where the researchers should be cautious about factors that affect the ranking order. They concluded that pediatric researchers should consider other journal quality metrics in addition to IF for publication of their researchers; we agree with this conclusion, the same for the anatomy and morphology researches.

The main differences between the journal IF and the SJR indicator derive mainly from differences in the scientific databases used as the sources of citations, as well as from differences in the methodology of estimation of these indices. Regarding the differences in the scientific databases, on which the compared indices of evaluation of scientific journals are applied, Scopus includes a substantially larger collection of journals, originating from remarkably more countries and published in a greater variety of languages (Falagas et al., 2008b). SJR considers all type of articles in its denominator and is less influenced by non-citable items. It is suggested to use SJR indicator, in addition to IF for quality evaluation of journals which publish high amount of noncitable items and, particularly so for those published in non-English languages (Ramin & Sarraf Shirazi).

For calculation of SJR and ES, the same algorithm similar to Google page rank is used with the major advantage of incorporating the source of citations: citations by more prestigious journals would have more influence compared to other journals (Brown, 2010). Another major difference is the time window of ES and SJR calculations. ES uses five previous years, and SJR uses three previous years as the time window. Both SJR and ES are freely available which can make them more available than IF (Ramin & Sarraf Shirazi; Jamali et al., 2014). ES has eliminated a number of the IF deficiencies by omitting journal self-citation impact, lengthening the time interval of calculation from two to five years, reflecting the impact of the prestigious citations as well as considering the indirect citations impact (Jamali et al.). Furthermore, ES does not have any denominator and is sensitive to total number of citable items. In other words journals with low number of articles are likely to have lower ES (Kianifar et al.), as noted in the journal Advances in Anatomy Embryology and Cell Biology with the higher IF, but the smaller ES.

It is assumed that publication in a high impact journal will enhance the impact of an article, the called free ride hypothesis. If the high impact journals had contributed 'free' citations, independently of the article contents, the relative difference would have been expected to diminish as a function of increasing journal impact (Seglen, 1994). These data suggest that the journals do not offer any free ride. The citation rates of the articles determine the journal impact, but not vice-versa (Elsaie & Kammer).

CONCLUSION

Impact Factor is the main index used by researchers for ranking the anatomy and morphology journals, but several shortcomings should be taken into account when they are using this index alone. The citation rate of an article determines the journal impact, but not viceversa. SJR and ES can be more accurate quality indexes in certain conditions. It is recommended considering all these indices when judging quality of the anatomy and morphology journals.

REFERENCES

Brown, T. Journal quality metrics: options to consider other than impact factors. Am. J. Occup. Ther., 65(3):346-50, 2011.

Cantín, M.; Muñoz, M. & Roa, I. The h-index in Academic Morphology. Int. J. Morphol., 33(2):706-11, 2015.

Cantín, M. Scientific production and h-index of Chilean specialists in digestive surgery. Int. J. Med. Surg. Sci., 1(4):319-25, 2014.

Elsaie, M. L. & Kammer, J. Impactitis: the impact factor myth syndrome. Indian J. Dermatol., 54(1):83-5, 2009.

Falagas, M. E.; Kouranos, V. D.; Arencibia-Jorge, R. & Karageorgopoulos, D. E. Comparison of SCImago journal rank indicator with journal impact factor. FASEB J., 22(8):2623-8, 2008.

Falagas, M. E.; Pitsouni, E. I.; Malietzis, G. A. & Pappas, G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J., 22(2):338-42, 2008b.

Galdames, I. S. Evidence based morphology; Quo Vadis? Int. J. Morphol., 31(4):1524-5, 2013.

Garfield, E. The history and meaning of the journal impact factor. JAMA, 295(1):90-3, 2006.

Jamali, J.; Salehi-Marzijarani, M. & Ayatollahi, S. M. Factors Affecting Journal Quality Indicator in Scopus (SCImago Journal Rank) in Obstetrics and Gynecology Journals: a Longitudinal Study (1999-2013). Acta Inform. Med., 22(6):385-8, 2014.

Kianifar, H.; Sadeghi, R. & Zarifmahmoudi, L. Comparison Between Impact Factor, Eigenfactor Metrics, and SCimago Journal Rank Indicator of Pediatric Neurology Journals. Acta Inform. Med., 22(2):103-6, 2014.

Krell, F. T. The Journal Impact Factor as a performance indicator. Eur. Sci. Ed., 38(1):3-6, 2012.

Manterola, C.; Otzen, T. & Calo, L. Electronic resources for searching health scientific information. Database publication type. Int. J. Morphol., 32(4):1484-91, 2014.

McKerahan, T. L. & Carmichael, S. W. What is the impact factor, anyway? Clin. Anat., 25(3):283, 2012.

Oosthuizen, J. C. & Fenton, J. E. Alternatives to the impact factor. Surgeon, 12(5):239-43, 2014.

Ramin, S. & Sarraf Shirazi, A. Comparison between Impact factor, SCImago journal rank indicator and Eigenfactor score of nuclear medicine journals. Nucl. Med. Rev. Cent. East Eur., 15(2):132-6, 2012.

Rosenstiel, S. F. Journal metrics and other measures. J. Prosthet. Dent., 113(4):A7-8, 2015.

Seglen, P. O. Causal relationship between article citedness and journal impact. J. Am. Soc. Inform. Sci., 45:1-11, 1994.

Weale, A. R.; Bailey, M. & Lear, P. A. The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Med. Res. Methodol., 4:14, 2004.

---

Correspondence to:

Dr. Ignacio Roa Henríquez

Morfology Unit
Basic Biomedical Sciences Departament
Faculty of Health Sciences
Universidad de Talca
Av. Lircay s/n
CHILE

Email: iroa@utalca.cl

Recibido : 11-05-2015
Aceptado: 19-07-2015