A critical review of SCImago Journal & Country Rank

Author / Creator

Mañana-Rodríguez, Jorge

Part of

Research evaluation, 2015-10, Vol.24 (4), p.343-354

DOI

10.1093/reseval/rvu008

Summary

• Since the use for evaluation purposes of the SCImago Journal & Country Rank (SJR) is increasingly being considered, it is pertinent to carry out a qualitative and quantitative analysis from a theoretical and practical perspective. The aim of this article is the analysis of the advantages and limitations of SJR. Some weaknesses from the perspective of its use for research evaluation purposes are identified and detailed, such as the absence of construct definition, lack of coherence between statements of its developers and observable features in data, gaps in the coverage of journals and its consequences for comparative purposes, issues related to the comparability of different citation networks, lack of a trustworthy ordinal position of journals in the output ranking, the use of retrospective backups in the underlying data and its consequences for longitudinal data stability, methodological issues in quartile construction, comparatively low discriminative capacity of the indicator, issues related to parameter fixing procedures, degree of transparency and reproducibility of the results, and the existence of errors in the assignment of documents in Scopus to countries and its consequences for the Country Ranks. It is concluded that, although the idea behind SJR is laudable and the indicator is a sophisticated alternative to other quality indicators, SJR omits a large amount of information, putting into question its transparency, reliability, and suitability for evaluative purposes in its current form, although most of the identified problems can be solved and might be the object of future improvements.