Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland

Thumbnail Image





Journal Title

Journal ISSN

Volume Title


Title alternative


This article discusses the Polish Journal Ranking, which is used in the research evaluation system in Poland. In 2015, the ranking, which represents all disciplines, allocated 17,437 journals into three lists: A, B, and C. The B list constitutes a ranking of Polish journals that are indexed neither in the Web of Science nor the European Reference Index for the Humanities. This ranking was built by evaluating journals in three dimensions: formal, bibliometric, and expert-based. We have analysed data on 2035 Polish journals from the B list. Our study aims to determine how an expert-based evaluation influenced the results of final evaluation. In our study, we used structural equation modelling, which is regression based, and we designed three pairs of theoretical models for three fields of science: (1) humanities, (2) social sciences, and (3) engineering, natural sciences, and medical sciences. Each pair consisted of the full model and the reduced model (i.e., the model without the expert-based evaluation). Our analysis revealed that the multidimensional evaluation of local journals should not rely only on the bibliometric indicators, which are based on the Web of Science or Scopus. Moreover, we have shown that the expert-based evaluation plays a major role in all fields of science. We conclude with recommendations that the formal evaluation should be reduced to verifiable parameters and that the expert-based evaluation should be based on common guidelines for the experts.



This work was supported by the National Programme for the Development of Humanities in Poland [Grant Number 0057/NPHR3/H11/82/2014].


journal ranking, multidimensional evaluation, structural equation modelling, expert-based evaluation, citations, Poland


Kulczycki, E. & Rozkosz, E.A. Scientometrics (2017). doi:10.1007/s11192-017-2261-x


Title Alternative

Rights Creative Commons

Creative Commons License

Uniwersytet im. Adama Mickiewicza w Poznaniu
Biblioteka Uniwersytetu im. Adama Mickiewicza w Poznaniu
Ministerstwo Nauki i Szkolnictwa Wyższego