Please use this identifier to cite or link to this item:
Title: Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland
Authors: Kulczycki, Emanuel
Rozkosz, Ewa A.
Keywords: journal ranking
multidimensional evaluation
structural equation modelling
expert-based evaluation
Issue Date: 2017
Citation: Kulczycki, E. & Rozkosz, E.A. Scientometrics (2017). doi:10.1007/s11192-017-2261-x
Abstract: This article discusses the Polish Journal Ranking, which is used in the research evaluation system in Poland. In 2015, the ranking, which represents all disciplines, allocated 17,437 journals into three lists: A, B, and C. The B list constitutes a ranking of Polish journals that are indexed neither in the Web of Science nor the European Reference Index for the Humanities. This ranking was built by evaluating journals in three dimensions: formal, bibliometric, and expert-based. We have analysed data on 2035 Polish journals from the B list. Our study aims to determine how an expert-based evaluation influenced the results of final evaluation. In our study, we used structural equation modelling, which is regression based, and we designed three pairs of theoretical models for three fields of science: (1) humanities, (2) social sciences, and (3) engineering, natural sciences, and medical sciences. Each pair consisted of the full model and the reduced model (i.e., the model without the expert-based evaluation). Our analysis revealed that the multidimensional evaluation of local journals should not rely only on the bibliometric indicators, which are based on the Web of Science or Scopus. Moreover, we have shown that the expert-based evaluation plays a major role in all fields of science. We conclude with recommendations that the formal evaluation should be reduced to verifiable parameters and that the expert-based evaluation should be based on common guidelines for the experts.
Sponsorship: This work was supported by the National Programme for the Development of Humanities in Poland [Grant Number 0057/NPHR3/H11/82/2014].
DOI: 10.1007/s11192-017-2261-x
Appears in Collections:Artykuły naukowe (WNS)

Files in This Item:
File Description SizeFormat 
art%3A10.1007%2Fs11192-017-2261-x.pdf1.78 MBAdobe PDFView/Open
Show full item record

This item is licensed under a Creative Commons License Creative Commons