Methodology for development of “in-house indicators” for scholarly journals in consolidation

Patrícia Zeni Marchiori, Full Professor (retired) of the Federal University of Paraná/SCSA/Department of Information Science and Management, Curitiba, PR, Brazil

Eduardo Michelotti Bettoni, Collaborator – Methodologies in Information Management Research Group, Curitiba, PR, Brazil

Andre Luiz Appel, PhD student in the Post-Graduate Program in Information Science/School of Communication of the Federal University of Rio de Janeiro (in cooperation with the IBICT), Rio de Janeiro, RJ, Brazil

A research team from the Universidade Federal do Paraná, Brazil, named Methodologies in Information Management, worked on the development of indicators, namely “in-house indicators”, to measure performance of scholarly journals in consolidation, i.e., those not yet indexed by highly recognized databases, throughout the use of publicly available data regarding such journals.

The results of the study, recently published by the Journal Transinformação (Vol. 30, No. 3) in the article named “Bringing out indicators for open scientific journals”, present a set of criteria to evaluate journals (listed on the literature or by indexing agencies) (SCIENTIFIC ELECTRONIC LIBRARY ONLINE, 2014), followed by a detailing of the selected indicators which were tested and validated in the context of the journal “AtoZ: novas práticas em informação e conhecimento”.

The research team explored open source software (ZEARIN, 2016; THE R FOUNDATION, 2017) to extract data, and to generate relationships between citations available on the journal’s articles (an APA/BibTeX database) and a metadata set structured according to OJS/OAI-PHM. Another sort of open source software was used to perform consistency tests, and variable handling, as well as to calculate the indicators and generate data visualizations. Seven out of ten indicators were generated from OJS metadata only, and five of them are described on the article (namely, authors per article, international collaboration, most cited authors, bibliometric statistics, and citation-based graphs).

The results highlighted some issues to be considered by the editors of the journal selected for the study, such as: to diminish endogeny, to attract authors/submissions from a broader geographical range, as well as to foment submissions from authors/coauthors with doctoral degrees, among other aspects.

Figure 1. Representation of cited journals, according to macro areas of knowledge of the citing articles – AtoZ Journal (2011- 2016). Source: The authors – based on AtoZ Journal metadata and BibTeX files (2011-2016), May 2017.

Based on the premise that the monitoring of indicators could increase the power of an editorial team to adapt current practices, to project and achieve goals, and to refine decision making processes, the researchers pointed out that data selection and procedures should be performed in semi-automated basis and non-pending of special access permissions (which is suited by the use of the Open Journal Systems (OJS) platform). The study was designed in a way to allow further replication by other journals’ teams who can harvest their own data organically available through OJS, whether or not combined with complimentary data sets.

The study was funded by the Edital de Apoio à Editoração e Publicação de Periódicos Científicos, 2016 (UFPR/PRPPG/SIBI).

References

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Critérios SciELO Brasil: critérios, políticas e procedimentos para a admissão e a permanência de periódicos científicos na Coleção SciELO Brasil. 2014. Available from: http://www.scielo.br/avaliacao/20141003NovosCriterios_SciELO_Brasil.pdf

THE R FOUNDATION. The R Project for Statistical Computing. 2017. Available from: https://www.r-project.org/

ZEARIN. BibTeXML. San Francisco: GitHub Repository, 2016. Available from: https://github.com/Zearin/BibTeXML

To read the article, access

MARCHIORI, P. Z., BETTONI, E. M. and APPEL, A. L. Geração de indicadores para periódicos científicos abertos.Transinformação [online]. 2018, vol.30, n.3, pp.324-335. ISSN 0103-3786. [viewed 7 January 2019]. DOI: 0.1590/2318-08892018000300005. Available from: http://ref.scielo.org/sm4zbc

External links

Transinformação – TINF: www.scielo.br/tinf