This summer the Italian National Agency for the Evaluation of University and Research Systems (ANVUR) denied scientificity and excellence (“classe A”) to Open Research Europe (ORE) for general sociology. Recently, however, ANVUR has updated its rules for classifying journals, adding an article 18 entitled ‘Transitional regulation for open peer review’.

The ORE affair was the first step in ANVUR’s systematic violation of the principles of COARA, the European coalition for the reform of research evaluation towards more open science and a less irresponsible and extensive use of bibliometrics, of which ANVUR is a member. And it was also a diplomatic incident: by denying the scientific quality of ORE, ANVUR told Italian sociologists who wanted to pursue an academic career in Italy that publishing their texts on a website set up by the European Commission to submit articles written by recipients of European funding to open peer review was tantamount to throwing them in the trash.

However, ANVUR now states, albeit provisionally, that

1. As an alternative to the requirements of Art. 13 c. 4.1, for journals using open peer review procedures, the following requirements shall be provided, subject to compliance with the previously defined rules on ethics and management of conflicts of interest
a. the tracking of all versions of papers produced during the review process
b. the assignment of a specific status to papers that have passed at least two successful peer reviews and their subsequent indexing;
c. clearly indicating any additions, changes, or corrections made by authors to previous versions. In accordance with the provisions of c. 1, journals that adopt appropriate open review procedures are considered eligible for the classification procedure for inclusion in the lists of scientific journals and class A journals, with reference only to those articles that have been definitively accepted or have successfully passed peer review and have subsequently been indexed.

Could Open Research Europe finally be included in the Italian agency’s lists? Not yet: Article 2(2) states that “(a) provide for the publication of multiple units (fascicles, volumes, issues), with continuity and without a predetermined end date; (b) provide formally identifiable and citable units of publication (numbered and dated) that are self-contained and allow the identification of individual contributions within them (through progressive page numbering and/or DOI code assigned to each article)”.

Perhaps ORE could – but why should it? – bother to hire a box-ticker to periodically produce self-contained, numbered and dated fascicles to satisfy the Italian agency. But beyond the pragmatic interest, we must ask at least two questions:

.
1. Why is it so important to the agency that articles that are already identifiable in other ways, e.g. by DOI, be organized in fascicles as if they were going to be printed?
2. Could an article that was rejected after open peer review, but remained public with the reviewers’ critiques and the author’s responses, still be considered by committees evaluating his scientific qualifications for a professorship (ASN) or deciding whether to hire him? Imagine, for example, that a young Galileo Galilei had a manuscript entitled “Sidereus Nuncius” rejected by an open peer review of the open-access Journal of Ptolemaic Studies, which was rated excellent (“di classe A”) by ANVUR. Galileo, however, is so sure of his theory that he decides to leave his “Sidereus Nuncius” with the reviewers’ negative opinions and his replies on the journal’s website. Could a Copernican ASN or a hiring committee recognize the scientific value of the “Sidereus Nuncius” even though ANVUR’s evaluation of journals is required to ignore it, according to the last paragraph of Article 18? Galileo perhaps could not meet the bibliometric thresholds required for application: but this circumstance avoids the problem administratively, but does not solve it scientifically.

As Alessandro Figà Talamanca points out in Il fattore d’impatto nella valutazione della ricerca e nello sviluppo dell’editoria scientifica, irregularity in publication was reason enough to exclude even well-established but artisanal journals from ISI’s commercial database (now in the hands of Clarivate Analytics) on which the Journal Impact Factor (JIF) is calculated. This exclusion had nothing to do with scientific quality, but only with the convenience and cost of calculating the JIF. And, according to the ANVUR regulations (Article 15(3) and (4)), the impact of a journal is still an aspect to be taken into account when awarding a “di classe A” rating.

A former president of ANVUR argued that a predicament like that of the young Galileo Galilei is rare and unlikely. Under state evaluation, scholars are precociously trained to behave as petty clerks of research and to submit orthodox texts to the Ptolemaic journals evaluated by ANVUR. In this way, such excellent journals will never have the chance to become scientifically stimulating venues for the discussion of Copernican provocations that could never see the light of day.

And yet the imaginary Galilean predicament suggests that ANVUR’s new regulation on journals has not a scientific purpose but a bibliometric one, and that even the transitional Article 18 works to make open peer review so bureaucratic that it can be reconciled with a mainly bibliometric and centralized evaluation of research, whose principle is that a government-appointed agency is entitled to determine what is scientific and what is not. ANVUR is a participant in COARA – and one of its members sits on its Steering Board – but, despite the third commitment of the agreement, it continues to evaluate researchers, at the service of the Italian Ministry of Universities and Research, with metrics based on journals and their rankings, taking them as decisive criteria and not merely complementary. In fact, a scenario in which the peers of the scientific community, outside ANVUR’s control, actually evaluated published texts without ANVUR’s bibliometrics would weaken the power of ANVUR and the Ministry of Universities and Research behind it. And this may be the reason why ANVUR would rather try to reduce open peer review back to bureaucracy than to reduce itself to science.

This text is licensed under CC BY-SA 4.0 license

Accessi: 660