As reported by Roars, the Italian National Agency for the Evaluation of University and Research Systems (ANVUR)  has refused to recognize Open Research Europe (ORE) as a scientific “journal” because it does not publish in single, self-contained fascicles and does not practice anonymous peer review.

Open Research Europe is an infrastructure provided by the European Commission to authors whose research is the result of European funding. ORE can be simultaneously an archive, a public discussion forum, and a series of journals from different disciplines, since it is free from the technological constraints of print, for which the publication of texts selected ex ante also signals their value (filter, then publish).

There are many kinds of open peer review. Among them, ORE has chosen the one in which specially selected experts are appointed, but the whole discussion is open to the public: the public debate helps to recognize their responsibilities and merits, to expose any conflicts of interest, and to transform an opaque, hierarchical process prone to failure even catastrophic into an open scientific conversation .

ORE is objectionable because its management has been entrusted not to public libraries and archives, but to F1000 Research, which is owned by a private commercial oligopolist that is part of the so-called surveillance publishing, which has very little to do with the protection of independent and public knowledge. But this is not what the ANVUR is concerned about: for those who want to use the articles deposited there for the purposes of their academic career in the sociological field, ORE is, according to the agency, neither scientific nor “excellent”, whatever that means, because it does not publish in “clear, self-contained fascicles that are not open to further updating” and because it does not practice anonymous review.

Let us resist the temptation to laugh at those who, more than thirty years after the invention of the World Wide Web, demand that online publishing should not make texts available as soon as they are ready, but should manage to mimic the technical and economic limitations of printing by publishing them in “fascicles”. Let us, for the moment, avoid criticizing their attempt to harness scientific discussion by means of processes inspired by ideas that are newer and more questionable than they probably imagine. And let us also forget the debate that is raging beyond the Italian border, on the parasitism and obsolescence of commercial scientific journals. Instead, let us focus on who evaluated ORE and why they were entitled to do so.

The current system of research evaluation in Italy is administrative and centralized, rather than scientific and decentralized, under the pervasive control of a National Agency for University and Research Evaluation, whose board of directors is appointed by the government. One of its many tasks is to determine which journals in the humanities and social sciences are scientific and which are not, and which of the latter are “excellent” (“di classe A”) and which are not. Despite Article 33 of the Italian Constitution, Caesar est supra grammaticos: it is not the researchers who decide what is scientific and what is not, but an authority appointed by the government.

The same bureaucratic and hierarchical principle inspires the regulation for the rating of journals in the so-called non-bibliometric fields, written by the ANVUR board: in addition to having the final say, the board controls the process from the beginning by appointing the members of the working groups responsible for the preliminary evaluation.

The ANVUR is one of the formal signatories of the European Agreement on Reforming Research Assessment, and one of its board members also sits on the steering committee of the Coalition for Advancing Research Assessment (COARA). But, with its decision, it not only tells Italian sociologists that publishing on the platform provided by the European Commission is like throwing their papers in the trash: it also violates some of the principles and commitments of the COARA itself.

First, the ANVUR violates COARA’s second principle for overarching conditions, which is to “safeguard the freedom of scientific research”.

Safeguard freedom of scientific research. By putting in place assessment frameworks that do not limit researchers in the questions they ask, in their research implementation, methods or theories. By limiting the assessment frameworks to only those necessary, as assessment must be useful for researchers, institutions and funders.¹

If what is scientific and what is not is determined by lists written by a government-appointed agency, the freedom of researchers to choose the topics, methods, theories, and applications of their investigations is trapped within the agendas and editorial policies of journals approved by the ANVUR.

Second, among the principles for evaluation criteria and processes, the ANVUR violates at least the first one, which links research quality to openness: “openness corresponds to early knowledge and data sharing, as well as open collaboration including societal engagement where appropriate”.² Again, how can we promote open collaboration when what is scientific and what is not is decided by a government-appointed authority in a zero-sum game designed to pit researchers and institutions against each other?

The ANVUR is also in specific violation of at least three of the commitments in the COARA Agreement:

• the first commitment (p. 5), which requires recognition of all the “valuable contributions that researchers make to science and for the benefit of society, including diverse outputs beyond journal publications and irrespective of the language in which they are communicated”. If evaluation should be about content, wherever it is made available, rather than about its containers, why continue to compile lists of journals administratively labeled “scientific”?
• the second commitment (p. 5) recognizes that “peer review is the most robust method known for assessing quality and has the advantage that it is in the hands of the research community”. Therefore, “it is important that peer review processes are designed to meet the fundamental principles of rigor and transparency” – a transparency that is not possible if the only type of review that the ANVUR considers “scientific” is anonymous.
• the third commitment (p. 6) requires to “abandon inappropriate uses in research assessment of journal- and publication- based metrics”. But what would be the use of ANVUR’s lists of scientific journals if not the evaluation of researchers on the basis of the number of publications in the journals included in the lists, and thus on the basis of journal- and publication-based metrics?

Its decision on ORE was not essential in order to recognize that the administrative system of journal lists was in contradiction with the European commitments formally signed by the ANVUR. It was already clear that the Italian regulation on research evaluation conflicted with COARA’s principles. To be consistent, the ANVUR should have at least updated its regulation to make its lists more inclusive or, better yet, abolished the list system altogether. But the ANVUR did neither, as if it did not know the meaning of the agreement it had signed, or, knowing it well, it had the hidden intention of changing things only to keep them the same.

1. Agreement on Reforming Research Assessment, I , p. 3.
2. Agreement on Reforming Research Assessment, I , p. 3.

Accessi: 104