#CoARA, #DORA etc. want to (rightfully) broaden the evaluative criteria for academic performance - to not just reward the quantity of publications (which led to paper mills and fake publications that you can buy).

See Goodhart's law in action: Now academics can also buy patents:

"Patents, like peer reviewed publications, look impressive in CVs, and many institutions reward them. [...] A research team with lead at Northwestern University in the United States reports a new type of academic fraud: fake patents. They have found multiple companies which sell credits on UK registered designs to academics, mostly in India, who can use them to boost promotion scores and institutional rankings."

 https://zenodo.org/records/14889136

Found on Sabine Hossenfelder’s blog: https://sabine.beehiiv.com/p/patent-fraud-a-record-battery-satellite-pollution-and-energy-from-the-vacuum

#ResponsibleResearchAssessment

Das #Bundesverfassungsgericht hat die #Zweitveröffentlichungspflicht im Hochschulgesetz BW für nichtig erklärt.

Was bedeutet das für die Steuerung der #OpenAccess Transformation in der Zukunft?

🔍 Ich habe dazu ein paar Gedanken beim @verfassungsblog notiert.

💡 Zentrale These:
Verpflichtende rechtliche Steuerungsmodelle stoßen schnell an kompetenz-, grund- & unionsrechtliche Grenzen.
Ein wirksamer Hebel ist dagegen die Reform der #Forschungsbewertung #CoARA

👉 
https://dx.doi.org/10.59704/e39eae0d69d843a7

As is the case for nearly all complex systems, changing a process is going to be not only difficult, take a long time but also meet a "chicken & egg" problem (causality dilemma). The goal of science is to create and disseminate knowledge. At least for publicly funded research projects, the deliverables should be publicly available. In the past decade, much progress has been made to reach #openscience, including moving from a closed journals to an "open-access" model (whether it actually reduced the financial burden on organizations is another matter entirely).

In the case of the open science reform, the "chicken & egg" problem asks whether researchers should practice open science first, or should the research ecosystem adopt open science first?
1) With strict resource constraints (time, funding, workforce), researchers don't have time to dedicate to deliverables that don't provide a direct value. The values of open science are laudable, but they are not rewarded. Why would researchers spend time creating and sharing a FAIR dataset when publishing another paper with that time is preferable for career advancement?
2) If the research ecosystem adopts open science (one way or another), e.g., researchers MUST publish code, data, register protocols, etc., then there is a huge non-adoption risk, or even worse a malicious compliance risk. A highly valued right for researchers is their academic freedom. In some ways, project-based funding is already threatening academic freedom, then what would it look like if grant agencies started withholding further funding because they didn't publish X or Y dataset?

In an era and a society where work is often meaningless or people are not convinced by their company's values and purposes, research is probably one of the remaining job where the work is driven by the person themselves. For that reason, the culture is very strong and/or opinionated because people actually like their jobs.

Hidden in the problem above is the interdependency between culture, trust and research reforms. Jonny Coates points many issues in a thoughtful piece [1], including that researchers MUST be at the center of the system reform: "Many reform initiatives are designed for researchers, but not often with them. If open science, assessment reform, and research culture initiatives are to succeed, active researchers need to be more than subjects. They need to be participants in designing the systems they are expected to use.".

[1] https://upstream.force11.org/connecting-reform-movements-linking-research-culture-trust-and-open-research-publishing/

#OpenScience #CoARA #ReformingRA

An unintended but positive side effect of the rise of AI and LLMs is that it may accelerate the long overdue reform of research assessment. If the "journals" are flooded with slop academia will be forced to take DORA seriously.

#ResearchAssessment #COARA

RE: https://mastodon.social/@OpenAIRE/116283445949332979

Excited to share this @OpenAIRE interview with @mathijsvleugel, head of our office & Chair of the #CoARA National Chapter Germany, reflecting on research assessment & sharing insights into our survey "Valuing what matters?" (further information & data set: https://doi.org/10.5281/zenodo.18944700) #ResponsibleResearchAssessment (lmf)

What should research assessment really value?

In our latest interview, Dr Mathijs Vleugel reflects on the gap between what is rewarded in evaluation and what researchers believe should count.

🔗Read the interview: https://shorturl.at/Ou1L3

#OpenAIRE #ResearchAssessment #OpenScience #CoARA

RE: https://wisskomm.social/@WissRat/116211637900668257

Zwei Beobachtungen aus der Pressekonferenz:

1) Quantitative Rankings sieht die Jury beim internationalen Wettbewerb kritisch. So fiel die Anmerkung, dass qualitative Begutachtungsverfahren gestärkt werden sollen und die Forschungsbewertung generell eine willkommene Veränderung durchlaufe (Stichwort #CoARA).

2) Die @BerlinUAlliance erhielt insbesondere Lob für ihren kooperativen Ansatz (Forschung, Lehre, Studium, Infrastruktur). Und die Ausweitung auf @b_r_50 wurde explizit begrüßt.

Quantitative Metriken setzen falsche Anreize. Was in vielen Disziplinen längst ein strukturelles Problem ist, hat die #Rechtswissenschaft noch nicht erfasst. Sie kann #Bewertungslogiken sowie #Publikationsinfrastruktur noch so gestalten, dass diese den epistemischen Anforderungen der Disziplin entsprechen & eine belastbare Grundlage für #Forschungsbewertung schaffen.

Wie? Dazu habe ich mit @philippfalkenburg bei #IntRechtDok ein #Preprint veröffentlicht:
https://doi.org/10.17176/20260309-114602-0
#CoARA #DOA #OA

In the #RESQUE evaluation scheme („Research Quality Evaluation scheme for psychological research“), there is an indicator for publications: “Correctness of computational results has been independently verified“.

https://www.resque.info

With CODECHECK, you can request and upload a verification certificate. This gives you "extra points“ in our evaluation scheme, as this is a real quality criterion.

https://codecheck.org.uk

#researchAssessment #CoARA @aufdroeseler https://nerdculture.de/@aufdroeseler/116165190898955396