Mastodawn

“𝗬𝗼𝘂𝗿 𝗽𝗮𝗽𝗲𝗿 𝗶𝘀 𝗻𝗼𝘁 𝗻𝗼𝘃𝗲𝗹 𝗲𝗻𝗼𝘂𝗴𝗵.”🙅
We have all seen this infamous comment from Reviewer #2.

But 𝗵𝗼𝘄 𝗿𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗶𝘀 𝘁𝗵𝗲 𝗻𝗼𝘃𝗲𝗹𝘁𝘆 𝗮𝘀𝘀𝗲𝘀𝘀𝗺𝗲𝗻𝘁 𝘁𝗼𝗱𝗮𝘆?🤔

Novelty is central to scientific progress. Assessing it is getting harder as we speak: 𝘂𝗻𝗰𝗹𝗲𝗮𝗿 𝗰𝗿𝗶𝘁𝗲𝗿𝗶𝗮, 𝘀𝗸𝘆𝗿𝗼𝗰𝗸𝗲𝘁𝗶𝗻𝗴 𝗻𝘂𝗺𝗯𝗲𝗿 𝗼𝗳 𝘀𝘂𝗯𝗺𝗶𝘀𝘀𝗶𝗼𝗻𝘀 𝗮𝗻𝗱 𝗮 𝗳𝗿𝗮𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗹𝗶𝘁𝗲𝗿𝗮𝘁𝘂𝗿𝗲.

Show thread

UKP Lab Mar 11

📄 In our 𝗘𝗔𝗖𝗟 𝟮𝟬𝟮𝟲 paper, we propose a 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲𝗱, 𝗟𝗟𝗠-𝗮𝘀𝘀𝗶𝘀𝘁𝗲𝗱 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲 for novelty assessment.

📍We study novelty as a 𝘀𝘁𝗮𝗻𝗱𝗮𝗹𝗼𝗻𝗲 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝘁𝗮𝘀𝗸 (vs. a by-product of AI-based review generation).
🤖 The pipeline produces 𝗹𝗶𝘁𝗲𝗿𝗮𝘁𝘂𝗿𝗲-𝗮𝘄𝗮𝗿𝗲, 𝗲𝘃𝗶𝗱𝗲𝗻𝗰𝗲-𝗯𝗮𝗰𝗸𝗲𝗱 novelty claims.
✅ Validated with real human reviews: 𝟴𝟲.𝟱% 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁 with human reasoning.
😌 𝗡𝗼 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴. 𝗡𝗼 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴. Just task design grounded in 𝗿𝗲𝘃𝗶𝗲𝘄𝗲𝗿 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴.

Show thread

UKP Lab Mar 11

🎯 𝗪𝗵𝘆 𝘁𝗵𝗶𝘀 𝗺𝗮𝘁𝘁𝗲𝗿𝘀
As submissions grow, transparent novelty support can help reviewers justify decisions and reduce “hand-wavy” novelty claims.

Looking forward to discussions at #EACL!

📄 Paper: https://arxiv.org/pdf/2508.10795
💻 Code and data: https://github.com/UKPLab/eacl2026-assessing-paper-novelty
🔗 Project: https://ukplab.github.io/eacl2026-assessing-paper-novelty/

Show thread

UKP Lab

Follow the authors Osama Mohammed Afzal and Iryna Gurevych from the Ubiquitous Knowledge Processing (UKP) Lab, Technische Universität Darmstadt, as well as Preslav Nakov from MBZUAI (Mohamed bin Zayed University of Artificial Intelligence) and Tom Hope from Ai2.

See you this #EACL2026 in Rabat 🕌!

#UKPLab #NLP #NLProc #AI #PeerReview #AIforScience #ScientificPublishing #NoveltyDetection #Evaluation #OpenScience