Mastodawn

EMNLP 2025 глазами аналитика из Яндекса: мировые тренды и наши решения

Всем привет! Меня зовут Катя Еникеева, я руковожу командой аналитики перевода в Яндексе. Мы занимаемся оценкой качества машинного перевода — моделей, которые работают в Яндекс Переводчике, Браузере, Поиске и во множестве других сервисов. Качество перевода можно измерять по‑разному, но можно выделить два основных направления: экспертная разметка и автоматические метрики. В последние годы автометрики всё чаще строятся поверх LLM: фактически это отдельный пайплайн, который анализирует исходный текст и полученный перевод. Поэтому нас интересует не только способность моделей переводить, но и их умение анализировать качество перевода, что может быть заметно сложнее. Под катом вас ждёт обзор самых интересных решений, представленных на конференции EMNLP 2025. Наша команда перевода приехала на EMNLP 2025 не только слушать, но и рассказывать о своей работе. В этом году у нас приняли две статьи: одну — в Findings основной конференции, вторую — на WMT. О них я тоже подробно расскажу.

https://habr.com/ru/companies/yandex/articles/991144/

#яндекс #машинный_перевод #конференции #ml #языки #перевод #llm #emnlp

EMNLP 2025 глазами аналитика из Яндекса: мировые тренды и наши решения

Хабр

Andrei Kucharavy Nov 5

If you are at EMLP'25, I will be presenting shortly our work on LLM disinformation in low-resource languages, using Walliserdeutsch Swiss German dialect as an example.

TLDR; We show that a competent attacker can wreak havoc with model that have no declared performance in Germanic languages let alone WD. Since WD shares a lot of features with other low-resource languages, this means LLM disinformation in low-resource languages is much closer than thought.

#Science #EMNLP #Disinformation

Show thread

UKP Lab Nov 4

And consider following the authors Sheng Lu, Ilia Kuznetsov, and Iryna Gurevych (all Ubiquitous Knowledge Processing (UKP) Lab/Technische Universität Darmstadt).

See you at the #EMNLP conference in Suzhou 🏯

(3/3)

#EMNLP2025 #UKPLab #PeerReview #LLM #AIResearch #NLProc

Show thread

UKP Lab Nov 3

📊 𝗦𝗘𝗘𝗘𝗗 outperforms #GPT-4o and #Phi-4 by up to +𝟴 𝗽𝗽 across multiple datasets.

📄 𝗣𝗮𝗽𝗲𝗿: https://www.arxiv.org/abs/2509.10833
💻 𝗖𝗼𝗱𝗲: https://github.com/UKPLab/emnlp2025-automatic-error-discovery
🔗 𝗣𝗿𝗼𝗷𝗲𝗰𝘁: https://ukplab.github.io/emnlp2025-automatic-error-discovery/

Be sure to follow the authors: Dominic Petrak, Thy Thy Tran, and Iryna Gurevych from Ubiquitous Knowledge Processing (UKP) Lab/Technische Universität Darmstadt.

See you at the #EMNLP in Suzhou!

(2/2)

#NLProc #ConversationalAI #Agents #EMNLP2025

Towards Automated Error Discovery: A Study in Conversational AI

Although LLM-based conversational agents demonstrate strong fluency and coherence, they still produce undesirable behaviors (errors) that are challenging to prevent from reaching users during deployment. Recent research leverages large language models (LLMs) to detect errors and guide response-generation models toward improvement. However, current LLMs struggle to identify errors not explicitly specified in their instructions, such as those arising from updates to the response-generation model or shifts in user behavior. In this work, we introduce Automated Error Discovery, a framework for detecting and defining errors in conversational AI, and propose SEEED (Soft Clustering Extended Encoder-Based Error Detection), as an encoder-based approach to its implementation. We enhance the Soft Nearest Neighbor Loss by amplifying distance weighting for negative samples and introduce Label-Based Sample Ranking to select highly contrastive examples for better representation learning. SEEED outperforms adapted baselines -- including GPT-4o and Phi-4 -- across multiple error-annotated dialogue datasets, improving the accuracy for detecting unknown errors by up to 8 points and demonstrating strong generalization to unknown intent detection.

arXiv.org

Show thread

UKP Lab Oct 31

And consider following the authors Haishuo Fang (UKP Lab), Xiaodan Zhu (Department of Electrical and Computer Engineering, Smith Engineering at Queen's University), and Iryna Gurevych (UKP Lab/ATHENE Center) if you are interested in more information or an exchange of ideas.

See you at the #EMNLP conference in Suzhou 🏯

(3/3)

#NLProc #AI #EMNLP2025 #LLMAgent

UKP Lab Nov 27, 2024

𝗧𝗵𝗿𝗼𝘄𝗯𝗮𝗰𝗸 𝘁𝗼 Hashtag#EMNLP2024 𝗶𝗻 𝘀𝘂𝗻𝗻𝘆 𝗠𝗶𝗮𝗺𝗶🌞
The UKP Lab had an amazing time at this year’s #EMNLP in Florida🌴!

Our team presented a total of 13 papers, including 11 in the Main track and 2 in the Findings track, showcasing our latest research to a vibrant international audience.
(1/🧵)

WikiResearch Nov 12, 2024

RT @aroraakhilcs: Disappointed to not be at #EMNLP owing to a dislocated shoulder 😢
@DebjitPaul2 will present our poster on Multilingual E…

via https://twitter.com/WikiResearch/status/1856126524354900436

WikiResearch (@WikiResearch) on X

RT @aroraakhilcs: Disappointed to not be at #EMNLP owing to a dislocated shoulder 😢 @DebjitPaul2 will present our poster on Multilingual E…

X (formerly Twitter)

UKP Lab Oct 23, 2024

⏳ Countdown to #EMNLP2024 in Florida🌴
The #EMNLP in Miami begins on 12 November and starting tomorrow, we will be posting UKP Lab's contributions to the conference daily.

Follow along for in-depth insights and key contributions to the field of NLP innovations.

ALTA Sep 23, 2024

❗ #CfP EXTENDED to 27th Sep

Did your #NLP #RAG #NLU paper get rejected from @emnlpmeeting #EMNLP?

Consider submitting instead to #ALTA2024, affiliated with @aclmeeting.

📅 Submission deadline for short/long papers, presentation abstracts and industry demonstrations: 27 Sep 2024, 23:59 Anywhere on Earth, UTC -12

📅 Author notification: 29 Oct 2024

📅 Camera ready: 12 Nov 2024

📅 2nd-4th Dec 2024, #ANU, #Canberra, Australia.

#Hybrid in person and online.

https://alta2024.alta.asn.au/calls/papers

Call for Papers

Official website for the 2024 Workshop of the Australasian Language Technology Association

ALTA 2024

Harald Klinke Jun 24, 2024

The 4th International Conference on Natural Language Processing for Digital Humanities (NLP4DH) will be co-located with EMNLP in Miami, USA, on November 15-16, 2024.
Topics include text analysis, dataset creation, cultural heritage collection research, and more. Submission deadline: September 1, 2024. Proceedings will be published in the ACL Anthology.

More info: https://www.nlp4dh.com/nlp4dh-2024

#NLP4DH #EMNLP #DigitalHumanities #NLP #Research #Conference2024

NLP4DH - NLP4DH 2024

The 4th International Conference on Natural Language Processing for Digital Humanities (NLP4DH 2024) will be organized together with EMNLP 2024. The proceedings of the conference will be published in the ACL anthology. The conference will take place in Miami, USA. The focus of the conference is on