UKP LabDec 7, 2023

New Paper Alert! 🚨 Ever wondered what shapes #NLProc research over time?

We explore the evolution of research in our #EMNLP2023 paper – and, delve into the »WHEN, HOW, and WHY« of paradigm shifts in Natural Language Processing. (1/🧵)

📃 https://arxiv.org/abs/2305.12920

A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why?

Understanding the fundamental concepts and trends in a scientific field is crucial for keeping abreast of its continuous advancement. In this study, we propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques. We define three variables to encompass diverse facets of the evolution of research topics within NLP and utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data. Subsequently, we leverage this structure to measure the intensity of these relationships. By conducting extensive experiments on the ACL Anthology corpus, we demonstrate that our framework effectively uncovers evolutionary trends and the underlying causes for a wide range of NLP research topics. Specifically, we show that tasks and methods are primary drivers of research in NLP, with datasets following, while metrics have minimal impact.

arXiv.org
Show threadUKP LabDec 7, 2023

In our paper, we use causal discovery algorithms to reveal intricate relationships among research entities in #NLProc: Tasks, Datasets, Methods, and Metrics.

With causal inference, we evaluate the strength of these relationships. (2/🧵) #EMNLP2023

Show threadUKP Lab
Grounded on research papers from #ACLAnthology between 1979 and 2022, our framework unveils the primary drivers behind research trends in NLP. (3/🧵) #EMNLP2023
Dec 7, 2023 at 9:20amWeb
Show threadUKP LabDec 7, 2023
It highlights the central influences driving #NLProc research:
🥇 Tasks and Methods (1st place)
🥈 Datasets (2nd place)
🥉 Metrics (3rd place) (4/🧵) #EMNLP2023
Show threadUKP LabDec 7, 2023
Exciting insights 🤩
Our paper analyzes #NLProc Tasks (such as “Sentiment Analysis” or “Word Sense Disambiguation”) revealing their common modeling as “Text Classification” task. (5/🧵) #EMNLP2023
Show threadUKP LabDec 7, 2023

Datasets such as “Penn Treebank” play a pivotal role, leaving marks on the tasks like "Language Modeling," "POS Tagging," and "Semantic Parsing”, whereas tasks like “Speech Recognition” and “Summarization” witness the evolution of datasets over time.

Stay tuned for the deep dive into #NLProc Research! (6/🧵) #EMNLP2023

Show threadUKP LabDec 7, 2023

You can find our #EMNLP2023 paper here:
➡️ https://arxiv.org/abs/2305.12920

Check out the work of our authors Aniket Pramanick, Yufang Hou (IBM Research Europe), Saif M. Mohammad (National Research Council Canada) and Iryna Gurevych for more info. See you at #EMNLP2023 this week in Singapore! (7/7)

A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why?

Understanding the fundamental concepts and trends in a scientific field is crucial for keeping abreast of its continuous advancement. In this study, we propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques. We define three variables to encompass diverse facets of the evolution of research topics within NLP and utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data. Subsequently, we leverage this structure to measure the intensity of these relationships. By conducting extensive experiments on the ACL Anthology corpus, we demonstrate that our framework effectively uncovers evolutionary trends and the underlying causes for a wide range of NLP research topics. Specifically, we show that tasks and methods are primary drivers of research in NLP, with datasets following, while metrics have minimal impact.

arXiv.org