Lukas Galke

@[email protected]
586 Followers
385 Following
423 Posts

Assistant Professor of Data Science and Advanced Machine Learning at the University of Southern Denmark in Odense

Machine Learning, Natural Language Processing, Interpretability

Previously:
Postdoc @mpi_nl
PhD @ Kiel University, Germany

#ML, #NLProc

Websitehttp://lpag.de
ORCIDhttps://orcid.org/0000-0001-6124-1092
Google Scholarhttps://scholar.google.de/citations?hl=en&user=AHGGdYQAAAAJ&view_op=list_works&sortby=pubdate
Show threadLukas GalkeJan 6, 2025
When analyzing the learning trajectory of RNNs throughout training, we make several other interesting observations: medium-structured languages have an learnability advantage early in training (likely due to ambiguous terms in those languages) but fall behind high-structured languages later.
Show threadLukas GalkeJan 6, 2025
We find a similar effect when looking at memorization errors. In the memorization test, the task for in-context LLMs boils down to copying a word that is present earlier in the prompt. But even here, we can see an advantage of language structure.
Show threadLukas GalkeJan 6, 2025
All these learning systems, small RNNs, pre-trained LLMs, and humans, show *very* similar memorization and generalization behavior -- with more structured languages leading to generalizations that are more systematic and more similar to the generalization of human participants.
Lukas GalkeJan 6, 2025

🗞️ Now out in Nature Communications:

Deep neural networks and humans both benefit from compositional structure.

w/ Yoav Ram and Limor Raviv

Lukas GalkeOct 29, 2024
🗞️ New paper out at the European Conference on Artificial Intelligence 2024 (ECAI) under Yousef Younes lead and with @ansgarscherp We simplify hierarchical text classification by training a lightweight decoder on top of a pre-trained encoder-only language model. The decoder learns the hierarchical structure just from the training sequences of class labels -- showing that various hacks, like integrating the hierarchy into text classification models, are not needed. https://ebooks.iospress.nl/doi/10.3233/FAIA240661
IOS Press Ebooks - RADAr: A Transformer-Based Autoregressive Decoder Architecture for Hierarchical Text Classification

Show threadLukas GalkeOct 19, 2023

Next up, an introduction to the avalanche library for continual learning by Albin Soutif

#unconf23

Show threadLukas GalkeOct 19, 2023

First invited talk of the Unconference. Marco Gori proposes *time* as the new protagonist for machine learning in his talk on Collectionless AI.

#unconf23

Lukas GalkeOct 19, 2023

I'm so excited for the Unconference :) It just started but you can still register/join.

https://unconf.continualai.org/register

#unconf23 #ContinualLearning #LifelongLearning #MachineLearning

ContinualAI

CLAI Unconf is an open-access, multi-timezone, 24 hours long event which brings together ideas at the intersection of machine learning, computational neuroscience, robotics and more!

Lukas GalkeSep 27, 2023
Dean Falk just gave the first keynote of #Protolang8 about the 'putting the baby down'-hypothesis, importance of bipedalism, and how learning to keep a beat affects musical and linguistic abilities, starting already in the womb
Lukas GalkeAug 9, 2023
We just made friends with a bat in our garden 😳🦇💖