sayzardFeb 3

Tsinghua University (@Tsinghua_Uni)

Xu가 ICML 2025(CCF A급)에서 발표한 논문에서 Xattention을 도입해 알고리즘에서 13.5× 속도 향상을 달성했고, 실시간·무한 길이 비디오 이해를 위한 StreamingVLM을 개발했다고 보고했습니다. 해당 연구는 100+ 인용과 300k+ 미디어 조회를 기록했습니다.

https://x.com/Tsinghua_Uni/status/2018565287906517400

#xattention #streamingvlm #icml2025 #videounderstanding #efficientattention

Tsinghua University (@Tsinghua_Uni) on X

As the first author, Xu published a paper at @icmlconf 2025, a CCF A–ranked conference. He introduced Xattention, achieving 13.5× speedup for algorithms & developed StreamingVLM for real-time, infinite-length video understanding, gaining 100+ citations & 300k+ media views.

X (formerly Twitter)
Institute for AIJul 19, 2025

Is complex query answering really complex? A paper at the International Conference on Machine Learning (#ICML2025) presented by Cosimo Gregucci, PhD student at @UniStuttgartAI @Uni_Stuttgart, discussed this question.

In this paper, Cosimo Gregucci, Bo Xiong, Daniel Hernández (@daniel), Lorenzo Loconte, Pasquale Minervini (@pminervini), Steffen Staab, and Antonio Vergari (@nolovedeeplearning) reveal that the “good” performance of SoTA approaches predominantly comes from answers that can be boiled down to single link prediction. Current neural and hybrid solvers can exploit (different) forms of triple memorization to make complex queries much easier. The authors confirm this by reporting the performance of these methods in a stratified analysis and by proposing a hybrid solver, CQD-Hybrid, which, while being a simple extension of an old method like CQD, can be very competitive against other SoTA models.

The paper proposed a way to make query answering benchmarks more challenging in order to advance science.

https://arxiv.org/abs/2410.12537

#KnowledgeGraphs #QueryAnswering #ArtificialIntelligence #MachineLearning #Benchmarking #CQA

ExLabJul 17, 2025

We will be advertising for a postdoc position soon, to work on #generative #models #structure #induction and #uncertainty with Michael Gutmann as part of the GenAI Hub (@genaihub)!

Keep an eye out, and get in touch! (#ML #AI #ICML2025 )

👉 https://homepages.inf.ed.ac.uk/snaraya3/
👉 https://michaelgutmann.github.io/
👉 https://www.genai.ac.uk/

Siddharth - Home

Siddharth's page

ExLabJul 17, 2025

Are you compositionally curious 🤓

Want to know how to learn embeddings using🌲?

In our new #ICML2025 paper, we present Banyan:
A recursive net that you can train super efficiently for any language or domain, and get embeddings competitive with much much larger LLMs 1/🧵

#embeddings #structure #nlp #semantics #efficient #lowresource

Daniel HernándezJul 15, 2025

Is Complex Query Answering Really Complex? We evaluate models that predict answers to queries over knowledge graphs on a reduced set of standardized benchmarks. This Wednesday at 16:30, Cosimo Gregucci (PhD student at @UniStuttgartAI @Uni_Stuttgart) will discuss this question and our paper at the poster session at the International Conference on Machine Learning #ICML2025.

https://arxiv.org/abs/2410.12537

Joschka BraunJul 13, 2025
1/ Can steering vectors reliably control text properties during summarization?
Our paper "Beyond Multiple Choice: Evaluating Steering Vectors for Adaptive Free-Form Summarization" at the #ICML2025 Workshop on Reliable and Responsible Foundation Models investigates this! 🧵👇
Gatsby UnitJul 11, 2025

👋 Attending #ICML2025 next week?

Don't forget to check out work involving our researchers!

Krzysztof KołaczJun 21, 2025

Apple rozwija technikę, która pozwala AI lepiej naśladować styl pisania użytkownika

Apple zaprezentowało nową metodę personalizacji modeli językowych o nazwie PROSE (Preference Reasoning by Observing and Synthesizing Examples), która ma na celu poprawę dopasowania stylu pisania AI do indywidualnych preferencji użytkownika.

Prace zostaną przedstawione podczas konferencji ICML 2025. Jak działa PROSE?

Zamiast polegać wyłącznie na promptach czy ręcznej edycji, PROSE:

  • analizuje próbki tekstów użytkownika (np. e-maile, notatki),
  • tworzy wewnętrzny profil preferencji pisarskich (np. „krótkie zdania”, „ironia na początku”),
  • iteracyjnie porównuje własne odpowiedzi z oryginałami i dostosowuje się,
  • sprawdza spójność stylu na większej liczbie próbek, by nie opierać się na pojedynczych przykładach.

Choć badanie nie wspomina bezpośrednio o produktach Apple, PROSE idealnie wpisuje się w strategię Apple Intelligence – systemu inteligentnych funkcji, które mają być bardziej spersonalizowane i działać lokalnie na urządzeniu. Technika ta może w przyszłości trafić do aplikacji dzięki Frameworkowi Foundation Models, umożliwiającemu dostęp do AI na poziomie systemowym.

Nowy benchmark: PLUME

Apple opracowało także PLUME – zbiór danych do testowania jakości dopasowania stylu, oparty na autentycznych e-mailach i notatkach. PROSE pokonało wcześniejsze metody, np. CIPHER, aż o 33%, a w połączeniu z klasycznym „in-context learning” (ICL) zyskało dodatkowo 9% skuteczności.

PROSE wpisuje się w coraz silniejszy nurt personalizacji AI – od dopasowywania preferencji po uczenie kontekstu. To nie tylko kwestia użyteczności, ale też strategii utrzymania użytkownika w ekosystemie – im lepiej AI zna Twój styl, tym trudniej będzie Ci z niej zrezygnować.

#AppleAI #AppleIntelligence #ChatGPTVsApple #FoundationModelsApple #ICML2025 #personalizacjaAI #PLUMEApple #PROSE #przyszłośćAIWIPhone #stylPisaniaAI

Gatsby UnitJun 5, 2025

🥁 "Training Dynamics of In-Context Learning in Linear Attention" by Yedi Zhang, Aaditya Singh, Peter Latham, @SaxeLab

This work reveals how in-context learning abilities emerge during gradient descent training of linear attention, revealing abrupt acquisition or progressive improvements depending on how the key and query are parametrized.

Spotlight paper at #ICML2025
👉 https://arxiv.org/abs/2501.16265

Training Dynamics of In-Context Learning in Linear Attention

While attention-based models have demonstrated the remarkable ability of in-context learning (ICL), the theoretical understanding of how these models acquired this ability through gradient descent training is still preliminary. Towards answering this question, we study the gradient descent dynamics of multi-head linear self-attention trained for in-context linear regression. We examine two parametrizations of linear self-attention: one with the key and query weights merged as a single matrix (common in theoretical studies), and one with separate key and query matrices (closer to practical settings). For the merged parametrization, we show that the training dynamics has two fixed points and the loss trajectory exhibits a single, abrupt drop. We derive an analytical time-course solution for a certain class of datasets and initialization. For the separate parametrization, we show that the training dynamics has exponentially many fixed points and the loss exhibits saddle-to-saddle dynamics, which we reduce to scalar ordinary differential equations. During training, the model implements principal component regression in context with the number of principal components increasing over training time. Overall, we provide a theoretical description of how ICL abilities evolve during gradient descent training of linear attention, revealing abrupt acquisition or progressive improvements depending on how the key and query are parametrized.

arXiv.org
Daniel HernándezMay 11, 2025

Cosimo Gregucci (PhD student at @UniStuttgartAI @Uni_Stuttgart) will present our paper "Is Complex Query Answering Really Complex?" at the International Conference on Machine Learning, #ICML2025. We observed that we can answer many of the current benchmarks for complex queries by reducing queries into basic ones. Are they really complex? How could we advance the field by proposing a more complex set of queries?

Old preprint: https://doi.org/10.48550/arXiv.2410.12537

Is Complex Query Answering Really Complex?

Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current benchmarks for CQA might not be as complex as we think, as the way they are built distorts our perception of progress in this field. For example, we find that in these benchmarks, most queries (up to 98% for some query types) can be reduced to simpler problems, e.g., link prediction, where only one link needs to be predicted. The performance of state-of-the-art CQA models decreases significantly when such models are evaluated on queries that cannot be reduced to easier types. Thus, we propose a set of more challenging benchmarks composed of queries that require models to reason over multiple hops and better reflect the construction of real-world KGs. In a systematic empirical investigation, the new benchmarks show that current methods leave much to be desired from current CQA methods.

arXiv.org