Representations and Computations in Transformers that Support Generalization on Structured Tasks

Yuxuan Li, James McClelland

Action editor: Stefan Lee.

https://openreview.net/forum?id=oFC2LAqS6Z

#attention #learns #representations

Representations and Computations in Transformers that Support...

Transformers have shown remarkable success in natural language processing and computer vision, serving as the foundation of large language and multimodal models. These networks can capture nuanced...

OpenReview