Long short-term memory (1997) [pdf]

본 문서는 1997년에 발표된 Long Short-Term Memory (LSTM) 논문의 PDF 파일입니다. LSTM은 순환 신경망(RNN)의 한 종류로, 장기 의존성 문제를 해결하여 자연어 처리, 음성 인식 등 다양한 시퀀스 데이터 처리에 혁신적인 영향을 미쳤습니다. 이 논문은 AI/ML 모델 및 알고리즘 분야에서 매우 중요한 기초 연구로, 현재까지도 LLM 파인튜닝과 AI 에이전트 구축에 널리 활용되고 있습니다.

https://www.bioinf.jku.at/publications/older/2604.pdf

#lstm #rnn #sequencelearning #deeplearning #neuralnetworks

Raven: Memory as a Set of Slots

Raven은 고정 크기 메모리 모델이 장기 기억 유지에 겪는 문제를 해결하기 위해 슬롯 단위로 상태를 분할하고, 각 슬롯을 독립적으로 선택적 갱신하는 새로운 순환 모델 구조를 제안한다. 기존 SSM은 메모리를 균일하게 감쇠시키고, SWA는 고정 슬롯을 강제로 교체하는 반면, Raven은 학습된 희소 라우터를 통해 어떤 슬롯을 갱신할지 결정하여 중요한 정보를 전용 슬롯에 저장하고 간섭을 최소화한다. 이 접근법은 이론적 한계에 근접하는 상수 메모리 재호출 능력을 가능하게 하며, 후속 연구에서 구체적 아키텍처와 성능 결과가 공개될 예정이다.

https://goombalab.github.io/blog/2026/raven-part1/

#recurrentmodel #memory #sequencelearning #slotmemory #ssm

Raven Part-1 - Memory as a set of Slots | Goomba Lab

Homepage of the Goomba AI Lab @ CMU MLD.

This paper by Raju et al. proposes a unified model – “clone‑structured causal #graphs” (#CSCG) – for #hippocampal #SpatialCoding. It suggests that #SpatialMaps arise from #learning #latent higher‑order sequences rather than representing #EuclideanSpace directly. The model elegantly explains phenomena like #PlaceFields, #SplitterCells, #contextual #remapping, and predicts when #PlaceFieldMapping may mislead.

🌍 https://www.science.org/doi/10.1126/sciadv.adm8470

#Hippocampus #CognitiveMaps #SequenceLearning #Neuroscience

Hot off the press! Lind, Ghirlanda and Enquist, the authors of the above-mentioned book ⬆️, have now published their study on #SequenceLearning in #bonobos , showing that the #GreatApes struggle to tell short sequences apart https://doi.org/10.1371/journal.pone.0290546: https://doi.org/10.1371/journal.pone.0290546 We are currently conducting a study on #Chimpanzees at the moment and hope to be able to contribute to the discussion soon.
A test of memory for stimulus sequences in great apes

Identifying cognitive capacities underlying the human evolutionary transition is challenging, and many hypotheses exist for what makes humans capable of, for example, producing and understanding language, preparing meals, and having culture on a grand scale. Instead of describing processes whereby information is processed, recent studies have suggested that there are key differences between humans and other animals in how information is recognized and remembered. Such constraints may act as a bottleneck for subsequent information processing and behavior, proving important for understanding differences between humans and other animals. We briefly discuss different sequential aspects of cognition and behavior and the importance of distinguishing between simultaneous and sequential input, and conclude that explicit tests on non-human great apes have been lacking. Here, we test the memory for stimulus sequences-hypothesis by carrying out three tests on bonobos and one test on humans. Our results show that bonobos’ general working memory decays rapidly and that they fail to learn the difference between the order of two stimuli even after more than 2,000 trials, corroborating earlier findings in other animals. However, as expected, humans solve the same sequence discrimination almost immediately. The explicit test on whether bonobos represent stimulus sequences as an unstructured collection of memory traces was not informative as no differences were found between responses to the different probe tests. However, overall, this first empirical study of sequence discrimination on non-human great apes supports the idea that non-human animals, including the closest relatives to humans, lack a memory for stimulus sequences. This may be an ability that sets humans apart from other animals and could be one reason behind the origin of human culture.