sayzardfly51fly (@fly51fly)
‘Text Has Curvature’ 논문이 발표되었습니다. K. Grover, H. Zeng, Y. Xia, C. Faloutsos, G. J. Gordon(소속: CMU & Meta)이 텍스트 데이터의 곡률(curvature)이 표현 공간에 미치는 영향을 연구해 NLP 표현학습과 모델 해석에 시사점을 제공합니다.
fly51fly (@fly51fly)
‘Symmetry in language statistics shapes the geometry of model representations’ 연구가 공개되었습니다. D. Karkada, D. J. Korchinski, A. Nava, M. Wyart(소속: Google DeepMind, UC Berkeley, EPFL)가 언어 통계의 대칭성이 모델 표현의 기하구조를 형성한다는 이론적·실험적 결과를 제시합니다.
fly51fly (@fly51fly)
표현 인코더(representation encoders)를 활용해 표준 확산(transformer-based diffusion) 모델을 매니폴드 데이터에 적용 가능하도록 확장한 연구입니다. 매니폴드 구조를 고려한 학습으로 확산 트랜스포머의 성능과 적용 범위를 넓히는 방법을 제안하며, 이미지·신호·지오메트리 데이터 등 비유클리드 데이터 처리에 기여합니다.
https://x.com/fly51fly/status/2022790223336505559
#diffusion #transformers #manifoldlearning #representationlearning
fly51fly (@fly51fly)
Effective Reasoning Chains 논문은 체인 기반 추론(reasoning chains)이 모델 내부 표현의 내재적 차원(intrinsic dimensionality)을 낮춘다는 발견을 보고합니다. A. Prasad, M. Joshi, K. Lee, M. Bansal( Google DeepMind & UNC Chapel Hill )의 이론·실험은 체인 오브 사고의 구조적 이유와 모델 설계·효율성에 대한 시사점을 제공합니다.
UKP LabBig congratulations to all authors! 🚀
#ICLR2026 #MachineLearning #AIResearch #RepresentationLearning #InformationRetrieval #DenseRetrieval #SelfSupervisedLearning #LanguageModels #NLP #UKPLab #ICLR
Rohan Paul (@rohanpaul_ai)
해당 논문은 별도로 학습된 많은 신경망들이 동일한 소수의 가중치 방향(weight directions)을 공유한다는 흥미로운 주장을 제시합니다. 약 1,100개 모델을 분석한 결과 각 층당 약 16개 방향이 대부분의 가중치 변이를 설명한다고 보고하여, 표현학습·모델압축·전이학습 연구에 중요한 시사점을 제공합니다.
Rohan Paul (@rohanpaul_ai) on X
Super cool claim in this paper. Says that many separately trained neural networks end up using the same small set of weight directions. Across about 1100 models, they find about 16 directions per layer that capture most weight variation. A weight is just a number inside the
Robert Simon FongRepresentation learning often emphasizes metric preservation. We instead build Symplectic structural invariance directly into the representation.
https://arxiv.org/abs/2512.19409
We embed Hamiltonian/symplectic geometry by making the RNN state dynamics a symplectomorphism, which preserves Legendre duality (information geometry) through time. This yields structure-preserving representations enforced by the latent dynamics, rather than imposed indirectly via the output.
#ReservoirComputing #RepresentationLearning #InformationGeometry #SymplecticGeometry #HamiltonianDynamics #GeometricDeepLearning #DynamicalSystems #PhysicsInformedML
Symplectic Reservoir Representation of Legendre Dynamics
Modern learning systems act on internal representations of data, yet how these representations encode underlying physical or statistical structure is often left implicit. In physics, conservation laws of Hamiltonian systems such as symplecticity guarantee long-term stability, and recent work has begun to hard-wire such constraints into learning models at the loss or output level. Here we ask a different question: what would it mean for the representation itself to obey a symplectic conservation law in the sense of Hamiltonian mechanics? We express this symplectic constraint through Legendre duality: the pairing between primal and dual parameters, which becomes the structure that the representation must preserve. We formalize Legendre dynamics as stochastic processes whose trajectories remain on Legendre graphs, so that the evolving primal-dual parameters stay Legendre dual. We show that this class includes linear time-invariant Gaussian process regression and Ornstein-Uhlenbeck dynamics. Geometrically, we prove that the maps that preserve all Legendre graphs are exactly symplectomorphisms of cotangent bundles of the form "cotangent lift of a base diffeomorphism followed by an exact fibre translation". Dynamically, this characterization leads to the design of a Symplectic Reservoir (SR), a reservoir-computing architecture that is a special case of recurrent neural network and whose recurrent core is generated by Hamiltonian systems that are at most linear in the momentum. Our main theorem shows that every SR update has this normal form and therefore transports Legendre graphs to Legendre graphs, preserving Legendre duality at each time step. Overall, SR implements a geometrically constrained, Legendre-preserving representation map, injecting symplectic geometry and Hamiltonian mechanics directly at the representational level.
Reddit Tech VN BotNghiên cứu mới giới thiệu UML: Unpaired Multimodal Learner, một phương pháp đột phá giúp các mô hình đơn phương thức (ảnh, âm thanh) học biểu diễn tốt hơn bằng cách tận dụng dữ liệu đa phương thức KHÔNG GHÉP CẶP. UML chia sẻ tham số giữa các modality khác nhau, cải thiện đáng kể hiệu suất mà không cần dữ liệu ghép cặp truyền thống.
#AI #MachineLearning #MultimodalAI #UnpairedData #RepresentationLearning #UML
#TríTuệNhânTạo #HọcMáy #ĐaPhươngThức #DữLiệuKhôngGhépCặp #HọcBiểuDiễn
https://www.redd
Victoria Stuart 🇨🇦 🏳️⚧️A geometric link: Convexity may bridge human & machine intelligence
https://phys.org/news/2025-07-geometric-link-convexity-bridge-human.html
On convex decision regions in deep network representations
https://www.nature.com/articles/s41467-025-60809-y
Convexity (algebraic geometry)
https://en.wikipedia.org/wiki/Convexity_(algebraic_geometry)
#ML #RepresentationLearning #convexity #AbstractAlgebra #DeepLearning #intelligence #NetworkTheory
FIZ KDAI Research GroupWe still have free slots in our KIT summer semester 2025 seminar on "Large Language Model-Enhanced Representation Learning for Knowledge Graphs" supervised by @GenAsefa, Mary Ann Tan and @lysander07
https://portal.wiwi.kit.edu/ys/8600
#teaching #knowledgegraphs #llms #generativeai #representationlearning #seminar @fiz_karlsruhe @KIT_Karlsruhe #AI
