GitHub - Compresr-ai/Context-Gateway: Context Gateway is an agentic proxy that enhances any AI agent workflow with instant history compaction and context optimization tools

Context Gateway is an agentic proxy that enhances any AI agent workflow with instant history compaction and context optimization tools - Compresr-ai/Context-Gateway

GitHub - mksglu/claude-context-mode: Stop losing context to large outputs.

Stop losing context to large outputs. Contribute to mksglu/claude-context-mode development by creating an account on GitHub.

Brent’s Encapsulated C Programming Rules

A bunch of tips and rules I’ve created for myself for developing programs in the C programming language

Ursula K. Le Guin — 3. The Absent Silence

A year or two ago I was asked to review a novel by José Saramago, and in looking up facts about him on Google I found over and over the same quotation from him — God is the silence of the universe, and man is the cry that gives meaning to that silence. It’s from his Lanzarote journals, which are

Principles of Vasocomputation: A Unification of Buddhist Phenomenology, Active Inference, and Physical Reflex (Part I) – Opentheory.net

What the hell have you built.

ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference

Transformer-based large models excel in natural language processing and computer vision, but face severe computational inefficiencies due to the self-attention's quadratic complexity with input tokens. Recently, researchers have proposed a series of methods based on block selection and compression to alleviate this problem, but they either have issues with semantic incompleteness or poor training-inference efficiency. To comprehensively address these challenges, we propose ChunkLLM, a lightweight and pluggable training framework. Specifically, we introduce two components: QK Adapter (Q-Adapter and K-Adapter) and Chunk Adapter. The former is attached to each Transformer layer, serving dual purposes of feature compression and chunk attention acquisition. The latter operates at the bottommost layer of the model, functioning to detect chunk boundaries by leveraging contextual semantic information. During the training phase, the parameters of the backbone remain frozen, with only the QK Adapter and Chunk Adapter undergoing training. Notably, we design an attention distillation method for training the QK Adapter, which enhances the recall rate of key chunks. During the inference phase, chunk selection is triggered exclusively when the current token is detected as a chunk boundary, thereby accelerating model inference. Experimental evaluations are conducted on a diverse set of long-text and short-text benchmark datasets spanning multiple tasks. ChunkLLM not only attains comparable performance on short-text benchmarks but also maintains 98.64% of the performance on long-context benchmarks while preserving a 48.58% key-value cache retention rate. Particularly, ChunkLLM attains a maximum speedup of 4.48x in comparison to the vanilla Transformer in the processing of 120K long texts.