#context-memory — Agentic AI

Context window and agent memory: count, truncate, summarize, index.

What you'll play with

Welcome to #context-memory. A LLM has no persistent memory: on every call you re-feed it everything it must know inside its context window, a bounded token queue (8k, 128k, 1M depending on the model). An agent that runs for a while fills it fast: system prompt, few-shot, user and assistant history, tool observations, RAG documents. On screen: the bar = your window, the white line = the model limit, the gauge above = your fill ratio, and at the bottom the external memory (indexed documents you can fetch on demand). You are going to fill it, then compress it.
Start small. Type /model llama-3: the limit drops to 8,000 tokens. On small-window models every token is expensive — that is where you can see the management strategies at work.
Add a user message. Type /add user A long test user message with some explanation. An indigo block appears; the gauge climbs.
Few-shot examples are question/answer pairs put at the start of the context to demonstrate the expected format. Type /few-shot 3: three green blocks land at once.
Now RAG (Retrieval-Augmented Generation). Instead of stuffing everything into the context, you index documents in an external memory, and only bring the most relevant ones on each query. Type /rag REST API documentation: the 3 closest documents (by cosine similarity on embeddings) light up in the bottom grid and get added to the context in purple.
The window starts filling up. First compression strategy: the sliding window. Type /truncate sliding: the oldest user/assistant messages disappear, system and few-shot are preserved.
A subtler alternative: summarize instead of dropping. Type /summarize 3: the 3 oldest user/assistant blocks merge into a single summary block whose cost falls to about a quarter of the original tokens.
And what if you had 1M of context? Type /model gemini-flash: the limit explodes, everything you added looks tiny in the bar, the gauge drops back near zero. But watch out — cost: a 1M-token context at ~$0.30 per million input tokens is ~$0.30 per call just for the context.
Your turn. Try /strategy summary to enable auto-summarization when you overflow, /purge-memory to empty the RAG index, or /reset to start over. Premium channels #tool-calling (free) and #planning-reflection (Premium) show what comes next: how an agent picks its tools and plans its steps while accounting for its context.

Channel commands

/model <gpt-4o|claude-sonnet|gemini-flash|llama-3> — Switch model and therefore the context-window size.
/add <system|user|assistant|observation> <text> — Append a block to the context (message, observation, system prompt).
/few-shot <n=1..5> — Append n few-shot examples (question/answer pairs).
/rag <query> — Fetch the 3 closest documents in the external memory and append them to the context.
/truncate <sliding|system-only> — Purge the oldest messages (keeps what is stable).
/summarize <n=2..5> — Replace the n oldest user/assistant blocks with a summary (~1/4 the tokens).
/strategy <sliding|summary|none> — Set the strategy applied automatically when the limit is exceeded.
/purge-memory — Empty the external memory (RAG). Rag blocks already in the context stay.
/reset — Reset the context (current model is kept).

Glossary

context window: The token queue a LLM reads on every call: system instructions, conversation history, tool returns, RAG documents. Its size is bounded by the architecture (8k for a llama-3, 128k for a GPT-4o, 200k for a Claude, 1M for a Gemini Flash). Anything that does not fit is invisible to the model for that call.
token: The chunking unit a LLM uses (usually a BPE sub-word). In English one token is about 0.75 word on average — hence the common approximation tokens ≈ words × 1.3. Tokens are the billing unit of the APIs (per million input and output).
tokenization vs token counting: Tokenization splits a text into precise tokens using a learned vocabulary (BPE, SentencePiece, tiktoken). Counting can be exact (through the model's tokenizer) or approximate (words × 1.3). The #tokenization channel details the algorithm; here we approximate to stay lightweight.
sliding window: Purge strategy: when the window is full, you drop the oldest user/assistant messages while preserving the system prompt and the few-shot examples. Fast, free, but the removed information is lost for the model. Used by Claude Code and Cursor to sustain long sessions.
summarization: Alternative to truncation: instead of dropping the old blocks, you ask a LLM to summarize them, and replace the originals with the summary (~1/4 of the tokens). Information is compressed, not lost. Price: one extra LLM call, and some loss of fidelity.
RAG (retrieval-augmented generation): Rather than stuffing everything in the context, you index the documents in a vector database (embeddings), and on every query only load the top-k most relevant (cosine similarity). This is the main workaround for the context-length limit. Detailed in the #rag channel (Premium).
embedding: Representation of a text as a vector in a high-dimensional space (typically 384 to 3072). Two texts with close meaning yield close vectors. Here we use a deterministic toy embedding (size 8, derived from the vocabulary) — the cosine-similarity → top-k mechanic stays faithful to real models.
cosine similarity: Closeness metric between two vectors, computed as the cosine of their angle: a · b / (‖a‖ · ‖b‖). Bounded in [−1, 1]: 1 = identical, 0 = orthogonal, −1 = opposite. It is RAG's elementary operation: it sorts indexed documents by relevance to the query.
working memory vs long-term memory: Working memory = what is in the context window on this call (immediate, expensive, bounded). Long-term memory = what is indexed in an external vector database, fetched on demand by RAG (persistent, nearly unlimited, but needs a good query). A useful agent combines both.
cost per million tokens: The billing unit for LLM APIs. In 2026 a GPT-4o costs about $2.50 per million input tokens; a Claude Sonnet ~$3; a Gemini Flash ~$0.15. The economic reason to manage the context: a 10× larger context = a 10× more expensive call. An agent running 20 times with 100k tokens of context costs more than a movie ticket.

Other channels in Agentic AI

#react-loop — An agent's ReAct loop: Thought → Action → Observation, live.
#tool-calling — Tool calling (function calling / MCP): the JSON that makes the LLM act.
#slash-commands — Slash commands, Claude Code / Cursor style: templates, arguments, chaining.
#context-memory — Context window and agent memory: count, truncate, summarize, index.
#planning-reflection — Planning, reflection and self-correction: from 60% to 90% success.
#multi-agents — Multi-agents: planner, workers, verifier. A DAG that beats the monolithic agent.