Context Engineering: What You Feed the Model Sets Its Ceiling

Context engineering determines an AI agent's ceiling by controlling what the model sees. This guide explains four core operations—write, select, compress, and isolate—along with RAG versus long context, memory, KV-cache optimization, practical patterns, and a self-checklist.

Anyone building agents eventually hits the same wall: the prompt has been tuned to exhaustion, yet the results remain unstable. The problem is often not the wording of that instruction, but what the model can see at each step. This article is about managing the model's field of view. It takes engineers and technical decision-makers from concepts to practical patterns, with a checklist you can use directly.

1. A counter-intuitive fact

Many people tune an agent like this: poor result → rewrite the prompt → still poor → add more instructions → cram in every potentially useful document, conversation, and tool description → get an even worse result.

The problem is often not whether the prompt is well written, but whether the model sees the right context at that particular step.

In 2025, the industry converged on a term for this work. Karpathy put it most plainly:

"Context engineering" is a better term than "prompt engineering." People associate prompts with short, everyday task descriptions, but in any production-grade LLM application, the real craft is filling the context window with exactly the right information for the next step.

「+1 for "context engineering" over "prompt engineering". People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step.」

Shopify CEO Tobi Lütke has expressed the same preference. "Context engineering" more accurately describes the core skill: "the art of providing all the context for the task to be plausibly solvable by the LLM."

One sentence to distinguish:

prompt engineering: Write a relatively static instruction. context engineering: In a multi-step, ever-changing Agent loop, dynamically decide what to put and what not to put into the limited context window at each step.

Prompt engineering is a subset of context engineering. Once a task grows from a single question and answer into dozens of autonomous steps, tool calls, and results, the main challenge shifts from writing one instruction to managing the entire window.

2. Why context is a scarce resource

Intuitively, the bigger the context window, the better, and 1M token sounds like it can fit anything. But in practice, context is the scarcest resource for three reasons:

1) Attention is limited, and models can "forget the middle." The classic Lost in the Middle study found that models often overlook key information placed midway through a long context: they attend more strongly to the beginning and end than to the middle. Later, Chroma and others used more systematic experiments to describe context rot: performance on the same task declines steadily as context grows, even when needle-in-a-haystack tests suggest the model can still retrieve the fact. Finding information is not the same as using it well.

2) Irrelevant information will actively interfere. Every extraneous piece of content in the context is competing for attention with the correct answer. Three common symptoms:

• Distraction: Being biased by irrelevant history;
• Poisoning: An early wrong conclusion remains in the context, and the wrong conclusion will be wrong all the way;
• Clash: The data inserted are contradictory to each other, and the model is at a loss.

3) Cost and delay increase linearly with token. Every additional token costs money and time. An Agent that fills up the context is slow, expensive, and inaccurate.

The conclusion is counter-intuitive, but extremely important:

More context is not necessarily better; a higher signal-to-noise ratio is. Context engineering is the practice of keeping only the high-signal information needed for each step.

3. What is in the context?

Let's break down the "context" first. What you see when the model is inferring is far more than the user's words, but a whole assembled window:

context engineering is to make the decision of "what to add, what to subtract, and how long to keep" for each cell in this table. One of the things that is most likely to get out of control is the "tool return result" - a web page crawl or a database query may contain thousands of tokens.

4. Four basic operations: write, select, compress, isolate

LangChain summarizes all the methods of context engineering into four types of operations. This is the best mental map currently available:

1. Write—store information outside the window

Not everything has to stay in context. Write intermediate conclusions, plans, and notesoutside the window and retrieve them when needed:

• Scratchpad: Temporary notes within the current task, allowing the model to remember "how many steps have I taken and what have I arrived at".
• Memory: Persistent information (user preferences, project conventions) across tasks and sessions.

2. Select—retrieve only what is relevant

This is the essence of RAG (retrieval-augmented generation), tool selection, and example selection:

• Instead of stuffing in the entire knowledge base, retrieve the passages most relevant to the current question;
• When there are too many tools, first use a layer of search to filter out "several tools that may be used in this step" instead of listing hundreds of tool definitions;
• Even few-shot examples can be dynamically selected based on similarity.

3. Compress—express the same information with fewer tokens

When the context is almost full, it is not roughly truncated, but compressed:

• Summary / compaction: Summarize the previous dozens of rounds of dialogue into a "current progress summary", discard the original process, and retain the conclusions and unresolved matters;
• Crop: Delete the oldest and most irrelevant parts according to rules.

4. Isolate—split context across independent spaces

Split a large task to multiple sub-Agents. Each sub-Agent takes a clean and exclusive context window to do the sub-task, and only returns the "condensed conclusion" to the main Agent. The main Agent's window therefore always looks clean. Sandboxed, standalone runtime environments also fall into this category - isolating operations that may produce massive output.

Remember these four verbs—write, select, compress, isolate—and you’ll have a toolbox for nearly any contextual problem.

In the next few sections, we will pick out the most frequently asked questions and the most common mistakes in this framework, and go into depth one by one.

5. RAG or long context? The answer in 2026

When long context window first appeared, many people asserted that "RAG is dying - why bother searching when you can fit the entire book into it?" Two years later, the conclusion is clear: long context did not kill RAG, but allowed both to return to their respective places.

• The weakness of long context is the context rot mentioned in the second section: the more stuffed it is, the easier it is to lose memory in the middle, the slower and more expensive it is. Putting all 500,000 tokens into it does not mean that the model has really "understood" 500,000 tokens.
• The value of RAG is precisely that it can select the relevant 5,000 tokens from a corpus of 500,000, producing the high-signal, low-noise context that context engineering aims for.

Practical options:

The more cutting-edge approach is agentic retrieval (intelligent retrieval): It is no longer "retrieve once, get the results and go", but let the agent conduct multiple rounds of searches by itself and judge "whether it is enough and whether to check again", turning the retrieval into a feedback loop.

6. Memory: Make Agent smarter across sessions

Memory is the part of context engineering that specifically deals with "time" and is divided into two layers:

• short-term memory: Continuity within a single session/task is mainly maintained by compaction and scratchpad in Section 4.
• long-term memory: Information retained across sessions - who the user is, what their preferences are, and what commitments the project has. Behind the "remember your preferences" you see in ChatGPT and Claude is a mechanism of writing (what is worth remembering) + retrieving (what should be remembered this time).

The difficulty in memory is never "storage", but two judgments:

1. What to write: Not all dialogues are worth remembering, and random memorization will contaminate future context;
2. What to retrieve: Which items should be remembered this time? If you retrieve too many, it will be noise.

After all, the memory system is the application of the two operations "Write + Select" in the time dimension.

7. One of the most overlooked practices: Make context cache-friendly

The previous sections have been talking about "what to put and what not to put", and there is another path that almost directly determines the cost that is often ignored - KV-cache (hint cache).

When the model processes the context, it calculates the prefix into a reusable intermediate state (KV-cache). As long as the context prefix of this round is exactly the same as that of the previous round, this part can directly hit the cache without having to recalculate - it is fast and economical (the hit token is usually only charged at a fraction of the original price). In a multi-step Agent, each step carries almost the same system prompts and history, and the cache hit rate almost directly determines billing and latency. There is even a saying in the industry: "KV-cache hit rate is the most important single indicator for production-level Agents."

This leads to several construction principles that are equally important as "signal-to-noise ratio":

• Stable things are placed in the front, and volatile things are placed in the back. System prompts and tool definitions that remain unchanged in each round are fixed at the front to ensure the stability of the prefix.
• Only append, not rewrite. New information is appended to the end; once you go back and change the previous content (even if only one timestamp is changed), all subsequent caches will be invalid.
• Compression requires timing. Compaction and cropping will rewrite the prefix and invalidate the cache, so don't trigger it too frequently, and do it once at a cost-effective point.

In a word: context engineering not only determines "whether the model is accurate", but also directly determines "whether the system is expensive or not."

8. In practice: two common patterns

Pattern 1: Compaction. This is the lifeline of long-running tasks. Take Claude Code as an example: when the context window is nearly full, it does not simply stop. It summarizes the work so far into a compact handoff record—files changed, current state, and next steps—then continues from that record. The key is to retain open issues and important decisions while discarding the raw process. Poor compaction makes an agent forget its progress and repeat work.

Pattern 2: Sub-agent context isolation. The main agent orchestrates while a sub-agent handles a context-heavy task, such as reading 20 files to find bugs. The sub-agent works in a clean window and returns only its condensed conclusions. The main agent's context is never flooded by all 20 files. This is the power of isolation—and one of the most underrated benefits of multi-agent systems: they are a context-management technique, not merely a way to run work in parallel.

9. What is its relationship with these concepts?

context engineering is surrounded by a ring of confusing words. There is no need to memorize it by rote, just sort it out according to the "type of relationship", and the boundaries will naturally be clear:

① Upstream and downstream - MCP supplies materials from below, and Harness takes care of the details from the outside.

• MCP (Model Context Protocol): Solve the problem of "what can be connected", using a unified protocol to connect external tools and data sources in a standardized manner, which is the "source" of context.
• Harness (the engineering shell of the agent): In addition to the model weight, all the engineering (tools, status, verification, context management...) that determine whether the intelligence can be stably implemented. Context management is only one of the subsystems, but it is often the first to overturn.

② Subset - prompt engineering and RAG, are part of it, not itself.

• prompt engineering: Predecessor and subset of context engineering. The task has changed from "asking and answering" to "running dozens of steps independently", and the main battlefield has expanded from "writing that sentence" to "managing the entire window."
• RAG (Retrieval Enhancement): It is just a specific method in the "select" type of operation, responsible for "fetching relevant knowledge on demand". It is a subset of context engineering, not a synonym - equating the two misses out the other three categories of writing, compression and isolation.

③ Alternate route - context engineering vs fine-tuning.

There are two fundamentally different ways to inject knowledge and capabilities into the model: feeding it into the context (in-context, all discussed in this article), or training into the weights (in-weights, that is, fine-tuning).

• Knowledge is updated quickly, sources must be brought, and additions and deletions must be made at any time → use context (RAG/long context);
• To solidify a stable style, format or proprietary capability, and the data is sufficient → consider fine-tuning.

Most mature systems use both: fine-tuning determines the "base color", and context engineering feeds the "current thing".

Concluding in one sentence: context engineering determines the "field of view" of each step of the model - prompt engineering and RAG are its parts, fine-tuning is its other way, MCP supplies materials below, and Harness takes care of the outside.

10. Ready-to-use context checklist

• Can I explain clearly what components the model has in the context it sees at each step?
• Have the results returned by the tool been "slimmed down" (leaving only conclusions and truncating overly long output)? Or just stuff it back as it is?
• Is there any "history that is no longer useful" in the context? Is there any regular compression/cropping?
• Is there a compaction mechanism for long tasks? Are open items and key decisions retained during compression?
• Is knowledge retrieval "full stuffing" or "retrieval on demand"? Do the search results include sources?
• Are the tools/examples "available in full"? Is it possible to dynamically filter based on the current step?
• Have you offloaded the heavy work to the sub-Agent and used independent windows to isolate the massive intermediate information it generates?
• Is the context "stable content first and only appended at the end" to hit the KV-cache as much as possible and suppress costs?
• Am I on the right side between "longer context is better" and "higher signal-to-noise ratio is better"?

Models will keep getting smarter and context windows will keep growing. But as long as attention remains limited and information remains noisy, an agent's ceiling will depend not only on the model itself, but on the context you feed it.

References

• Andrej Karpathy, Tweet about "context engineering" (2025)
• LangChain, Context Engineering for Agents (Write/Select/Compress/Isolate framework)
• Anthropic, Effective context engineering for AI agents
• Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance (2025)
• Liu et al. (arXiv), Lost in the Middle: How Language Models Use Long Contexts (2023)