Rewording a prompt can't fix what the model never saw, or what it lost among everything else it was given. This guide explains what context engineering is, why extra information can make AI worse, the four techniques teams use to control it and a six-step audit for a system you already run.
✦Key Takeaways
- The prompt can be the smallest part of what a model reads. Tool descriptions, history, documents and tool results compete for the same space, and software chooses most of them.
- More context isn't always better. Anthropic reports that recall falls across all models as context grows, so treat the context window as a budget rather than a storage cupboard.
- Long-context failures come in four types: poisoning, distraction, confusion and clash. Naming the type tells you where to look for the fix.
- Four techniques cover most fixes: write notes outside the window, select what each step needs, compress what you keep and isolate side jobs.
- Start an audit by reading the full context from ten bad runs and ten good ones. Look for the fact that was missing, buried or contradicted.
- Context engineering can't rescue wrong information. If your documents are out of date or your business terms were never defined, fix those first.
Connect an AI agent to five common software tools and it can use more than a quarter of its working memory before anyone has typed a word. When Anthropic's engineers added up a setup of GitHub, Slack, Sentry, Grafana and Splunk, the 58 tool descriptions alone came to about 55,000 tokens. A token is the unit AI models read text in, roughly three-quarters of an English word, so that's about 40,000 words of instruction manual on a model that holds 200,000 tokens at once.
Now compare that with the prompt you spent an afternoon rewording. It might be 300 words. When an agent picks the wrong tool, quotes an old policy or forgets what it was doing, the cause often sits in that other material, and a better-worded prompt can't reach it.
Managing that material is the job of context engineering. This guide explains what it is, how it differs from prompt engineering, why extra information can make a model perform worse, the four techniques teams use to control it and how to audit a system you already run.
What Is Context Engineering?
Context engineering is the practice of deciding what an AI model sees at each step of a task: its instructions, the conversation so far, retrieved documents, tool descriptions, tool results and saved notes. The goal is to give the model what it needs for the next step and leave out whatever gets in the way.
Think of a new hire on their first day. The prompt is the brief you hand them. Context is everything else on their desk: the handbook, the customer file, yesterday's notes, the list of people they can call. A clear brief on a desk buried in old paperwork still produces poor work. An AI model's desk is its context window, the fixed amount of text it can read at once, a term we define in our AI glossary for executives.
The phrase took off in June 2025. Shopify CEO Tobi Lutke called it "the art of providing all the context for the task to be plausibly solvable by the LLM", and Andrej Karpathy backed the term, describing "the delicate art and science of filling the context window with just the right information for the next step". Simon Willison explained why a new name was needed: most people's working definition of prompt engineering was "a laughably pretentious term for typing things into a chatbot".
Prompt Engineering vs Context Engineering
Anthropic describes context engineering as "the natural progression of prompt engineering", and that framing is useful. Prompts still matter. They have become one input among several.
| Prompt engineering | Context engineering | |
|---|---|---|
| Main question | How should I word this? | What should the model see right now? |
| Scope | Instructions and examples | Instructions, history, documents, tools, results and memory |
| When it happens | When you write the prompt | At every step of every run |
| Typical failure | A vague or brittle instruction | The right fact is missing, buried or contradicted |
For a one-off task in a chat window, prompting is most of the job. It stops being enough once a system runs in a loop, because each step produces new material and something has to decide what stays.
What Goes Into an AI Model's Context
A modern AI system assembles up to six kinds of material before each model call. Most of it is chosen by software rather than typed by a person.
- Instructions: the system prompt that sets the role, rules and tone.
- Conversation history: earlier messages, including the model's own replies and mistakes.
- Retrieved information: passages pulled from your documents or the web, the approach we compared in advanced RAG vs long context windows.
- Tool definitions: descriptions of each action the model can take, often supplied by MCP servers.
- Tool results: whatever those actions return. Anthropic's example is a two-hour sales call transcript, which could add 50,000 tokens as it passes through the model.
- Memory: notes saved from earlier work and pulled back in when they seem relevant.
Why More Context Can Make AI Worse
The obvious fix is a bigger window. That helps with capacity, but not with attention. Anthropic calls the effect context rot: as the amount of text grows, a model's ability to recall any one detail from it falls. It says this "emerges across all models", although some degrade more gently, and it expects "context windows of all sizes" to face the same problem "for the foreseeable future".
Position matters too. The 2023 paper Lost in the Middle found that models often performed best when the relevant information sat at the start or end of their input, and that performance "significantly degrades" when it sat in the middle of a long context. That held even for models built for long inputs.
In a 2025 essay, Drew Breunig sorted long-context failures into four types, and all four can show up in business systems:
- Context poisoning: an error enters the context and keeps being reused. Google DeepMind found that its Gemini agent, while playing Pokémon, could become "fixated on achieving impossible or irrelevant goals" after false information about the game reached its goals and summary. A support agent could do the same with a misread order number.
- Context distraction: there's so much material that the model leans on it and neglects what it learned in training. An agent fed years of old tickets may copy a past answer instead of working through the new problem.
- Context confusion: irrelevant material changes the answer. Breunig cites a benchmark where a small, compressed model failed a query when offered 46 tools, even though everything fitted in its window, and succeeded when offered 19.
- Context clash: two sources disagree. If the search returns both the 2022 price list and the 2026 one, the model may quote either.
Tools are usually the quickest win. Anthropic calls bloated tool sets "one of the most common failure modes" it sees, and its test is a good one: "If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better."
In internal tests published in November 2025, Anthropic loaded tools only when they were needed instead of all at the start. Accuracy on its tool evaluations rose from 49% to 74% on Claude Opus 4 and from 79.5% to 88.1% on Opus 4.5. These are a vendor's tests of its own feature, so treat the size of the gain with caution.
Four Core Context Engineering Techniques
LangChain groups the common approaches into four moves: write, select, compress and isolate. Real systems often combine several.
Write: save it outside the window
Have the agent keep notes in a file or database and read them back when needed. On complex tasks, the AI agent Manus creates a todo.md file and rewrites it as it goes. This keeps the plan at the end of the context, close to the model's recent attention, and helps avoid what its team calls "lost-in-the-middle" issues. Notes also let a run survive a crash, as we covered in why AI agents get stuck.
Select: fetch only what this step needs
Instead of loading everything at the start, give the agent references, such as a file path, a record ID or a saved query, and let it open what it needs. Anthropic calls this "just in time" context. An agent processing an invoice needs that supplier's record, not your whole CRM.
Compress: shrink what you keep
When the window fills up, summarise the history and carry on from the summary, which Anthropic calls compaction. Start with the easy wins: once a tool result has been used, the raw output rarely needs to stay. The danger is losing a detail that turns out to matter, because, as the Manus team puts it, "you can't reliably predict which observation might become critical ten steps later". Their answer is to make compression restorable. Drop the text of a web page but keep its URL, so the agent can fetch it again.
Isolate: give side jobs their own desk
A sub-agent takes one focused job, works in its own clean context and returns a summary. Anthropic says a sub-agent might use tens of thousands of tokens and hand back only 1,000 to 2,000. This suits research and searching. It has limits, though. Cognition, the company behind the coding agent Devin, argues against multi-agent designs because agents that can't see each other's work make conflicting decisions. Its example is two sub-agents building a Flappy Bird clone, where the bird and the background come out in completely different styles. Our rule of thumb: split out work that reads and gathers, and keep work that makes decisions in one thread.
Context Also Decides Cost and Security
Model providers generally charge for the text you send, so context choices show up on the bill, one of the production costs we covered in the AI ROI trap. Caching changes the sums. Text the provider has recently processed can cost far less, as long as the start of the context hasn't changed. Manus reported in July 2025 that cached input on Claude Sonnet cost a tenth of the uncached price, and argues that cache hit rate is the single most important metric for a production agent. One timestamp at the top of a system prompt can break the cache.
Context is also an attack surface, because everything in it can influence the model, including web pages, emails and files written by strangers. OWASP lists prompt injection first among its risks for language model applications, including the indirect form that arrives through "external sources, such as websites or files". Treat retrieved text as something to read, not orders to follow, and limit what an agent can do after reading it. Memory deserves the same care, since a planted instruction saved as a note can come back in a later session.
How to Audit Your AI System's Context
You don't need a rebuild to start. These six steps work on most custom chatbots or agents already in use.
- Read what the model actually received. Log the full context for ten bad runs and ten good ones, then compare them. Look for the fact that was missing, buried or contradicted.
- Measure each source. Count tokens by type: instructions, history, retrieved text, tool definitions and tool results. Anthropic suggests loading tools on demand once definitions pass about 10,000 tokens.
- Cut the tool list. Remove tools that overlap and rename any that are easy to confuse.
- Shrink tool results. Return the fields the next step needs, plus an ID to fetch the rest, rather than a full record or raw log.
- Set rules for memory. Decide what gets saved, who can correct it and when it expires. Keep unverified claims out of summaries, where they harden into facts.
- Test every change. Keep a fixed set of real questions and rerun them after each change, as our guide to testing an AI system before launch explains. A change that fixes one case can quietly break another.
Expect several rounds. The Manus team calls context engineering "an experimental science" and says it rebuilt its agent framework four times, each time after finding a better way to shape context.
When Context Engineering Won't Help
Context engineering arranges information. It can't supply information that doesn't exist or correct information that's wrong.
- The source is wrong. No arrangement fixes an outdated policy. Fix the knowledge base first.
- Business terms were never defined. In a September 2026 VB Pulse survey, 69% of 109 respondents whose organisations run agents on company data had traced at least one wrong answer to bad business context in the past six months. The respondents were self-selected, so it isn't a picture of all companies, but the problem is easy to recognise. If nobody has defined an "active customer", the agent has to guess.
- The failure is elsewhere. Crashes, rate limits and retry loops call for recovery logic around the model.
- The job needs fixed rules. If a decision must come out the same way every time, write it as ordinary code.
Conclusion
For a while, getting good results from AI looked like a writing skill. For anything that runs over many steps, it's now closer to systems design. The prompt still matters, but it sits on a desk that software fills, and that desk decides what the model can use and what it overlooks.
The work can also outlast any one model. Manus chose context engineering partly to keep its product independent of the models underneath: "If model progress is the rising tide, we want Manus to be the boat, not the pillar stuck to the seabed."
Start by reading what your system sent the model before its last ten bad answers. If you'd like help with what you find, or you're planning a new agent, talk to AI Native Agency.
Frequently Asked Questions
- What is context engineering in simple terms?
- It means deciding what an AI system gets to read before each response. That covers instructions, past messages, documents, available tools, tool results and saved notes. Done well, the model has enough to finish the step without the important details being buried.
- Is context engineering replacing prompt engineering?
- No. Clear prompts still matter, and they form part of the context. Prompting alone is fine for a one-off task in a chat window. Systems that run over many steps also need a way to manage what builds up along the way.
- What is context rot?
- Context rot is the decline in a model's ability to recall details as the text it's reading grows. Anthropic uses the term and says it affects all models, though some degrade more gently. It helps explain why a long chat or agent run can miss something it was told earlier, even before it reaches its limit.
- How is context engineering different from RAG?
- RAG is one technique within it. RAG finds relevant passages in your documents and adds them to the context. Context engineering also covers how many passages to include, what to remove, and how to handle tools, history and memory.
- Do we need context engineering if we only use ChatGPT or Copilot?
- Much less, because the vendor handles most of the design. You still control some inputs: start a new chat when the topic changes, share only the documents that matter, keep custom instructions short and limit what connected tools can reach. If you build your own agents, it becomes a core engineering job.
- How can I tell a context problem from a model problem?
- Read the full context from a failed run. If the fact the model needed was missing, buried or contradicted, it's a context problem. If the fact was clearly there and the model still got it wrong, try tighter instructions or a different model.
