Context Engineering Replaced Prompt Engineering, and 1M Tokens Explain Why

Anthropic made a 1 million token context window generally available in March 2026, at standard per-token pricing. Its own best model scores 78.3% on MRCR v2 at that length, which is the highest published figure and still leaves more than a fifth of the benchmark unearned. Cheap context did not buy good output. What the model can see, and in what order, now decides more than how the instruction is phrased.

What context engineering actually means

Context engineering is the practice of deciding what enters the model's context window on every call. Prompt engineering is the practice of wording one of those inputs well. The first contains the second.

Anthropic published the definition that stuck on 29 September 2025. It describes context engineering as the set of strategies for curating and maintaining the optimal set of tokens during inference, including all the other information that may land there outside of the prompts.

Read that last clause twice. It is where the shift lives. The prompt is one input among many, and in a production agent it is usually the smallest one.

The definition arrived from practitioners, not vendors

Shopify's Tobi Lütke used the phrase in June 2025. Andrej Karpathy amplified it days later, writing that context engineering is the delicate art and science of filling the context window with just the right information for the next step.

Two details in that sentence do the work. It treats the window as a budget to allocate. And it scopes the job to the next step, not the whole task.

That second point is the one teams skip. A context assembled once at the start of a run is a configuration file. A context reassembled per step is an engineering system with its own failure modes.

Prompt engineering is not dead, it was demoted

The claim that prompt engineering is dead is wrong, and the correction is boring. Wording still matters. It stopped being the variable with the largest effect on output quality.

In a chat window, the prompt is close to 100% of what the model sees. In an agent running tool calls across 100 turns, the system prompt might be 2% of the context. Optimising 2% of the input is not where your remaining error lives.

Here is my position, stated plainly. If your team still describes this work as prompt engineering, the job title is a lagging indicator of your architecture. The label matters less than the budget line it sits under.

The million token window did not make the problem go away

The obvious objection to all of this is that context scarcity was a temporary hardware condition. Windows got bigger. Prices came down. Why curate anything?

Because the evidence says models do not use long context uniformly, and the vendors shipping the long context say so too.

What context rot is, measured

Chroma published a technical report on 14 July 2025 by Kelly Hong, Anton Troynikov and Jeff Huber. They evaluated 18 models on tasks extended from Needle-in-a-Haystack, holding task complexity constant and varying only input length.

Every model degraded as the input grew, including on simple tasks. The effect worsened when the target information was semantically similar to distractors rather than a literal string match, which is exactly the case in real retrieval.

That last finding is the practically important one. Needle-in-a-Haystack demos use an obviously out-of-place sentence. Your production corpus is 400 near-identical contract clauses.

What the 1M announcement actually claims

Anthropic made 1M context generally available for Opus 4.6 and Sonnet 4.6 on 13 March 2026, and removed the long-context price premium. A 900K token request is now billed at the same per-token rate as a 9K one.

The same post reports Opus 4.6 at 78.3% on MRCR v2 at that context length, and calls it the highest among frontier models. Take the claim at face value. It still means the best available model gets more than a fifth of a retrieval benchmark wrong at full window.

So the economics of long context changed and the reliability did not change enough. That combination is precisely what makes curation an engineering discipline rather than a cost workaround.

The four levers that now decide output quality

Anthropic names four techniques in its context engineering guidance. They are worth learning as a set, because each one fails differently and most teams reach for the wrong one first.

Just-in-time retrieval beats loading everything

The pattern is to keep lightweight identifiers in context, such as file paths, record IDs or query handles, and load the underlying data at runtime through a tool call. The model decides what it needs at the step where it needs it.

This is the same instinct behind retrieval augmented generation, applied per step instead of once. The difference matters because a single up-front retrieval has to guess what the whole task will require. Retrieval quality is downstream of data quality, which is why the order in which you remediate an AI-ready data estate tends to decide how well any of this works.

Compaction and structured note-taking handle long runs

Compaction summarises the conversation so far and restarts the window from the summary. Note-taking writes state to a file the agent can read back later, so memory lives outside the window rather than inside it.

Anthropic reports that on an internal agentic search evaluation, the memory tool combined with context editing improved performance 39% over baseline, with context editing alone at 29%. Those are vendor figures on a vendor eval, and I say so in the limitations section below.

Tool design is the most under-priced lever

Every tool definition sits in context on every call. Anthropic's November 2025 guidance notes that agents connected to thousands of tools need to process hundreds of thousands of tokens before reading a request.

Two fixes have published numbers. Loading definitions on demand through a tool search step cut token usage 85%. Rebuilding one workflow so the model writes code against filesystem-backed MCP APIs took it from about 150,000 tokens to about 2,000, a 98.7% reduction. If you are choosing between assembling this yourself and buying it, the tradeoffs are the same ones in the case against building your own agent framework.

What context bloat costs you in cash

Token reduction is not a tidiness exercise. It is a gross margin line, and for anyone shipping an AI product it is the fastest one to move.

Anthropic's own multi-agent research published on 13 June 2025 gives the scale. Agents use about 4 times the tokens of a chat interaction, and multi-agent systems about 15 times. On the BrowseComp evaluation, token usage alone explained 80% of the performance variance.

Sit with that number. If most of the measured performance difference tracks token consumption, then how you spend the budget is the product decision, not a deployment detail.

The cost column is the one vendor guidance tends to leave out. None of these techniques is free, and compaction in particular trades a hard failure for a quiet one.

The wider point is that per-token prices falling does not mean your bill falls, because context engineering choices set how many tokens each request consumes. That gap between falling unit prices and rising invoices is covered in why cheap tokens produce rising AI bills, and the margin consequences in what AI COGS do to real gross margin.

Retrieval or long context: the research does not settle it

The most common question this topic produces is whether retrieval augmented generation is obsolete now that windows are large. The honest answer is that the peer-reviewed work points both ways, and the split is systematic.

Li and colleagues at Google, in work accepted to EMNLP 2024, found that long context consistently outperforms retrieval on average when resourced sufficiently, while noting that retrieval's much lower cost remains a real advantage. They proposed routing each query to one or the other based on model self-reflection.

The LaRA benchmark, published February 2025, tested 11 models across 2,326 cases and four task types at 32K and 128K. Its finding is in the subtitle: no silver bullet for long context or retrieval routing. Weaker models gained from retrieval at extreme lengths, stronger ones preferred direct context, and retrieval won on hallucination detection across every model tested.

Where this argument is weakest

Five of the eight sources under this post were published by Anthropic. That is a company selling context management as a product feature, reporting results on evaluation sets it defined and has not released.

I still used them, for a specific reason. They are the only figures anyone has published with enough method detail to argue with, and the direction they point matches the independent work from Chroma and the two academic papers. Corroboration across source types is what makes a vendor number usable. One vendor restating itself in three blog posts is not.

Two further weaknesses worth naming. The Chroma study is a technical report, not peer reviewed, and it predates the current model generation by more than a year. And long-context leaderboards for MRCR v2 disagree with each other badly enough that I have cited only Anthropic's own claim about its own model, rather than any aggregated ranking.

The strongest counterargument runs like this. Every reliability gap listed here has narrowed with each model release, so the discipline of curating context may be a temporary workaround for a temporary limitation. That case is coherent. It also has to explain why the vendor that shipped the largest cheap window is simultaneously shipping tools to put less into it.

The order to try these in, and why order matters

Most teams start with the hardest technique. The sequence below is cheapest first, and it is deliberately unglamorous.

Start by counting what is already in your context window on a typical call. Split it into system prompt, tool definitions, retrieved content and conversation history. Most teams discover that tool definitions are the largest block and nobody had looked.

Then prune the tool catalogue before doing anything clever with it. Removing tools an agent never calls is free and reversible. Loading definitions on demand comes after that, not instead of it.

Compaction and memory come last, because both introduce new failure modes. A lossy summary produces an agent that is confidently missing something, which is far harder to debug than an agent that hit a limit and stopped.

None of this is checkable without measurement. You need a fixed task set you can rerun after each change, which is the argument for building an evaluation suite before you tune anything. Without one you are trading token count against a quality number nobody is recording.

What this changes for the person doing the work

This post is written for the practitioner shipping an LLM feature, not the executive approving the budget. Every recommendation here is something one engineer can test in a week.

The practical change is what you open first when output quality is bad. The old reflex was to rewrite the system prompt. The better reflex is to print the exact context the model received on the failing call and read it.

Nine times in ten, the answer is visible there. The retrieved chunk was wrong. Three tools had overlapping descriptions and it picked the wrong one. A stale tool result from turn 14 contradicted the fresh one from turn 40. None of those get fixed by better wording.

There is a second-order effect on how you buy. A vendor demo runs on a short, clean context. Your workload does not. Ask any agent vendor what their typical context length is at turn 50, and what they do when it approaches the limit. The answer separates products that were built for this from products that were demoed for it.

The interoperability layer matters here too, because tool definitions arrive through it. What the standard does and does not guarantee is set out in the breakdown of MCP as an agent interoperability standard.

Frequently asked questions

Where to start this week

One measurement, then one deletion. Both fit in an afternoon.

Log the full context payload for ten failing calls in your worst-performing agent flow. Count the tokens in each block: system prompt, tool definitions, retrieved content, conversation history. Write the four numbers down. That table is usually the whole diagnosis, and almost nobody has it.

Then delete every tool your agent has not called in the last thousand runs. It is reversible, it costs nothing, and it is the only change on this page that needs no new infrastructure. Re-run your evaluation set and record whether accuracy moved.

Related reading