Budget a tool-using chat request before truncating history. Count instructions, current user intent, prior messages, tool definitions, retrieved content and output reserve using the selected model's accounting. Keep complete tool-call/result relationships and preserve recent corrections. A larger context window can postpone overflow, but it does not decide which evidence is relevant or which instructions still govern the task.
Inventory the request, not only the conversation text
A long report can consume more context than many chat turns, while a large tool catalogue adds tokens before the first call. Images, documents and thinking-related behavior can have provider-specific accounting. Record the exact model and current API contract rather than assuming one universal window size. Prompt caching can reduce some repeated processing costs without making cached input disappear from capacity accounting.
Claude's context-window documentation describes input, output and tool-related accounting and directs developers to model-specific limits and token counting. It also distinguishes long-running context-management strategies such as compaction. Use the current provider guidance for the actual model; this worksheet deliberately uses invented capacity values instead of presenting a current product limit as a permanent constant.
Claude: context accounting, tool use and model-specific limits
A synthetic capacity worksheet
Suppose a fictional model budget is 32,000 tokens and the application reserves 4,000 for output. Instructions use 2,000, tool definitions 3,000, relevant recent history 5,000 and a report result 24,000. The planned input totals 34,000, exceeding the 28,000 input allowance by 6,000. Those numbers are synthetic arithmetic, not tokenizer measurements or a real model specification.
| Component | Planned tokens | Policy |
|---|---|---|
| Instructions + tools | 5,000 | Retain required contract |
| Relevant history | 5,000 | Preserve current correction |
| Report result | 24,000 | Select evidence, disclose limits |
| Output reserve | 4,000 | Subtract from 32,000 capacity |
| Initial input total | 34,000 | Exceeds 28,000 allowance by 6,000 |
Shortening the report to a selected 8,000-token evidence slice would bring planned input to 18,000 in this example. That calculation says the request fits the invented capacity; it does not prove the shortened report supports the answer. Preserve source identifiers, filters, completeness and any facts needed for the user's question. A summary that omits the decisive exception can fit perfectly and still produce the wrong result.
Keep the active task and message relationships coherent
The latest user correction should remain visible with the instruction or state it supersedes. Summarize older decisions into a traceable state record, distinguishing accepted choices, unresolved questions and discarded proposals. Do not let a compressed memory silently turn a tentative suggestion into authorization. Durable user preferences and per-request context have different lifetimes and deletion rules.
Proposed compacted state
current task: compare the revised reporting interval
latest correction: use workspace-local dates
accepted constraints: permitted workspace only; no write requested
evidence: report-7, selected rows, completeness=false
pending tool relationship: call-id-12 paired with its resultFollow the provider's required tool-message structure when removing old turns. Retaining a result without its identifying call, or dropping a result while keeping a dependent assistant claim, can make the transcript invalid or misleading. Treat a tool interaction as a coherent unit unless the API explicitly supports another representation. Keep authoritative application state outside an improvised natural-language summary when it controls later writes.
Evaluate behavior after the budget decision
Create fixtures in which the decisive fact appears early, late and inside a large tool result. Add a recent user correction that conflicts with older history. Compare the selected evidence, tool arguments and final answer after truncation or compaction. A lower input count is not the acceptance criterion; the task must still respect current intent and supported evidence.
No provider token-count or model-quality experiment was executed for this article. The worksheet is a synthetic planning tool, and the inspected history-loading code establishes ordering and limits rather than a complete context policy. For release evidence, count the exact serialized request with the provider-supported method and retain redacted task fixtures. Choose bounded retrieval, summaries and output reserve as explicit application decisions, then measure whether the assistant still completes the intended task correctly.
An AI coach should remember what the user can correct.
Building UltraFit360: the app, coach portal and shared backend
Bring one tool workflow, supported hosts and a sanitized failure trace for a scoped AI integration review.
AptixLabs ai agent development
How should tool-using chat manage context-window budgets?
Budget every request component and output reserve, preserve active instructions and complete tool interactions, then evaluate task behavior after summarization or truncation.
Does prompt caching make input free of context limits?
No. Follow the provider’s accounting; cached tokens can still count toward the context window.
Is fitting the token limit enough?
No. Test whether shortened context still supports the answer and preserves current user intent and tool relationships.