Revenue? $20m. Checked.
Profit? $2m. Checked.
Loan? $9m. Checked.
OK, everything’s done. Time to have a coffee.
Wait. $9m? $9m?
I swear that I saw $9.6m, like literally 15 minutes back.
Wait. Can’t be. This model is strong and should not make mistakes.
No, better to take a look. OK, page 232. Long-term... here you go.
Loan $9.6m.
Wait, where did the $0.6m go? Did I mistype something in the chat? Did the AI misread?
I can’t tell that from the answer on screen. I need to go back through the chat from the top. The number could have been wrong long before the last exchange. Then I’d have to look further back.
Ah, found it. In the final compaction, the AI model decides to write a summary to shorten the context. It changes $9.6m to $9m. You read it right. $0.6 million is gone because of a compaction.
Welcome to the world of compaction.
An AI agent that has been doing a long task starts filling up the window with notes, files and output from tools. At some point it needs space for the next thing. So it may summarize what’s happened so far, keep some other material, and carry on from there.
This blog is about what’s compaction and why it’s a blessing and curse for compliance. In short, compaction saves long prompt and tokens, but it creates other issues that we may not expect. Ultimately, we need to monitor not just the deliverable but the process in long context window with AI model when AI compacts context. Just like what we do with other control tools.
. . .
The case for putting everything in
At the dawn of GPT, people like me struggle to feed long financial statement PDFs into the AI model. There are multiple reasons. First, it does not have a context window that is long enough to feed the whole document, which may run up to 100,000-200,000 tokens (assuming 1 token is about 4 characters). Remember the old days when GPT-3 had about 2k tokens, and ChatGPT about 4k at launch, and we need to constantly start a new window when chat ran long?
Second, it is not clear where the attention of the LLM model will go. It may decide to read the first 1,000 characters and give you a hallucinated summary. Or it may read the first and last 1,000 characters, and decide to skip the middle altogether. Well, telling an AI model to focus on the first 1,000 characters sometimes work. Sometimes we only realized it failed post-mortem. If you wanted your GPT to prepare a correct summary, but somehow it ran off the rails and read only the last 1,000 characters, well, that would be a problem.
The need to read long documents later popularizes the RAG approach. RAG is retrieval-augmented generation. The idea is to cut a long document, in text, into multiple smaller tranches of text documents. Then, instead of feeding the one long text into the model, we would let the AI find the ones that are similar to our query. Maybe based on text similarity. Maybe based on other ranking algorithms or criteria, like BM25. Then you would do a retrieval phase where you’d select the pieces that were relevant to your question. Then your model would only see the relevant pieces. That will save compute and tokens.
If you didn’t find the right piece in the search, then the model didn’t see it at all. It saves a lot of tokens and compute.
The idea works perfectly in theory and mostly in practice. So we see various tools like LangChain, in which various innovative methods are created to find the most accurate and most efficient way to pin down the exact result. This was especially valuable back then, as some users were facing mountains of documents, so we see the rising popularity of Pinecone and other providers, who specialize in providing the vector index, to scale up the search and storage.
It is not without issue though. For one, how to decide on the cut between tranches? Based on the natural stopping point of a sentence? Based on a hard cut-off of word or character count? Or based on the return of a new paragraph? But the field is moving extremely fast. There are now multiple methods to tackle this, like overlapping chunks, semantic chunking and splitting at headings.
A year back, you could already give Gemini 2.5 Pro a million tokens in one go. For a 10-K, that’s plenty of room. Put it in and ask for the figures. (Again, one token is about 3/4 of a word in English. So that’s roughly 750,000 words.)
There’s room for more than one 10-K, or multiple years of board minutes. So the compliance people can ask it to “test the whole population”! You drag in the documents and ask. No need to save all those small chunks or set up the search first. Assuming the bill is not a big concern, of course. Caching can make it cheaper to ask about the same files again. (Or so it sounds. I’d still check whether it read every item.)
I can see why you’d want this. I do research with language models where I am trying to read all these corporate disclosures. It’s easier to just load the entire report. It helps the language model associate a footnote on page 80 with a table on page 12.
Or across a sustainability report and financial statements, for example. Retrieval can do this too, if it finds the relevant passages in both reports. I’d compare them on the same files.
Mind you, a prompt with a million tokens has a lot of stuff that you didn’t type or were possibly too busy to read. You might not notice a hidden line in a PDF from the supplier that tells the model to stop checking and approve the contract, or a line in a financial statement that instructs the model to disregard all the red flags during technical assessment when approving bank facilities. Maybe white font in font size 1. Hidden somewhere at the back of a footnote. Worse, it may be embedded in a picture that models with picture reasoning capacity may digest and blindly comply, even if you search and find nothing like it in the text. That’s what we call prompt injection. That line could change the model’s answer.
The file also contains client data. An external vendor receives the file. With a local model, it may stay on your machine. Check where it goes. Who can see the data? How long are AI vendors going to retain data? An AI company promising “We will not use your data to train our AI model” is not necessarily the same as “We will not use the summarized version or anonymized version or rewritten version to train our AI model.” Note the difference.
But you can have a document in the window and it can still get very little attention from the model.
What the research found
In July 2025, Chroma, which makes AI search tools, tested 18 models. They gave the models simple tasks, like “Find a fact in this long text. Only one.” or “Repeat this text verbatim.”
Performance generally got weaker with longer inputs, though the size of the drop differed across models and tasks. Also depending on whether reasoning is switched on. The term context rot refers to this exact situation and it first shows up in a reply on Hacker News in June 2025. Reasoning models generally perform better, but long contexts still have a worse context rot issue, as the name suggests.
Here are two things that affect compliance. First, when the question used a different form of words from the passage with the answer. For example, if your query is “what’s the liquidity of this client” but the loan document only says a covenant on the $9.6m loan was breached. The bank may decide not to recall the loan because the word “liquidity” never appears. The general rule is that the lower similarity the query word and document word, the difficulty to retrieve and get the correct information increases with the content length. Now, put the compaction here, the problem will be even worse because AI model cannot see the original chat but a summarized version of it, or multiple summarized versions that drift over time.
Second, when there were passages that could have been the answer, but weren’t. For example, continuing from the same illustration, a paragraph of general risk factor in a firm’s annual report may contain every keyword of risk, but it is just referring to the general difficulty of meeting financial obligations that is universal across all companies. The drop was greater in both cases.
Some LLM models often gave you the lookalike as the answer. Claude in the past would be more willing to say they couldn’t find one, if you offer them a way out “Tell me if you cannot find it” in the prompt. There are tons of lookalikes in a compliance file. And your question will seldom be exactly as worded in the ledger.
Something that feels a little more compliance-adjacent. It’s a monitoring test done by Sam Martin (Anthropic Fellow) and Fabien Roger (Anthropic) in a May 2026 paper. They benchmarked Claude Opus 4.6, GPT 5.4 Thinking, and Gemini 3.1 Pro. All three missed subtle unsafe actions 2 to 30 times more often when preceded by 800,000 tokens of harmless activity, compared with seeing the unsafe actions on their own. Bear in mind that the history here came from a coding agent. We would have to run our own test to get a number for a loan review.
And detection also depended on where in the log the violation was. The middle of the log was where the violations were least likely to be detected. Probably because LLM attention is strong at the head and tail, and weak in the middle.
For the monitoring study, the model had the best success identifying the violation immediately after it happened. More reasoning also helped. Frequent reminders helped most models with obvious violations, but did little for subtle ones.
Some researchers also find the order of prompt and content would trigger a difference. Some models like to see context first. Some models like to see the prompt first. Some even prefer to see question in xml format.
You should treat the model’s review as any other control. Plant known exceptions at the beginning, middle and end of some actual files (of various lengths). This eval is gold because it tells you what model sees and where it fails. A small error in one agent can be carried over to the next agent down the pipeline and it is difficult to track and trace where things go on. Count the misses and false alarms. Keep every run. Keep the failed ones too.
Split the review by month, entity or contract, and check anything that crosses from one part to another separately. I’d ask for the actual quotes before the verdicts, do a text search to find them in the source, then check the model’s interpretation.
What happens when the agent summarizes
So, back to the $9 million.
Anthropic describes compaction keeping architectural decisions and unresolved bugs and implementation details, and removing redundant tool output. They also warn that overly aggressive compaction may leave out some detail that becomes important in hindsight. Different AI companies use different ways to compact.
Take an example with 3,000 invoices and four compactions. Invoice 2417 has been cleared. You’re using last year’s policy, though. It should have failed. If the summary just says “Batch 1: no exceptions”, and leaves out which invoice and which policy, the next step takes that mistake as fact.
And there’s the issue of documentation. ISA 230 requires the file to be such that an experienced auditor, unfamiliar with the engagement, can understand what’s happened and what’s been found and why. It also requires auditors to document the identifying characteristics of the individual items which have been tested.
External logs and source files may survive compaction. I’d check those records. Keep all input, the PDF text, the full prompt, model version and complete output for every call. The summary comes from the model I’m checking. We need the underlying record.