The State of Context Engineering

It's nice that this article goes beyond the "context too long" scapegoat diagnostic. There is more to it than just reducing content in context. LLMs attention mechanism works by finding graphs of related words and concepts and it works best when the relevant relationships are close together in the text. Length matters but how well the information is organized matters as well. It's useful to be familiar with the concept of dependency length. The two sentences in this image ( https://benoitessiambre.com/img/dependencylength.png ) have the same words and same meaning but sentence C is organised to minimize the distance between related words. This can also apply to paragraphs and sections. Ordering them so that related information is recursively closer together, helps AIs think better (also helps humans, Dependency Length Minimization, DLM is an idea from psycho-linguistics). This is one reason progressive disclosure and sub agents are important. Including unnecessary information spreads apart surrounding information and makes it more difficult for AIs to figure things out. For eg. optimizing skills, it can help to put more important information at the bottom of a skill. Usually after a skill is loaded, the AI will start “thinking”, talking to itself to try to reason through a problem. The thinking block will have an easier time considering the text closest to it, the last part of the skill, than information farther up. Thinking blocks can produce a lot of text (a lot of it often hidden from users) that also spreads information apart. Sometimes, when an early instruction is still relevant but there’s been a lot of thinking and discussion added to the context since, it’s worth repeating a summary the early information ( keeping in mind that too many reminders can also spread other information apart ). I think however, LLMs are tuned to overweight very early information so that they don't disregard system prompts. You might be able to exploit this by placing some important facts near the very beginning of a conversation. If you've ever written compression algorithms where repeated blocks are referenced instead of encoded, you'll be familiar with how related information being clustered allows for better compression. It's another example of the parallels between intelligence and compression and it hints at the information theoretic aspect of DLM.

I like the idea of VictorTaelin's UniiChat [1] which summarizes every message into a single line and on top of that a binary tree of lines combined from lower levels. The context is then a view of that tree with more detail in recent messages and less detail in historical messages, plus a `zoom` tool that enables the agent to navigate the tree and read messages. An implementation of this idea based on the pi harness: https://github.com/jonaslsaa/pi-optchat [1]: https://gist.github.com/VictorTaelin/91837951a5ce5b38f341ec1...

Is it just me or others also feel like “no respect” for this kind of knowledge? Before I was proud to know about TCP connection protocols, 2FC, parsers, compilers… and much of that knowledge had applications on the software I had to build. But nowadays I have very little interest in knowing how “to prompt better” or why after around 100K tokens of context things start to suck for agents, or compiling tons of MD as “skills”. I don’t find that stuff challenging or interesting (granted, I don’t mean that I know all about it) and I have the feeling that it’s not like learning about operating systems or relational dbs. I think part of my overall rejection to LLMs comes from this (of course the actual knowledge of how LLMs work under the hood is super interesting, but not their usage Id say)

> Several times, Gemini 2.5 became convinced it needed to get tea, even though it > was playing the older game. The tea task for FireRed was in its weights, so it > poisoned its context. It spent hours trying to find or give the guard tea. Once > it had been added to its to-do list, the model could not get past it. I don't understand why this is called context poisoning when the information was in the weights

> As we focus more on memory, we also need to think about strategic forgetting. Over time, overly precise memories become conflicted and messy, and nobody has yet figured out how to handle this. > I think that’s the context problem of tomorrow. This passage is explains why humans have imperfect memory. Seems the perfect memory we wish LLMs had impedes creativity and is maladaptive. Strategic forgetting helps put the non-deterministic machine to best use. LLMs don't need perfect memory; that's covered by the computer itself. I like that this justifies the continuing relevance of context engineering - a seemingly dated term - in the face of trends like loop engineering.