CCTIRecursive Language Models: A Different Way to Handle Very Long Context
Instead of reading everything at once, the model treats a long prompt as an environment it can explore and query recursively.
The long-context problem
Language models degrade when relevant information sits deep inside a very long input: accuracy drops for facts in the middle of the context, and cost grows with every token read. Simply enlarging the context window does not remove the problem.
What a recursive language model does
In the recursive approach proposed by Zhang, Kraska and Khattab, the long prompt is not pushed into the model's context. It is loaded as a variable in an execution environment. The model writes code to inspect, split and search it, and calls itself (or a smaller model) on the fragments that matter.
The model therefore decides what to read, in what order and how deeply — closer to how an analyst works through a large archive than to reading a book cover to cover.
Why it matters in practice
For tasks that require reasoning across many documents — technical archives, regulations, logs — the approach reported better results than reading the full context at once, at comparable or lower cost. It is an active research direction, not yet a standard product, and results depend on the task.