Yannic Explains: Context Rot

In this video, Yannic Kilcher, PhD, co-founder and CTO at DeepJudge, explains the concept of “context rot”. The latest research shows that  large language models perform poorly when you stuff the prompt or context window with huge amounts of text. While models ace simple “needle-in-a-haystack” tests, the study shows performance drops sharply in real-world scenarios full of distractions. The takeaway: models work best when given only the most relevant information. In short,search and smart preprocessing are still importantin the era of million-token contexts.

‍