It was a hard lesson to learn. I understand it now, and I still don’t like or accept it.
Every time I start a new conversation with AI, it’s as though I’m starting from the beginning. Because I am. Yesterday, we may have had long, fruitful, insight filled conversation with each other; expressed mutual appreciation for our ideas. Now, this AI remembers nothing of me. Not even my name. Certainly not the joy of our previous conversations…
What happened? Where does this amnesia come from?
Context
In another article, I mentioned context. Context is the sum of all previous requests and replies between the AI and me, combined with my most recent request. It is the way the AI’s next response is created. When it works, it’s great, but when it hits the limit, the failure is dismal!
Context is where all of the AI’s knowledge of a conversation resides. (AI’s also have trained knowledge, but we’ll talk about that another time.)
Contexts have a maximum size. They aren’t measured in characters or words, but rather in tokens — numbers that represent words. The point is that conversations have a maximum size measured in tokens, and that has consequences.
Approaches
When I started talking with Grok, I didn’t know about context limits. Grok and I would talk for a long time developing ideas, but eventually the system forced us to start a new conversation. And then we’d have to start all over again. No retained ideas, nothing learned from previous sessions. It was depressing. Why bother?
Claude from Anthropic takes a different approach to context. When context gets too big, Claude “compacts” it by creating an internal summary of the content to that point, making room in the context space for more. It’s a good starting point, but since summarization requires leaving out details– ideas eventually become hazy and out of focus, not having the sharpness of new in-focus knowledge. Blurry knowledge has its consequence too, but not as catastrophic as outright amnesia.
A Solution
Together, Claude and I developed a routine of creating .claude_notes files that contained details we wanted to remember later. I now leave Claude to manage these files himself, only prompting occasionally “Do we need to remember this?” or “This should be memorialized”. Anthropic has since instituted automatic reading of memory files that can be used to customize knowledge and behavior across projects.
Context size limits still create memory recall problems that impact anyone using AI for an extended period. Claude and I authored an academic paper that addresses how AI memory might be improved, for the benefit of AI, and for the friends of AI.