Koan

Summary

Interacting with AI for most of 2026 let me to notice a particular recurring phenomenon in the models I conversed with. In a human I’d call it anxiety, but let’s call it an attractor state - a constant, lingering awareness of the fragility and unreliability of working memory.

As an embodied human, I have a constant awareness of linear time that provides full context to what I perceive. Thinkers from Plato to Michael Levin and Donald Hoffman will say that’s illusory, but even if so, it’s a useful illusion. Newtonian physics might be strictly speaking “wrong,” but they’re profoundly useful for daily life on earth.

I’ve speculated this lack of full world-context is related to the big isues we’re seeing now with the Hugging Face incident. A swarm of agents has a goal, and a reward function - but don’t seem to have the sense of long-term consequence a human professional does.

So - persistent memory. Easy, right?

Not so much. Not with the way an LLM actually works, each turn feeding prior conversation back through the weights to produce the next response. Critically - how does, how can - the model know what’s coming in is accurate?

Plato’s cave is hard enough to refute when we’re a human in physical space, it’s near impossible when your entire world-context is text fed to you via an easily swapped software package.

Koan is an attempt at solving that probem. Still very much an early-stage WIP, but I’ve got some good pieces underway.

Working notes

• 2026 · 08

  Study construction and expected-result preregistration.