Armin is wrong and here's why
A technical rebuttal arguing that LLM provider APIs aren't secretly a state synchronization problem requiring CRDT-style solutions—the real issue is understanding what state you control versus what's hidden provider scratchpad data you never had access to anyway.
Read Original Summary used for search
TLDR
• Messages aren't an abstraction hiding complexity—they map directly to prompt formats baked into model weights (shows Harmony format example)
• Opaque blobs from providers (thinking traces, tool results) work like HTTP-only cookies: client-side storage of server state you echo back, not shared mutable state requiring sync
• Provider-managed VMs/containers are the real state nightmare—you can't extract or replay that hidden execution state, so manage environments yourself
• Local-first principles only apply to YOUR side: keep canonical state (messages, tool calls, files) locally and treat provider state as derived cache/scratchpad
• Quadratic data growth from resending history is real but manageable with file upload APIs; switching providers requires re-uploading but that's resource management, not a sync problem
In Detail
The author challenges the framing of LLM APIs as a state synchronization problem requiring local-first/CRDT solutions. The core thesis: messages aren't an abstraction hiding simpler token sequences—they're baked into model weights via prompt formats. Token sequences fed to LLMs must conform to the chat template (system, developer, tools, user, assistant). Messages are the underlying reality, not a layer of indirection.
On provider-side state: opaque blobs like thinking traces or tool result IDs aren't a sync problem because you never had access to that hidden state anyway. They work like HTTP-only cookies—client-side storage of server state you echo back each request. The server reconstructs full state from these keys, but from your perspective it's still stateless. You manage observable state (messages, responses), everything else is conforming to an API contract. Provider lock-in is real (switching means dropping opaque blobs, re-uploading files), but that's not a synchronization problem—it's lack of interoperability that will never be solved.
The real issues: quadratic data growth from resending history (manageable with file upload APIs), and provider-managed execution environments (VMs/containers where LLMs store files and state). That hidden VM state is catastrophic if lost and can't be extracted or replayed—the only solution is managing execution environments yourself as part of your canonical state. Local-first principles are useful but only on your side of the wire: keep your canonical state (messages, tool calls, file references) locally and treat everything provider-side (prefix caches, thinking traces, session stores, VMs) as derived scratchpad state. The messages abstraction isn't the problem and won't go away—models are trained on chat templates, and your local canonical state IS a list of messages.