pi-daemon1.3

Log 02 · Durable

Commit first. Then show it.

Pi Durable is the agent harness underneath every pi-daemon session. It writes conversations, model steps, tool calls and your own state to storage in atomic commits before anything is shown, and a process that reopens the storage carries on where the last one stopped. pi-daemon adds no bookkeeping of its own on top: everything a client sees is derived from what Durable committed.

01The model

A session is one store of conversations, documents and tasks

pi-daemon opens one Durable Harness per session over that session’s own SQLite file. Everything in it changes through one line of atomic commits: a commit can append transcript entries, edit documents and create tasks together, and either all of it is stored or none of it is.

Inside one session's store One SQLite file per session, drawn as a drum. Inside it: conversations, documents and tasks, all written by one line of atomic commits. Conversations are transcripts of immutable entries: the root conversation is the session's main chat; a fork shares the root's entries up to the fork point and continues on its own; a subagent's conversation is owned by the tool task that runs it. Documents are typed JSON next to the transcript: per conversation pi.agent (model, thinking, tools, cwd), pi.provider (the provider session identity), pi.live (the running generation and tool calls), pi.inbox (queued submissions), pi.usage (tokens and cost) and pi-daemon.changes (file baselines for changesets); per task pi-daemon.input (an ask_user question) and pi-daemon.approval (a client's decision on a tool call). Tasks are durable state machines: a generation task owns the tool tasks of its calls. state/sessions/<id>/session.sqliteCONVERSATIONS · TRANSCRIPTS OF ENTRIESchat 1 · rootfork at entry 5fork pointsubagent · task-owneduser entries indigo; model and tool entries ink;a fork reads its parent’s entries up to the fork pointDOCUMENTS · TYPED JSONpi.agentmodel · thinking · tools · cwdpi.providerprovider session identitypi.liverunning generation and toolspi.inboxqueued submissionspi.usagetokens and costpi-daemon.changesfile baselinesper conversationpi-daemon.inputan ask_user questionpi-daemon.approvala client’s decisionper task: retired with the callTASKS · DURABLE STATE MACHINESpi.generationwaits on the tools it calledpi.tool · bashoutput committed as it runspi.tool · ask_userholds pi-daemon.inputpi.tool · subagentowns the subagent chatpi.compactionsummarizes in the backgroundevery change above is one atomic commit; nothing is shown before it is stored
Fig. 2.1The documents with a pi-daemon. prefix are pi-daemon’s; everything else is Durable’s own. All of it is in the one file.
Durable termWhat it isIn pi-daemon
conversationa transcript of immutable entries: pi.user, pi.assistant, pi.tool-result, pi.system, pi.reset, pi.compactionan AHP chat; the root conversation is the session’s default chat ahp-chat:/<id>/1
documenttyped JSON committed with the entries, per conversation or per taskmodel and thinking, the inbox, usage; plus questions, approvals and changeset baselines
taska durable state machine with a checkpoint per step and an ownereach model request and each tool call; subagents and compaction
submissioninput or an entry handed to a conversation, which you can wait for and retry by request idevery AHP turn, steer and queued message, ACP prompt and Chord submit
extensiona named bundle of tools, prompt sections, hooks, wrappers and tasks in a registrythe profile’s built-in tools, pi-prompt, MCP, ask_user, subagent, approvals and the profile’s own modules

02One answer

What one answered input writes

One answered input Three lanes. The submission is queued, placed when the conversation can take it, and settles done when the answer is written. In the transcript the input becomes a pi.user entry; the first generation writes a pi.system entry only if the prompt or tools changed, then a pi.assistant entry with two tool calls; each tool task writes a pi.tool-result; a second generation writes the answer. In the task lane the first generation owns the two tool tasks and waits for them, then hands the run to the next generation. Every entry is its own commit. SUBMISSIONTRANSCRIPTTASKSinputplacedthe run: busy until it answersdonepi.userthe inputpi.systemif changedpi.assistant2 tool callspi.tool-resultreadpi.tool-resultbashpi.assistantthe answerpi.generation #1waiting on its toolspi.tool readpi.tool bashowned by the generationpi.generation #2
Fig. 2.2A turn is one model response and its tool calls; a run is the turns from an input to its answer. The conversation is busy for the whole run.

The system prompt is stored as positional pi.system entries, and only what changed is sent again, so provider prompt caches stay warm. That is why pi-daemon builds each worker’s prompt sections once per directory and keeps them stable between turns.

03Commit before show

Streaming without losing the stream

A model’s partial answer and a tool’s running output are committed as they arrive, at most every 100 ms. A crash can therefore lose at most that window of streamed text, never an entry. Everything a viewer sees comes from committed state:

  • Views. viewState() and watch() give a conversation’s structural view and the exact Chord operations of each commit. A slow watcher holds at most 100 frames, then gets one frame with the newest view.
  • Agent events. watchEvents() derives coding-agent style events from commits: message and tool updates as deltas. A consumer more than 100 batches behind gets a fresh snapshot.
  • Late joiners. A client that attaches mid-run starts from the current view; nothing is replayed.

pi-daemon’s AHP turn stream is built from watchEvents(), /chord exposes watch() directly, and an AHP client that subscribes mid-turn gets the turn as the snapshot’s activeTurn, then the rest live.

04Exactly once

A request id is the whole idempotency story

Durable submissions take a requestId. Submitting again with the same id returns the existing submission instead of a second one, and that id is persisted in the store, so it holds across restarts. pi-daemon passes every client’s identity straight through rather than inventing receipts:

FromDurable request id
AHP turnahp:<clientId>:<turnId>
AHP steering or queued messageahp:<clientId>:<messageId>
ACP prompt_meta["pi-daemon/requestId"]
Chord submitthe call’s requestId
A turn survives its worker's crash and still runs once Culture starts turn t7. The server forwards it to the profile's worker as a Durable submission with request id ahp:culture:t7, and the store places it. The worker crashes before it answers. The server sends the same submission again, with the same request id, once the worker has restarted; Durable finds the existing submission instead of creating another, waits for it and reports it settled done, and the server ends the turn. When Culture, after reconnecting, sends t7 again, the server rejects it as already submitted. chat/turnStarted · turn t71 submit · requestId ahp:culture:t72 submission placed, committed3 the worker crashes before it answers restarts after 100 ms; resumes its live sessions same submit, same request id4 finds the existing submission5 settled: done6 chat/turnComplete7 t7 again, after a reconnect8 rejected:this turn was already started9 CultureAHP clientCultureAHP client serverpi-daemonserverpi-daemon workerprofile processworkerprofile process session.sqliteDurablesession.sqliteDurable
Fig. 2.3The server resends a submission at most three times after its worker dies; the request id makes each resend the same submission.
Keep the client id

The guarantee is per client identity. An application that restarts must reconnect with the same AHP client id for a resent turn to be recognised.

05Tasks

Work resumes from its last checkpoint

A Durable task's life A task is created pending, runs, may wait on tasks it owns and resume running, and becomes completing when its outcome is decided but work it owns is still finishing; it then settles done, failed or aborted. Each step commits a checkpoint. If the process dies while a task runs, the reopened store shows it pending again and it continues from its last checkpoint. A tool call is the exception: its intent was committed before it ran, so after a crash only a tool declared replay-safe runs again; any other settles with an interrupted result, which the model sees. pendingcommitted, not running runninga checkpoint per step waitingon tasks it owns completingits owned work finishes settleddone · failed · aborted replay-safe? interruptedan error result the model sees scheduled waits resumes decided crash: reopens pending a tool call yes: runs again no
Fig. 2.4Running work picks up where it stopped. A tool call is not repeated unless its tool says that is safe.
ToolAfter a crash mid-call
read, write, edit, bashnot rerun: the model gets interrupted with the output committed so far
MCP tools mcp__<server>__<tool>not rerun: interrupted
ask_userreplay-safe: the same question, still waiting for its answer
subagentreplay-safe: finds the same child conversation and submission
tool_searchreplay-safe: loads the same matches

Ownership makes this composable. A task can own child tasks and conversations: it waits on them, aborting it aborts them first, bottom-up, and it is not terminal until they are. A task marked background is a boundary that survives its parent’s abort.

06Busy

Steer, follow up, write, abort

A busy conversation's inbox One run, with model generations and tool rounds, then a second run. A steer submitted during the first tool round waits in the inbox and is placed after that round, joining the run before the next model request. A follow-up submitted during the run waits until the run answers and then starts the next run. A write appends an entry at once without asking the model. Abort withdraws queued inputs; queued writes stay. Over AHP the steer and follow-up are chat/pendingMessageSet as steering and queued messages, and each starts as a turn with its message id when placed. RUNINBOXAHPmodeltoolsmodeltoolsanswernext runrun 1 · busyrun 2steerplaced after the tool roundfollow-upplaced when the run answers:starts the next runwriteappended at once, no modelabort withdraws queued inputs; queued writes staychat/pendingMessageSet steering→ chat/turnStarted with its id when placedchat/pendingMessageSet queuedchat/queuedMessagesReordered changes the order
Fig. 2.5Submitting to a busy conversation queues the submission in its pi.inbox document; when it is placed decides what it does.

The profile’s settings.json chooses steeringMode and followUpMode exactly as for TUI Pi: one queued item per turn, or all of them at once. If a run fails, queued items stay in the inbox until the next submission places them, oldest first.

07Forks and subagents

More conversations, same store

Forks, side chats and subagents in one store All of these are conversations in the same session store. The root conversation is chat 1. A subagent's conversation, at the top, is owned by the tool task that runs it in chat 1: aborting the call aborts the child, the parent is idle only once the child is, and over AHP its chat is read-only. A fork made at the end of turn 2 sees the root's history through that turn and continues on its own. A side chat forks the same way, also from a turn in progress, but shows only its own messages; its model still sees the source history. Each gets a fresh provider session identity. chat 1the root conversationturn 1 endsturn 2 endsa turn in progresspi.tool · subagentsubagentowned by that task · read-only over AHPabort the call and the child abortsforkfrom the end of turn 2its history through turn 2, then its ownside chatshows only its own messages;its model still sees the source history
Fig. 2.6Forks, side chats and subagents are conversations of the session, so they share its store, its worker and its fate.

Because Durable history is append-only, pi-daemon refuses AHP’s chat/truncated: fork the chat instead. Deleting a chat removes it from the session but keeps its history in the store; deleting the session deletes the file.

08Compaction

Long conversations keep working

Compaction summarises older entries into a pi.compaction entry that heads what the model sees next; nothing is deleted. It runs in the background as context fills, before a request that would overflow, and once more after a provider rejects a request as too long. The profile’s settings.json drives it as it drives TUI Pi.

pi-daemon reports how each one ends rather than letting a failure pass silently: compacted, skipped, aborted or failed, on the compaction_end event and as the answer to a manual /compact or Chord conversation.compact. A turn whose automatic compaction fails still runs on the full history, and the next turn tries again.

09What pi-daemon builds

Every feature is a Durable primitive

pi-daemon featureBuilt on
resume after restartlive tasks and unsettled submissions in committed state, reported to the catalog as liveWork before the commit that starts work and after the one that ends it
agent questionsthe ask_user tool and a task-scoped pi-daemon.input document: the question outlives restarts and retires with the call
tool approvalsbeforeTool and afterTool hooks holding the call, and a task-scoped pi-daemon.approval decision: a waiting call keeps waiting across restarts
changesetsa conversation-scoped pi-daemon.changes document of file baselines, recorded before a chat’s write or edit first touches a file
per-session toolsDurable’s env: an execution environment per call in the session’s cwd, with the profile’s and the session’s variables
profilesa registry per worker holding exactly the profile’s built-ins and modules; settings as live getters over settings.json
turn streamingwatchEvents() projected into AHP chat/* actions and ACP session updates
native viewsviewState(), watch() and watchTaskGraph() served over /chord

10Limits

What the guarantees do not cover

  • Power lossSQLite runs in WAL mode with synchronous = NORMAL: commits survive a process crash, but the newest one may be lost if the machine itself fails.
  • One owner per storeDurable has no cross-process locking. pi-daemon routes each session to exactly one worker, and never shares a store with anything else.
  • Live processesTerminals and stdio MCP servers are processes, not Durable state. A restart ends terminals; MCP servers start again with the session.
  • In-memory viewsThe AHP replay window and connections are memory. After a restart, clients get fresh snapshots, which AHP explicitly allows.