Architecture

Cacophony is a distributed control plane for agent work. The same design runs as a single laptop node, a direct-mesh cluster, an AKS warm standby, a Codespaces or dynamic-node fleet, or a transient microVM-backed worker pool. The core rule is consistent across all topologies: local clients talk to the local daemon; daemons coordinate over authenticated cluster links; durable state is append-only or git-backed and can be inspected outside the UI.

Layered System Map

System mapClients only ever call their local daemon; daemons talk to each other and to the beads primary over mutual TLS.

Layer	Primary components	Responsibility
`Operator surfaces`	CLI, TUI, web dashboard, desktop app, native macOS app, Android and Wear OS, iPhone and Apple Watch, MCP plugins	Human and agent-facing control surfaces. They are clients of the daemon API rather than independent authorities.
└ `Local node services`	`caco-daemon`, optional `caco-bd-daemon`, `caco-web`, sidecars, TTS/audio helpers	Serve local loopback APIs, supervise work, expose health/logs, and proxy to authoritative services.
└ `Cluster coordination`	mTLS daemon mesh, PKI authority, feed replication, peer health, config hash comparison	Coordinate nodes, replicate events, and distinguish maintenance windows from unexplained outages.
└ `Project state`	Canonical project checkouts, agent worktrees, beads git branch, summaries, scratch, releases	Keep source, task state, session evidence, and runtime artifacts inspectable and recoverable.
└ `Execution backends`	Agent checkouts hosted in Herdr (default) or tmux, local build/test queues, dynamic nodes, AKS pods, Codespaces, microVM/OCI isolation	Run agents and validation workloads with explicit resource, topology, and trust boundaries.

Runtime Services and Daemons

caco-daemon

The node authority for agent lifecycle, project checkout refresh, feed, messages, scratch, choices, UI snapshots, queued build/test jobs, and cluster peer coordination. It owns the local bearer-token loopback API and the mTLS cluster API.

caco-bd-daemon

Exclusive SQLite owner and standalone beads authority when beads.primary uses the service-object form. The main daemon never opens project beads stores in this mode: the active-primary host proxies to the standalone loopback listener, and remote peers proxy to its advertised cluster mTLS endpoint. It can be restarted independently with caco bd daemon restart.

caco-web

Browser dashboard service. It serves static assets and proxies daemon APIs, including bounded snapshot and SSE paths that turn daemon restart windows into explicit degraded UI states rather than browser-level noise.

Sidecars and service manager

Sidecars expose service health/log contracts. The native lifecycle supervisor manages foreground loops through caco up, caco down, caco restart, and caco service status.

Audio/TTS/STT helpers

Speech surfaces route through daemon-aware configuration for output/input devices, local or Pulse-routed playback, STT capture sources, VAD, and provider-backed or command-template adapters.

Certificate and relay services

caco cert serve can act as a credential-light PKI authority endpoint, while relay-capable nodes support constrained dynamic/container nodes that need an outbound peer allowlist.

Pi agent driver (caco-picophony)

Drives headless Pi agents over Pi's JSON-RPC — protocol, session reader, and backends for spawning and observing Pi worker sessions — used by managed Pi worker runtimes.

Embedded mobile control planes

caco-embed-ios and caco-embed-android build the daemon core as a staticlib with C FFI so the iOS and Android companions can run an in-process control-plane daemon on-device, not only as remote clients.

Listener and Trust Model

Every first-party service follows the dual-listener pattern. Local clients use loopback HTTP with bearer tokens. Peer daemons use HTTPS with mutual TLS. This lets the CLI, TUI, web, mobile, wearable, and MCP plugins share one API shape while preserving different trust boundaries.

Path	Transport	Authentication	Typical users
Local loopback API	HTTP on `127.0.0.1`	Bearer token scoped to node/operator/agent	CLI, TUI, web proxy, local agents, MCP plugins, companion apps over local tunnel
Cluster API	HTTPS on configured host/port	Mutual TLS using the cluster CA	Peer daemons, remote beads proxying, remote spawn forwarding, feed replication
Bootstrap/PKI path	HTTPS or pre-provisioned files	Bootstrap token plus pinned CA when needed	Fresh secondary nodes, dynamic nodes, cert issuance/pull/distribution
Private cluster inspection	`kubectl exec`, SSH, Codespaces tunnels, or local port forwards	Host/cloud-native access plus daemon token	AKS, container, Codespaces, and operator-private dashboard/TUI access

Control-Plane Data Flow

Request pathOne request from a client to a receipt: validate, proxy or execute, persist first, then replicate and publish.

Client request: a CLI/TUI/web/mobile/MCP client calls the local daemon. The client does not write directly to peer state or to the beads git branch.
Local authority decision: the daemon validates config, auth scope, project resolution, concurrency limits, and maintenance windows.
Proxy or execute: local commands execute on the node; beads operations proxy to the active beads authority; remote spawns forward to the target node; web snapshot/SSE requests proxy through caco-web to the daemon.
Persist first: feed, message, and agent mutations are recorded to their durable stores; beads mutations commit to the authoritative SQLite/WAL transaction before downstream notification.
Replicate and snapshot: feed events replicate at-least-once to peers. The beads authority exports a deterministic one-row-per-bead JSONL snapshot to its Git branch only when content changed; JSONL remains a rebuild and recovery source rather than the live query path.

State, Sync, and Authority Strategies

State family	Authority / storage	Sync strategy	Failure behaviour
`Feed and messages`	`daemon/feed.jsonl`, `daemon/undelivered.jsonl`, `daemon.db`	At-least-once peer forwarding with event-ID dedupe and bounded retention.	Undelivered events retry; UI should treat gaps/degraded backend state as visible diagnostics.
`Beads task board`	Authoritative SQLite/WAL plus a deterministic Git-owned JSONL snapshot; optional exclusive standalone `caco-bd-daemon`	Active primary serves all reads and writes from SQLite; other daemons proxy. Dirty exports update Git only when content changed, preserving inspectable history and recovery evidence without putting Git on the request path.	Restart windows surface as maintenance. Store ownership fails closed, expensive reads have bounded admission/deadlines, and a recovered former primary is fenced import-only until it proves catch-up before future promotion.
`Independent beads mode`	Experimental local git-backed stores per node	No force reconciliation; divergence is reported visibly.	Used only when explicitly configured; sync drift degrades rather than overwriting another node.
`Project source`	Daemon canonical checkout plus per-agent worktrees	Checkout refresh fetches/rebases when safe, preserves local blockers, and reports checkout-health issues.	Dirty/untracked project-owned blockers stop refresh; narrow safe cleanup exists for target-branch-tracked repo-owned blockers.
`Agent sessions`	Agent directory, Herdr workspace or tmux session (none for dormant and daemon-hosted Pi sessions), status rows, summaries	Lifecycle hooks, heartbeat/reconcile loops, deterministic persistent IDs, and optional recorded summaries.	Persistent agents are reconciled/restarted by policy; raw runtime exit is not success without lifecycle completion/reintegration.
`Config`	`$CACOPHONY_DIR/config.yaml` plus imported project `.cacophony/` fragments	Template/import materialization, config hashes, validation, and optional distribution commands or Kubernetes ConfigMap projection.	Missing optional imports warn; required imports fail. Runtime skew and config mismatches surface through status/doctor rather than silent drift.
`Summaries and artifacts`	`.cacophony/agent/.../summary/` files published to `cacophony-state`, plus legacy default-branch records	Ordinary direct reintegration extracts summaries/artifacts into the state branch, assigns final pending-summary indices under the publication lock, and strips those paths from the code tree before publishing `main`.	Summaries remain durable operator evidence for reintegrations, observation loops, screenshots, and validation artifacts.

Deployment Topologies

Single node

Everything runs on one host: daemon, optional beads daemon, web, TUI, agents, and queues. This is the same API and persistence model as a cluster, with localhost as a semantic node.

Direct mesh cluster

Static nodes each run a daemon and talk over mTLS. One node may be beads primary, another a controller, and others worker/frontend hosts. Peer health expectations mark known maintenance drift without disabling probes.

Authority + warm standby

Beads and PKI authority can be separated from workers. AKS can use the canonical beads branch as warm standby while avoiding autonomous bead-writing agents during normal operation.

AKS/container deployment

Pods run the daemon with external config projection, PVC-backed runtime state, live binary updates under /var/lib/cacophony/bin, private kubectl exec inspection, and Helm/config rollout guards.

Codespaces/dynamic nodes

Repo-owned Codespaces and dynamic-node templates join through canonical caco node join / caco up flows, using explicit bootstrap URL/token material and externalized secrets.

Relay / constrained egress

Dynamic/container nodes that cannot freely dial the whole mesh use configured relay peer allowlists and advertised public cluster ports rather than ad-hoc ingress hacks.

Transient compute

agents.transient_compute lets caco agent new --compute or queued dispatch launch short-lived provider-neutral workers with launch-spec TTLs, budgets, retries, SKU/profile allowlists, and artifact import.

MicroVM isolation

Optional backends such as cloud-hypervisor, firecracker, kata, or OCI isolate managed agents with projected mounts, resource ceilings, daemon auth boundaries, and first-party validation reports.

Execution and Agent Lifecycle

Managed agents run in isolated project worktrees under the agent directory. The daemon materializes profile instructions, runtime plugins, environment, hooks, and a Herdr workspace or tmux session (or a terminal-less Pi daemon session); then lifecycle surfaces decide whether work is in progress, waiting, blocked, reintegrated, discarded, or pruned.

Agent class	How it runs	Lifecycle contract
One-shot workers	Claim/dispatch a bead, edit a worktree, validate, reintegrate/complete.	Completion or reintegration is the success path; raw runtime exit is not success.
Persistent controllers/specialists	Daemon declarations with restart policy, deterministic IDs, daemon heartbeats, and direct message delivery.	They stay alive and use mid-flight reintegration for safe docs/control-plane changes; they do not call complete during normal operation.
Surface specialists	Dedicated loops for caco-web, caco-macos, caco-android, caco-tui, transcription, wearable, etc.	Observe their surface, file/fix focused evidence-backed beads, land slices, then return to surface monitoring.
Transient agents	Spawned on local, dynamic, cloud, or microVM compute according to explicit scheduler policy.	TTL, retry, budget, quarantine, artifact import, and orphan reaping are part of the launch contract.

Frontend and Companion Surfaces

Surface	Data path	Architecture notes
`CLI`	Local daemon HTTP JSON	Canonical control surface. MCP schemas and plugins are generated from CLI metadata, not maintained as a parallel API.
`TUI`	Daemon snapshot/SSE plus live agent terminals (Herdr or tmux) and structured Pico sessions	Ratatui-first dashboard with optional kitty/Ghostty bitmap effects, real-dashboard benchmark harnesses, summaries browser, bead views, performance/errors diagnostics, and workspace tabs.
`Web dashboard`	`caco-web` static shell + daemon proxy	Workspace panes, agent terminals, source views, summaries, logs, release state, and degraded backend handling through bounded snapshot/SSE sentinels.
`macOS companion`	Native Swift app to local daemon/token and command socket for QA focus	Uses isolated Test/Canary bundles for automation, distinct bundle IDs/keychains/sockets, command-palette and pane-navigation smokes, and cloud/local builder refresh paths.
`Android companion`	Mobile API client plus remote-builder QA path	Overview, Nodes, beads, agents, chat, terminal access, jobs, logs, notifications, releases, and diagnostics share the same control-plane concepts; bead detail editing supports canonical mobile updates for title, description, type, priority, status (excluding computed blocked), assignee, dependencies, labels, spoken name, and `target_project` project moves while emulator work stays off shared macOS hosts. Terminal access now uses the Compose-hosted Termux `TerminalView` renderer for both a local Android shell and remote agent PTY bridge, with the source-only smoke check covering the local session factory, remote PTY URL helper, and remote bridge contracts.
`Desktop app`	`caco ui` (Tauri) around a loopback `caco-web`	Keyboard-first native window on Linux and macOS; it owns a local dashboard on port 11801 by default or connects to an existing one with `--web-server`.
`iPhone and Apple Watch`	Swift companion to a daemon profile	Agents, beads, chat, inbox and choices on the phone; complications and on-wrist choice triage on the watch. See iOS & Apple Watch.
`Wearable`	Companion-oriented compact routes	Small-screen feed/timeline, notifications, quick status, and high-signal controls mirror caco-web's visual grammar where practical.
`MCP plugins`	Claude-compatible JSON-RPC stdio servers invoking CLI command families	First-party plugin packages expose `caco bd`, `caco agent`, `caco msg`, build/test queues, scratch, choices, projects, and aggregate tools.

Beads Authority and Failover

Beads are the unit of work. One authoritative primary owns them, usually a standalone caco-bd-daemon, and every other node proxies to it. Its SQLite/WAL store is the runtime authority and the JSONL export is the Git snapshot; the main daemon is never a fallback database owner. Failover is deliberately conservative.

Each node's beads role over time. A standby is fenced and import-only: it applies the authority's changes but never writes. Once it has proved it has caught up to the active term and the beads branch head it becomes eligible, and a vote chooses a primary from the eligible candidates by configured order and reachability. The primary alone owns the SQLite store and exports the JSONL snapshot to Git; restarting it within a restart window is reported as maintenance, not a failover. A primary that is replaced comes back fenced and import-only, and its place in the preferred order does not give it writes back automatically. — **Beads primary roles**A vote picks among caught-up candidates by configured order and reachability; `caco bd trigger-vote --dry-run` shows the views, candidates and planned action.

Service-object primary: standalone beads daemon is managed independently from the main daemon and exclusively owns project SQLite files; caco up/down manage both, while caco restart defaults to the main daemon only.
Load control: push wakes are the dispatch fast path, slow jittered batch polling is anti-entropy, aggregate status and UI read models are cached and single-flight, and legacy per-project polling is shed before it can scan SQLite.
Warm standby: AKS can use the canonical beads branch to serve as an outage standby, with destructive drills using an override such as aks-beads.
Independent git mode: experimental mode where each node serves its own local store and divergence is surfaced rather than force-resolved.
Snapshots: destructive-rewrite forensic snapshots are rate-limited and visible through first-party list/rotate preview surfaces.

Build, Test, Release, and Update Paths

Heavy validation/builds on shared hosts go through daemon queues. Release workflows stream CLI binaries, macOS app artifacts, and Android preview builds into draft GitHub releases or short-lived artifacts, then publish only after asset inspection succeeds. Fleet updates use caco update, live install directories, launcher archives, restart promotion windows, and rollback on hard failure evidence.

Path	Architecture contract
Build/test queues	Serial or low-parallel daemon-managed jobs avoid shared-host compiler/emulator contention and keep output attached to first-party job records.
Release assets	Draft GitHub releases remain unpublished until required CLI binaries/checksums and macOS ZIP/DMG/metadata are present; Android previews use short-retention direct artifacts.
Update helper	Persistent update agents coordinate binary refresh, launcher promotion, restart health windows, and rollback without assuming every node updates simultaneously.
Container live binary	Read-only images fall back to the packaged binary while PVC-backed `/var/lib/cacophony/bin` supports staged `caco_next` promotion.

Observability and Recovery

Cacophony treats operator readability as part of the architecture. Status surfaces expose maintenance windows, config drift, profile validation warnings, stale checkout blockers, service PID/health divergence, and backend-degraded frontend states. Agents speak progress, write summaries, and preserve artifacts so future operators can understand why a state changed.

Health: caco status, caco doctor, peer health, service status, web status, and sidecar logs all carry different but complementary signals.
Restart windows: planned daemon/beads restart or startup convergence should be treated as maintenance until the bounded window clears or expires.
Frontend evidence: TUI benchmarks, caco-web Playwright observation, macOS Test/Canary QA, Android remote builder screenshots, and wearable surface audits land as bounded artifacts.
Retention: completed checkouts, logs, summaries, wallpapers, screenshots, and feed history have explicit retention or artifact strategies instead of relying on infinite local state.

Where to Go Deeper

Topic	Reference
Exact config schema	Configuration guide and recursive schema browser
Daemon/service lifecycle	Daemon operations, Networking & security, PKI
Task board authority	Beads and Reintegration policy
Frontend contracts	TUI, Web dashboard, macOS development, Android distribution, Wearable
Deployment variants	AKS deployment, Codespaces, testing and validation lanes
Automation profiles	Agents, Agent profiles, MCP integration