tendril0.0.4

agents

Built to be called by agents

Tendril's users are mostly agents. The command line and the MCP server share one typed surface; every result is a JSON envelope with structured errors; and input is serialised, guarded and reversible, because the desktop is shared.

MCP tools

tendril mcp stdio serves the same surface over the Model Context Protocol. The tool list, as the server reports it:

ToolArgumentsDoes
listall_apps, x11_displayDiscover targets
list_elementswindow, display, camera, include_offscreen, x11_displaySemantic elements
capturewindow, display, camera, format, compression, max_width, max_height, timeout_msA screenshot or camera frame
runinput_definition, a target, restore_focus, no_lock, lock_timeout_ms, lock_stale_ms, lock_pathInput, under the execution lock
listensource, duration_ms, format, outputRecord audio, or probe
clipboard_get, clipboard_setselection, text, serve_msThe X11 clipboard or primary selection
permissionsrequestPlatform permission status
self_update_*, feedback_*Updates and feedback from the shared CLI stack

The x11_display argument lets one server work against a disposable headless display without changing its environment. Through --remote, the MCP stream is passed through to a remote Tendril unchanged.

Envelopes and errors

With --json, and always over MCP, a result is {status, meta: {schema_version, command}, data} or, on failure, {status: "error", meta, error}. Errors are typed, so an agent can branch on them instead of parsing prose. The guard that refuses to type a bare key name returns:

{  "status": "error",  "meta": { "schema_version": 1, "command": "run" },  "error": {    "category": "validation",    "code": "invalid_run_input",    "message": "validation error: ambiguous run input `Return` looks like a DSL key token; refusing to type it as literal text",    "details": {      "action": "Return", "ambiguity": "bare_key_token", "ambiguous_token": "Return", "field": "input_definition", "stage": "parse",      "hint": "Use a DSL sequence containing a comma for key taps, for example `wait(1ms),Return`. If you intended to type the literal text, use `send(\"Return\")`."    }  }}
CategoryTypical codes and details
validationinvalid_input, invalid_run_input, with the field and a hint
unsupported_capabilityThe capability, platform, reason and a suggested action
missing_permissionWhich permission, for what, and how to grant it
target_not_foundThe target kind and id
platform_adapter_failureThe operation and platform
timeoutplatform_adapter_timeout, with the backend and timeout
execution_failure, config_error, serialization_errorWhat failed, and where

The process exits 0 on success and 1 on any error; the envelope carries the detail. Successful results carry notes for anything an agent should know, such as a fallback taken or a focus restore that was not possible.

The execution lock

The execution lockThree run commands from different agents arrive at once. Each writes a queue ticket and waits its turn. The first ticket takes the lock: it writes an owner record and refreshes a heartbeat while its input runs, then releases. The next ticket then proceeds. A holder whose heartbeat stops for 30 seconds is reaped as stale, and a waiter gives up after 60 seconds by default. Reads such as list and capture do not take the lock. queue/ ticket · agent A ticket · agent B ticket · MCP run held/owner.json heartbeat while running then release input runs one run at a time stale holder reaped after 30 s without a heartbeat a waiter times out after 60 s (both configurable) list and capture never wait for the lock
One input stream at a time.Several agents can share a desktop without interleaving keystrokes, clicks and waits.

Every run, including MCP run calls, writes a ticket and waits until it is first in the queue, then takes the lock atomically and keeps a heartbeat while its input runs. The lock is per user and desktop session, under the temporary directory by default ($TMPDIR/tendril-execution-lock-user-session/), so separate processes and separate MCP servers share it. A run's envelope reports its place in the queue, how long it waited and whether stale locks were reaped. --no-lock opts out for workflows that serialise themselves.

Capture, act, verify

Tendril cannot know whether an action worked; only the screen can. The working rule for agents:

  1. Capture before acting

    Find the target and its coordinates from a fresh capture, and convert them to source space.

  2. Act in small runs

    A few actions per run, with waits where the interface needs time.

  3. Capture again

    Confirm the change before the next step. If a run reports that focus was not restored, pause and recapture before sending more input.

  4. Prefer elements to pixels

    Where an accessibility tree exists, list-elements and click(id) survive layout changes that coordinates do not.

Updates and feedback

Tendril shares the same small libraries as the owner's other command-line tools: one typed surface for the CLI and MCP, tendril update for checked self-updates from tagged releases, and optional feedback that routes breakages to the owning project's board through a webhook. Feedback is off unless configured.