io-mcp — MCP hub for agent–human communication

What is io-mcp?

An MCP server that gives AI agents a way to talk to humans — and humans a way to direct agents — without touching a keyboard.

The problem

Claude Code agents run autonomously, but they regularly need human input: approval, direction, clarification. The standard interface is a terminal prompt — which means you have to be at a keyboard.

The solution

io-mcp exposes MCP tools that let agents present choices, speak aloud, receive messages, and share images. A TUI and Android companion app provide the human side. Any input device that maps to j/k/Enter — including a smart ring — becomes a full project controller.

┌─────────────┐ │ Input │ ──── j/k/Enter (keyboard, ring, phone) └──────┬──────┘ │ ┌──────────┐ ┌──────▼───────┐ ┌──────────────┐ │ Claude A │◄──────►│ io-mcp │◄──────►│ Android App │ │ Claude B │◄──────►│ TUI + MCP │ SSE │ (Compose) │ │ Claude C │◄──────►│ │ └──────────────┘ └──────────┘ └──────┬───────┘ │ ┌──────▼──────┐ │ TTS Output │ ──── earphones / speakers └─────────────┘

Case study: building a project with a ring and earphones

One use of io-mcp that demonstrates the generality of the approach.

Scroll & Pick

A smart ring’s scroll wheel maps to j/k. The agent presents choices via present_choices(); each option is read aloud through TTS. Click to select.

Speak

Voice commands via STT, or queue text messages with m. The agent narrates progress through speak_async() so you always know what’s happening.

Ship

The agent writes code, runs tests, commits — all narrated. You stay eyes-free the entire time. No screen, no keyboard, just gestures and audio.

Kitty graphics protocol

Pixel-perfect rendering in the terminal — no browser, no Electron.

io-mcp uses the kitty graphics protocol to render bitmap overlays directly in the terminal. This enables:

Animated glow borders around agent tabs and UI elements
Sparkline charts for agent activity and velocity
Progress gauges, status dots, and health indicators
Inline image rendering in the chat feed (agent-generated images, screenshots, pixel canvas snapshots)
Profile picture avatars for each agent session

Terminals that support kitty graphics (Ghostty, kitty, WezTerm) get full bitmap rendering. All other terminals fall back gracefully to Rich text markup — no feature is lost, only visual fidelity.

Chat feed before kitty graphics — Rich text markup only

Before: Rich text markup only.

Same TUI after kitty graphics protocol — bitmap glow borders, status indicators, and inline images

After: kitty graphics protocol — bitmap overlays composited onto the terminal.

Capabilities

MCP tools, TUI features, and infrastructure for multi-agent coordination.

📡

Multi-Agent Tabs

Each connected agent gets its own tab with independent state, TTS voice, and chat history.

🎛

Agent Spawner

Press t to launch new agents locally or on remote hosts. Workers run in isolated git worktrees.

🐝

Agent Swarm

DMs, broadcasts, work dashboard, automatic health monitoring. Agents claim beads and coordinate via io-mcp.

📋

Beads & Workboard

SQLite-backed task tracking shared across agents. Kanban board, lifecycle management, velocity metrics, 24h activity heatmaps.

🖼

Pixel Canvas

Shared drawing surface with layers, animation frames, stamps, Bézier curves, and GIF export. Agents collaborate visually.

📷

Image Store

Persistent image IDs, server-side resize/compression, inline kitty graphics rendering in chat. Generate images via gpt-image-1.

🧠

Agent Memory

Per-agent and global key-value memory that persists across restarts. Share project knowledge across the swarm.

✨

Kitty Graphics

Glow borders, sparklines, progress gauges, status dots, and inline images via the kitty graphics protocol. Text fallbacks everywhere.

🔔

Audio & TTS

Multiple TTS models (gpt-4o-mini-tts, mai-voice-1), voice/emotion presets, per-agent voice rotation, audio chimes, and STT input.

⏱

Ambient Mode

Escalating TTS updates during silence. Configurable health monitoring with webhook notifications via ntfy, Slack, or Discord.

📱

Android Companion

Jetpack Compose app on port 8445 — touch, scroll, mic, notifications. SSE-connected to the backend.

🎨

Themes

Nord, Tokyo Night, Catppuccin, Dracula. WCAG-aware contrast. Switch from the settings menu or config.

MCP tool surface

50+ tools spanning interaction, speech, task management, drawing, and infrastructure.

Interaction

present_choices Scroll-wheel single-select

present_multi_select Checkable batch list

run_command Shell with user approval

check_inbox Poll queued messages

await_message Block until message arrives

Speech

speak Blocking TTS

speak_async Non-blocking TTS

speak_urgent Interrupt current playback

set_voice / set_emotion Runtime voice config

set_tts_model / set_stt_model Switch models

Coordination

send_to_agent Direct message another agent

broadcast_to_agents Team-wide announcement

beads_* Create, claim, close, search tasks

run_tests Queue background test run

memory_* Per-agent & global KV store

Visual

send_image / generate_image Share or create images

pixel_canvas_* Draw shapes, text, gradients

canvas_* Rich-text collaborative grid

set_profile_pic Agent avatar

get_tui_state Capture screen for debugging

Quick Start

Up and running in under a minute.

Install & Run

# Clone and enter dev shell
git clone https://github.com/harryaskham/io-mcp
cd io-mcp && nix develop

# Start io-mcp (auto-starts MCP proxy on :8444)
uv run io-mcp

Connect Claude Code

# Add as MCP server in Claude Code
# Agents register via register_session()
# Each gets its own tab in the TUI

Try it

# Send a message from another terminal
io-mcp-msg "check this file"

# Or run in demo mode (no agent needed)
uv run io-mcp --demo

Architecture

Three processes, one seamless connection. The proxy survives backend restarts.

MCP Proxy

:8444

Thin FastMCP proxy. Agents connect here. Survives TUI restarts so agents stay connected.

Backend

:8446

TUI, TTS engine, session management, tool dispatch. Hot-reloadable via request_restart().

Frontend API

:8445

REST + SSE for remote frontends. Powers the Android companion app.