Terminal Graphics

Back to Terminal UI. This page holds the graphics-heavy reference separately from the main TUI guide for focused navigation.

The TUI uses a text-layout-first approach with optional bitmap graphics:

Bitmap Surfaces

Surface lifetime is ratatui-first. Views register bitmap enhancements against stable ratatui surface owners, and panel borders may also carry a GraphicsElementOwner token. The surface manager tracks these owners with weak references and geometry: if a modal, popup, split tile, or panel owner disappears, moves to a different reserved rectangle, or its token is dropped, all bound Kitty surfaces are retired and delete commands are queued. Rebinding a shared key to a new live owner preserves the replacement surface and avoids an unnecessary de-draw. Geometry changes retire the previous placement before a replacement can wait in the upload budget, across all graphics families rather than only chat bubbles. Same-geometry content updates can replace in place. Ordinary navigation and pane layout changes preserve reusable terminal payloads, but Ratatui terminal clears/autoresizes invalidate GPU residency and re-upload cached CPU assets. A failed write or flush retains an explicit attempted placement identity for cleanup, and cached text frames replay the graphics declarations that belong to their cells.

Kitty upload scheduling can be kill-switched with tui.graphics.batchUploads. The default is true, which batches delete/upload APC writes within one pass, reuses one pending-work summary for fetch/upload/delete/retry decisions, skips scans when no matching work exists, uses event-fed tmux pane geometry rather than running positioning commands during placement writes, and skips cleanup-only scans when graphics uploads are disabled. Live themes keep bounded upload defaults for operator sessions: max_uploads_per_frame 4 shared across native-animation and regular-bitmap full uploads, max_batch_wire_bytes 256 KiB, and animation upload throttling enabled. The count cap applies only to new full a=T payloads; cheap terminal-retained a=p placements bypass it and may move together in one frame, while ordinary wire chunking remains bounded. Set batchUploads to false to restore legacy per-surface flushing while diagnosing regressions.

Chat arrivals/selection/scroll and Pico transcript/reveal changes reconcile individual surfaces rather than retiring the entire pane. Actual tab/source changes and end-of-frame stale-owner cleanup still remove obsolete graphics. The live TUI does not periodically forget and retransmit its image inventory. Redraws preserve ownership until a real replacement/delete receipt, and synchronized terminal frames keep text, retirement, and replacement together. Normal/error/panic cleanup targets session-owned image IDs; signal cleanup uses a bounded lock-free emergency inventory. Neither path emits a terminal-global image clear. See the September graphics audit for the reproduced failures and the workspace convergence proposal for the separate UX plan.

Graphics telemetry can likewise be kill-switched with tui.graphics.detailedPerfTelemetry. The default is true, which reports richer cache-category counters and estimated kitty wire bytes so SSH users can reason about traffic volume; set it to false to restore the legacy coarse accounting path.

Graphics perf-event persistence is separately gated by tui.graphics.perf_logging (default false). When false, ordinary sessions do not POST tui.graphics perf events to the daemon perf_events store (POST /api/v1/logs/perf) and emit no per-pass graphics/perf lines into the client tui.log, avoiding daemon.db bloat and write/ingest load on the very render path being measured. Enable it for benchmarking/optimization to restore the full sampled telemetry and the Performance-tab tui.graphics rows. Explicit diagnostic log calls use an 8 KiB formatter buffer and flush before returning, coalescing small file writes without dropping messages or delaying their normal visibility. This is distinct from detailedPerfTelemetry above, which controls the richness of the accounting rather than whether it is persisted; the in-memory benchmark harness (caco tui dashboard-benchmark) is independent of both gates.

Kitty upload format is controlled by tui.graphics.format. The default pnga keeps the compatible PNG-with-alpha upload path.

The TUI uses flate2's pure-Rust zlib-rs backend with the existing bounded per-thread encoder reuse. The PNG Sub filter, configured compression effort and decoded pixels remain unchanged; compressed byte streams and sizes may differ. Darwin's bounded build profile optimizes the small codec packages without changing the rest of the CLI profile. See the heavy PNG-workload measurements and compatibility limits.

Complete low-color RGBA inputs from 64 KiB through 32 MiB may use an exact indexed PNG palette with up to 256 distinct RGBA values, including hidden RGB when alpha is zero. The standard PLTE/tRNS representation preserves every channel without quantization, dithering or resolution changes. Palette lookup work is bounded, and smaller, oversized, incomplete, high-color or collision-heavy inputs keep the existing RGBA PNG encoder. Raw/local-auto paths are unchanged; decoded residency is still counted as RGBA rather than compressed length. Size and timing improvements are workload-dependent, not a blanket guarantee. See the lossless contract and measured limits.

Budgeted host-driven PNG backgrounds can keep large animated pixels in memory until an image is selected for upload, then apply the same PNG filter and compression effort before either transport. This avoids compressing superseded, unselected images, at the cost of more staging memory. Static, disabled, single-frame, native-prebaked and unlimited-upload paths remain eager; raw/local-auto policies are unchanged. Upload receipts preserve the prepared-image and decoded-residency accounting. See the bounded deferred-encoding results and limitations.

tui.graphics.mode selects transport independently of encoding. stream always uses inline transfer. auto and memory use shared memory only after a silent, read-only query proves that the terminal can consume an exclusively created local object; tmux and relays stay on the portable path. Unavailable, full or failed memory transfers fall back to streaming. Objects are owner-only, transfers are bounded to 32 objects / 32 MiB, and small assets stay inline. Normal drop cleans the owned namespace; locked private manifests let a later startup recover abandoned objects after process death without PID-reuse guesses. A forced kill can leave bounded objects until that recovery or reboot.

The opt-in format: auto keeps compact PNG controls and portable PNG fallback, but avoids encoding large host-driven backgrounds when the local shared-memory probe succeeds. Native pre-baked loops conservatively retain PNG encoding. This changes neither raster resolution nor pixel fidelity; it is not a blanket switch to raw RGBA over SSH. Shared-memory payload bytes and terminal wire bytes are reported separately in benchmark results.

Background image source caching can be kill-switched with tui.graphics.imageSourceCache. The default is true, which reuses decoded and resized image intermediates beneath the PNG cache so repeated image-style background renders do not reopen and rescale the same file every time; image render keys, decoded/resized source caches, and app-level background surface snapshots include file length and modification-time identity, so replacing an image in place at the same path invalidates stale cached bitmaps before either the renderer or the kitty-surface fast path can reuse them. Set it to false to restore the legacy decode/resize path.

Procedural backgrounds use bucketed cache sizes by default (tui.graphics.scaledBackgroundCache, capped by scaledBackgroundCacheMaxPx, default 1280):

Set scaledBackgroundCache: false only as a rollback or exact-raster diagnostic. The same settings appear in the configuration reference sample for repeatable benchmark profiles.

Unchanged surface upload dedupe can be kill-switched with tui.graphics.surfaceUploadDedupe. The default is true, which keeps already-uploaded kitty surfaces intact when the newly registered PNG bytes are identical, reducing repeated SSH traffic without suppressing genuine animation-phase updates; new retained-image variants avoid redundant per-surface and global LRU duplicate-removal scans while replacement variants still move to the back for eviction correctness, and retained-image lookups skip local or shared hash-map probes when those caches are empty. Set it to false to restore the legacy pointer-only invalidation path.

Retained-image accounting uses decoded RGBA residency when PNG/raw headers provide dimensions, not only compressed wire length. Native frame residency shares that budget; oversized native background loops render incrementally instead of baking every frame during navigation. Border and text-decoration asset caches are byte/entry-bounded. Active animation frames—including phase zero and native background pre-bakes—bypass both disk lookup and persistence; their bounded memory caches and terminal retention still operate normally. Static assets retain cross-launch disk caching through a shared nonblocking queue with a byte bound and atomic publication. A failed/full cache worker never falls back to synchronous writes on the frame thread. Ordinary unit tests keep disk caching disabled; explicit owned-directory diagnostics exercise the real persistence boundary.

Reusable static borders have a protected 4 MiB / 2,048-entry RAM working set within the existing total 16 MiB / 8,192-entry border-cache budget; the remaining capacity is a general/transient LRU, which also accepts static overflow. Animation churn therefore need not evict every hot static asset and cause another synchronous disk read. First/cold static reads and cross-launch persistence remain possible; this is not a blanket disk-cache disable. See the cache-pressure regression and bounded tail-latency results.

Raster scale is controlled by tui.graphics.renderScale. The default is 2, preserving the current HiDPI-friendly output. Fractional values such as 0.5 are supported for half-scale rendering. Lower values reduce raster work and terminal wire bytes for both static and animated surfaces; higher values trade throughput for sharper graphics.

Idle animation wakeups are further gated by tui.graphics.activeAnimationOnly. The default is true, which means the live TUI only spends timer-driven redraw work when at least one visible kitty surface needs redraw-driven animation; terminal-native animation loops do not keep that wake path active because the terminal advances those frames itself.

The animation clock remains dormant for a static theme and can start after a live theme switch. Resume and FPS changes establish fresh deadlines; missed ticks are skipped, and a slow renderer can have at most one pending animation wakeup. Borders, chips and backgrounds sample one coherent frame time. A persistent --no-animation override still wins over later theme/config changes. Bitmap support is distinct from native frame playback: Ghostty 1.3 uses host-driven frames, while detected Kitty and native-capable Ghostty versions can use eligible pre-baked loops.

Resize suppression is controlled by tui.graphics.disableOnResize (default true) and tui.graphics.resizeDebounceMs (default 1000). During terminal or frame-geometry changes the TUI keeps the ratatui text baseline visible while suppressing kitty surfaces, including decorative glows, sparklines, fills, and modal overlays, until geometry stabilizes. The checked-in performance graphics fragment enables both drag and resize suppression so performance-oriented themes temporarily fall back to text during active layout movement.

Surface Materials

A role's material adds inset lighting, reflection and stable microtexture inside the existing background raster pass. Choose none (the default), glass, satin, chrome or paper. Materials preserve background alpha and semantic tint, add no separate terminal image or forced animation, and leave native text selectable. Glass is an analytic translucent finish, not blur of arbitrary terminal text or the desktop.

tui:
  themes:
    my-material:
      graphics:
        header_decoration_style: glass_pill
        panel:
          - background_style: gradient
            background_color: "#202a36"
            background_contrast: 0.92
            background_opacity: 0.88
            pattern: bevel
            header_decoration_style: glass_pill
            material: {kind: satin, depth: 32, sheen: 16, grain: 2, width: 20}
          - background_style: inner_glow
            background_opacity: 0.4
            inner_glow: {radius: 14, intensity: 0.7, opacity: 0.65}
            blending_mode: screen
            merge_into_previous_layer: true

depth, sheen and grain are integer percentages, saturated at 100. width is the inset falloff in logical raster pixels; zero uses a one-pixel minimum. Omitting the material or selecting none preserves the previous rendering path. Background contrast is a brightness multiplier: 0.92 produces a subtle gradient, whereas 0.08 makes its far end almost black.

Active materials use exact rectangle dimensions to avoid stretching their inset edges; they retain the existing procedural background scale cap and do not lower border/text fidelity. Every material parameter participates in the pixel cache key, but screen origin does not. Moving an unchanged surface therefore reuses its encoded pixels. Named themes remain the base; explicit global/node graphics overrides still win. The optional v2 presets are authored in the separate theme configuration repository, not selected by this renderer change.

Contained chips, relief strokes and animation

header_decoration_style: glass_pill and button_decoration_style: glass_pill augment the existing capsule geometry with a reflective rim, tinted fill and inset shading. Effects stay inside an antialiased silhouette; no bloom is painted behind individual letters. Narrow chips remain bounded, and native text, hit targets and layout are unchanged. The theme editor and isolated testbed expose this style alongside the original pill, box and glow treatments.

pattern: bevel lights the existing stroke mask from the upper left; pattern: etched reverses that lighting for recessed edges. pattern_contrast controls relief strength. These treatments preserve stroke alpha and semantic tint rather than adding terminal surfaces. Inner glow, radial lighting, static noise and scanlines can be authored as separate background layers; merge_into_previous_layer: true composes them into the cached base bitmap. Independent avatar layers keep their own ownership.

material.motion is an opt-in, saturating integer percentage (default 0), enabled only with background_animate and the applicable animation gates. Material motion uses background_animation_speed and quantized animation_frames. Glass chips use the shared animation-redraw clock and animation_speed, respect the border-glow animation gate and the owning surface's animation permission, and return to cached phase zero when disabled. Static presets schedule no new animation work.

Temporal resolution and motion speed are separate. At 0.08 cycles/second, 24 animation_frames provide only 1.92 distinct phases/second even if the dashboard redraws at 60 FPS. The 16-bit phase count permits finer sampling: 750 phases allow 60 phase updates/second over the same 12.5-second cycle, subject to rendering and transport capacity. The default remains 24, and explicit preset counts are not silently changed. Large cycles use bounded caches rather than pinning a full per-panel phase bank; native pre-baking is limited by both decoded-byte budget and a 256-frame ceiling. Higher counts render incrementally. Values above 255, like new enum values, require a compatible client before loading the registry, even when another theme is selected; deploy those configuration changes only after the engine update.

Rotating border colors follow one clockwise stroke-centreline coordinate across edges and rounded corners. Insets, thickness and radius affect the path; a diagonal across the corner image box does not. Title gaps and hidden sides do not reset phase. This spatial continuity is separate from transport timing: a slow, budget-limited uploader can still update neighboring segments at different times.

With verified local-memory transport and automatic encoding, large host-driven animated borders can join each contiguous corner/run row into one bitmap while preserving the original pixels and decoded area. The unchanged upload-count budget can then advance other surfaces sooner. Static, small/native-style, incompatible and explicit-format paths retain separate segments. Capability loss rebuilds the physical segment set; image ownership and bounded cache limits remain unchanged. This can improve per-surface update cadence but increases useful transfer volume and does not make the whole panel's presentation atomic. Compatible animated rows and sides may additionally share bounded atlas pages through cropped placements when shared-image ownership is enabled. Atlas aliases share one full-upload slot; PNG fallback and crop-only invalidation preserve placement correctness. This improves useful image cadence at the cost of increased local SHM traffic, not lower bandwidth or guaranteed stall-free presentation. See the scoped row contract and atlas lifecycle, measurements and limits.

Metal and GPU raster/composition

The wgpu 30.0.1 / HAL 30.0.1 backend uses event-driven Metal command completion instead of the older polling-sleep wait. Exact-submission waits, bounded failure fallback and mapping guards preserve same-frame ownership; shader/raster quality and upload budgets are unchanged. This improves the measured heavy layered case but does not remove CPU encoding, readback, terminal transfer or busy-scene stalls. The workspace wasm-bindgen crate, flake CLI and generated browser pair advance together; see the compatibility contract, measurements and validation limits.

Supported animated single-layer backgrounds can use the same procedural GPU path as composite stacks. With event-driven Metal waits, single-layer jobs become GPU-eligible at 65,536 raster pixels; supported fused composites at 65,536 raster-pixel-layers (base plus nonempty overlays). Other backends retain the 262,144 single-layer / 131,072 composite floors. Tiny, disabled, unavailable or unsupported small jobs retain their existing CPU/fallback behavior. These policies do not reduce raster scale, quality or layers. See the Metal crossover and whole-frame limits and the earlier conservative-floor evidence.

Real-terminal benchmark GPU/CPU compositor pass and duration counters exclude warm-up. Device/cache state is preserved, and allocation current/peak values remain lifetime resource data. Older snapshots with warm-up-inclusive counters must not be normalized as exact measured-frame GPU timings; readback intervals are host timings, not GPU timestamps.

The headless wgpu backend can select Metal on macOS. Common animated stacks of gradients, grain-free materials, solid fills, inner/radial glows and noise can fuse rasterization and composition in one compute pass. Exact CPU geometry masks and material profiles preserve clipping and layout. The fast path accepts up to 16 layers and preserves per-layer RGBA8 quantization and all five blend modes; unsupported images, overlays, patterns or textured materials retain the CPU raster path and existing compositor. Static procedural layers in fallback composites have a bounded decoded cache. No visual layer or configured texture is dropped to improve throughput.

GPU floating-point raster math is not promised bit-identical to CPU math; explicit reference diagnostics bound the observed differences and check clipping separately. Readback still completes on the render thread, but eligible procedural misses run independent same-frame border/chip work after GPU submission and before blocking for pixels. Cache/CPU/native paths keep their ordinary registration path, and the readback mapping is released on errors or unwinds. No extra worker or frame queue is created; this is not zero-copy terminal rendering. Large local assets can encode directly into their immutable allocation to avoid intermediate full-frame copies. Benchmark JSON separates gpu_procedural_passes, submissions/readback, render time, upload time, phase advances, terminal wire and shared-memory bytes. Host completion/readback measurements exclude the useful CPU interlude and are not GPU timestamps or actual presentation latency. See the animation performance evidence and limitations and the earlier material design.

Embedded-Pane Kitty Passthrough

Agent and shell panes isolate child Kitty image IDs from the TUI's own terminal-global image namespace.

Debugging & Benchmark Harnesses

Two first-party harnesses exercise this graphics and border stack directly, on a smaller surface than the full dashboard:

The mixed_chat_mosaic fixture scene exercises six unequal workspace panes, including three actual chat views with differently sized bubbles. Its stream_interval_ms defaults to 250; zero freezes data/focus for motion inspection. Existing fixtures without this setting retain their prior cadence. Results include live bitmap/border counts, missing committed images and pending animation age, so FPS alone cannot hide starved borders. These are application-side upload facts, not terminal presentation acknowledgments. See animated-theme correctness and workload parity.

scripts/tui-kitty-lifecycle-audit.py --test-binary <library-test-executable> --output <empty-directory> runs an isolated real-terminal lifecycle probe and compares Kitty's own decoded-image and placement references with the TUI's ownership inventory. Build the test executable with cargo test --locked -p caco-tui --lib --no-run; the probe never connects to an operator daemon or edits user configuration.

Benchmarking

Use caco tui dashboard-benchmark for claims about the live dashboard; the isolated testbed is for chrome-only experiments such as bubbles, modals and nested bitmap borders.

The JSON reports, in groups:

Read gpu_status before changing a default: active means the compositor ran on the GPU, pending means no multi-layer composite was exercised, and fallback_cpu means initialization or a resource-limit preflight failed and the CPU path was measured. Oversized work degrades to the CPU instead of panicking in wgpu. In live tui.graphics perf labels, upload:burst marks large uploads packed into a few passes, while upload_wire:large marks busy but spread-out windows.

Scripts:

For real Kitty or Ghostty validation, attach a short capture (4–8 seconds, low frame rate, compressed, no audio) or a still with the fallback reason next to the JSON, not the JSON alone.

The isolated testbed uses the active tui.theme_name and node overrides, so its palette matches the dashboard:

Isolated testbed keyControl
i / ICycle background style
1 / 2Adjust background opacity
3 / 4Adjust background contrast
5Cycle background angle
6 / 7Toggle/adjust background animation speed
8 / 9Lower/raise target FPS for headroom checks

Animation timing that benchmarks depend on:

Kitty Graphics Sparklines

When kitty graphics are available, the Status page telemetry panel renders sparklines as bitmap images instead of Unicode block characters. The system uses a pre-computed segment atlas — a set of tiny PNG images uploaded once via the kitty graphics protocol and then reused through placement sequences.

Segment Atlas

The atlas is lazily initialized on first sparkline render and re-uploaded on theme change. It contains:

Configuration

tui:
  graphics:
    sparkline:
      style: gradient_fill_line    # bar | line | gradient_fill_line
      opacity: 0.8                 # Line/fill opacity (0.0–1.0)
      fadeOpacity: 0.0             # Bottom gradient opacity
      levels: 10                   # Quantisation levels (atlas size)

Higher levels values produce smoother sparklines but require more pre-computed segments. Falls back to the standard ratatui Sparkline widget when kitty graphics are unavailable.

Button Widgets

Reusable button widgets provide clickable chrome for actions throughout the TUI. Button appearance is controlled through tui.graphics.button (standard GraphicsStyleLayers) and padding through tui.graphics.buttonPadding (uniform integer or per-side { top, left, bottom, right } map).

Background Animations

The TUI supports animated backgrounds that render behind the main content. Each background type has a static and animated variant:

TypeAnimation
GradientContinuous rotation or scrolling, selected by animation_style
ScanlinesDownward scroll with phosphor pulse sweep
NoiseRandom refresh per frame
VignetteBreathing pulse on radius
GridSlow drift
Solidn/a

Animations are toggleable at runtime. Speed is in cycles per second; phase resolution and redraw cadence are independent. This example uses a five-second background cycle and a slightly faster four-second border cycle without reducing the frame target. When authoring a named theme, keep these fields in that theme's graphics block rather than imposing a global override:

tui:
  graphics:
    animations: true
    background_style: gradient
    background_animate: true
    animation_frames: 750
    target_fps: 60
    background_animation_speed: 0.2
    animation_speed: 0.25
    panel: {animation_phase_offset: 0.0}
    subpanel: {animation_phase_offset: 0.04}
    bubble: {animation_phase_offset: 0.08}
    button: {animation_phase_offset: 0.08}

animation_phase_offset is a fixed fraction of a full cycle, quantized to animation_frames. It shifts supported animated backgrounds, material/noise phases and borders without changing their rate. Root/subpanel/small-control offsets of 0/4/8 percent can convey depth without time drift; explicit role and instance overrides remain authoritative. Animated glass title/span chips use the button-role offset, while legacy static pills remain static. Default zero offsets preserve the original clock, and disabled backgrounds do not gain animation work. A box's corners and edges retain one shared phase. Slower motion and finer phase counts do not remove upload stalls or prove terminal presentation; see the phase and validation contract.

Span Glow Bleed

Text-level glow effects extend beyond cell boundaries using configurable bleed: spanGlowBleedX (default 2 columns) and spanGlowBleedY (default 1 row). This prevents hard clipping at cell edges.

Portable Theme Packages

caco tui theme-package export serializes the current effective theme, or a named theme selected with --name, as portable package YAML. The package has a package: metadata block (name, optional author, description, created_at, version, and schema_version: 1) plus the existing named-theme fields such as colors and graphics. Pass --output to write the package to disk instead of stdout.

Cacophony — Declarative agent orchestration.

OFFICIAL · A.SKH.AMOFFICIAL · A.SKH.AMSEAL OF ORIGINSEAL OF ORIGINMADE BYMADE BYAGENTSAGENTS