Agent Surfaces — Design & Implementation Spec

1 · Overview

The current /voice page is a generic voice-chatbot surface — violet "S" circle, waveform strip, "Start Voice Session" button. It's functional but impersonal and does not match the "mission control / command center" feel the product requires. Additionally, there is no tiered presence model — Shakeeb is either absent (every non-voice screen) or all-consuming (the voice page). This spec replaces that binary with a four-tier presence stack:

The four surfaces share the same agent state, transcript, and trust contract. A user swaps between them without losing conversational continuity. There is no "session" that starts and ends — presence tier is a lens on the same persistent context (matches JARVIS-feel signature #7 "no session boundaries").

2 · Problem Statement

1. The current /voice UI does not communicate that Shakeeb is a character — it reads as a generic chatbot.
2. Shakeeb has no presence outside /voice — when working in other screens, he effectively doesn't exist unless explicitly summoned. This is the opposite of an always-on assistant.
3. There is no planned topology for multi-agent interaction. Phase 2 (BRX) and Phase 3 (Jean Otti, Nadzingo, Kamini) will need a shared surface, and bolting it on late is more expensive than architecting for it now.
4. The "draft-first trust contract" requires visible surfaces (draft queue, approval prompts, proactive suggestions). None of these exist in today's /voice.

3 · Scope

In scope (this spec)

• UI and interaction design for all four surfaces
• Shared state model (agent state, transcript, presence tier)
• Toggle / transition rules between surfaces
• Zod contracts for new IPC channels and web API endpoints introduced
• Trust/privacy affordances (always-on mic, mute, persistent listening indicator)
• Mobile/PWA degradation strategy
• Phasing: what ships in Phase 1.6+ (single-agent surfaces) vs Phase 2+ (multi-agent)
• Accessibility: WCAG AA minimum, keyboard-navigable, screen-reader labels

Out of scope

• Character mark / illustrated portraits for each agent (the orb is the presence — no portrait work in this spec)
• Multi-language voice support (English only for v1)
• Third-party agent plugins / external agents
• Parallel voice-conversations (you talk to one agent at a time even in multi-agent mode)
• Native mobile apps (PWA only)

4 · User Journeys

J1 — Morning walk-in. User opens the desktop app. Dashboard landing shows all 5 agents as portraits. Shakeeb strip up top shows "Morning Wassim — three items: Sarah's Dubai reply, Sam at three, Mariam's pricing follow-up." User clicks Shakeeb card → enters war room (single) with context already loaded. No login, no "start session."

J2 — Working in Email. User navigates to /email. Right rail docks with Shakeeb listening + last 3 turns of transcript. User types "draft a reply to Sarah, apologetic tone" into rail input. Shakeeb drafts in the queue module. User hits − to collapse rail to corner orb; orb pulses when draft is ready for review. User clicks orb → rail expands back.

J3 — Extended focus session. User hits ⌘J (or clicks "War Room →" in rail) from anywhere. Enters war room (single-agent, v3 JARVIS). All HUD modules materialize. Session persists; closing the app and reopening tomorrow picks up with "continuity banner: picking up from yesterday 18:42."

J4 — Multi-agent parallel work (Phase 2+). User starts BRX on a car research task, Jean Otti on a Reddit scan, Kamini on an arsenal audit. Enters multi-agent war room at /war-room. All three visible as 1×3 adaptive grid. When BRX has results, his orb pulses in the grid. User clicks BRX name in the top bar → grid animates into focus+satellites with BRX in center, Jean Otti + Kamini as satellites at the corners. User then clicks Jean Otti's satellite → BRX swaps to satellite, Jean Otti blooms to center.

J5 — Mobile/PWA quick check. User opens PWA on phone. Dashboard landing degrades to single-column agent cards. Tapping Shakeeb opens a mobile-adapted subtle presence (bottom sheet variant, not corner orb). No war room on mobile — too dense. Prompt: "open on desktop for the full war room."

5 · Architecture — Four Surfaces

Surface 1 — Dashboard landing

• Route: / (desktop) and / in PWA
• Layout: Existing option-d portrait-card spine, with portraits replaced by color-coded breathing orbs (atlas violet, karma orange, flux red, forge teal, apex green). 5-column grid on desktop, 2-column on tablet, 1-column stacked on mobile.
• Per-card: Agent name + role + status (Live / Soon) + model + hover reveals "Enter War Room" pill
• Soon-agents: Grayscaled orbs (grayscale 78%, opacity 55%), "Soon · P2/P3" badge, non-clickable
• Shakeeb strip: Above the card grid. Pill with violet left-accent showing current ambient-state message. Clickable CTA "Enter war room →"
• Top bar: Mission clock (Mon · 20 Apr · 09:47:03), ⌘K command palette hint
• Sidebar: Standard app nav — Dashboard (active), Email, Calendar, Commitments, Memory, Chat, Agents list

Surface 2 — Subtle presence (rail ↔ corner orb)

• Where: Every non-war-room screen (Email, Calendar, Commitments, Memory, Chat, Settings)
• Default state: Right rail · 300px wide
  • Header: agent orb · name · live state · minimize − button · "War Room ↗" button
  • Body: scrolling transcript (last ~10 turns) with who-markers (YOU / SHAKEEB) and timestamps
  • Footer: text input + mic button (circular, agent-colored)
• Collapsed state: Corner orb · bottom-right
  • 48×48 orb with halo. Optional whisper card (truncated latest line) appears on agent-initiated surfacing
  • ⌘\ keyboard shortcut and tapping the orb both expand the rail back
• Toggle affordance: − in rail header collapses · tap orb OR ⌘\ expands
• Persistence: Last state remembered per session and per user profile. First-run always opens in rail.
• One agent only: Subtle presence is tied to the current agent (Atlas/Shakeeb in Phase 1). In Phase 2+ if the user has a default agent, the rail shows that one; switching default is a setting, not an in-rail control.

Surface 3 — War room · single (JARVIS-feel, v3 density)

• Route: /war-room (defaults to current/default agent) OR /war-room/[codename] for explicit agent
• Old /voice route redirects to /war-room/atlas for backwards compat
• Layout anatomy (matches v3 JARVIS preview):
  • Top bar: Mission clock · priority heatmap (H/M/L segments) · session timer T+18h 23m · agent tag with breathing dot
  • Continuity banner: "picking up from yesterday 18:42 — we left off on X"
  • Center: Orb (260px) wrapped by 3 rotating rings (outer data-node ring CCW 60s, middle segmented-arc ring CW 80s with tangent labels LISTENING / RECALL / PRESENT, inner tick ring CCW 40s) + radial speech spectrum
  • State label: LISTENING / THINKING / SPEAKING / IDLE (mono, letterspaced, below orb at ~180px offset)
  • Thought-stream: One line of faint italic whisper below state label — current inner monologue
  • Proactive-suggestion chip: Floating card offset right of orb with accept/leave-it buttons (appears only when agent has a proactive suggestion)
  • Focus tether: Cyan animated beam from orb to currently-active HUD module
  • 6 HUD modules at positions NW (Active Recall) · NE (Draft Queue, focused) · W (Memory Trail) · E (Live Feeds, with red-pulse on inbound) · SW (Context Threads) · SE (Voice Telemetry)
  • Ambient telemetry: 9 scattered tiny readouts (ctx-depth, mem-hit, vad, rtt, tok/s, t+, cost, focus, mood)
  • Tool-readiness strip: 5 chips above transcript — gmail ✓ · calendar ✓ · commits ✓ · memory ✓ · draft · composing
  • Teletype transcript: Bottom 168px. Mono font, timestamps [09:46:58], who-markers (WASSIM / SHAKEEB), partial-turn cursor
• No "Start Voice Session" button. Mic is always live while the room is open. A persistent MUTE affordance is required (top-right), with clear muted-state visual (orb dims, ring stops breathing, "MUTED" label).
• Proactive opener: First transcript line is always a proactive greeting with pre-loaded context (never a dead mic). Hook fires on room open if no turn in the last 5 minutes.

Surface 4 — War room · multi (adaptive + focus-swap)

• Route: /war-room/multi (Phase 2+)
• Default topology: Adaptive grid sized to active agents
  • 2 → 1×2 horizontal halves (full-size orbs, rings)
  • 3 → 1×3 three columns (compact orbs)
  • 4 → 2×2 quadrants (the "default shape")
  • 5 → Hero-top-row spans 2 columns, bottom 2×2 quadrant below (asymmetric to preserve "one-primary" feel)
• Per-cell: agent orb · name · role · state (dot-pulsing) · current-task summary one-liner
• Top bar: all active-agent chips (colored, clickable) · mission clock · ⊞ Back to grid button (only visible in focus mode)
• Shared transcript ribbon: 36px height bottom bar showing currently-speaking agent's latest turn · agent chip tabs on the right
• Focus mode: Click an agent chip → grid animates (400ms cross-fade) into focus+satellites with the chosen agent at center (full JARVIS rings, 180px orb) and the other 1–4 agents as 64-80px satellites at corner positions NW/NE/SW/SE with their own halos and state pills
• In focus mode: Transcript shows only the focused agent's turns (not interleaved). Others visible as [bg] tags or through their satellite state pills.
• Swap focus: Click another chip in top bar OR click any satellite orb → cross-fade swap
• Exit focus: Click ⊞ Back to grid → return to adaptive grid

6 · Shared Components

7 · IPC & API Contracts

All new contracts follow the existing pattern: Zod schema in src/contracts/, response envelope { ok: true, data: T } | { ok: false, error: string }, validated at the IPC bridge before handler execution.

New contracts

src/contracts/agent-presence.ts

export const AgentStateSchema = z.enum(['idle','listening','thinking','speaking','muted'])
export const PresenceTierSchema = z.enum(['dashboard','subtle-rail','subtle-orb','war-room-single','war-room-multi','war-room-focus'])

export const AgentPresenceSchema = z.object({
  codename: AgentCodenameSchema,   // existing
  state: AgentStateSchema,
  tier: PresenceTierSchema,
  sessionStart: z.string().datetime(),
  lastTurnAt: z.string().datetime().nullable(),
  thoughtStream: z.string().nullable(),
  activeModule: z.enum(['recall','queue','memory','feeds','context','telemetry']).nullable(),
})

export const ProactiveSuggestionSchema = z.object({
  id: z.string().uuid(),
  codename: AgentCodenameSchema,
  message: z.string(),
  actions: z.array(z.object({
    label: z.string(),
    kind: z.enum(['primary','secondary']),
    commandId: z.string(),
  })),
  expiresAt: z.string().datetime(),
})

src/contracts/agent-hud.ts

export const HUDModuleDataSchema = z.object({
  id: z.enum(['recall','queue','memory','feeds','context','telemetry']),
  title: z.string(),
  mainLine: z.string(),
  subLine: z.string().optional(),
  pills: z.array(z.object({ label: z.string(), tone: z.enum(['neutral','warn','ok','danger','info']) })).optional(),
  serial: z.string().optional(),
  incoming: z.boolean().default(false),
  focused: z.boolean().default(false),
})

export const ToolReadinessSchema = z.object({
  tools: z.array(z.object({
    id: z.enum(['gmail','calendar','commits','memory','draft']),
    state: z.enum(['warm','composing','cold','error']),
    sub: z.string().optional(),
  })),
})

New IPC channels (16 total)

Add to src/contracts/ipc-channels.ts under IPC:

Web API (Phase 1.5+)

Mirrors the IPC surface over HTTP for the web PWA. All under /api/agent-presence/*, /api/hud/*, /api/proactive/*, /api/tools/*. Request/response shapes are the same Zod schemas — we export both sides from src/contracts/.

8 · State Machines

Agent state (per agent)

  idle ──(mic detects VAD)──> listening
  listening ──(user stops speaking, agent starts reasoning)──> thinking
  thinking ──(agent starts TTS)──> speaking
  speaking ──(TTS complete, no further turn)──> idle
  {any} ──(mute toggle)──> muted ──(unmute)──> idle

The orb color, halo intensity, ring speed, and thought-stream visibility all derive from this state. listening = atlas-violet halo bright, slow breathing. thinking = karma-orange undertone, turbulent orb. speaking = apex-green accent, ripples project outward. muted = orb dims to 40% opacity, halo hidden, rings static.

Presence tier (per agent)

  dashboard ──(click card)──> war-room-single
  any ──(focus tether click / war-room button)──> war-room-single
  war-room-single ──(navigate away)──> subtle-rail (default) OR subtle-orb (if last was collapsed)
  subtle-rail ──(minimize − button)──> subtle-orb
  subtle-orb ──(tap orb / ⌘\)──> subtle-rail
  subtle-* ──(⌘J / war-room button)──> war-room-single
  war-room-single ──(open multi-route)──> war-room-multi  [Phase 2+]
  war-room-multi ──(click chip)──> war-room-focus
  war-room-focus ──(click ⊞ back)──> war-room-multi

Invalid transitions (e.g., subtle-rail → war-room-multi without going through multi-route entry) are rejected by the presence handler.

9 · Persistence Rules

Sessions are never explicitly closed. Closing the app suspends them; reopening resumes with the continuity banner (T+18h 23m).

10 · Data Feeds Required

Six feeds must be live and reactive for the war room HUD modules to show real data (vs. placeholder):

1. Active Recall — memory service emits most-recent-recalled episode(s); target: event on memory search > confidence threshold
2. Draft Queue — email service emits drafts awaiting approval; target: event on email.draftCreated + email.draftRevised
3. Memory Trail — memory service emits episode-chain for the current reasoning context (last ~3 pulled)
4. Live Feeds — inbound email + calendar-event + external-notification stream; fires red-pulse on arrival
5. Context Threads — commitment/topic service emits the top ~3 active contexts with last-touched timestamps
6. Voice Telemetry — Pipecat/browser-native voice layer emits VAD level, RTT, cost, mood classification

Feeds 1, 2, 5 exist today. Feeds 3, 4, 6 are new work in Phase 1.6.

11 · Trust & Privacy

The "always-on mic" posture requires explicit, always-visible affordances. Non-negotiable:

1. Persistent mute pill in the top-right of every war-room surface. Shows unmuted state with a pulsing dot; muted state with a filled pill, distinct color, and MUTED label.
2. Muted-state orb visual — orb dims, halo removed, rings freeze. Impossible to confuse with unmuted.
3. System-level indicator parity — on desktop, OS mic indicator (macOS menu bar mic dot) must match our shown state. If OS says mic is live and we say muted, that's a bug.
4. First-run consent gate — before the war room ever opens, a one-time modal: "Mission Control's war room keeps the mic open for continuous conversation. You can mute anytime (top-right or ⌘M). OK to proceed?"
5. Subtle presence mic policy — rail + corner orb do NOT keep mic open by default. Mic only engages when user clicks the mic button. (Different posture from war room, which is immersive.)
6. No stored audio — transcripts are stored (text), raw audio is not.
7. Draft-first trust contract — no tool call (email send, calendar event create, etc.) fires without user approval. Proactive suggestions chip is suggestion, not action.

12 · Performance Budgets

13 · Mobile/PWA Degradation

14 · Accessibility

• All orbs have descriptive aria-label matching state: "Shakeeb is listening", etc.
• All state changes announced via polite aria-live region
• Full keyboard nav: Tab through cards on landing, ⌘K command palette, ⌘J enter war room, ⌘\ toggle subtle, ⌘M mute, ⌘1..5 switch agent in multi-focus
• No color-only state conveying. State pills use text + dot + color redundantly.
• WCAG AA contrast enforced on all text. Orb glow is decorative — no content relies on it for meaning.
• Reduced-motion media query: prefers-reduced-motion: reduce disables ring rotation, orb breathing pulse, and tether particle animation. State changes become instant transitions instead.

15 · Test Strategy

16 · Implementation Phases

Order below is build sequence, not release order. All surfaces except multi-agent land together in Phase 1.6.

Phase 1.6 (now → 4 weeks, single-agent only)

• P1.6.1 — Shared components library (AgentOrb, AgentRings, TeletypeTranscript, HUDModule, etc.)
• P1.6.2 — Presence state machine + 16 IPC channels + Zod contracts
• P1.6.3 — Subtle presence (rail + corner orb + toggle) layered onto all existing screens
• P1.6.4 — Dashboard landing refresh (replace flat portraits with breathing orbs + Shakeeb strip)
• P1.6.5 — War room single (replace current /voice with v3 JARVIS layout)
• P1.6.6 — Data feeds wiring — memory trail, live feeds, voice telemetry (net-new); rest hook into existing services
• P1.6.7 — Trust gates — mute affordance, first-run consent modal, reduced-motion support
• P1.6.8 — Mobile/PWA degraded variants
• P1.6.9 — Integration tests + E2E + a11y audit + perf audit

Phase 2 (when BRX lands)

• P2.1 — Multi-agent war room route /war-room/multi
• P2.2 — Adaptive grid (AgentGrid component) for 2/3/4/5 agent counts
• P2.3 — Focus swap animation + shared transcript tab system
• P2.4 — Per-agent proactive suggestions + HUD feeds

Phase 3 (when Jean Otti, Nadzingo, Kamini land)

• P3.1 — 5-agent full grid testing + tuning
• P3.2 — Per-agent thought-stream customization (each agent has different personality register)

17 · Open Questions

1. 5-agent layout confirmation. Spec ships "hero-top-row + 2×2 quadrant" for 5. User indicated preference for this vs. 3+2 rows vs. 5 equal columns. Confirm before Phase 2 start.
2. Agent switching in subtle rail. Phase 2+ — does the rail always show the default agent, or does it auto-switch based on what the user is doing? (e.g., viewing a car page in the app → BRX rail)
3. Proactive suggestion cadence cap. How many proactive chips per hour before it feels naggy? Default proposal: 3/hour max, cooldown of 5min between.
4. Session-end visualization. If the user goes 48+ hours without interacting, does the continuity banner still say "picking up from 2 days ago at 18:42"? Or does it propose a fresh start? Default: always resume, with option to "start fresh" in the banner.
5. Mute-state mic icon on PWA. Browsers show their own mic indicator in the tab — can we suppress / layer ours so they don't look contradictory?

18 · Companion Documents

The four HTML design previews, rendered live via Vercel (deployed from /public):

Related specs + decisions:

• Mission-Control-Product-Spec.md — parent product spec (add reference under Companion Documents)
• docs/ARCHITECTURE.md — system overview
• src/contracts/agents.ts — 5-agent registry with codename/name/color mapping (source of truth)
• dev_docs/decisions/genesis-decisions.md — D17 dual-surface lock, D21 voice split, D22 display names