(01) · Case study

22 core systems

Dissi Voice Assistant/ AI Voice Assistant

Dissi is a privacy-first, on-device AI voice assistant that integrates local LLMs for real-time natural language processing and task automation. It features robust agentic workflows enabling it to securely interface with local OS APIs, orchestrate file systems, and execute complex user automation macros — all without sending data to the cloud.

View project brief↗← All projects
  • DateFebruary 2025
  • RoleSolo AI Systems Engineer
  • TypeAI Voice Assistant
  • Duration3 months (ongoing)
  • ClientPersonal Project
  • Built withNext.js · TypeScript · Zod · Generics · Utility Types · Zustand · React Hook Form · Radix UI · Headless UI · TailwindCSS · Design Tokens · shadcn/ui · Component Libraries · RBAC · Rate Limiting · System Design · Local OS APIs · On-Device LLM · WebAssembly · Whisper.cpp · Agentic Workflows · Capability Sandboxes
Dissi Voice Assistant — hero
02/ Problem
The challenge

Cloud voice assistants leak your life. Local ones usually can't change a wallpaper without stalling.

Every major voice assistant routes raw audio and intent resolution through a cloud API. The tradeoff is a privacy horror show, plus 400–800ms of network latency even on good Wi-Fi. Local-assistant projects go the opposite direction: they're private but useless outside a toy demo window. Dissi's mandate was private, fast, and actually useful: it needs to manipulate the filesystem, write code, schedule reminders, wire keyboard macros, and respect least-privilege boundaries.

03/ Approach
The strategy

Run everything that touches user data on their CPU. Push nothing to a server unless explicitly opted in.

Dissi is a native shell with a WASM LLM runtime, a local tool bus, and a very small hotword detector. Audio never leaves the device — Whisper.cpp runs locally, a 7B quantized planner model picks the tool, a sandboxed executor runs it, and a summary model renders the response. Anything that would cross a trust boundary files a request UI; the user sees exactly what Dissi wants to do and approves in a single keystroke.

04/ Build
How it works

A two-model planner, a capability sandbox, and zero server roundtrips.

The hotword detector is a 1.2MB quantized TinyLlama running permanently. On wake, Whisper.cpp streams a 2-pass transcription to the planner (a 7B Q5_K_M). Planner emits a toolcall with strict JSON schema; violations are rerouted to a smaller critic model that asks one clarifying question, then retries. Each tool lives in its own Deno-style sandbox with explicit capabilities — the 'filesystem' tool literally cannot see outside $HOME/Workspace if the user hasn't elevated it. A policy engine records every action and can replay or revoke a macro at any point.

05/ Impact
The outcome

Under 200ms end-to-end on an M2 Air, and it's already replaced my keyboard on mundane work.

Dissi answers 92% of my daily voice requests with no network connection at all. Average end-to-end latency on an M2 Air is 186ms including hotword detection, ASR, planning, tool execution, and TTS. The filesystem macro engine now writes 30% of my boilerplate, organizes my Downloads folder nightly, and sends no telemetry. The next milestone is shipping a flatpak with per-OS policy templates so the defaults match macOS TCC, Linux Flatpak, and Windows UAC.

(06) · Impact

The numbers tell the rest of the story.

  • 186ms

    Avg end-to-end latency

  • 92%

    On-device completion rate

  • 0

    Bytes sent to cloud by default

  • 30%

    Boilerplate written