Frugal AI & Harness Engineering: The Token-Conscious Engine

How Dori uses Frugal AI principles and Harness Engineering to minimize token waste, maximize prompt caching, and enforce deterministic reliability.

Most AI applications treat LLM APIs like infinite resources—sending massive context windows, re-parsing raw files on every turn, and burning millions of unnecessary tokens.

Dori takes the opposite approach: Frugal AI. At the core of Dori is the Token-Conscious Engine, built using Harness Engineering to ensure that every LLM call is lean, schema-bounded, and necessary.

1. The Frugal AI Philosophy

Frugal AI means designing intelligence systems that accomplish maximum work with minimal compute, latency, and cloud dependency.

In Dori, Frugal AI is enforced by three core principles:

Never use an LLM for what code can do in 0ms.
Never resend raw files when targeted snippets suffice.
Never accept unverified output that requires manual human cleanup.

2. Harness Engineering: Pre-LLM, In-Flight & Post-LLM

Harness Engineering is the technical discipline of wrapping non-deterministic LLMs in deterministic software boundaries before, during, and after model invocation.

A. Pre-LLM Harness Engineering (Pre-Flight)

Before a request leaves your machine:

Zero-Token Gatekeeper: If a query can be solved via local SQLite lookup or CLI execution (./dori), Dori short-circuits execution with 0 tokens spent.
Tiered Model Routing: Routes routine classification or data extraction to fast models (flash_lite / flash), reserving frontier models (pro) strictly for complex synthesis.
Context Compression: Uses SQLite FTS5 BM25 and vector search to extract only the exact relevant note sections rather than dumping entire files into context.
Prompt Cache Normalization: Freezes static system prompt headers so LLM providers hit Prompt Caching (reducing token costs by up to 90% and latency by up to 80%).

B. In-Flight Harness Engineering (Execution)

During LLM execution:

Strict Schema Boundaries: All tool calls and output objects are validated against Zod schemas (guardrails.ts). Invalid structures trigger immediate internal parameter corrections.
Egress Privacy Shield: Redacts local API secrets and machine credentials before sending payloads over the network.

C. Post-LLM Harness Engineering (Verification & Convergence)

After the LLM responds:

Edge Verifiers: Execution outputs pass through mandatory @guard (security review) and @qa (test verification) nodes.
Test Evidence Harnesses: Document conversions, Whisper audio transcripts, and OCR extractions run through automated test evidence pipelines to verify output fidelity.
Cyclic Convergence Loops: If a verifier node rejects an output, the post-LLM harness feeds the exact failure log back into a retry loop (@guard → @dev) until verifiable criteria pass.