sui Free tiers, verified before use.

A gateway that proves the route before it takes it.

A route answers a probe before it answers a prompt, and every failover is traced in plain language. Paid credentials sit in a sealed vault that is not opened unless you open the gate.

tests
529
no network, no keys
p95 overhead
1.13 ms
budget 15 ms
invariants
19
each with a test
catalogue
29
endpoints, all awaiting a probe

Install

Node 20 or newer. No runtime dependencies, no build step. Not yet on npm, so npx has nothing to download — clone it and run the wrapper that ships with the code.

git clone https://github.com/drsamonline/sui
cd sui && npm install

# macOS / Linux
./sui

# Windows (PowerShell or CMD)
.\sui.cmd

That is the whole setup: the wrapper runs the source directly, no global install. Want a sui command anywhere? npm link once. Then point anything that speaks OpenAI at http://127.0.0.1:8787/v1 — the first run ends with a real round-trip through the gateway. Windows guide · cURL, Node, Python, PHP.

What it does

Routing

  • OpenAI-compatible endpoint, streaming and not
  • Four strategies: fallback, weighted, cheapest, latency
  • Per-target circuit breaker, three states
  • Budgets per day and per provider
  • Free-first ordering: a free target outranks a paid one, always
  • Speculative cascade: a 50 ms classifier picks cheap or strong
  • Quota windows: a 429 with Retry-After closes a target for exactly that long
  • Fallback learning from measured health, not declared order
  • Two-tier cache: exact, plus lexical or hashed-vector similarity
  • Quota-aware prompt compression, seven engines, code preserved
  • Savings ledger: dollars avoided, tokens never sent
  • Verification escalation: weak answers are never called verified
  • Explain-my-route, in sentences, without sending a request
  • Red-team self-test: 11 obfuscations, a random canary per run
  • Native prompt caching: the hint each provider understands
  • Grounding check: a claim that is not in the context costs confidence
  • Residency: a request can require a region, and an unmarked provider never satisfies it

Trust

  • Paid air-gap: gate in code, vault sealed with AES-256-GCM
  • tos: avoid is never auto-routed
  • Prompts hashed, never stored — and health history has no field for one
  • Host-header allowlist, SHA-256 API keys, constant-time comparisons
  • Bounded runs: spend, steps, time and loop limits per agent run
  • Two-plane split: the agent-facing half holds no provider key
  • Owner licence grant, enforced by a test
  • WCAG AA verified numerically on every token pair

Operating it

  • Dashboard: budget, savings, free-tier weather, topology, breaker states
  • Ledger with cost, latency, first token, cache, verification
  • CSV export, gzip via compress-kit
  • Onboarding that detects your tools and writes their config
  • Migration from another router with a preview diff
  • MCP server over stdio and HTTP, and MCP servers as providers
  • Recipes: five named setups that never overwrite your work
  • sui local finds a model server on this machine
  • Drift detection (EWMA + CUSUM) and trend history on disk
  • Tenant console: what each key spent, and what it may reach
  • Bounded runs: spend, steps, time and loop limits per agent run
  • Sandbox: sui sandbox, a separate process, 14-case attack suite
  • Two-plane split: the agent-facing half holds no provider key

Proving it

  • 50-prompt golden set, nine deterministic graders
  • Self-test that proves the graders grade
  • Soak harness gating p95 against a 15 ms budget
  • Invariant table: 19 statements, each with an enforcer and a test
  • Two audits in the repository, with accepted risks stated
  • Regression gate on every PR

Where it stands

A row says done when a command proves it, and the evidence column names the command.

AreaStateEvidence
Gateway, streaming, SSEdone14 server tests
Routing + breaker + budgetsdone17 router tests
Invariants as datadone19 enforced, audited by CLI
Paid gate + sealed vaultdone15 kit and vault tests
Eval harness + regression gatedone56 self-checks, baseline gate
Dashboard + onboarding + migrationdonescreenshots, 22 UI tests
Cataloguepartialentries loaded; resolution needs egress
Canary proberdone3 levels, 16 tests, auto disable
Worker schedulerdoneprober + cache sweep, /api/workers
Empty-turn retrydoneretry once, then fail over
Compression enginesdone7 engines, 4 presets, cache-aware
Quota budget managerdone60/30/10% thresholds
Output axis + savings ledgerdone4 styles, /api/savings
Prompt caching + per-key limitsdoneprefix hints, 429 + Retry-After
Audit + legal docsdone8 checks in CI, PRIVACY, TERMS
MCP server + clientdone6 tools, stdio and HTTP, 21 tests
Bounded runs + approvalsdone4 bounds, single-use gates, 15 tests
Two-plane splitdonecredential boundary, 14 tests
Recipes + local discoverydone5 recipes, loopback scan, 18 tests
Drift + trend historydoneEWMA/CUSUM, /api/history, 21 tests
Sandboxed code executiondoneseparate process, netns, 14-case suite
Windows supportdonewrappers, installer, guide
Legal setdoneLICENSE, NOTICE, DISCLAIMER, PRIVACY, TERMS, SECURITY, index
Budget correctnessdoneper-day buckets, not the record window
Verification + explain-my-routedone7 signals, /api/explain
Savings + explain panelsdonedashboard, /api/savings
Canary + SLO rollbackdoneoutages excluded, 12 tests
Weather Map + tutorialsdonetrend lines, 7 walkthroughs

Honest limits

Not done yet

  • 29 catalogue endpoints are seeded but unverified. sui probe --write on a networked machine is what promotes them.
  • No eval baseline is recorded, because that needs one working provider key.
  • The 72-hour soak has not been run.
  • Nothing is published to npm, so npx has nothing to download.

Stated plainly

  • The vault passphrase sits beside the vault unless you supply SUI_VAULT_PASSPHRASE
  • The lexical cache tier is lexical, not semantic
  • Upstream kits keep their PolyForm NC terms; this repository distributes them under a written owner grant, which is not available to a third party
  • No first-chunk clone, by choice: two providers billed for one answer is a real cost for a fake latency win, so a mid-stream drop is reported rather than spliced