Wire-Speed AI Routing. Deterministic Zero-GC Footprint.
Built in Rust for massive throughput: zero GC pauses, instant RAII reclamation, and just +0.42 ms added latency.
npm i -g exoroute p50 added gateway overhead
Cold startup time to TCP ready
Idle working set (up to 20x lighter)
When Gateways Become the Heavy Bottleneck
Traditional gateways add massive RAM, CPU, and latency bloat before a single token is generated.
Massive Runtime Bloat
Most gateways built with Python or Node.js mandate multi-container Docker deployments, consuming 300MB–600MB of RAM even when completely idle. Heavy dependencies like Redis and PostgreSQL make simple self-hosting a nightmare.
Unpredictable GC Pauses
When thousands of streaming tokens fly across the gateway, sudden Garbage Collector stops spike p99 latency, jittering client connections and breaking the fluidity of real-time streaming experiences.
Resource Leaks & Security Vulnerabilities
Loose buffer management causes runaway memory leaks when clients abruptly drop connections. Many gateways expose unauthenticated endpoints or fail to encrypt secrets at rest.
Direct Confrontation
Traditional AI Gateways
Bloated Stack
- • Runtime: Python / Node.js + Docker stack
- • RAM Idle: 160MB – 560MB+ RAM at Idle
- • Dependencies: Docker, Redis, PostgreSQL mandatory
- • Startup: 6 – 8 seconds to open TCP port
- • Overhead: +5ms to +25ms+ added gateway lag
ExoRoute (Rust Engine) Native
Zero Overhead Architecture
- Runtime: Native Rust 2021 (Single binary)
- RAM Idle: Only 26.4 MiB Working Set (15.0 MiB Private)
- Dependencies: Zero dependencies (Embedded LMDB & Svelte 5)
- Startup: 1.208 seconds cold boot time
- Overhead: +0.42 ms p50 overhead
Engineered for Your Bottom Line
Slash infrastructure costs, eliminate vendor lock-in, and guarantee 24/7 AI reliability.
Slash Infrastructure Bills
Requires merely ~15MB of private memory. Runs effortlessly on the cheapest $3–$5/mo VPS instead of bloated server clusters hogging gigabytes of RAM.
Zero Vendor Lock-In
Keep your existing OpenAI, Anthropic, or Responses client SDKs. Switch between OpenAI, Claude 3.7, or DeepSeek-R1 anytime via a single configuration line.
Local-First Data Privacy
Binds strictly to loopback (127.0.0.1) on your own infrastructure. API keys encrypted with ChaCha20-Poly1305. Prompts never pass through third-party SaaS proxies.
Sub-Millisecond Wire Speed
Built in native Rust without Garbage Collection pauses. Added gateway overhead is just +0.42ms — so instantaneous your users will never detect the difference.
Unbreakable Continuity
Automatically rotates keys and fails over to backup providers during rate limits. Buffers token streams for up to 24 hours if a client connection drops.
Instant 5-Second Launch
A single, self-contained binary (~18MB) containing an embedded ACID database and compiled Svelte 5 Web UI. Download and run — no Docker, Python, or Node needed.
Ready to Experience Real-World Benefits?
Launch ExoRoute in under 5 seconds with a single command and see the difference immediately.
Real-World Performance & Footprint
Direct comparison of ExoRoute against 9router and OmniRoute.
Time to Port Ready (Cold Start)
From process start to TCP connection ready.
One Unified Gateway for Every LLM
Use any client SDK (OpenAI, Claude) to talk to any model without changing code.
Engineered for Unforgiving Production
Equipped with stream continuity, intelligent key rotation, combo failover, and zero-compromise response shaping.
Optional Stream Continuity
When enabled, if the client drops connection mid-stream, a bounded background worker continues consuming the upstream response for up to 24 hours. Reconnect with x-exoroute-stream-id and Last-Event-ID without losing tokens.
Combo Fallback & Automatic Key Rotation
Configure combo aliases with ordered or round-robin failover. ExoRoute fails over only when an upstream response definitively rejects the request—never resending blindly on ambiguous timeouts. Automatically switches provider keys on credential errors.
Response-Shaping Output Styles
Enable static response-shaping styles without mutating client request tools or parameters:
One Command. No Docker. No External Dependencies.
Download the signed native executable verified via Sigstore Cosign and launch your local control center in under 30 seconds.
curl -fsSL https://raw.githubusercontent.com/nghiaomg/ExoRoute/main/install.sh | sh3-Step Quickstart
1. Launch the Gateway
Run exoroute. It prepares ~/.exoroute/, generates a starter .env, outputs the temporary admin password, and serves the UI at http://127.0.0.1:8686.
2. Configure Providers & Combos
Sign in to the dashboard. Add your OpenAI, Anthropic, or OpenRouter keys. Set up your combo alias (e.g. coding-model) with fallback rules.
3. Send Requests Seamlessly
Point your SDK or application base_url to http://127.0.0.1:8686/v1. Enjoy instant wire-speed routing.
curl http://127.0.0.1:8686/v1/chat/completions \
-H "Authorization: Bearer YOUR_EXOROUTE_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "coding-model",
"messages": [{"role": "user", "content": "Explain Rust ownership in one sentence."}]
}'