AI Gateway Use Cases
Organized by "what you want to achieve" — strings gateway capabilities into actionable plans, each linking to the full step-by-step in the docs.
These use cases come from what developers and architects actually ask about on Reddit, StackOverflow, and Zhihu — cost, high availability, multi-tenancy, compliance/audit, and troubleshooting. Each ties community pain points to GateLLM capabilities and links to the full how-to and config reference in the docs.
Reduce LLM Cost
Pain: Monthly invoice arrives, then you find out you burned $10k
Fix: BYOK, no token markup + Composition + real-time cost attribution
High Availability
Pain: One provider outage takes the whole app down
Fix: Multi-node failover + cross-provider fallback + shared state across instances
Multi-Tenant Isolation
Pain: One shared API key, no idea who spent what
Fix: Key groups + Header ACL + per-tenant rate limit & billing
Compliance & Audit
Pain: Data can't leave the country, PII must be redacted, audit logs required
Fix: Self-hosted + two-layer redaction + logging + retention
Troubleshoot by Symptom
Pain: Streaming broke / 429 / 502, no idea where it stuck
Fix: z-request-id end-to-end tracing + normalized error codes
Mixed Multi-Vendor Model Stack
Pain: Claude Code on Opus all the way is too costly; dropping to Sonnet degrades planning
Fix: Pin top models per agent role across vendors, zero client change
AI in Office, Data Stays In-house
Pain: Copilot routes contracts & financial models through the cloud; sensitive files can't leave the network
Fix: Self-hosted Office add-in to your own gateway, zero SaaS dependency
Token Usage Visibility & Attribution
Pain: One key shared by a dozen agents — you can only guess who is burning money
Fix: Break usage and cost by person / key / key group × model
Turn AI Spend into an Allocable Ledger
Pain: Vendor list price is not your real cost, and it can't be charged back to teams
Fix: Self-owned rate card + tag allocation + daily/monthly cost quotas
MCP Tool Governance
Pain: Agents wired to a pile of MCP servers, no control over who can call which tool
Fix: Unified /mcp endpoint + tool-level ACL + call audit
Web Search for Every Model
Pain: Open models have no search; standing up RAG just for that is too heavy
Fix: Gateway search injection + native search blocks + not billed
Multi-Key Rotation & Standby
Pain: One key hits 429 rate limits and the business just waits
Fix: Weighted multi-key + sticky binding + standby keys
Admission Control for Connected Clients
Pain: Any key that leaks lets any client — a script, a rival SDK, a scraper — talk to your models
Fix: Header ACL rules: allow/deny by request header across global / group / key scopes
Enforce Which Model Agents Actually Run
Pain: A dev flips Claude Code to Opus to summarize a changelog, and no one can stop it
Fix: Ingress allowlist + catch-all model mapping + plan-mode script split
No Model Choice on the Client
Pain: Everyone has a different model in their settings, so the same prompt comes out differently
Fix: One model name on the client; the gateway tiers automatically by plan vs execution