LLM Gateway Architecture: The Control Plane for AI Apps
An LLM gateway architecture for production AI: routing, semantic caching, rate limits, budgets, fallbacks, and observability across multiple model providers.
LLM Semantic Router: An Inference Routing Pattern
The LLM semantic router pattern in 2026: route requests by intent and cost to the right model, with vLLM Semantic Router, embeddings, and a reference design.