Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins

Respan is an open-source Rust AI gateway designed around a tiny core and a plugin ecosystem so every provider, protocol, modality, and gateway feature is implemented as a plugin. It was born from operational pain maintaining a large LiteLLM wrapper - over 100k lines and brittle upgrades - so the design emphasizes extensibility without touching the core. Providers (OpenAI, Anthropic, Gemini, etc.), protocols (chat, realtime voice, MCP), modalities (images, video), and reliability/security features (fallbacks, smart routing, PII redaction) are all plugins. Adding a provider is intentionally simple - a complete provider implementation can be a short file (Mistral’s example is 22 lines) and quirks are isolated and tested as separate plugins. The project is pre-alpha, MIT/Apache-2.0 licensed, runnable locally from source, and also offered as a hosted platform with observability and unified billing.

Technically, adapters hand HTTP or WebSocket connections to a client format that exposes a bidirectional stream of typed input/output events; the core dispatches these events and runs plugins in registration order. Plugins can choose providers, filter or wrap streams, make extra calls, or refuse requests. If client and provider share a format and plugins don’t change payloads, bytes and protocol-specific fields can passthrough unchanged; otherwise formats translate via shared types for messages, tool calls, streams, and usage. The team aims to grow a public plugins ecosystem (documentation and examples in PLUGINS.md) and welcomes contributions for routing, tools, filtering, observability, and security.