Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Granite Switch 4.1 is a modular model family that serves chat, decision-making, and speech-to-text from a single hosted endpoint. The 8B and 30B variants include three composed adapters: a general Decision aLoRA, a specialized Guardian aLoRA that outperforms several competing decision models on safety-oriented tasks, and Granite-Speech-5.0-Turbo-CTC - currently the fastest ASR on the OpenASR leaderboard. These components are embedded as cache-efficient Activated LoRAs so the combined model occupies roughly the same GPU VRAM as the base model, and reuses the base model KV cache across adapter switches, allowing near-instant switching between conversational, classification, and transcription requests with accelerated inference from vLLM optimizations and a co-located ASR encoder.

Evaluation and demos demonstrate that specialized granitelib adapters outperform generalist decision models on targeted classification tasks while incurring minimal additional cost because adapters share the KV cache. In benchmarks, a 30B Granite-Switch with four specialized aLoRAs completed a five-step RAG workflow faster than Jev-27B, and a custom 3B Granite executed game-play decisions at 60-100 decisions/sec versus Jev’s reported 10 decisions/sec while also narrating and answering questions. The release ships “batteries included” but supports user composition and custom adapter training via an upcoming Adapter SDK, with native inference support in vLLM (plugin) and llama.cpp and with model cards, a Speech-to-Decisions demo, and a GitHub repo available for immediate use.