Show HN: LLMxRay an open-source observability tool for LLMs

LLMxRay is an open-source local observability interface for Ollama-run LLMs that exposes token-level behavior, quality analysis, performance profiling, and cost tracking without cloud services or API keys. Launch with one command (npx llmxray) or Docker (docker run -p 5174:5174) and connect to a local Ollama daemon. It streams tokens live with confidence/latency coloring, supports multi-turn chat, file attachments, vision models, and reasoning effort controls. Key user-facing tools include a side-by-side model comparison workbench, a RAG/embeddings lab with local knowledge base and vector visualization, a visual tool-workshop for building and testing tools, a fill-in-the-middle playground, and an archival IndexedDB history for experiments and training-data curation.

Observability features include automated quality gates (repetition, refusal, gibberish, truncation), detailed performance analytics (latency percentiles, TTFT, usage heatmaps, cold vs warm starts), and a cost dashboard that estimates cloud-equivalent spend. Specialized labs provide actionable insights: Cache Lab detects KV-cache reuse losses and demonstrated a 3.5× prefill speedup on a 324-token prompt by reordering a timestamp; Decision Lab displays decision-model probability distributions; Protocol Observatory compares native, OpenAI- and Anthropic-compatible endpoints side-by-side. Surgical Benchmark scores models using true token logprobs. The tool is multilingual, stores embeddings locally, and lists Ollama endpoint/version compatibility (tested on Ollama 0.40.x; some features require Ollama 0.33.3+).