If you don't run your own LLM proxy, how do you know whats going on?

Jared Siegel insists that anyone building with large language models should run their own LLM proxy because provider dashboards and polished analytics are useless for real engineering decisions. He frames the problem with concrete examples - production automation, transforming images into CAD for manufacturing, or simply querying a model for nutrition facts - and criticizes consumer-facing analytics as distractions rather than actionable telemetry. Off-the-shelf metrics and shiny UI screenshots don't tell engineers which key, client, or model caused a failure or why a cache miss happened, so relying solely on vendor tooling leaves teams blind to critical issues.

He lays out practical reasons to self-host a proxy: a single controllable endpoint that can front multiple providers and models, detailed logs tying API keys to models/clients/timestamps/statuses, virtual API keys to avoid leaking a master key, fast pivoting between providers, and infrastructure under your control. The proxy should be moddable and scriptable - his proxy exposes an API, custom endpoints, and instrumented telemetry so he can pinpoint problems in seconds and automate workflows. The bottom line: own the data path, visibility, and storage so you can reliably serve and debug inference rather than trusting opaque vendor analytics.