hn.today

Is this the best inference for agents?

runinfra.ai4 points1 comments
Screenshot of Is this the best inference for agents?

RunInfra is an inference platform built for agent workflows that emphasizes fast responses, cached context to cut input costs, and unified access to open models through a single API key compatible with OpenAI and Anthropic SDKs. It implements session-aware routing so repeat calls hit the same replica and remain “warm,” reports cache hits in usage and bills cached tokens at lower rates, and publishes a cache-measurement methodology. Security and governance features include zero data retention by default, TLS/AES-256 encryption, SOC 2 Type II attestation, role- and row-level workspaces, and an explicit “no training” policy on prompts and completions. The site also offers CLI setup for coding agents, pricing plans, and a marketplace-style model library.

Model-specific metrics and effective pricing are central: GLM 5.3 FLASH lists $0.11/1M input ($0.03 cached), $0.45/1M output, 99.3% cache hits, 254.1 output tok/s, effective input $0.031/1M; DEEPSEEK V4.1 FLASH $0.14/$0.03/$0.58, 99.2% hits, 378 tok/s, effective $0.031; NEMOTRON 3.5 LIGHTNING 30B $0.05/$0.01/$0.15, 540.9 tok/s (cache rate not published); QWEN3.8 27B $0.10/$0.01/$0.40, 71.1% hits, 140 tok/s, effective $0.037; ORNITH 1.5 35B $0.10/$0.01/$0.40, 94.2% hits, 216 tok/s, effective $0.016. Cache-hit rates are presented as best-effort and used to compute “effective input” prices that can be multiple times lower than list rates.

Read on runinfra.ai1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.