Inferock Bench

Your model provider counts the tokens, sets the price, decides what counts as a failure and keeps the only detailed log. Inferock Bench is a small tool that gives you an independent record. Run it with one npx command on a laptop or server, point your application at the local address instead of the provider, and every call is logged with provider-reported usage, pricing evidence, status, latency and retry history. A set of detectors then flags the things that quietly cost money: responses that came back empty but were billed, refusals and truncations that cost as much as an answer, token recounts that do not match, duplicate request IDs, and prompt-cache discounts that may not have applied. The dashboard rolls that into four labeled numbers, what you spent, money loss it can tie to the bill, time lost waiting, and an invoice-check exposure, and it is careful not to add unlike things together. Your API keys stay on your machine; the proxy attaches them only to the provider request. The license is the Functional Source License: you can use and modify it freely but not sell a competing service, and each release converts to Apache 2.0 two years after it ships, so it is source-available today rather than open source in the strict sense. The maintainers sell a hosted version, Inferock Watch, Serve and Dedicated, and the benchmark grades receipts against a standard they wrote, which they disclose up front. For spend caps and routing rather than audit, see Helicone; for tracing what an agent did, Langfuse; for the cloud bill as a whole, Vantage. Inferock Bench is the one to run when you suspect the AI line on the invoice is wrong and want evidence.