Cost and speed
Cutting our inference bill by 68% by being obsessive about optimization
Our short summary of Cloudflare’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on blog.cloudflare.com (opens in a new tab)Problem
Inference costs for code reviews kept rising, hurting the budget.
Solution
They traced token usage, improved the reviewer code, switched models, and increased cache hits, cutting review cost by 68% and speeding up reviews.
Cost per review $0.77
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on blog.cloudflare.com (opens in a new tab)