Evaluation
Stop Building Models. Start Building Systems.
Our short summary of Scribd’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on tech.scribd.com (opens in a new tab)Problem
Need a scalable, cheap way to generate and validate labels as LLM models and prompts evolve.
Solution
Created a repeatable human‑in‑the‑loop pipeline: fast model generates candidates, a higher‑capability judge model reviews samples, SMEs edit a slice, and results are ingested for iterative improvement.
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on tech.scribd.com (opens in a new tab)
Gusto
PostHog