Evaluation
Workbench: Building an AI Content Generation and Quality Assurance System at Scale
Our short summary of Thumbtack’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on medium.com (opens in a new tab)Problem
Creating and reviewing hundreds of thousands of landing page descriptions manually was impossible.
Solution
Thumbtack built Workbench, an AI pipeline that writes, automatically checks, and publishes marketing copy at scale, using automated quality checks and a bandit selector for variants.
97% drafts passed QA, $13K cost
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on medium.com (opens in a new tab)
PostHog