Evaluation
Search Quality Assurance with AI as a Judge
Our short summary of Zalando’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on engineering.zalando.com (opens in a new tab)Problem
New markets lacked user data, so search quality was only checked after launch, causing slow fixes.
Solution
They built an LLM-as-a-judge system that automatically evaluates search relevance at scale, using clustered real queries and NER to cover all search scenarios before launch.
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on engineering.zalando.com (opens in a new tab)
PostHog
Gusto