Evaluation
How we used DSPy to turn AI evaluations into better responses in Dash chat
Our short summary of Dropbox’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on dropbox.tech (opens in a new tab)Problem
Chat agent often gave incomplete answers and used many tokens, hurting user experience.
Solution
They built a layered evaluation system using LLM judges, calibrated it with human labels, then used DSPy to tune the agent’s prompts, creating a feedback loop that improved answer completeness and cut token use.
Fewer incomplete answers and reduced token usage
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on dropbox.tech (opens in a new tab)
PostHog
Gusto