Recommendations
Advancing Generative Recommendation Inference: our collaboration with NVIDIA on TensorRT LLM
Our short summary of Spotify’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on engineering.atspotify.com (opens in a new tab)Problem
Personalized recommendations needed real‑time, multi‑surface inference that could scale beyond a proof‑of‑concept.
Solution
Spotify built a generative recommendation system that uses a large language model with beam search to generate semantic IDs, then switched from vLLM to NVIDIA TensorRT LLM for GPU‑optimized inference.
Research behind this, on AI Papers DailyTAMEing the Open-World Personalization: Towards Open-Set Personalized MLLM Assistant
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on engineering.atspotify.com (opens in a new tab)