Cost and speed
Accelerating Gemini Nano models on Pixel with frozen Multi-Token Prediction
Our short summary of Google’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on research.google (opens in a new tab)Problem
On-device LLMs were too slow and drained battery, hurting user experience.
Solution
They added Multi-Token Prediction to frozen Gemini Nano models, letting the phone generate several words at once and cut energy use.
Research behind this, on AI Papers DailyPhrase-Level Speculative Decoding on a Frozen Target
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on research.google (opens in a new tab)