Agent platforms
TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor
Our short summary of NVIDIA’s engineering write-up. The full story, with all the detail, is on their blog.
Read the full write-up on developer.nvidia.com (opens in a new tab)Problem
Edge devices struggled to run multi‑step AI agents quickly within tight power and memory limits.
Solution
Created TensorRT Edge‑LLM that runs a 27B model on Jetson AGX Thor using FP4 quantization, tree‑based multi‑token prediction, and KV‑cache reuse, cutting inference time dramatically.
6.4× faster than reference
Want the architecture, code and numbers? They’re in the original post.
Read the full write-up on developer.nvidia.com (opens in a new tab)
AWS