# Benchmark Prediction with Sparse and Unpaired Cached Responses

**Authors:** Hayden Helm, Avijit Ghosh, Ben Johnson, Carey Priebe  
**Lab:** Johns Hopkins University · **Published:** 2 Nov 2026  
**Links:** [arXiv](https://arxiv.org/abs/2611.03614) · [PDF](https://arxiv.org/pdf/2611.03614) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2611.03614.html)

## Abstract

Evaluating a generative model on all queries from all available benchmark tasks is prohibitively expensive. As such, models are often evaluated on different subsets of tasks and/or different subsets of queries from a given task. The resulting collection of responses and the corresponding (model, task) score matrix are sparse and unpaired. In this paper, we introduce the Product Kernel Perspective Space (PKPS), a collection of low-dimensional representations of models based on their embedded responses. PKPS generalizes existing methods for representing black-box generative models by comparing responses to similar -- rather than just identical -- queries and is therefore able to utilize sparse and unpaired information. Theoretically, we show that benchmark prediction using PKPS is query-efficient relative to methods that only use information from dense and paired portions of the evaluation data. Empirically, we show that PKPS-based methods outperform baselines on benchmark prediction across two large benchmark suites.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.