All papersML Theory & Optimization

Data-Efficient Scaling-Law Discovery for Recommendation Systems via Coupled Hyperparameter Scaling

/ Paper arXiv 2611.05643

Key numbers

0.66%the workflow yielded a mean 0.66% offline normalized-entropy improvement and 0.36% cumulative online lift
0.36%the workflow yielded a mean 0.66% offline normalized-entropy improvement and 0.36% cumulative online lift

From the paperCopied from the abstract, not written by AI.

Abstract

From the paperThe authors' own words.

Large-scale ranking and recommendation models of Web platforms benefit from increased compute much as large language models do, and their scaling behavior is broadly power-law in form. Yet ranking models have heterogeneous architectures which makes their scaling law less straightforward than that of LLMs. We address how to discover the scaling law of an arbitrary ranking model efficiently. We introduce a scaling layer, which groups primitive architecture parameters into semantically coherent groups and defines how they co-vary, inducing a mapping from a low-dimensional space of hyper-hyperparameters into the full architecture space. On top of it, Axia fits scaling-informed surrogates, selects configurations by a posterior-based acquisition criterion, and returns a Pareto frontier from which per-model operating points are validated -- replacing exhaustive sweeps with a small number of informative runs. A fleet-level evaluation protocol, computed on a fixed representative model set under matched trial budgets, makes refinements to either component measurable. Our contribution is an automated system that learns model-specific complexity-quality relationships and gives uncertainty-aware guidance for allocating compute across a heterogeneous model ecosystem. Across 34 model iterations in two development cycles on Meta's ads ranking and retrieval system, the workflow yielded a mean 0.66% offline normalized-entropy improvement and 0.36% cumulative online lift.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Where it fits

The most cited papers it builds on, and the papers that build on it.

/ Builds on 3 most cited

  1. 201210.4k citedPractical Bayesian Optimization of Machine Learning Algorithms ↗
  2. 20209.6k citedScaling Laws for Neural Language Models ↗
  3. 20001.8k citedPredicting the output from a complex computer code when fast approximations are available

/ Built on by None yet

No papers cite it yet. We check again every week.

From Semantic Scholar, updated weekly.

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      Read the PDF