All papersLLMs & Language Models

PolyRoute: A Twelve-Language and Code-Mixed Benchmark for Typed-Decision Routing, with Baselines and a Leaderboard

/ Paper arXiv 2611.13843

Key numbers

91%Labels come from Qwen3.8-27B with a blind second pass (91% of candidates kept)
97.2%agrees on 97.2% of test items
61.2%A leaderboard of 13 systems gives 61.2% for the English-root checkpoint

From the paperCopied from the abstract, not written by AI.

Why this ranked

MeasuredPublic numbers, not AIHow ranking works

5 / 100#136 of 1200 papers that day, from GitHub, Hugging Face, Hacker News and Semantic Scholar, looked up the morning it was ranked.

Builder evidence0 / 40

  • No public code found

Community attention4.8 / 30

  • ▲ 2 upvotes on Hugging Face

Author track record0 / 15

  • Nothing found

What the paper shows0 / 15

  • Nothing found

Abstract

From the paperThe authors' own words.

Typed-decision encoders answer a question about a text in one forward pass, but their behaviour outside English is rarely measured on a fixed yardstick. We release PolyRoute: 322 native-language support messages in 12 languages and 9 writing systems, 156 code-mixed or romanised messages (Hinglish, Spanglish, Arabizi), 967 optional native training rows, and a protocol for a three-way department-routing decision. Labels come from Qwen3.8-27B with a blind second pass (91% of candidates kept); an independent model, gemma4-31b, agrees on 97.2% of test items, and the largest character 5-gram similarity between any test text and any training text is 0.33. A leaderboard of 13 systems gives 61.2% for the English-root checkpoint, 77.3% for the shipped multilingual one and 96.6% for the best fine-tune; TF-IDF with logistic regression reaches 91.9%. On four withheld languages the three multilingual systems score 77%, 90% and 93%. On code-mixed text the English root (85%) is nominally ahead of the shipped multilingual checkpoint (79%; p=0.134). The split resolves differences only above about 4 points. All texts and labels are LLM-generated.

From the full paper

From the paperCopied from the paper itself by our code.

Reading the paper on arXiv…

Ask this paper

/ Ask a questionAnswers come from the full paper and name the section

    / Your notes

    Select text in the abstract to highlight it or add a note. They stay in this browser.

      Read the PDF