# Benchmarking color judgment in multimodal LLMs with CIE-computable ground truth

**Authors:** Shuwei Yue, Kaida Xiao, Zhihua Wang, Qiao Chen  
**Lab:** Shenzhen Polytechnic University · **Published:** 29 Oct 2026  
**Links:** [arXiv](https://arxiv.org/abs/2610.40212) · [PDF](https://arxiv.org/pdf/2610.40212) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2610.40212.html)

## Abstract

Multimodal large language models (MLLMs) are now used as zero-shot judges of image quality, but their color judgments are rarely tested with colorimetric ground truth. We present a five-task benchmark based on the CIE colorimetric framework: AWB pair judgment, skin-tone bias direction, CCT estimation, CIE Delta E 2000 pair ordering, and color naming under changing illuminants. We test eight MLLM endpoints, including four Qwen3-VL variants, GLM-4.5V, GPT-5.4, Claude-Opus-4.7, and Gemini-3.1-Pro. The models often solve coarse binary AWB judgments, but they remain weak on numeric CCT estimation and on some fine color-difference judgments. On ColorChecker-24 stimuli, the three closed-weight frontier models reach CCT MAE of 715-1056 K, compared with 1358-2133 K for the five open-weight models. A non-ColorChecker control shows that this advantage is partly template-dependent. The results suggest that current MLLMs have useful color words and coarse visual priors, but their color judgments are not reliably calibrated to CIE-computable color differences.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.