# Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models

**Authors:** Xinhe Zhang, Arnau Marin-Llobet  
**Lab:** Harvard · **Published:** 29 Oct 2026  
**Links:** [arXiv](https://arxiv.org/abs/2610.40328) · [PDF](https://arxiv.org/pdf/2610.40328) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2610.40328.html)

## Abstract

Language models often need to maintain multiple contextual variables that evolve independently within a document. Prior work has identified structured representations of individual temporal or contextual quantities, but their joint organization remains less understood. We study this question using controlled documents with independently phased language-switch and topic-change timers. Across ten pretrained checkpoints, both timers are linearly decodable from the same hidden states. Removing a timer-associated centroid plane substantially reduces access to its associated variable while having little effect on the concurrent timer, although the planes are nonorthogonal and contain cross-timer information. Separate timer contributions also explain most reproducible variation among joint-state centroids. These signatures persist from 3 to 51 timer values and across Qwen3 checkpoints from 0.6B to 14B. Schedule position affects continuation probabilities, but the tested centroid displacement produces no consistent timer-specific output effect. Together, these results support approximate factorization based on selective accessibility and approximately additive joint-state geometry in overlapping subspaces.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.