# Latent-Lens: Visual Perception in Small Language Models Through Latent Communication

**Authors:** Mo Shahdloo, Faegheh Sardari, Mathew Salvaris  
**Lab:** Microsoft · **Published:** 30 Oct 2026  
**Links:** [arXiv](https://arxiv.org/abs/2610.41751) · [PDF](https://arxiv.org/pdf/2610.41751) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2610.41751.html)

## Key numbers

- **1.6M**: while training only ~1.6M parameters

## Abstract

Bridging independent vision-language models (VLMs) and small language models (SLMs) in a modular multi-agent system typically forces them to exchange information as discrete text, which creates a lossy bottleneck for spatial and fine-grained details. This constraint is especially acute for SLMs, whose limited capacity makes them more reliant on the quality of the input signal. Recent latent-communication methods exchange states between language models. How a VLM can pass visual information to a separately pretrained text-only SLM though remains unexplored. We propose \textbf{Latent-Lens}, a lightweight codec that bridges the representation gap between a frozen VLM and a frozen SLM directly in latent space, enabling them to communicate purely through hidden states. The codec consists of a domain encoder that projects hidden states from the VLM into the SLM's embedding space, and an attention decoder formalised as a LoRA that teaches the SLM to attend to these visual prefixes, while training only ~1.6M parameters (0.2\% of the combined model size). On visual question answering (CLEVR, GQA), Latent-Lens achieves 81.0±7.9\% and 65.4±5.4\%, compared with 58.3±5.7\% and 61.8±1.0\% for text communication with the same LoRA budget (+30.2--69.8pp relative to frozen text communication), while on image captioning (Flickr8k) it improves over the same baseline by +16.5 (+18.8 frozen). We also evaluate Latent-Lens with TinyLlama-1.1B and InternVL2-1B, which use different architectures, vocabularies, and hidden dimensions.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.