AI PAPERS/DAILY
Subscribe free Open sourceView on GitHub Star This project is open sourceRead the code or use it in your own project. If you like it, don’t forget to give it a ★ star on GitHub.
Qdrant
RAG and search

UltraBERT Fits 32K Tokens in One Vector. RIP Chunking?

Our short summary of Qdrant’s engineering write-up. The full story, with all the detail, is on their blog.

Read the full write-up on qdrant.tech (opens in a new tab)

Problem

Long documents either lose context when split into small chunks or get cut off by embedding limits.

Solution

They created a 32K‑token UltraBERT model that embeds whole documents into one vector, compared it to traditional chunking in Qdrant, and added hybrid search and 1‑bit quantisation to reduce size.

Research behind this, on AI Papers DailyContextual Embeddings Beyond the Gold Chunk
+ Suggest a write-up

Want the architecture, code and numbers? They’re in the original post.

Read the full write-up on qdrant.tech (opens in a new tab)

More from RAG and search

All RAG and search write-ups