Better Vector Search for Long Documents: Chunking Inside Manticore Search

Manticore Search adds automatic chunking inside model-backed vector columns so long documents aren’t silently truncated by a model’s input window. By setting chunk_strategy on a float_vector or float_vector_array column, Manticore splits each stored document, embeds chunks, and searches them without requiring an external splitter, an extra table, or post-query aggregation. Five strategies are provided: truncate and mean (produce one vector per document and use float_vector), and fixed, recursive, and sentence (produce multiple vectors and require float_vector_array). Chunks compete individually but a document is returned once with knn_dist() reflecting its closest chunk; k counts documents. Tunables include max_tokens (chunk size), overlap_tokens, and max_chunks. Queries themselves are not chunked.

Measured on a 189‑page manual with content intentionally placed beyond a model’s window, automatic chunking raised recall@5 from 55.1% to 83.3% and MRR from 0.44 to 0.70, at a cost of roughly 2.5× RAM and ~4× ingest time. A compact example contrasts truncate, mean, and sentence strategies to show how buried content becomes retrievable. The feature is presented as a minimal, declarative change to table definitions that preserves document-level results while improving retrieval of long documents.