LensVLM-9B by Apple

LensVLM is an inference framework and post-training recipe that lets vision-language models read rendered text as compressed images by scanning low-resolution renderings and selectively expanding only the regions needed for accurate recognition. It addresses the core failure mode where characters shrink below the vision encoder’s effective resolution under high compression. Implemented on top of Qwen3.5-9B-Base and released as a 9B model variant, the approach uses learned tools to restore relevant text or image patches on demand, preserving a fixed-size visual token budget while avoiding full-resolution encoding of entire pages.

Empirical results show LensVLM preserves accuracy comparable to an uncompressed text upper bound at about 4.3× effective compression and outperforms retrieval-based, text- and visual-compression baselines up to 10.1× across seven text question-answering benchmarks. Performance gains increase on multimodal document and code-understanding tasks as compression intensifies. Analysis demonstrates that training makes visual compression robust to rendering choices and that the model progressively relies on expanded content rather than unreliable visual reading as compression grows. Practical guidance emerges: expand text tokens for rendered text and use high-resolution image expansion for native documents where layout cues matter.