Show HN: A decent text-to-image model trained for ~$20K (Apache 2.0 Weights)

Linum’s two-person research team presents Pyramid-JiT, an experimental text-to-image model released with Apache 2.0 weights and code. The $21,038 project is intended as a stepping stone toward faster, cheaper generative video, where attention’s quadratic cost makes reducing image tokens especially valuable. Unlike conventional diffusion systems that generate from a separate VAE’s latent space, P-JiT works in pixel space and uses 32×32 patches, cutting the attention context to a quarter of that in the team’s earlier model. Intermediate prediction heads and feature-alignment training help strengthen learning through the network. The 2.18-billion-parameter model was trained for 660 million samples: 610 million at 512-pixel resolution and 50 million at 1024 pixels, using 8,415 H100 hours in total.

The team reports that the longer training run produced image quality that exceeded its expectations for a model of this size, and treats the result as a proof of concept rather than a finished product. Training cost about $20,348, with another $690 spent on preference-based post-training. The system uses a frozen Qwen text encoder and combines image-prediction losses with perceptual and feature-alignment objectives. Its limitations are significant: it was not trained on text-heavy images, so it cannot reliably render words or make posters, slides, or document pages. It also struggles with full-body subjects, crowds, multiple interacting people, and some world knowledge. The team suggests that larger models and further post-training could improve these weaknesses.