Compress any LLM 3x in one click, near-lossless, in under 50 MB RAM

SPECTRA is a 4-bit model-compression system and GPU inference runtime designed to run large language models on low-memory consumer and edge GPUs. It claims to work across transformer, state-space, and hybrid architectures, compressing Hugging Face or local models without calibration data. Its method quantizes eligible two-dimensional weight matrices in groups of 32, uses special handling for outlier channels, and retains structural vectors such as normalization weights and biases in their original precision. Shard-by-shard processing is designed to keep host RAM below 50 MB. Models can be compressed through a Python API or command line, then loaded through a CLI or web dashboard; the project also advertises rapid GPU loading and a quantized key-value cache.

Reported tests on a 4 GB RTX 3050 Ti cover models from about 1.7 billion to 4.15 billion parameters. Compressed files range from 1.00 to 2.63 GB, with compression taking two to 4.4 minutes and cold loading around 3.2 to 3.9 seconds. The reported generation rates vary widely, from 9.58 to 82.53 tokens per second, with all models fully resident in GPU memory. For DeepSeek-R1-Distill-1.5B, the project reports 35.58% ARC-Challenge accuracy versus 34.73% for the BF16 baseline, 91% first-token agreement, and matching semantic task completion across its test. The software is marked for non-commercial research use.