Show HN: Maple – Sub-2-BPW Moe Pruning to Run Gemma 4 26B at 45 TPS on a 6GB GPU

MAPLE is a pipeline for shrinking mixture-of-experts language models by removing experts that appear rarely or contribute little activation energy in a chosen domain. It profiles actual model forward passes with llama-imatrix, combines expert-routing frequency with mean input activation energy into an importance score, then selects experts and assembles a reduced GGUF model. For Gemma 4 26B-A4B, the project removes 80 of 128 routed experts in each of 30 layers, retaining 48 experts and, it says, 79.88% of observed active-forward-pass mass. The resulting checkpoint is 4.75 GB, compared with 9.92 GB for the original, and is intended for English technical reasoning and code generation. The scripts are designed to let users repeat the process with their own calibration corpus and target domain.

On an RTX 4050 laptop GPU with 6 GB of memory, the README reports the compressed model runs entirely in VRAM at 45.4 tokens per second, versus 6.9 tokens per second for the original, which offloads 14 layers. A four-task test covering Fibonacci code, binary search trees, Rayleigh scattering, and syllogistic logic yields a reported mean fidelity of 93.8%; the tree task scores 75% without longer context and 100% with at least 500 tokens. The documentation also explains three implementation pitfalls involving inaccurate proxy routing estimates, compounding expert-scale reductions, and sampling penalties that disrupt Python syntax. It provides downloadable weights, setup instructions, and an interactive console.