Strands Decider: Why Not an Encoder?

Marc Brooker compares using a modified causal decoder (strands-decider-2B v19) versus encoder and encoder-decoder architectures to decide which is better for strands-decider tasks. He tests five designs: v19 (Qwen3.5-based, ~2B), B1 (T5Gemma2 encoder-decoder 1B+1B), B2 (4B+4B ≈8B), E1A (encoder-only T5Gemma 2.6B), and E1B (a latency-optimized masked encoder variant that shares state via a masked cache). Frozen ModernBERT and ettin-encoder-1b were measured but underperformed T5Gemma per parameter and were not pursued further.

Results show B2 wins on raw accuracy and calibration but is much larger. E1B slightly outperforms v19 on JevBench accuracy (within v19 run variance) with marginally worse calibration and ties on JF100, while encoder-only models deliver much lower median latency (~35ms vs v19’s 65ms). Encoders dominate at short prompts but scale worse as prompt length grows because E1B uses 26 full-attention layers (O(N^2)) versus v19’s mix of Gated DeltaNet layers with linear cost. E1A outperforming v19 on seven of nine tasks but performing worse on JevBench and JF100 implies masked-state handling affects generalization. Conclusion: no clear winner at this scale; results are confounded by architecture, size, and tuning differences, and the experiment code remains in a branch.