WorldTUI: I trained a 360M model to pretend to be a terminal

A 360M-parameter language model was fine-tuned to act as a terminal emulator that predicts screen updates per keypress rather than running a shell or filesystem. The terminal is represented as a character grid and the model consumes sequences describing initial snapshots, raw keypress bytes (including escape codes and WAIT tokens), and outputs compact screen deltas: row writes, COPY operations to implement efficient scrolling, and cursor state. Training data were synthetically generated by running real programs in Docker with randomized prompts, files, personas, and realistic typing (including typos), producing 336k episodes (~2B tokens). Supervised fine-tuning began from Qwen2.5-Coder-0.5B and was augmented with a DAgger-like self-correction procedure where the model predicts several steps, differences are detected, and repair deltas are taught so it learns to fix its own mistakes; this improved benchmark performance substantially. Total fine-tuning cost was about 75 GPU-hours on an RTX 4090.

Making it interactive required three optimizations: parallel decoding by generating each row as a separate batch element (reducing large-frame p95 latency from 6.7s to 0.87s on an RTX 4090), speculative drafting+verification to restore cross-row consistency, and shrinking the vocabulary to 333 tokens so the model could be quantized to int4 and run in-browser (208MB). Benchmarks show major gains: on a 106-command shell test the fine-tuned model scores 80 vs 52-60 for prompted bases, and it recalls more file values (9/13 vs 2-3), but persistent filesystem behavior remains unreliable in complex sequences. A live web demo runs in desktop browsers with WebGPU.