I had Gemini train its own replacement for $9

A developer scraped Reddit knife threads to extract brand, model and steel mentions, had Gemini label 4,290 comments for $9, and fine-tuned GLiNER large v2.5 (459M) to replace paid API calls. Labels were requested as exact substrings so code could compute offsets; the dataset contained about 1,575 positive examples, 675 negatives, 3,907 entity spans (1,720 brands, 1,345 models, 842 materials) and a locked 225-comment validation set. Zero-shot GLiNER scored roughly 0.65 F1 versus Gemini; the plan was to pay once for labels, train a local model, and stop billing per comment. Training used a Tesla T4 with gradient accumulation, small batches and a 0.45 detection threshold.

Ten training runs revealed that most failures were plumbing, not modeling. Three config mistakes and two runs broken by a misused words_mask tensor prevented learning: words_mask is a per-word index (0 for special/pad, then 1,2,3…) used to pool subtokens, not an attention mask, and filling it with ones made the loss flat. After fixing that and tuning (per-class thresholds, negative sampling, early stopping), results stabilized: a 209M checkpoint hit 0.80 F1 and the 459M model 0.83 F1 against Gemini labels, with material recall of 0.911 after per-class thresholds. Total cost was about $11.50; the main lesson is that small fine-tuning projects typically fail in the glue code, and explicit assertions on hand-built tensors save days of debugging.