Running local LLMs on your Mac: what fits, what's free, and what's overkill

This explains how virtually any Apple Silicon Mac can run useful language models locally, why Macs are especially well-suited (unified memory, Neural Engine, MLX), and which model sizes fit which machines. Concrete footprints are given for 4-bit quantized models: 1-3B ≈1-2GB (any Mac), 7-8B ≈4-5GB (8GB+), 14B ≈9GB (16GB+), 30B ≈18-20GB (32GB+), and 70B ≈40GB+ (64GB+). Two practical rules are emphasized: leave RAM headroom for other apps, and prefer a tuned smaller model over a generic larger one for focused tasks. The piece points to two free, mature runtimes - Ollama (CLI) and LM Studio (GUI) - as the easiest ways to run open models locally.

The central argument is that local models aren’t replacements for frontier cloud models on hard reasoning or long-form research, but they excel at latency, privacy, cost, and reliability for daily work. The most productive use isn’t a chat window but embedding small, fine-tuned models into everyday tools: local transcription, private file search, and especially on-device autocomplete that learns your writing. The recommended setup is a hybrid workflow - pay for cloud access for the hardest 5% of tasks and run local models for the constant 95% of typing, with on-device fine-tuning required for truly personal assistants like TypeTab’s 1B autocomplete model.