Kevin Lewis runs his own AI models on a Mac mini in his house — the same machine that backs his Hermes agent, his phone chat apps, and his coding assistant. His essay is a practical case for why he did it, and why he thinks local is no longer a hobbyist compromise.
His core argument is that cloud AI is rented land:
- Providers can change pricing, throttle your usage, or silently swap the model behind the endpoint — he was regularly maxing out two $200/month subscriptions while getting inconsistent quality
- Sending sensitive code or client data to a third party is a decision you can’t undo
- Governments can restrict model availability; owning your compute is the only guaranteed remedy
- After the hardware purchase, every query is free — flat cost, no rate limits, works offline
The piece also teaches you how to read a model name like Qwen3.6-35B-A3B-OptiQ-4bit. The number that matters isn’t 35 billion — it’s the ~3 billion parameters actually activated per token. Mixture-of-experts models spread weights across many “experts” but only wake a few at a time, which is why a 35B-class model fits in 20GB of memory. Compression to 4-bit precision costs only a couple of benchmark points versus the full-quality baseline.
His practical numbers: about 34 tokens per second of generation, model swaps that are just a download and restart, and SSD-backed session caching that makes local machines genuinely workable for agent-style coding loops.
The takeaway is a shift in the default: for most day-to-day AI work — not frontier research, but real tasks — a mid-range machine with open-weight models now competes on quality while winning outright on control, privacy, and cost.