A packaged local AI stack
- A new local AI stack switches between chat and image models — A small container-based project offers a way to try both on one machine without keeping both models in graphics memory.
- Solène Rapenne’s project combines a model server, image server, SearXNG search and the Open WebUI interface using Podman containers.
- The author says the active model loads when needed and unloads after a few idle minutes. Search requests still leave the computer.
- The setup requires Linux and rootless Podman 5.0 or newer. The author tested AMD graphics cards with up to 12 GB of graphics memory; the smallest suggested models need roughly 4 GB of graphics memory and 8 GB of system memory.
- The default model downloads total about 8 GB. The author describes a larger model running partly in system memory and taking minutes to answer.
- Caveat: The author has not tested the project on Nvidia graphics cards, so it is not a verified drop-in for an Nvidia machine. (lobste.rs · author’s guide and project repository)
For earlier developments, see the actual Morning edition.
Gathered items not included (6)
- DeepSeek Super-Agent report → DEDUP: Morning’s coding-tools section already covers the same prototype; the new article adds no checked release change.
- Small-model team workflow → DEDUP: Morning’s coding-tools section already covers the experiment and limitations; more HN engagement does not change its findings.
- Framework cost experiment → DEDUP: Morning’s coding-tools section already covers the same measurements.
- Deepgram and Daily → STALE: the report dates to October 29; its reappearance provides no checked afternoon development.
- OpenAI policy plans → UNVERIFIABLE: the full report was blocked in the available retrieval, so the collected headline alone is insufficient for an item.
- Brazilian court filing case → STALE: Morning noted that the report dates to July despite its new HN listing.