
3 hours
What’s stuck for me after trying too many things:
- Ollama (or llama.cpp server if you want control over flags) as the model server, bound to localhost only
- Open WebUI as the front-end; it also handles OpenAI-compatible APIs, so local and hosted models live in the same history
- LiteLLM in between once you have more than one backend: one endpoint, per-key budgets, and logs
- SearXNG as the search tool for anything agent-ish; keep concurrency low or upstream engines start blocking it
Biggest lesson: RAM decides what you can run, and document upload/embeddings in Open WebUI eat more memory than chat does. On small boxes, point embeddings at an API or give it 4 GB.
ARM is a non-issue these days for the usual self-hosted stuff; most popular images ship arm64 builds (check the tags on Docker Hub before you commit to one). For a box at your mum’s, I’d honestly consider staying on macOS + Docker or OrbStack rather than Asahi, purely because remote fixes are easier when the OS is the boring one. Add Tailscale so you can reach it without opening ports. 8 GB is plenty for things like a photo backup, an ad blocker, or a small search engine like SearXNG; it’s tight only if you try to run local LLMs on it.