Local AI on a NAS

A NAS (network attached storage) is already switched on all day, has plenty of disk space and runs containers, so it makes a good always-on home for a small model and a chat page everyone in the house can open. It is a poor fit for big models: almost every NAS runs models on its CPU, and generation speed follows memory speed (bandwidth), not the number of drive bays. This guide covers which boxes qualify, what they can really run, how to set one up and how to keep it private. We have not run these steps on every NAS system ourselves; where a figure comes from a maker’s page, a published test or an owner’s report, we say so.

1. Check that your NAS qualifies

• A 64-bit processor and containers. You need the maker’s container app: Container Manager on Synology (DSM 7.2 or later), Container Station on QNAP, Apps on TrueNAS, Docker on Unraid, and the Docker app on UGREEN, ASUSTOR, TerraMaster and ZimaOS. QNAP’s own Ollama tutorial asks for Container Station 3 and an Intel or AMD model, and says only a few ARM models are supported.
• Enough memory. QNAP’s guidance is at least 8 GB for 3B models and 16 GB for 7B models. Entry-level units with 1 to 4 GB of soldered memory can’t run a useful model.
• Room to add memory. Many units ship with less than they can take. Synology’s DS925+ ships with 4 GB and takes 32 GB; the DS1825+ ships with 8 GB. Budget for a memory upgrade before anything else.
• A maker “AI” app is not always local. Assistants that ask for an OpenAI or Gemini key send your prompts to that company. Nothing runs on the NAS. On each NAS product page we list which maker apps run a model on the box itself.

2. What a NAS without a graphics card can run

Measured figures for an actual Synology, QNAP or UGREEN unit are rare. The closest published test is Jeff Geerling’s Ollama benchmark of an Intel N150 mini PC with 16 GB, the same chip family as many current NAS boxes: Llama 3.2 3B at about 9 tokens per second and a 14B model at about 2, drawing 26 to 30 watts. A Ryzen AI 5 340 in the same test ran the 3B model at about 24 tokens per second, which is the class a Core i5 or Ryzen AI NAS belongs to. Our own figures on each hardware page are estimates from memory speed.

• Good jobs: summarising, tagging, short questions, Home Assistant voice replies, embeddings for document search (local RAG) and overnight batch work.
• Poor jobs: coding agents, long documents and anything interactive with a 7B or larger model (how speed feels).
• The built-in graphics rarely help. Owners who passed the Intel graphics into the Ollama container report no speed-up. Count on the CPU (no graphics card).
• Use two matched memory sticks where the NAS has two slots and a two-channel chip. One stick roughly halves memory speed, and tokens per second with it. Intel N100, N150 and N305 boxes have a single memory channel, so a second stick adds room, not speed.

3. Install Ollama and Open WebUI

The recipe is the same everywhere: two containers. Ollama runs the model and Open WebUI is the chat page with user accounts.

1. Ollama: image ollama/ollama, a folder on the NAS mounted at /root/.ollama (the models live there), port 11434. The official command is docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama.
2. Open WebUI: image ghcr.io/open-webui/open-webui:main, container port 8080 published as 3000, a folder mounted at /app/backend/data. The first account you create becomes the administrator.
3. Join them: put both on one Docker network and set OLLAMA_BASE_URL=http://ollama:11434 in Open WebUI. Use the container name, not localhost and not https. With this layout you don’t need to publish port 11434 at all.
4. Pull a small model first: ollama pull llama3.2:3b in the Ollama container’s terminal, or from Open WebUI’s admin settings.
5. Check it: ollama run llama3.2:3b --verbose prints the speed in tokens per second, and ollama ps shows whether the model is on the CPU or a graphics card.

Where to click on each system:

• Synology DSM: Package Center, install Container Manager, make a shared folder such as docker/ollama, then Project, Create and paste a compose file with the two services.
• QNAP: Container Station, Applications, Create, paste the compose file. QNAP’s tutorial mounts /share/Container/ollama and recommends an SSD volume and a memory limit.
• TrueNAS (24.10 or later): Apps, Discover Apps, install Ollama and then Open WebUI from the Community train. The Ollama app publishes its API on port 30068, not 11434, and starts with a limit of 2 CPUs and 4096 MB. Raise both or anything bigger than a 3B model will fail or crawl.
• Unraid: install Community Applications, then the Ollama and Open WebUI templates. Keep their data under /mnt/user/appdata on the cache pool.
• UGREEN, ASUSTOR, TerraMaster, ZimaOS and others: open the Docker app and use the same two-service compose file.

Ollama is not the only choice. llama.cpp’s server, LocalAI and others publish container images too; see apps that run on a NAS.

4. Keep the models on an SSD

• Storage speed affects loading only. Once a model is in memory it generates at the same speed from any disk. QNAP says an SSD can load a model up to ten times faster than hard drives.
• Loading happens often. Ollama unloads a model after five idle minutes, so the first question after a pause reloads several gigabytes, and sleeping hard drives have to spin up first. Keep models on an SSD or NVMe pool, or raise OLLAMA_KEEP_ALIVE (-1 keeps the model loaded).
• A Synology SSD cache is not a place to store files. Make an M.2 storage pool and put the Docker folder there.
• Always map the folders. Without a folder mounted at /root/.ollama and /app/backend/data, models and chats are lost when a container is updated.
• Stay off the array during a scrub, parity check or rebuild. Both jobs compete for the same disks.
• Disk space is not memory. A 70B model is a 43 GB file that any NAS can store and almost none can run.

5. Leave memory for the NAS itself

Size the model against free memory, not installed memory: take off the NAS system, your other containers (Plex, Immich, Home Assistant, virtual machines) and the file cache. On TrueNAS the ZFS cache uses whatever memory is spare; it gives memory back under pressure, but file serving slows when it has to. Our NAS hardware pages already hold back a share of memory for this.

• Set a memory limit on the Ollama container so a model that is too big fails cleanly instead of stalling the whole NAS.
• Set OLLAMA_MAX_LOADED_MODELS=1. The default lets several models stay loaded at once.
• Leave the context at its default until you know how much memory is free (context and memory). Use the memory calculator to check.
• Keep generation away from backups and video transcodes. Generating text uses every core.

6. Keep it private

• Ollama has no login. Anyone who can reach port 11434 can use it. Internet scans keep finding open Ollama servers: Cisco Talos counted over a thousand in 2025, about a fifth of them answering prompts from anyone.
• Never forward port 11434 or 3000 on your router, and switch off UPnP port mapping on the NAS.
• Publish only Open WebUI to your home network. It has accounts. Leave Ollama reachable only inside the Docker network unless other computers need it.
• From outside the house, use a VPN such as Tailscale or WireGuard, not a port forward. If you use Tailscale, use Serve, which stays inside your own network; Funnel publishes to the internet.
• Chats stay on the NAS only while no cloud service is set up. Adding an OpenAI key or web search to Open WebUI sends those prompts out (private and offline).

7. Adding a graphics card

A model that fits entirely in a card’s memory runs many times faster than on the NAS processor. Three things decide whether a card will work:

• The slot and the system. A PCIe slot is not automatically a graphics slot. Synology lists its slots for network cards only and has no supported graphics path. QNAP supports NVIDIA cards in some models, through its GPU driver app and Container Station mode. TrueNAS and Unraid support NVIDIA cards with a driver option or plugin; on Unraid, add --gpus=all to the Ollama container.
• Space. Most NAS cases take only short, low-profile cards. QNAP’s TVS-h874, for example, takes cards up to 185 mm long.
• Power. Most have no spare power lead, so the card must run from the slot’s 75 watts. Cards that do: NVIDIA RTX A2000 (6 or 12 GB), RTX 2000 Ada (16 GB), RTX 4000 SFF Ada (20 GB) and Intel Arc Pro B50 (16 GB), all about 168 mm long and 70 watts. No NAS maker’s system ships Intel Arc drivers for containers, so plan on TrueNAS, Unraid or Proxmox for the Arc.
• External cards. NAS-style mini PCs with an OCuLink port, running TrueNAS, Unraid or Proxmox, can use a graphics card in a dock; one owner reports an RTX 3060 working this way on a Minisforum N5. Shut down before plugging OCuLink in. Thunderbolt ports on Synology, QNAP and similar systems are not documented for external graphics cards.
• Check it is being used. If the driver or the container setting is missing, Ollama quietly stays on the CPU. ollama ps or nvidia-smi shows the truth.

Each NAS product page says whether the unit has a usable slot. To choose a card, see choosing a graphics card. If your NAS has no usable slot, a mini PC or desktop with a graphics card beside it costs less than replacing the NAS.

8. Use the NAS as the model library or the front door

• Model library. Set OLLAMA_MODELS on a desktop (or your app’s model folder) to a share on the NAS. The desktop loads the file over the network once and runs it on its own graphics card, so the network affects load time only: a 5 GB model takes about 45 seconds over gigabit Ethernet and about 20 over 2.5 gigabit (our arithmetic). Let one computer do the downloading and deleting.
• Front door. Run Open WebUI on the NAS, which is always on and holds the accounts and chats, and point it at Ollama on a PC with a graphics card (OLLAMA_BASE_URL=http://PC-address:11434). The PC does the work when it is on (the local API).
• In a cluster. A NAS can join a llama.cpp RPC or LocalAI cluster as a worker or hold the shared model files, though a slow NAS processor adds little speed (clusters).

What to expect

• Speed. One to three tokens per second means the model is too big for the box. Drop to a smaller model or a smaller file (quantization).
• Power. An idle model uses no processor time. While generating, an N150-class box draws about 26 to 30 watts in total; a 70-watt card adds its draw only while it works.
• Heat and noise. NAS cases are built to cool drives, not a processor at full load. Owners report the chip running hot during generation, so check drive temperatures after a long session and keep the fan off its quietest setting.

See every NAS chip under Hardware and every unit under Products, with the models each can run and how fast. On a desktop or laptop, press Detect on My hardware; a NAS has no browser of its own, so pick yours from the list.