Deployment layers¶
Four ways to run this stack, each one a superset of the last:
| Layer | What runs | What it adds | What it hides |
|---|---|---|---|
| Local process | local-ai run from a shell |
Nothing between you and the binary. Paths, ports and logs are visible immediately | Nothing — this is the reference for what everything else does |
| Single container | docker run localai/localai:v4.8.2 |
A pinned userland, the accelerator driver stack, a HEALTHCHECK, a fixed WORKDIR / so paths stop moving |
Which directory the process considers "here", and therefore where state lands |
| Compose | LocalAI + LocalRecall + LocalAGI + PostgreSQL | Service DNS, dependency ordering, healthcheck gating, one env file per stack | Startup ordering failures — depends_on masks the fact that a model load is still in flight |
| Kubernetes | The same containers, scheduled | Restart policy, rolling replacement, GPU scheduling, secret injection, PVC lifecycle | Everything above, plus: probes now decide whether your pod lives, and this stack has startup and shutdown behaviour that defeats the defaults |
Read the layers in that order even if you intend to deploy only the last one. Each layer inherits a defect from the one below, and the defects are easier to recognise where they originate. Two examples that recur on every page here:
${basepath}is the process working directory. LocalAI's path defaults are computed fromkong.ExpandPath(".")(source-verified, v4.8.2). Runlocal-ai runfrom/home/youand models land in/home/you/models. The container image only appears to fix this: it setsWORKDIR /, so the defaults resolve to/models,/backends,/data,/configuration— which happen to match the declaredVOLUMEs. Change the working directory and the coincidence ends. See docker.- The HTTP listener binds last.
application.New()runs every download and preload beforeappHTTP.Start()is called (source-verified, v4.8.2). At the process layer this looks like a slow start. At the Kubernetes layer it looks likeCrashLoopBackOff, because probes get connection-refused rather than a - See kubernetes.
Where state lives in the smallest deployment¶
A single local-ai process contains three logical components and writes to four
directories. Nothing here is a separate service.
flowchart LR
subgraph P["local-ai process"]
INF["inference"]
AG["agent pool<br/>(LocalAGI library)"]
KB["collections<br/>(LocalRecall library)"]
end
M[("/models")]
B[("/backends")]
D[("/data")]
C[("/configuration")]
INF -.->|"fs: /models"| M
INF -.->|"fs: /backends"| B
INF -.->|"fs: /configuration"| C
AG -.->|"fs: /data"| D
KB -.->|"fs: /data"| D
The agent pool and the collections store are Go libraries linked into the same
binary, not sidecars. That is why a deployment which persists /models and
nothing else loses every agent it ever created while appearing to work.
The four volumes¶
Declared by the image as VOLUME /models /backends /configuration /data
(source-verified, v4.8.2, Dockerfile).
| Volume | Contents | Cost of losing it |
|---|---|---|
/models |
Model weights (GGUF, safetensors), per-model YAML, ._gallery_*.yaml metadata |
Re-download of every model. Gigabytes, and rate-limited by the upstream host |
/backends |
Backend OCI artifacts, each unpacked to <name>/run.sh plus metadata |
Re-download of every backend on the next cold model load. Since v3.2.0 backends live outside the binary, so this is not an optional cache |
/configuration |
api_keys.json, external_backends.json, runtime_settings.json, watched by fsnotify and hot-reloaded |
Statically issued API keys, external backend registrations, and every setting saved through the web UI |
/data |
Agent pool state, agent jobs and tasks, the agent collection DB and its assets, the SQLite auth database (database.db), the auto-generated HMAC secret (.hmac_secret), persisted traces, the MITM CA |
Accounts, agents, agent knowledge, job history — and every API key ever issued |
That last row is worth spelling out. LOCALAI_AUTH_HMAC_SECRET has no default:
when unset, startup generates one and writes it to <DataPath>/.hmac_secret with
mode 0600 (source-verified, v4.8.2, core/application/startup.go). API keys
issued through the auth database are stored as HMACs under that secret. Lose
/data and a fresh secret is generated on the next boot, so every key a user
holds stops validating — with no error that names the cause. Either persist
/data or set LOCALAI_AUTH_HMAC_SECRET explicitly from a secret store, and
prefer doing both.
Three more paths are written but not covered by VOLUME:
| Path | Env | Default |
|---|---|---|
| Generated images, audio, video | LOCALAI_GENERATED_CONTENT_PATH |
$TMPDIR/localai-<uid>/generated/content |
| Files API uploads | LOCALAI_UPLOAD_PATH |
$TMPDIR/localai-<uid>/upload |
| System backends (read-only) | LOCALAI_BACKENDS_SYSTEM_PATH |
/var/lib/local-ai/backends |
Upstream's own docker-compose.yaml still mounts /tmp/generated/images/, a
path the code no longer uses after the move to a per-UID temp directory
(source-verified, v4.8.2). Set LOCALAI_GENERATED_CONTENT_PATH explicitly rather
than inheriting that mount.
The other two processes have their own state. LocalAGI keeps everything under
LOCALAGI_STATE_DIR (/pool in its compose file); LocalRecall splits between
COLLECTION_DB_PATH and FILE_ASSETS. Neither declares a VOLUME in its
Dockerfile, and both have a runtime working directory of / — which is a
problem in LocalRecall's case, because its image is FROM scratch and has no
shell to inspect with. The complete map is in
persistence.
What each layer is for¶
Local process. Development, and any Apple Silicon host that wants GPU acceleration — Metal exists only in the native build. See gpu.
Single container. One machine, one accelerator family, state on named volumes. Correct for a workstation or a single GPU box. Recipes per hardware family in docker.
Compose. The point at which LocalRecall and LocalAGI become separate processes and the port collision between LocalAI and LocalRecall (both default to 8080) has to be resolved. The reference stack is described in docker-compose.
Kubernetes. Only worth the cost if you need scheduled GPUs, rolling replacement or multi-node scale-out. Read kubernetes before deciding — the horizontal scaling story is narrower than it looks, because agent pool state is local JSON and chromem vector files are node-local.
Before exposing any of the four to a network you do not control, read security: none of the three projects terminates TLS, LocalAI's default CORS policy is a wildcard, and LocalAGI and LocalRecall are unauthenticated out of the box.
Upstream references¶
Dockerfile—WORKDIR /,VOLUME /models /backends /configuration /data,HEALTHCHECK. Source-verified against v4.8.2, validated 2026-08-17.cmd/local-ai/main.go—${basepath}=kong.ExpandPath("."), dotenv load order. Source-verified against v4.8.2, validated 2026-08-17.core/cli/run.go— path flags and their defaults; listener started last. Source-verified against v4.8.2, validated 2026-08-17.core/application/startup.go— HMAC secret generation into<DataPath>/.hmac_secret. Source-verified against v4.8.2, validated 2026-08-17.docker-compose.yaml— the stale/tmp/generated/images/mount. Source-verified against v4.8.2, validated 2026-08-17.- LocalAGI
cmd/serve.go— state directory resolution. Source-verified against v2.9.0, validated 2026-08-17. - LocalRecall
main.go—COLLECTION_DB_PATHandFILE_ASSETSdefaults. Source-verified against v0.6.4, validated 2026-08-17.