Single containers¶
One container per project, on one host. This is the layer where tag selection, path configuration and port allocation are decided; Compose and Kubernetes only repackage those decisions.
Images that exist¶
| Project | Registry path | Current tag |
|---|---|---|
| LocalAI | quay.io/go-skynet/local-ai, mirrored at localai/localai (Docker Hub) |
v4.8.2 |
| LocalAGI | quay.io/mudler/localagi |
v2.8.1 — see below |
| LocalRecall | quay.io/mudler/localrecall |
v0.6.4, plus v0.6.4-postgresql |
None of the three publishes to ghcr.io. Queries against the GitHub packages API
for mudler/local-ai, mudler/localagi and mudler/localrecall all return
404 Package not found (observed 2026-08-17).
quay.io/mudler/localagi:v2.9.0does not exist. The highest published tag on the main LocalAGI image isv2.8.1;latestwas last modified 2026-02-16. The main image build has produced nothing sincemain-20260415165223, while the separatelocalagi-sshboximage did publish av2.9.0tag — which makes it easy to conclude the main image exists when it does not (observed 2026-08-17 against the Quay tag API). Pinv2.8.1, or build from source.
The eight LocalAI tag suffixes¶
A substring query for v4.8.2 against the Quay tag API returned exactly eight
tags (observed 2026-08-17). The same eight names exist in the latest… form and
on the Docker Hub mirror.
| Tag suffix | Accelerator | Platforms | Compressed size |
|---|---|---|---|
| (none) | CPU | amd64, arm64 | 0.29 GB |
-gpu-nvidia-cuda-12 |
NVIDIA, CUDA 12.8 | amd64 | 3.73 GB |
-gpu-nvidia-cuda-13 |
NVIDIA, CUDA 13.0 | amd64 | 2.84 GB |
-gpu-hipblas |
AMD ROCm, base rocm/dev-ubuntu-24.04:7.2.1 |
amd64 | 4.25 GB |
-gpu-intel |
Intel oneAPI/SYCL, base intel/oneapi-basekit:2025.3.2-0-devel-ubuntu24.04 |
amd64 | 4.63 GB |
-gpu-vulkan |
Vulkan, LunarG SDK 1.4.335.0 + mesa | amd64, arm64 | 1.11 GB |
-nvidia-l4t-arm64 |
Jetson / L4T (JetPack r36), CUDA 12 | arm64 | 6.13 GB |
-nvidia-l4t-arm64-cuda-13 |
Jetson / DGX Spark, CUDA 13 | arm64 | 4.50 GB |
The image size is a floor. Every v4.8.2 image ships with no models and no
backends — a clean instance reports [] from both /v1/models and /backends
(observed 2026-08-17). Backends are pulled as OCI artifacts on first use, pinned
by digest; installing granite-embedding-107m-multilingual also fetched
quay.io/go-skynet/local-ai-backends:latest-cpu-llama-cpp, 42 MiB (observed
2026-08-17).
AIO images are gone, and old tags still resolve¶
There is no aio/ directory anywhere in the v4.8.2 tree, and the 4.0 release
notes state the removal directly: the images existed to bundle a preset of models
and maintaining them per hardware variant stopped being worthwhile (documented).
The word "AIO" survives only as the name of a nightly end-to-end test suite that
builds the standard image.
A latest% query returns 31 tags; only the eight above are current (observed
2026-08-17). The other 23 resolve to frozen builds:
| Stale tag family | Frozen at |
|---|---|
latest-aio-cpu, latest-aio-gpu-{vulkan,nvidia-cuda-12,nvidia-cuda-13,hipblas,intel} |
2026-02-21 |
latest-aio-gpu-nvidia-cuda-11, latest-gpu-nvidia-cuda-11 |
2025-12-25 |
latest-{aio-,}gpu-intel-f16, …-f32 |
2025-07-28 |
latest-gpu-nvidia-cuda11, latest-gpu-nvidia-cuda12 (no dash) |
2025-07-25 |
latest-vulkan |
2025-07-24 |
latest-cpu |
2025-06-19 |
*-extras, *-ffmpeg-core, latest-nvidia-l4t-arm64-core |
2025-04 / 2025-05 |
latest-cpu is the trap that matters, because LocalAI's own compose files still
name it: docker-compose.yaml and docker-compose.distributed.yaml both offer
localai/localai:latest-cpu as a commented-out image alternative (source-verified,
v4.8.2). It has been stale for over a year. The current CPU tag is plain latest
or v4.8.2. LocalAGI's docker-compose.nvidia.yaml has the same problem with
localai/localai:master-cublas-cuda12-ffmpeg, a naming scheme that no longer
exists (source-verified, v2.9.0).
Two more dead knobs propagated by upstream compose files: IMAGE_TYPE=core is
passed as a build arg by both, and the Dockerfile no longer declares that ARG;
there is no FFMPEG build arg either, because ffmpeg is now installed
unconditionally (source-verified, v4.8.2). Drop both rather than copying them.
${basepath} is the working directory¶
LocalAI's storage defaults are kong template expansions of
${basepath}, which is kong.ExpandPath(".") — the process working directory
(source-verified, v4.8.2, cmd/local-ai/main.go).
| Flag | Env | Default |
|---|---|---|
--models-path |
LOCALAI_MODELS_PATH, MODELS_PATH |
${basepath}/models |
--backends-path |
LOCALAI_BACKENDS_PATH, BACKENDS_PATH |
${basepath}/backends |
--data-path |
LOCALAI_DATA_PATH |
${basepath}/data |
--localai-config-dir |
LOCALAI_CONFIG_DIR |
${basepath}/configuration |
Inside the official image this resolves to /models, /backends, /data,
/configuration, because the final stage sets WORKDIR /. That agreement
between WORKDIR and VOLUME is the only reason the shorthand recipes work.
It is not a guarantee: run the binary natively from a different directory, or
override WORKDIR, and the process silently uses a different tree. Upstream's
docker-compose.yaml sets MODELS_PATH=/models and leaves the other three to
the coincidence.
Set all four explicitly in anything you intend to keep:
docker run -d --name localai \
-p 8080:8080 \
-e LOCALAI_MODELS_PATH=/models \
-e LOCALAI_BACKENDS_PATH=/backends \
-e LOCALAI_DATA_PATH=/data \
-e LOCALAI_CONFIG_DIR=/configuration \
-e LOCALAI_GENERATED_CONTENT_PATH=/generated \
-v localai-models:/models \
-v localai-backends:/backends \
-v localai-data:/data \
-v localai-config:/configuration \
-v localai-generated:/generated \
localai/localai:v4.8.2
Note /configuration is declared as a VOLUME but is not among the
directories pre-created by the image's mkdir (which covers /models,
/backends, /data only) — source-verified, v4.8.2.
Use named volumes, not bind mounts from network shares, for
/backends. Backend OCI artifacts contain symlinks. On CIFS/SMB or any filesystem without link support every link is materialized as a full regular file (documented,containers.md).
Ports¶
| Process | Default | Knob | Dockerfile EXPOSE |
|---|---|---|---|
LocalAI (run, federated, explorer) |
:8080 |
LOCALAI_ADDRESS, ADDRESS |
8080 |
| LocalAGI | :3000 |
LOCALAGI_BASE_URL |
none |
| LocalRecall | :8080 |
LISTENING_ADDRESS |
8080 |
| LocalRecall PostgreSQL image | 5432 | — | 5432 |
LocalAI and LocalRecall collide on 8080. Both bind all interfaces by default.
Upstream resolves this by publishing LocalAI on 8081:8080 in both LocalRecall's
and LocalAGI's compose files; this handbook's reference stack keeps LocalAI on
8080 and moves LocalRecall. Pick one convention and hold it — mixed conventions
across tutorials are the single most common cause of "the embedding call goes to
the wrong service".
LocalAGI's README says the UI is at http://localhost:8080. That is the host
side of the 8080:3000 publish in its compose file; the container port is 3000,
as its own curl examples show (source-verified, v2.9.0).
Auxiliary LocalAI ports, all off by default:
| Port | Purpose | Knob |
|---|---|---|
| ephemeral UDP | WebRTC ICE media for /v1/realtime |
LOCALAI_WEBRTC_UDP_PORT, then publish -p N:N/udp |
| 50051 | Distributed worker gRPC base | LOCALAI_SERVE_ADDR |
| 50050 | Worker HTTP file transfer, gRPC base − 1; also serves the worker's /healthz and /readyz |
LOCALAI_HTTP_ADDR |
| dynamic | libp2p, when --p2p is on |
LOCALAI_P2P_LISTEN_MADDRS |
There is no separate metrics listener. /metrics is served on the main port.
LocalAI per hardware family¶
Every recipe below assumes the five -e/-v pairs from the explicit-paths
example; they are elided for readability. Driver-level detail, capability
resolution and the VRAM floor are in gpu.
CPU — the plain tag, amd64 and arm64:
NVIDIA, CUDA 12 or 13:
docker run -d --name localai --gpus all \
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility \
-p 8080:8080 localai/localai:v4.8.2-gpu-nvidia-cuda-12
utility is not optional. Without it nvidia-smi and NVML are unavailable in
the container, free-VRAM reporting breaks and the nodes UI misreports memory —
upstream says so in its own compose comments. The image already sets
compute,utility; preserve it if you override the variable.
NVIDIA Jetson / DGX Spark:
docker run -d --name localai --runtime nvidia --gpus all \
-p 8080:8080 localai/localai:v4.8.2-nvidia-l4t-arm64-cuda-13
Use -nvidia-l4t-arm64 for CUDA 12 JetPack r36 devices.
AMD ROCm:
docker run -d --name localai \
--device=/dev/kfd --device=/dev/dri --group-add=video \
-p 8080:8080 localai/localai:v4.8.2-gpu-hipblas
Host requirements are ROCm ≥ 7.0.0 with amdgpu-dkms, and upstream asks for at
least 100 GB free on the disk holding the container runtime before installing the
ROCm stack (documented, GPU-acceleration.md).
Intel oneAPI / SYCL:
docker run -d --name localai \
--device /dev/dri -e ZES_ENABLE_SYSMAN=1 \
-p 8080:8080 localai/localai:v4.8.2-gpu-intel
Upstream's README names individual device nodes
(--device=/dev/dri/card1 --device=/dev/dri/renderD128); passing the directory
is equivalent and survives renumbering. LocalAGI's Intel overlay hardcodes
card1 / renderD129, which is correct only on a host with both an integrated
GPU and a discrete card.
Vulkan:
The only GPU variant that also builds for arm64, and 1.11 GB against 3–6 GB for the vendor stacks.
Apple Silicon gets no GPU in a container at all. Verified: zero
metal/darwin/mps/apple matches across the Dockerfile and every image
workflow (source-verified, v4.8.2). See gpu.
LocalRecall¶
FROM scratch, a static CGO_ENABLED=0 binary, no WORKDIR in the final stage
(source-verified, v0.6.4). Unset paths therefore resolve to /collections and
/assets at the filesystem root, inside an image with no shell — you cannot
docker exec your way into diagnosing it. Set both explicitly.
docker run -d --name localrecall \
-p 8082:8080 \
-e LISTENING_ADDRESS=:8080 \
-e COLLECTION_DB_PATH=/db \
-e FILE_ASSETS=/assets \
-e EMBEDDING_MODEL=granite-embedding-107m-multilingual \
-e OPENAI_BASE_URL=http://host.docker.internal:8080 \
-e OPENAI_API_KEY=unused \
-e API_KEYS=change-me \
-v localrecall-db:/db -v localrecall-assets:/assets \
quay.io/mudler/localrecall:v0.6.4
Four things about that command:
OPENAI_BASE_URLcarries no/v1. That works against LocalAI because it registers un-prefixed aliases for/chat/completionsand/embeddings. Against a strict OpenAI-compatible server, append/v1yourself.OPENAI_BASE_URLis applied unconditionally. Leaving it unset does not fall back toapi.openai.com— it overwrites the client default with the empty string (source-verified, v0.6.4).API_KEYSis the only thing standing between the network and the whole REST surface. Unset means unauthenticated, including the web UI.COLLECTION_DB_PATHis required even withVECTOR_ENGINE=postgres: thecollection-<name>.jsonmanifests are still written there and are what collection discovery scans at startup.
quay.io/mudler/localrecall:latest-postgresql, which LocalRecall's own README
tells you to pull, is not a real tag — the workflow sets latest=false for
that build variant, and it is absent from all 218 live tags (observed
2026-08-17). Use v0.6.4-postgresql.
LocalAGI¶
docker run -d --name localagi \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
-e LOCALAGI_MODEL=qwen3-1.7b \
-e LOCALAGI_LLM_API_URL=http://host.docker.internal:8080 \
-e LOCALAGI_STATE_DIR=/pool \
-e COLLECTION_DB_PATH=/pool/collections \
-e FILE_ASSETS=/pool/assets \
-e EMBEDDING_MODEL=granite-embedding-107m-multilingual \
-e LOCALAGI_API_KEYS=change-me \
-v localagi-pool:/pool \
quay.io/mudler/localagi:v2.8.1
LOCALAGI_MODEL and LOCALAGI_LLM_API_URL are both required — an empty value
prints help and exits. The image declares no EXPOSE and no VOLUME, and the
final stage has no WORKDIR, so the state directory fallback
filepath.Join(cwd, "pool") resolves to /pool (source-verified, v2.9.0).
Once LOCALAGI_API_KEYS is set, every route requires the key, including /
and /app: the auth middleware is registered with a Next function that returns
false for all paths, so nothing is exempt (source-verified, v2.9.0). Any
HEALTHCHECK or probe must carry an Authorization: Bearer header.
Note the pinned tag. Running v2.9.0 from a published image is not possible; the source tag exists and the container does not.
Health checks¶
LocalAI ships one. HEALTHCHECK --start-period=60m --interval=1m --timeout=10s
--retries=3 CMD /healthcheck.sh, and the script is mode-aware — it reads argv
and probes /readyz on :8080 for run, /readyz on the gRPC base port minus
one for worker, and exits 0 for the subcommands with no HTTP surface
(source-verified, v4.8.2). The 60-minute start period exists because a startup
preload has been observed materializing 31 GB of artifacts before the socket
binds; failures inside the start period leave the container starting rather
than burning retries, and the period ends early on the first success.
Neither LocalAGI nor LocalRecall ships a healthcheck, and neither has a health endpoint. Hand-write one:
# LocalRecall — the project's own convention, used in its Makefile
--health-cmd 'curl -fsS -H "Authorization: Bearer $API_KEYS" http://localhost:8080/api/collections || exit 1'
# LocalAGI — /app is the cheapest 200
--health-cmd 'curl -fsS -H "Authorization: Bearer $LOCALAGI_API_KEYS" http://localhost:3000/app || exit 1'
Do not copy /readyz for either. LocalAGI's own end-to-end test calls
/readyz against LocalAGI and asserts only that the HTTP call did not error — a
404 passes it (source-verified, v2.9.0). The route does not exist.
Verifying a container¶
docker logs --tail 40 localai
curl -fsS http://localhost:8080/readyz && echo ready
curl -fsS http://localhost:8080/v1/models | jq .
curl -fsS http://localhost:8080/system | jq .
curl -fsS http://localhost:8082/api/collections -H 'Authorization: Bearer change-me'
curl -fsS http://localhost:3000/api/agents -H 'Authorization: Bearer change-me' | jq .
On a clean LocalAI, /v1/models returns {"object":"list","data":[]} and
/system returns {"backends":[],"loaded_models":[]} (observed 2026-08-17).
Empty is the correct answer, not a failure.
Upstream references¶
.github/workflows/image.yml— the eight-suffix build matrix. Source-verified against v4.8.2, validated 2026-08-17.Dockerfile—WORKDIR /,VOLUME,EXPOSE 8080,HEALTHCHECK, absentIMAGE_TYPE/FFMPEGargs. Source-verified against v4.8.2, validated 2026-08-17.scripts/build/healthcheck.sh— mode-aware probe selection. Source-verified against v4.8.2, validated 2026-08-17.docker-compose.yaml—latest-cpuandIMAGE_TYPE=corein comments and build args. Source-verified against v4.8.2, validated 2026-08-17.website/content/blog/what-landed-in-localai-4-0.md— AIO images dropped in 4.x. Documented, validated 2026-08-17.docs/content/getting-started/containers.md— no preconfigured models; symlink caveat for/backends. Documented, validated 2026-08-17.- LocalAGI
cmd/env.go—:3000default, required variables. Source-verified against v2.9.0, validated 2026-08-17. - LocalAGI
webui/routes.go— auth middleware with no exempt paths; the 45-route list with no health endpoint. Source-verified against v2.9.0, validated 2026-08-17. - LocalAGI
Dockerfile.webui— noEXPOSE, noVOLUME, no final-stageWORKDIR. Source-verified against v2.9.0, validated 2026-08-17. - LocalRecall
Dockerfile—FROM scratch,EXPOSE 8080. Source-verified against v0.6.4, validated 2026-08-17. - LocalRecall
main.go— env surface, unconditionalOPENAI_BASE_URLassignment. Source-verified against v0.6.4, validated 2026-08-17. - LocalRecall
.github/workflows/image.yml—latest=falsefor the PostgreSQL variant. Source-verified against v0.6.4, validated 2026-08-17. - Registry tag counts, tag timestamps, image sizes, ghcr 404s: observed 2026-08-17 against the Quay, Docker Hub and GitHub packages APIs.
- Clean-instance
/v1/models,/system,/backendsresponses and the backend artifact pull: observed 2026-08-17 onlocalai/localai:latestreportingv4.8.2 (5ff25d9d).