Skip to content

LocalAI Stack Handbook

An independent field manual for LocalAI, LocalAGI and LocalRecall.

Not official documentation

This handbook is not produced, endorsed or reviewed by the LocalAI, LocalAGI or LocalRecall projects. It is independently maintained. Every technical page ends with links to the exact upstream file, release or documentation page it was derived from; when this handbook and upstream disagree, upstream is authoritative and the disagreement is worth reporting.

What this handbook is for

Documentation for these three projects is spread across three repositories, a website, several sets of Docker examples, and a fourth project that neither README names prominently. The result is that a competent engineer can read everything available and still not be able to say which process owns agent state, who computes an embedding, or what happens between a request arriving and a response leaving.

This handbook answers those questions from first principles, and states how each answer was obtained.

Start here

If you read three pages, read these:

  • The ecosystem

    What each project actually is, why there are three of them, and the fourth project — cogito — that owns the agent loop.

  • Architecture

    The stack built up one component at a time. Every diagram labels its edges with the transport, so you can see which calls cross a process boundary.

  • Logical vs physical

    Three names, one to three processes. Which deployment shapes are actually supported, and what each one costs.

The model, and its three errors

LocalAI     = model / compute runtime
LocalAGI    = agent platform
LocalRecall = knowledge / retrieval layer
MCP         = external capability boundary

Correct about responsibilities, misleading about processes:

The model implies Actually
Three deployable services One local-ai process can be all three. LocalAGI links LocalRecall as a library; LocalAI links both.
LocalAGI runs the agent loop The loop is mudler/cogito, a shared library. LocalAI links a much newer version of it than LocalAGI does.
LocalRecall is a vector database It stores nothing itself and computes no embeddings. It calls an OpenAI-compatible embeddings endpoint and writes to a backend.

Learning path

Each step assumes only the ones before it.

Step Page Answers
1 Ecosystem What are these projects, and why three?
2 Architecture How do they fit together?
3 Logical vs physical Which is a service right now?
4 Terminology What do "memory", "agent", "knowledge" mean here?
5 Recipe 1 → Recipe 2 Inference, then embeddings.
6 Recipe 3 Ingest and retrieve documents.
7 Recipe 4 → 6 An agent, then tools, then knowledge.
8 Recipe 7 → 9 MCP, full stack, delegation.
9 Deep dives Why does it behave that way?

Questions this handbook answers directly

Ownership

Interfaces

Operations

Evidence policy

Every architectural claim belongs to one of four tiers, stated explicitly wherever it is not obvious:

Tier Support required
Documented Link to the exact upstream page or README anchor
Source-verified path/to/file.go:LINE at a stated commit or tag
Tested A tested: block with date, versions and environment
Unverified Said to be inference or an open question, in the prose

A page that presents an inference as a fact is a defect. Report it.

Versions

Project Version Notes
LocalAI v4.8.2 released 2026-08-07
LocalAGI v2.9.0 source tag only — no published container image; highest is v2.8.1
LocalRecall v0.6.4 released 2026-07-19

Validated 2026-08-17 on darwin/arm64 under Docker, CPU only. What was not validated is recorded just as explicitly in the version matrix.