Recipes¶
Nine recipes, in a fixed order, each introducing exactly one new capability. A recipe that introduced two would hide which one broke.
Work through them in sequence. Recipe 6 is where the whole architecture becomes visible, and it is only comprehensible if recipes 1 through 5 actually ran on your machine.
The path¶
| # | Recipe | Introduces | Components running |
|---|---|---|---|
| 1 | LocalAI chat | model install, Chat Completions | LocalAI |
| 2 | Embeddings | embedding model, vectors, dimensions | LocalAI |
| 3 | LocalRecall RAG | collections, ingestion, chunking, retrieval | LocalAI + LocalRecall |
| 4 | Simple agent | an agent, the agent loop, Responses API | LocalAI + LocalAGI |
| 5 | Agent with tools | tool/function calling | LocalAI + LocalAGI |
| 6 | Agent with knowledge | retrieval inside the loop | all three |
| 7 | MCP agent | MCP as a capability boundary | all three + an MCP server |
| 8 | Complete stack | the assembled system, traced | all three + tool |
| 9 | Multi-agent | delegation between agents | all three |
flowchart LR
R1["1. inference"] --> R2["2. embeddings"]
R2 --> R3["3. retrieval"]
R1 --> R4["4. agent"]
R4 --> R5["5. tools"]
R3 --> R6["6. agent + knowledge"]
R5 --> R6
R6 --> R7["7. MCP"]
R7 --> R8["8. full stack"]
R8 --> R9["9. multi-agent"]
Note the two roots. Recipes 1–3 build the data path; recipes 1, 4, 5 build the control path. They meet at recipe 6. If recipe 6 misbehaves, the question to ask is which of the two paths is at fault — and you will know how to test each in isolation because you built them separately.
Reference models, used throughout¶
The same two models appear in every recipe, so that a failure is attributable to your configuration rather than to a model swap.
| Role | Model | Size | Key property |
|---|---|---|---|
| LLM | qwen3-1.7b |
1.19 GiB Q4_K_M | gallery entry wired for native tool calling |
| Embeddings | granite-embedding-107m-multilingual |
211 MiB F16 | 384 dimensions, upstream's own default |
Both are Apache-2.0 and run on a CPU-only laptop. Reasoning for these choices, and two models to actively avoid, are in the version matrix.
If quality is insufficient, step up to qwen3-4b (2.33 GiB). The configuration is
identical, so nothing in these recipes changes.
Prerequisites for all recipes¶
- Docker with Compose v2
- ~4 GB free disk
curl.jqis optional throughout — it only formats output- No GPU. Acceleration is covered separately in GPU; requiring it here would confuse "the stack does not work" with "the stack is slow"
Recipe conventions¶
Every recipe has the same sections in the same order, so you can jump straight to the one you need:
| Section | What it gives you |
|---|---|
| Verify each dependency | one command per dependency, in start order, with expected output |
| What happened internally | numbered steps, each naming the component and whether it crossed a network boundary |
| Request flow | a sequence diagram matching those steps exactly |
| Persistent state | what was written, by which process, where, and whether it survives a restart |
| Failure modes | symptom → cause → check → fix, with the real error text |
| Troubleshooting | diagnostic ordering: inference, then embeddings, then retrieval, then orchestration, then tools |
Versions tested carries a real tested: block when the recipe was executed. When
it was not, it says so explicitly instead. Nothing is marked tested that was not run
— see evidence policy.
Layer isolation, the one habit worth forming¶
Every troubleshooting section in this handbook asks the same question in the same order. Learn it here and you will rarely need the rest:
1. Is the model runtime up? GET /readyz
2. Does inference work at all? POST /v1/chat/completions
3. Do embeddings work? POST /v1/embeddings
4. Does retrieval return chunks? POST /api/collections/<c>/search
5. Is the agent runtime up? GET /api/agents
6. Does the agent answer without tools? POST /v1/responses
7. Does the tool work when run directly? POST /api/action/<name>/run
A failure at step n makes every result after it meaningless. This is exactly what
scripts/verify-stack.sh
automates:
It stops at the first failing layer and names the likely cause.
Where the recipes deploy from¶
Recipes 1 and 2 use a single docker run, because introducing Compose before you
have seen one process would add moving parts to no purpose. Recipe 3 onward uses
compose/,
where each service is separated deliberately so its edges can be probed.
Kubernetes appears only after all nine recipes, in deployment. Scheduling, service networking, GPU device plugins and persistent volumes are genuinely difficult, and mixing them with "what does an agent do" makes both harder to learn.