The Responses API¶
POST /v1/responses is how an application talks to an agent. It borrows OpenAI's
Responses shape, and the borrowing is partial in ways that matter if you are writing a
client against it.
The single most important difference from every other endpoint in this stack:
The contract¶
curl -s http://localhost:8081/v1/responses \
-H 'Content-Type: application/json' \
-d '{
"model": "handbook-probe",
"input": "What is 2+2? Answer with just the number."
}'
Observed response:
{
"id": "b9ec1e4d-41c5-42b7-a18a-d12d21040be4",
"object": "response",
"created_at": 1786981102,
"status": "completed",
"error": null,
"model": "handbook-probe",
"output": [
{"type": "message", "id": "msg_1786981102342119587", "status": "completed",
"role": "assistant",
"content": [{"type": "output_text", "text": "4", "annotations": null}]}
],
"usage": {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0},
"store": false,
"temperature": 0,
"top_p": 0,
"tool_choice": "",
"parallel_tool_calls": false,
"previous_response_id": null,
"metadata": null
}
Send a model name instead of an agent name and you get HTTP 500 with
{"error":"Agent not found"}. Verified.
What is faithful, and what is not¶
Fields that behave as an OpenAI client would expect:
| Field | Behaviour |
|---|---|
input |
string or message array, both accepted |
output[].content[].text |
the answer |
status |
completed on success |
previous_response_id |
request side works — conversation is carried |
tools |
user-defined tools are accepted and separated from built-ins |
tool_choice |
honoured when {"type":"function","name":"…"} |
Fields that do not:
| Field | Reality |
|---|---|
usage |
always zero. Hardcoded, with // TODO: calculate actual usage beside it |
id |
a bare UUID, not resp_… |
previous_response_id in the response |
always null, even when you sent one |
stream |
accepted and silently ignored — no handler reads it |
store |
reported false; accurate, nothing is persisted |
temperature, top_p |
reported as 0; set them on the agent or the model, not here |
stream is the one that bites¶
The field exists on the request type — webui/types/openai.go:161 — and no handler
reads it. Verified by grep across the whole webui and core trees: there are no
references to it outside its declaration.
So a client asking for a stream receives a single complete JSON body, with a 200, and no
indication that its request was not honoured. A client written to consume SSE will hang
waiting for events that never arrive, then fail on parsing.
If you need streaming, use LocalAI's /v1/chat/completions, which does support it, or
subscribe to LocalAGI's own SSE channel — see alternatives.
usage being zero is not a small thing¶
Token accounting is impossible at this layer. If you need it — for cost attribution, for
quota, for anything — take it from LocalAI's side, where each underlying
/chat/completions call reports real numbers. Note that one agent request produces
several such calls, so you must sum them.
Conversation state¶
flowchart LR
R1["request 1"] -->|"returns id A"| T[("ConversationTracker<br/>in-memory map")]
R2["request 2<br/>previous_response_id: A"] -->|"reads A"| T
T -->|"stores under NEW id B"| T
Mechanics, verified:
previous_response_idlooks up prior messages in an in-memory map.- New input is appended and the whole list is passed to the agent.
- The reply is appended and stored under a freshly generated UUID, returned as
id.
Each turn therefore has a new identity. Conversation is a chain of per-turn IDs, not one session ID. Continuing from an older ID branches rather than errors.
Observed:
request 1 → id b9ec1e4d-41c5-42b7-a18a-d12d21040be4 "I remember your favorite color, teal."
request 2 (previous_response_id: b9ec1e4d…) → id 00c6abcf-a427-420b-a35d-2ef05f9cb160
"My favorite color is teal."
Three indistinguishable failure states¶
An unknown previous_response_id does not error. Verified:
curl -s http://localhost:8081/v1/responses \
-H 'Content-Type: application/json' \
-d '{"model":"handbook-probe","input":"What is my favourite colour?","previous_response_id":"does-not-exist-at-all"}'
Observed: I don't have access to personal information such as favorite colors. — HTTP
200, status: completed, no warning.
The same is true of an expired ID. History lives in memory with a TTL
(LOCALAGI_CONVERSATION_DURATION, falling back to 1 hour if unset or unparseable),
and expiry returns an empty conversation rather than an error.
| State | Response | Distinguishable? |
|---|---|---|
| Valid ID, history carried | 200, model knows the context | — |
| Expired ID | 200, model knows nothing | no |
| Never-existed ID | 200, model knows nothing | no |
If your application needs to know which, it must track conversation identity itself. This is the mechanism behind "the agent forgot everything over lunch": a valid-looking ID, a valid response, and no history.
There is one guard: continuing a conversation with no new input is rejected, because it would end the job on an assistant message and some backends reject that:
{"error":"previous_response_id was set but no new input was sent; send at least one user or tool message when continuing a conversation"}
Nothing is persisted¶
| State | Location | Survives restart |
|---|---|---|
| Agent definition | <stateDir> JSON |
yes |
| Conversation history | process memory, TTL'd | no |
Optionally, LOCALAGI_ENABLE_CONVERSATIONS_LOGGING=true writes each turn to
<stateDir>/conversations/<agent>-<timestamp>.json. That is an audit log — nothing
reads it back, and it does not restore history.
User-defined tools¶
Tools sent in the request are separated into built-in and user-defined. User-defined ones
produce a tool-call response rather than a text answer: the output carries a
function_call item, the conversation is saved without an assistant message, and your
client is expected to execute the function and send the result back as a
function_call_output on the next request.
This is the OpenAI-style client-side tool loop, and it coexists with the agent's own server-side tools from Recipe 5. The distinction:
| Tool kind | Executed by | Configured |
|---|---|---|
| Built-in action | LocalAGI, in-process | on the agent |
| MCP tool | the MCP server | on the agent |
| User-defined | your client | per request, in tools |
The other two entry points¶
/v1/responses is one of three ways to reach an agent, and they behave very differently.
| Endpoint | Shape | Returns |
|---|---|---|
POST /v1/responses |
Responses envelope | the answer, synchronously |
POST /api/chat/:name |
LocalAGI-specific | does not return the answer |
GET /sse/:name |
SSE stream | events, including the answer |
/api/chat/:name is asynchronous. If you poll it expecting a body you will wait forever.
Use /v1/responses for request/response, and the SSE channel when you want incremental
output — that is also the only way to get streaming behaviour out of LocalAGI, since
stream on /v1/responses is ignored.
Timeouts¶
The most common operational mistake with this endpoint.
An agent request is a loop. LOCALAGI_TIMEOUT (default 5m) bounds one model
call, not the request. Measured on CPU:
| Request | Model calls | Wall clock |
|---|---|---|
| No tools, no knowledge | 1 | 2–3 s |
| Knowledge, no tools | 1 | 2.27 s |
| One tool | 3 | 38.7 s |
| Knowledge and one tool | 2 | 24.1 s |
So order your timeouts:
A proxy with a default 60-second read timeout in front of a legitimately slow agent returns 504 to the client while the agent runs to completion and commits its tool side effects. Check before blaming the agent:
If that line is present, the agent finished and the timeout was yours.
Note also that a mistyped LOCALAGI_TIMEOUT silently falls back to 150 s — shorter
than the documented 5m default.
Authentication¶
LOCALAGI_API_KEYS, comma-separated. The key is looked up from any of:
The middleware is applied globally, so with keys configured every route requires one —
including /api/agents, which the reference environment uses as a healthcheck. Add the
header to the healthcheck or drop it.
Reference¶
| Property | Value |
|---|---|
| Path | POST /v1/responses |
| Port | 3000 inside the container — hardcoded, no variable changes it |
model |
the agent name |
| Auth | LOCALAGI_API_KEYS, four accepted locations |
| Streaming | not implemented; field ignored |
| Token usage | not implemented; always zero |
Upstream references¶
- LocalAGI
webui/app.go— the handler at 575-692: input normalisation,previous_response_idlookup, the no-new-input guard at 598-602, agent resolution at 604-608, tool separation at 616-626,tool_choiceat 628-633, the tool-call response path at 646-658, new-UUID storage at 644 and 666, and the hardcoded zerousageat 567-571. Validated against v2.9.0. - LocalAGI
webui/types/openai.go— the request body type, includingStreamat 161, which no handler reads. - LocalAGI
core/conversations— the in-memory tracker, TTL expiry returning an empty conversation, and opportunistic GC of other conversations. - LocalAGI
webui/options.go— the 1 hour fallback at 42-50. - LocalAGI
webui/routes.go— route registration at 83,/api/chat/:nameat 75,/sse/:nameat 58, andGetKeyAuthConfig's four key locations at 252-254. - LocalAGI
cmd/serve.go— the hardcoded:3000listener at 126. - OpenAI Responses API reference — the shape being approximated.
- Response envelope, bare-UUID id, zero
usage, unknown-id behaviour, conversation chaining and all latencies: observed 2026-08-17 against LocalAGI v2.8.1, see version matrix.