Storage engines and on-disk state¶
VECTOR_ENGINE picks one of three backends, read once at process start and
defaulted to chromem. The switch is four cases (routes.go:87-116):
| Value | Constructor | Extra requirement |
|---|---|---|
chromem (default) |
rag.NewPersistentChromeCollection |
— |
postgres |
rag.NewPersistentPostgresCollection |
DATABASE_URL |
localai |
rag.NewPersistentLocalAICollection |
a LocalAI /stores endpoint |
| anything else | — | error unknown vector engine: %q |
There is no milvus, qdrant, weaviate or faiss support, and no plugin mechanism — the switch is compiled in.
Discrepancy (#4).
README.md:18-21advertises two engines, Chromem and PostgreSQL, and the env table row forVECTOR_ENGINE(README.md:203) names only those two. Thelocalaiengine is a real, selectable third value (routes.go:99). It is arguably omitted on purpose — most of it is unimplemented — but it is undocumented, not removed. Believe the code that it exists; believe the README that you should not use it.
DATABASE_URL is the only environment variable not hoisted into main.go's var
block; it is read lazily inside the postgres case.
Filesystem state exists under every engine¶
Choosing PostgreSQL does not make a deployment stateless on disk. Independent
of VECTOR_ENGINE, every collection has:
| Path | Contents | Lost if you lose it |
|---|---|---|
<COLLECTION_DB_PATH>/collection-<name>.json |
only {"external_sources": [...]} |
the source registry, and the collection's very existence in GET /api/collections |
<FILE_ASSETS>/<collection>/<uuid>/<filename> |
the original uploaded bytes, mode 0644 | every original; downloads and entry-content break; Repopulate has nothing to re-chunk |
Three behaviours follow from that layout, and they explain most confusing symptoms:
- The document list is filesystem-derived, not index-derived.
ListDocumentswalks the asset directory taking every file inside every subdirectory. That is why entries list correctly while the vector store is down — and why deleting files underFILE_ASSETSout of band makes entries disappear while their chunks remain in the store, permanently orphaned. - Entry resolution falls back to basename.
findEntryKeytries an exactuuid/filenamematch, then matches onfilepath.Base. SoGET /entries/report.pdfworks without knowing the UUID — but if two UUID directories hold the same basename, the first match wins arbitrarily, by directory iteration order. - A one-time layout migration runs on open. If any file sits directly in the
asset directory,
migrateToUUIDLayoutrewrites the whole directory into the UUID scheme, copying then removing.
loadDB also tolerates three historical state-file formats: the current object,
an older object carrying an index field, and an even older bare JSON array of
strings. The index field was the persisted chunk index, dropped in v0.5.9;
chunk-to-file mapping is now resolved by querying the engine for
metadata.source.
What to back up: COLLECTION_DB_PATH and FILE_ASSETS under every engine,
plus the database under postgres. Backing up only PostgreSQL loses the source
registry and every original file.
chromem — the default¶
rag/engine/chromem.go, wrapping github.com/philippgille/chromem-go v0.7.0.
Opened as chromem.NewPersistentDB(path, true), where true means compress.
On-disk layout¶
- Base directory is
COLLECTION_DB_PATH— the same directory that holds thecollection-*.jsonstate files. - Each collection gets a subdirectory named
hash2hex(collectionName): the first 4 bytes of the SHA-256 of the name, hex-encoded — 8 hex characters. Collection names are therefore not recoverable from the vector store directory. Map them via thecollection-*.jsonfilenames. - One file per document, named
hash2hex(docID), gob-encoded and gzip-compressed because compression is on —.gob.gz. - Plus one
metadatafile per collection.
<COLLECTION_DB_PATH>/
├── collection-handbook.json # external sources only
├── 3f7a2b91/ # SHA-256[0:4] of "handbook"
│ ├── metadata
│ ├── 0a1b2c3d.gob.gz # one chunk
│ ├── 4e5f6a7b.gob.gz
│ └── … # one file per chunk
└── …
A 10,000-chunk collection is 10,000 small files in one directory. Every
AddDocuments call writes a file per document. Plan inode budget and directory
performance accordingly, and expect backup tools to be slow over this layout.
No ANN index — what that means¶
Retrieval is brute force. chromem-go iterates every candidate document, computes a dot product against the normalised query vector, and keeps a bounded max-heap of size K. Cost is O(N·d) per query, in memory, parallelised across goroutines.
There is no HNSW, no IVF, no product quantisation, no clustering — nothing that lets a query skip candidates. Concretely:
| Collection size | Behaviour |
|---|---|
| Hundreds to low thousands of chunks | fine; latency dominated by the embedding round trip |
| Tens of thousands | noticeable per-query CPU cost, growing linearly |
| Hundreds of thousands and beyond | every query scans everything, every time |
Add the per-collection mutex from architecture — searches serialise — and the ceiling arrives sooner than raw single-query timing suggests. Chunk count is the number that matters, not document count; 400-byte default chunks reach large N quickly.
Other chromem specifics¶
- IDs are a monotonic integer counter, not UUIDs: it starts at 1 and is set to
count+1on open. After deletionsCount()shrinks while old IDs persist, so the counter can collide with existing IDs. Do not treat chromem chunk IDs as stable or unique across a collection's lifetime. StoreDocumentsbatches into oneAddDocumentscall withruntime.NumCPU()parallelism — but each document still costs its own embedding HTTP request, see embeddings.GetBySourceis a workaround: it issues a query with the dummy string".",nResults = Count(), and awherefilter onsource. Correct, but it burns an embedding call and a full scan per entry lookup — which means viewing an entry in the UI costs an embedding call.GetEmbeddingDimensionsreads the document whose ID equals the current count and returns its vector length; it errors with"no documents in collection"when empty.
postgres¶
rag/engine/postgres.go, 977 lines, the largest file in the repository, on
jackc/pgx/v5 with a pool. This is where essentially all recent engineering
attention has gone.
Extensions¶
setupDatabase runs in a fixed order:
CREATE EXTENSION IF NOT EXISTS pg_textsearch— mandatory. A failure aborts construction. This is Timescale'spg_textsearch, providing thebm25index access method and the<@>/to_bm25queryoperators. Without it there is no hybrid search and no collection.- Vector extension probing, in preference order: an existing
vectorscaleorpgvectorscale; elseCREATE EXTENSION vectorscale CASCADE; elseCREATE EXTENSION pgvectorscale CASCADE; else plain pgvector (CREATE EXTENSION vector). Only that last failing is fatal.
So: pgvector is the baseline, pgvectorscale/DiskANN is the preferred upgrade,
pg_textsearch is non-negotiable. Not paradedb, despite the BM25 vocabulary.
If you bring your own PostgreSQL rather than using Dockerfile.pgsql, those three
extensions plus shared_preload_libraries = 'timescaledb,pg_textsearch' are your
responsibility. See installation.
Schema¶
One shared registry table:
CREATE TABLE IF NOT EXISTS collection_config (
collection_name TEXT PRIMARY KEY,
embedding_model TEXT NOT NULL,
embedding_dimensions INTEGER NOT NULL,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
)
And one table per collection:
CREATE TABLE IF NOT EXISTS documents_<sanitized> (
id SERIAL PRIMARY KEY,
title TEXT,
content TEXT NOT NULL,
category TEXT,
metadata JSONB,
word_count INTEGER,
search_vector TSVECTOR,
full_text TEXT GENERATED ALWAYS AS (COALESCE(title,'') || ' ' || content) STORED,
embedding VECTOR(<embeddingDims>)
)
collection_config is the migration ledger: it is what lets the engine notice
that your embedding model changed. Do not drop it.
Table naming¶
sanitizeTableName prefixes documents_ and replaces every rune outside
[a-zA-Z0-9_] with _. If the sanitised name does not start with a letter,
col_ is inserted before it — 9lives becomes documents_col_9lives.
The allowlist closes SQL injection, which matters because table names are
necessarily interpolated with fmt.Sprintf rather than bound (identifiers cannot
be parameters). It does not close collisions: a-b, a.b and a b all map to
documents_a_b and silently share one table. Nothing detects this.
The allowlist exists because the previous version stripped only a hardcoded set of
characters, so a : used as a namespace separator in per-user collection names
reached CREATE TABLE and produced syntax error at or near ':'. The fix shipped
in v0.6.3. Naming guidance is in
collections.
Indexes¶
| Index | Definition | On failure |
|---|---|---|
idx_<t>_search |
GIN(search_vector) |
warning only |
idx_<t>_bm25 |
USING bm25(full_text) WITH (text_config='<cfg>') |
fatal |
idx_<t>_embedding |
USING diskann(embedding) when vectorscale is present; falls back to USING hnsw(embedding vector_cosine_ops); if that also fails, no index at all and only a warning |
warning |
Two things to notice.
search_vector is dead weight at v0.6.4. It is populated on every INSERT via
to_tsvector('english', ...) and its GIN index is built — and nothing ever
queries it. The hybrid search uses full_text and the BM25 index. You are paying
write cost and storage for an unused index.
A collection can end up with no vector index and keep working, on sequential
scans, because the third fallback only warns. On a large table that is the
difference between milliseconds and a statement timeout. Check for
idx_<table>_embedding after creating a collection on a fresh database.
BM25 text configuration¶
BM25_TEXT_CONFIG, default english, is the v0.6.4 feature (PR #52). Two
supporting behaviours:
ensureTextSearchConfigauto-provisions exactly one custom configuration,de_en, copyingpg_catalog.simpleand mapping word tokens through bothgerman_stemandenglish_stem. Any other value is passed through as a built-in PostgreSQL text-search configuration name.ensureBM25IndexConfigmakes the setting idempotent: it readspg_get_indexdef, and if the live index does not containtext_config='<desired>'it drops the index so the followingCREATE INDEX IF NOT EXISTSrebuilds it.
Changing BM25_TEXT_CONFIG therefore triggers a full BM25 index rebuild on the
next startup. On a large collection, expect a long startup.
Neither the LocalRecall build linked by LocalAI v4.8.2 (v0.6.3) nor the one
linked by standalone LocalAGI has this variable — it landed in v0.6.4.
Connection timeouts¶
Set as pgx RuntimeParams before the pool opens:
| Parameter | Environment variable | Default |
|---|---|---|
lock_timeout |
POSTGRES_LOCK_TIMEOUT |
30s |
idle_in_transaction_session_timeout |
POSTGRES_IDLE_IN_TRANSACTION_TIMEOUT |
300s |
statement_timeout |
POSTGRES_STATEMENT_TIMEOUT |
unset |
"0" or "off" (case-insensitive) is an explicit opt-out. The long in-source
comment records the motivating incident: a corrupt BM25 index, left inconsistent
by a past pg_resetwal, made an INSERT spin forever on a buffer-content lock
while holding its relation lock, head-of-line-blocking the entire table.
lock_timeout is described in-source as the cascade-killer.
statement_timeout is deliberately left unset, because a legitimate DiskANN or
HNSW build can exceed any fixed limit. Index builds are additionally exempted via
execNoStatementTimeout, which wraps the DDL in a transaction with
SET LOCAL statement_timeout = 0. If you set POSTGRES_STATEMENT_TIMEOUT
globally on the server instead, you will break index creation.
Reset and delete¶
Reset drops the collection table CASCADE, deletes the collection_config row,
then re-runs setupDatabase(). Metadata-scoped deletes build metadata->>$1 = $2
conditions with both the key and the value bound.
What PostgreSQL does not hold: the uploaded files and the external-source list. Those are always on the local filesystem. PostgreSQL holds chunks, embeddings and metadata only.
The localai engine¶
Undocumented in the README, and largely unimplemented. Seven of its nine Engine
methods are stubs or no-ops:
| Method | Reality |
|---|---|
Reset |
not implemented |
Count |
hardcoded 0 |
GetEmbeddingDimensions |
not implemented |
Delete |
not implemented |
GetByID |
not implemented |
GetBySource |
not implemented |
Store |
prints a notice that LocalAI stores have no IDs so entries cannot be deleted, and returns an empty ID |
StoreDocuments |
loops calling Store — one embedding round trip per chunk |
Search |
works: embeds the query, calls stores/find with TopK; results carry no ID and no Metadata |
It talks to LocalAI's /stores/{set,get,delete,find} API through a vendored copy
of LocalAI's store client, whose own header notes it is a duplicate.
The knock-on effects make it effectively unusable.
NewPersistentLocalAICollection unconditionally calls Repopulate() at
construction, with an in-source TODO admitting it does not work because there is
no Reset() and LocalAI stores are neither persistent nor upsert-capable.
Repopulate reaches Engine.Reset() and gets the not-implemented error — but the
constructor discards the return value, so construction succeeds anyway.
RemoveEntry and GetEntryContent then hard-fail against this backend, and
Count() returning 0 makes PersistentKB.Count() meaningless.
Do not select localai. It is documented here so that a reader who finds it
in a config knows what they are looking at.
Choosing¶
| chromem | postgres | |
|---|---|---|
| Setup | none | a database with three extensions |
| Search | dense only | hybrid BM25 + vector, RRF |
| Index | none — brute force | DiskANN or HNSW |
| Embedding requests per N-chunk file | N | 1 |
| Scale ceiling | thousands to low tens of thousands of chunks | millions of rows, with the indexes healthy |
| Score returned | cosine [-1,1] |
RRF (0, ~0.0164] |
| Operational surface | a directory of small files | a database to run, back up and monitor |
| Migration on model change | none (dimension change forces a full re-ingest) | a real, transactional migration |
Start on chromem for a single-node deployment holding a few thousand chunks. Move to PostgreSQL when any of these becomes true: query latency grows with the corpus; you need keyword matching for identifiers and error codes; ingestion time is dominated by per-chunk HTTP; or more than one process needs the same collection. Migration guidance is in rag.
Upstream references¶
routes.go—newVectorEngineswitch at 87-116;DATABASE_URLat 103-106. Source-verified against v0.6.4, validated 2026-08-17.rag/engine/chromem.go—NewPersistentDB(path, true)at 23, integer ID counter at 31 and 42-46,StoreDocumentsat 150,GetBySourcedummy query at 177-180, dimensions at 67-79. Validated 2026-08-17.rag/engine/postgres.go—setupDatabaseat 212-313,collection_configat 250-258, per-collection DDL at 264-276,sanitizeTableNameat 112-133, indexes at 283-306 andcreateVectorIndexat 372-397,ensureTextSearchConfigat 320-336,ensureBM25IndexConfigat 342-366,applyConnTimeoutsat 174-191 with rationale at 151-173,execNoStatementTimeoutat 197-210,Resetat 595-612, metadata delete at 759-772. Validated 2026-08-17.rag/engine/localai.go— the unimplemented methods at 26-36 and 83-93;Storenotice at 76-80;Searchat 95-131. Validated 2026-08-17.rag/engine/localai/client.go— vendored LocalAI store client,stores/*calls at 59-98. Validated 2026-08-17.rag/collection.go— state-file paths at 27, 45, 70; the discardedRepopulate()at 53-55. Validated 2026-08-17.rag/persistency.go— asset layout at 353-355,listDocumentKeysat 137-160,findEntryKeyat 164-183,migrateToUUIDLayoutat 717-763, legacyloadDBformats at 42-72. Validated 2026-08-17.philippgille/chromem-go v0.7.0—hash2hexat 22-27; per-document gzipped gob files indb.goat 61 and 77-85. Read at the pinned tag, validated 2026-08-17.- LocalRecall release v0.6.4 —
BM25_TEXT_CONFIG(PR #52),pg_textsearchpin (PR #50). Validated 2026-08-17. README.md— two-engine claim at 18-21 and 203. Validated 2026-08-17.