Skip to main content

Knowledge base guide

The KB is where you put the documents Jarvis should be able to search: company handbooks, product specs, client SOPs, invoice templates. Every document is chunked, embedded (Qwen3-Embedding), and re-ranked (Qwen3-Reranker) so kb_search returns the passages most relevant to a user's question.

Collections​

Documents live in collections. A collection is just a name — claway-kb, accounting-sops, hospital-protocols. Groups get access to specific collections; they never see the others.

Create a collection from the KB page → New collection. Naming convention: lowercase, dashes, no spaces. The name will be visible in the scope tab of every persona so make it self-describing.

Uploading documents​

Three paths. Two are yours, on the KB page after you drill into a collection; the third is the user's, from inside the chat (see Letting a chat file its own documents).

  • Drag-drop a .pdf, .docx, .md, .txt, or .html file (max 500 MB) onto the upload zone. The dashboard streams it to the server, the ingest worker picks it up within about five seconds, and the document appears in the list once chunking + embedding completes (usually well under a minute for anything under 30 pages).
  • Paste a URL in the "Add URL" input. The worker fetches the page, strips boilerplate, and ingests. Good for public docs; not so good for anything behind SSO.

Progress hints:

  • The upload toast appears immediately (the file is queued).
  • The document row appears once the worker finishes.
  • If a document never appears after a minute, check System → Env for KB_EMBEDDING_BASE_URL — a bad embedding endpoint is the usual cause.

Letting a chat file its own documents​

Users can also hand Jarvis a document in the conversation — "put this price list in the KB", "remember this page" — and it is queued through the same worker. The agent uses kb_ingest, then kb_ingest_status to confirm indexing finished before telling the user it is searchable.

Where it lands. By default, a chat can only file into its own private collection, chat-<jid-prefix>, which no other chat searches. That is deliberate: anyone in a group can trigger this, so nothing a user says should be able to rewrite a corpus other chats depend on. The chat is granted read access to its own collection automatically, so the document is searchable in the same conversation that supplied it.

Granting more. On Personas → group → KB, the second chip row — "Chat may add documents to" — is the write grant. Tick a collection there and this chat can file into it; the first ticked collection becomes the default target. Read access (the top row) is deliberately not a write grant. In config.yaml this is kb_scope.<JID>.allow_write.

What the agent will accept. .pdf, .docx, .txt, .md, .html up to 20 MB, a URL, or text the user typed. A file must be one the gateway itself received in this chat, or one in this chat's workspace — the agent cannot point the ingester at an arbitrary path on the host, which would otherwise make anything on disk searchable by everyone in the chat. Re-sending the same file is a no-op rather than a duplicate.

Turning it off. Filing is its own toolset, kb_ingest (with kb_ingest_status); kb_search is search alone. A platform that lists kb_search in platform_toolsets.<platform> grants kb_ingest too, and every App tool preset (accounting, sales, support, admin) includes both, so ordinary chats search and file as before. To allow searching but not filing: for one App chat, pick a custom tool list without Add to Knowledge Base; for the whole gateway, set agent.disabled_toolsets: [kb_ingest]. Work-card roles never hold kb_ingest — a role reads the knowledge base and cannot write to it, whatever preset it was made from.

Attaching a collection to a group​

By default, groups have no KB access. This is a safety default: if you never opt in, you cannot accidentally leak documents to the wrong group.

To attach:

  1. Personas → click the group → KB tab.
  2. Tick the collections this group is allowed to search.
  3. Save.

Two things happen when you save:

  • kb_scope.<JID>.allow in config.yaml gains the collection names.
  • If platform_toolsets.<platform> was missing the kb_search tool, we add it automatically and show you a toast.

The current session ends so the next message picks up the new scope.

The security model (worth reading)​

Per-chat scope is a hard upper bound. The model cannot escape it by guessing collection names — every kb_search call, no matter what arguments the model constructs, is filtered against the chat's allowlist at the SQL level. Concretely:

  • Chat with allow: [claway-kb] calls kb_search(query="…") → only claway-kb searched.
  • Same chat calls kb_search(query="…", collections=["hospital-protocols"]) → request intersected with allowlist → empty set → tool returns "no matching collection in this chat's scope".
  • Chat with no kb_scope entry → tool returns "kb_scope is unset for this chat, no KB access" and never touches Postgres.

The intersection layer is enforced in tools/kb_search_tool.py and the SQL WHERE kb_collection = ANY($1) clause is the second wall. Both must be compromised for cross-chat leakage to occur, and both are covered by tests.

What can go wrong​

  • Uploaded but not visible in list. Refresh the KB page. The dashboard polls every five seconds but a manual refresh forces the check.
  • "I don't have kb_search" from Jarvis. Three silent gates possible. The Personas → KB tab banner tells you which one is the culprit:
    • JARVIS_KB_DB_URL not set in the gateway systemd env — fix in /etc/systemd/system/*.service EnvironmentFile=….
    • platform_toolsets.<platform> missing kb_search — Save on the KB scope tab auto-fixes.
    • kb_scope.<JID>.allow empty — tick at least one collection.
  • "I can't add that to the knowledge base." The chat asked for a named collection it has no write grant for. Tick it in the "Chat may add documents to" row, or let the document go to the chat's own collection.
  • Ingest worker stalled. systemctl status claway-<tenant>-kb-worker — if inactive, restart it; the queue at ~/.claway/kb_ingest_queue.jsonl will drain.
  • Wrong document ended up in a collection. Delete it from the collection's document list. The chunks are deleted transactionally; searches that were mid-flight complete with the old row cache but every subsequent search respects the deletion.

Best practices​

  • Don't dump everything. Ingesting /mnt/shared/ wholesale makes the reranker's job harder. Curate: the documents that answer questions your end-users actually ask.
  • One collection per audience. If accounting and sales need different answers, they need different collections, not one big pool.
  • Refresh cadence. For monthly changing documents (price lists, SOPs), delete + re-ingest at the end of the month rather than leaving both versions in the pool.
  • Pair with the KB-first rule. A KB with no persona rule to use it is wasted disk. See the persona guide.