R42 was missing knowledge-graph information (e.g. pension questions) because retrieval and context-building dropped relevant facts: - retrieval: exact-token TF-IDF could not match Dutch compound words, so a "pensioen" query scored 0 against "pensioenregeling" / "partnerpensioen" and never retrieved them. Add a compound-word fallback (shared >=6-char stem or containment, 0.4x weight) alongside exact matching. - rag: deep article content was only injected for verbatim-mentioned topics; retrieved topics contributed just a 200-char description. Inject ~1000 chars of content for up to 5 topics (mentions first, then top-ranked retrieved) and widen the description snippet to 320. - prompts: add a NAUWKEURIGHEID block (use all relevant facts, call lookup_topic before giving up) and relax the 4-sentence cap for detail/list answers so complete facts aren't summarised away. Also add a clear-history control: a trash button in the chat header (confirm dialog) wipes chat🧵{userId} and reseeds the greeting via clearThread() in useChat. Tests: compound-word matching + rag deep-content injection. Spec updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3.7 KiB
R42 spec: the in-app chatbot
R42 is a knowledge-base-grounded assistant available on every screen. It runs client-side and is grounded by local TF-IDF retrieval — no vector database.
- UI:
src/components/chat/—ChatLauncher.jsx,ChatWindow.jsx,useChat.js - Prompt/tool:
src/components/chat/prompts.js - Retrieval/validation:
src/components/chat/rag.js+src/lib/retrieval.js - KB writes:
src/lib/kbStore.js
Conversation
- The brand mark (
src/components/ui/Mark.jsx) renders R42 in idle / typing / error states. - Messages persist per user in
localStorageunderchat:thread:{userId}, capped at 50 messages. Only roughly the last 12 turns are sent to the API; older context is truncated with a notice. - A greeting message seeds an empty thread.
- Each turn calls
callLLM(fast/standard Claude tier — low latency matters for chat). - The chat header has a clear button (trash icon). It confirms, then wipes
chat:thread:{userId}and reseeds the greeting viaclearThreadinuseChat.js.
Grounding (RAG via TF-IDF)
buildKbContext in rag.js:
- Build / reuse the TF-IDF index over
topics(src/lib/retrieval.js). - Retrieve the top 10 topics for the user's message. Scoring is exact-token
TF-IDF plus a compound-word fallback: an unmatched query token (≥6 chars)
also matches a document term when they share a ≥6-char stem or one contains
the other, at a reduced weight. This recovers Dutch compounds — e.g. a
pensioenquery matchespensioenregelingandpartnerpensioen. - Always include topics whose
idorlabelappears verbatim in the message. - Include relations only when both endpoints are in the retrieved set.
- Inject up to ~1000 chars of generated content for up to 5 topics — verbatim-mentioned first, then the highest-ranked retrieved ones — so a query that never names a topic exactly still gets rich content for what it matched.
- Append a short KB hash so the cached context busts when topics change.
If the summarised context is still too thin, R42 can call the lookup_topic
tool to pull a topic's full description and learning content on demand.
The system prompt (prompts.js) is assembled as cacheable blocks: a stable
preamble (role, tasks, style, "answer only from the KB"), the KB context block, and
a per-turn tail with the user's name and admin/non-admin flag.
Proposing knowledge-graph edits
R42 may call propose_graph_delta with:
reason— one sentencetopics— max 3, each{ id (kebab-case), label, type (concept|role|process), description }relations— max 5, each{ source, target, type (related_to|depends_on|part_of|executed_by) }
validateDelta (rag.js) dedupes by topic id and case-folded label, rejects
relations with missing endpoints or self-references, and enforces the 3/5 caps.
Invalid deltas are dropped.
A confirmation chip appears inline:
- Admin → Yes:
kbStore.applyDeltawrites topics/relations to PocketBase immediately. - Non-admin → Yes:
kbStore.appendSuggestionqueues apendingentry inkb:suggestions(localStorage) for admin review.
Admins review the queue in src/components/admin/SuggestionsQueue.jsx (approve
re-runs applyDelta; reject marks it rejected). kbStore dispatches
respellion:kb-updated after writes so the D3 graph and queue refresh.
Quiz integrity
src/pages/Testen.jsx sets quiz:active:{userId}=true on quiz start and clears it
on every non-quiz phase and on unmount, dispatching a respellion:quiz-state event.
ChatLauncher hides the FAB while this flag is set, so users cannot consult R42
mid-quiz. Never bypass this.