Add comprehensive documentation for key organizational aspects
- Introduced "Pension Scheme & Benefits" detailing secondary employment benefits and pension specifics. - Created "Roles & Accountabilities" outlining the Holacracy role structure and responsibilities within Respellion. - Added "Security" section covering GDPR compliance and workplace safety protocols. - Established "Spending and Contracting" policy detailing expense categories and submission processes. - Documented "Who We Are" to define Respellion's identity, services, and operational model under Holacracy and ISO 9001.
This commit is contained in:
RaymondVerhoef
291
docs/handover.md
291
docs/handover.md
@@ -1,231 +1,114 @@
|
||||
# Handover: employee learning platform
|
||||
# Handover: Respellion Learning Platform
|
||||
|
||||
## Purpose of this document
|
||||
|
||||
This document captures every design decision made before implementation started.
|
||||
It is the authoritative source for rationale. When a spec file is ambiguous,
|
||||
resolve it against this document. Do not ask the human — the answers are here.
|
||||
This document captures the **design decisions as actually built**. The platform
|
||||
diverged substantially from its original design vision (a Next.js multi-service
|
||||
system with Qdrant and OpenAI embeddings). This handover reflects what shipped:
|
||||
a React/Vite SPA on PocketBase with local TF-IDF retrieval.
|
||||
|
||||
When sources conflict, trust the code in `src/` first, then this document, then
|
||||
`docs/data-model.md` (schema), then `docs/architecture.md`.
|
||||
|
||||
---
|
||||
|
||||
## What this application does
|
||||
|
||||
Employees of a tech company use this platform to build and maintain knowledge of
|
||||
the employee handbook, holacratic structures, and internal processes.
|
||||
Employees use the platform to build and maintain knowledge of the company's
|
||||
internal handbook, roles, and processes.
|
||||
|
||||
Core mechanics:
|
||||
- Admins upload source documents → AI extracts a structured knowledge base
|
||||
- The KB is organised into Themes (broad) and Topics (specific)
|
||||
- An AI generates 10 types of micro learning content per Topic
|
||||
- Employees follow a 26-week curriculum of weekly sessions
|
||||
- Each session covers one Theme (multiple related Topics)
|
||||
- Employees choose which micro learning type to use per Topic
|
||||
- After 26 weeks the curriculum restarts, varied to reinforce rather than repeat
|
||||
- A chatbot called R42 answers KB-grounded questions on every screen
|
||||
- A gamification system using developer-native language motivates completion
|
||||
- Admins upload source documents → Claude extracts a structured knowledge graph (topics + relations)
|
||||
- AI generates learning content and micro-learnings per topic
|
||||
- Each employee follows a 26-week curriculum, **starting whenever they enroll**
|
||||
- Each week presents an assigned topic; the employee completes micro-learnings and a test
|
||||
- After week 26 the cycle restarts at week 1 with the same content
|
||||
- R42, an AI assistant, answers KB-grounded questions on every screen
|
||||
- A gamification layer (points, badges, leaderboard) motivates completion
|
||||
|
||||
---
|
||||
|
||||
## All confirmed design decisions
|
||||
## Key decisions as built
|
||||
|
||||
### Architecture
|
||||
- **Single-page React app, not microservices.** All logic runs in the browser
|
||||
(`src/`). PocketBase is the only backend; the Anthropic API is reached through a
|
||||
reverse proxy (Caddy in prod, Vite in dev). The original `app/` Next.js scaffold
|
||||
was abandoned and is not deployed.
|
||||
- **PocketBase for everything stateful** — auth, structured data, file storage.
|
||||
SQLite is sufficient at this scale.
|
||||
- **No vector database.** Retrieval is a dependency-free TF-IDF index over the
|
||||
knowledge graph (`src/lib/retrieval.js`). Qdrant and the embedding service from
|
||||
the original design were never built.
|
||||
|
||||
### Knowledge base
|
||||
- **Extracted, not hand-authored.** Admins upload `.txt` / `.md` (≤5 MB). Claude
|
||||
(standard tier) extracts topics and relations chunk by chunk.
|
||||
- **Flat graph, not a Theme→Topic tree.** The KB is `topics` + `relations`. A
|
||||
topic's `theme` is a string used for curriculum grouping, not a separate entity.
|
||||
- **Relation types:** `related_to`, `depends_on`, `part_of`, `executed_by`.
|
||||
- **Topic relevance** (`core` / `standard` / `peripheral` / `exclude`) controls
|
||||
what enters learning/curriculum; `relevance_locked` protects admin overrides on
|
||||
re-ingestion.
|
||||
|
||||
**Decision: KB is extracted from source documents, not manually authored**
|
||||
Admins upload raw source material. Claude Sonnet 4 extracts Themes, Topics, and
|
||||
relationships. Admins review and approve in batches (one Theme at a time, not
|
||||
one Topic at a time). Topic bodies are AI-drafted and admin-editable after approval.
|
||||
|
||||
**Decision: two-level hierarchy — Theme → Topic**
|
||||
A Theme is a broad subject area. A Topic is one specific concept within a Theme.
|
||||
One weekly session = one Theme. Multiple Topics within that Theme per session.
|
||||
|
||||
**Decision: three relationship types between Topics**
|
||||
- related: Topics that complement each other
|
||||
- prerequisite: Topic A should be understood before Topic B
|
||||
- contrast: Topics representing opposing approaches
|
||||
|
||||
These relationships are stored as explicit PocketBase relation fields, not a
|
||||
generic junction table.
|
||||
|
||||
**Decision: source material format priority**
|
||||
Accepted formats: PDF, MD, TXT only. MD is the highest quality input —
|
||||
heading structure maps directly to Theme → Topic hierarchy. Admins should be
|
||||
recommended to provide MD where possible.
|
||||
|
||||
**Decision: embeddings from source chunks, not topic summaries only**
|
||||
R42 retrieves from original source material chunks as primary source, with
|
||||
topic summaries as secondary. This keeps R42 grounded and reduces hallucination.
|
||||
|
||||
---
|
||||
|
||||
### Micro learnings
|
||||
|
||||
**Decision: 10 types, all generated by AI as structured JSON**
|
||||
Types:
|
||||
1. concept_explainer
|
||||
2. scenario_quiz
|
||||
3. misconceptions
|
||||
4. how_to
|
||||
5. comparison_card
|
||||
6. reflection_prompt
|
||||
7. flashcard_set
|
||||
8. case_study
|
||||
9. glossary_anchor
|
||||
10. myth_vs_evidence
|
||||
|
||||
Each type has a defined JSON schema in data-model.md. Generation uses
|
||||
Claude Sonnet 4. Output is validated against Zod schemas before storage.
|
||||
|
||||
**Decision: employees choose type per topic per session**
|
||||
Employees are not locked to one type globally. Each session, per Topic, the
|
||||
employee selects from all published types for that topic. Multiple types can
|
||||
be completed in one session.
|
||||
|
||||
**Decision: pre-generate, don't generate on demand**
|
||||
All 10 types are generated when a Topic is approved, not when an employee
|
||||
requests them. This controls quality and cost. R42 is the only on-demand
|
||||
AI interaction.
|
||||
|
||||
---
|
||||
### Learning content
|
||||
- **Long-form content is generated on demand**, three types: `article`, `slides`,
|
||||
`infographic` (the `content` collection). New types shallow-merge into the cached
|
||||
object. **No podcast type.**
|
||||
- **Micro-learnings**, three types: `concept_explainer`, `scenario_quiz`,
|
||||
`flashcard_set` (the `micro_learnings` collection). A former `reflection_prompt`
|
||||
type was dropped.
|
||||
- **Employee chooses the format** per topic per session. Completion is not
|
||||
quality-gated; engaging with the full micro-learning counts.
|
||||
|
||||
### Curriculum
|
||||
|
||||
**Decision: AI generates curriculum, admin edits**
|
||||
Claude Sonnet 4 reads the full KB graph (Themes, Topics, relationships,
|
||||
complexity weights) and produces a 26-week schedule. Admin reviews, reorders,
|
||||
and finetunes. Admin does not build from scratch.
|
||||
|
||||
**Decision: one Theme per week session**
|
||||
A session covers all Topics under one Theme. Topics within the session are
|
||||
ordered by the curriculum generator based on prerequisites and complexity.
|
||||
|
||||
**Decision: perpetual curriculum with versioning**
|
||||
The curriculum runs indefinitely. After week 26, cycle 2 begins on the latest
|
||||
curriculum version. Cycle 2+ varies sequence, surfaces unused micro learning
|
||||
types, and increases coverage of low-engagement topics.
|
||||
|
||||
**Decision: completed weeks are immutable**
|
||||
Regeneration only affects future unstarted weeks. An employee's completion
|
||||
history is never altered regardless of curriculum version changes.
|
||||
|
||||
**Decision: regeneration requires admin confirmation**
|
||||
When new Topics are approved, the system queues a regeneration but does not
|
||||
apply it until the admin explicitly confirms. Admin sees a preview of the
|
||||
proposed new schedule before confirming.
|
||||
|
||||
**Decision: rolling starts**
|
||||
Each employee has their own start date. There are no cohorts or shared
|
||||
start dates.
|
||||
|
||||
---
|
||||
|
||||
### Gamification
|
||||
|
||||
**Decision: developer-native visual language**
|
||||
Inspired by GitHub (heatmap), Stack Overflow (badges, reputation), and
|
||||
Duolingo (streak, XP, levels). Language uses developer vocabulary throughout.
|
||||
|
||||
**Decision: XP unit is called commits**
|
||||
Every completed Topic earns commits. Quantity varies by micro learning type.
|
||||
|
||||
**Decision: levels use developer rank names**
|
||||
Intern → Junior → Medior → Senior → Staff → Principal
|
||||
Based on cumulative commits across all cycles.
|
||||
|
||||
**Decision: streak is weekly, not daily**
|
||||
Consecutive weeks with at least one completion. Resets on a skipped week.
|
||||
|
||||
**Decision: activity heatmap covers 26-week cycle**
|
||||
GitHub-style contribution graph. Cell darkness = number of types completed
|
||||
that week.
|
||||
|
||||
**Decision: no social layer**
|
||||
No comments, reactions, or direct messaging. Gamification is visible but
|
||||
not interactive between employees.
|
||||
|
||||
**Decision: public milestone cards, not ranked leaderboard**
|
||||
At weeks 13 and 26, a public card is posted to the shared activity feed.
|
||||
Language: "shipped", not "graduated". The leaderboard shows multiple
|
||||
dimensions (commits, streak, types used, badges) — not a single ranking.
|
||||
|
||||
**Decision: named content badges**
|
||||
Examples: governance_nerd, process_architect, deep_reader. These are seeded
|
||||
at startup, not user-generated. See data-model.md for badge schema.
|
||||
|
||||
---
|
||||
- **AI generates, admin confirms.** Claude proposes a 26-week schedule from the
|
||||
themed/weighted topic set; the admin previews and activates it. Versions move
|
||||
`draft → active → superseded`; exactly one is active.
|
||||
- **Per-user, self-paced start (current behavior).** Each employee enrolls on first
|
||||
login; their week/cycle is derived from `curriculum_started_at`. There is **no
|
||||
shared calendar week**. Week 1 is the first 7 days after they enroll.
|
||||
- **Perpetual, repeating cycles.** After week 26, the cycle restarts at week 1 with
|
||||
the same content. Completion history (`micro_learning_completions`) is append-only.
|
||||
- **Hash fallback.** If no curriculum version is active, topic assignment falls back
|
||||
to a deterministic hash of user id + week. Keep this fallback.
|
||||
|
||||
### R42 chatbot
|
||||
- **KB-grounded via TF-IDF**, not vector search. Context = top-K topics + verbatim
|
||||
mentions + filtered relations + limited deep content.
|
||||
- **Conversation persists per user** in `localStorage` (cap 50 messages; ~12 turns
|
||||
sent to the API). It is not stored server-side.
|
||||
- **Can propose graph edits** (`propose_graph_delta`, ≤3 topics / ≤5 relations).
|
||||
Admins apply immediately; non-admins queue a suggestion for admin approval.
|
||||
- **Hidden during quizzes** to protect test integrity.
|
||||
|
||||
**Decision: functional only, no personality**
|
||||
R42 answers questions grounded in the KB. It does not have a defined persona,
|
||||
tone, or name story beyond the label R42.
|
||||
### Gamification
|
||||
- **Points:** +2 per correct quiz answer, in the `leaderboard` collection.
|
||||
- **Badges** computed at render time: First Steps (1 test), Veteran (5 tests),
|
||||
Perfectionist (a 100% score).
|
||||
- Admins are excluded from the public leaderboard.
|
||||
|
||||
**Decision: stateless per session**
|
||||
No chat history is persisted between sessions. This avoids privacy complexity
|
||||
and keeps the implementation simple.
|
||||
|
||||
**Decision: internal KB scope only**
|
||||
R42 cannot search external sources. If a question cannot be answered from the
|
||||
KB, R42 says so explicitly.
|
||||
|
||||
**Decision: context-weighted retrieval**
|
||||
R42 knows the employee's current curriculum week. Retrieval boosts chunks
|
||||
from the current week's Theme. General KB questions are not restricted.
|
||||
|
||||
**Decision: always cites source Topic**
|
||||
Every R42 response includes the Topic title(s) its answer draws from.
|
||||
|
||||
**Decision: Haiku 4.5 for R42, Sonnet 4 for generation**
|
||||
Low latency matters for chat. The retrieval layer compensates for Haiku's
|
||||
smaller knowledge base. Sonnet 4 is reserved for generation tasks where
|
||||
structure and quality matter more than speed.
|
||||
### Auth & infrastructure
|
||||
- **PIN auth** against `team_members`; the session id lives in `sessionStorage`.
|
||||
Role `admin` unlocks the Admin panel.
|
||||
- **Claude model tiers:** `fast` = Haiku 4.5, `standard` = Sonnet 4.6,
|
||||
`reasoning` = Opus 4.7. Admins can override per tier from Settings.
|
||||
- **Simulation mode** (`admin:use_simulation`) returns stub LLM output for UI work.
|
||||
- **Deploy:** Docker image (Caddy serving the built SPA) + PocketBase container;
|
||||
Ansible playbooks under `infra/` for dev and prod.
|
||||
|
||||
---
|
||||
|
||||
### Infrastructure
|
||||
## Notable divergences from the original vision
|
||||
|
||||
**Decision: PocketBase as primary backend**
|
||||
Auth, file storage, structured data, and admin UI in one binary. SQLite is
|
||||
sufficient for ~150 users. No PostgreSQL needed at this scale.
|
||||
| Original design (not built) | What shipped |
|
||||
|---|---|
|
||||
| Next.js 14 PWA + 6 Fastify services | Single React/Vite SPA, no backend services |
|
||||
| Qdrant + OpenAI embeddings | Local TF-IDF, no embeddings |
|
||||
| Theme/Topic entity hierarchy, batch approval | Flat `topics` + `relations` graph |
|
||||
| 10 micro-learning types | 3 micro-learning types |
|
||||
| `employee_curriculum_state`, `badges`, `milestone_cards`, etc. | `team_members` fields + `leaderboard` + render-time badges |
|
||||
| Shared calendar-week curriculum | Per-user start, self-paced |
|
||||
|
||||
**Decision: Qdrant for vector storage**
|
||||
Separate Docker container. Keeps vector operations out of SQLite.
|
||||
pgvector was rejected — adding Postgres just for vectors is unnecessary overhead.
|
||||
|
||||
**Decision: Next.js 14 PWA for frontend**
|
||||
Single codebase for admin and employee app. PWA covers mobile without a native
|
||||
app. Learning platforms do not require native device APIs.
|
||||
|
||||
**Decision: five discrete backend services**
|
||||
Ingestion, generation, curriculum, chat, progress. Each is a separate Fastify
|
||||
service with its own port and responsibility. They do not call each other
|
||||
directly — they read/write shared PocketBase collections.
|
||||
|
||||
**Decision: PDF parsing starts with pdf-parse (Node.js)**
|
||||
Switch to pdfplumber Python sidecar only if pdf-parse quality is insufficient
|
||||
for actual source documents. Do not over-engineer the extraction layer upfront.
|
||||
|
||||
---
|
||||
|
||||
## What is not yet specced
|
||||
|
||||
The following spec files still need to be written before their phases begin:
|
||||
- /docs/generation-spec.md — micro learning generation service
|
||||
- /docs/curriculum-spec.md — curriculum generator + versioning
|
||||
- /docs/r42-spec.md — chat service
|
||||
- /docs/gamification-spec.md — progress service + gamification mechanics
|
||||
- /docs/frontend-spec.md — employee app, admin app, PWA config
|
||||
|
||||
Do not begin a phase without its spec file. Flag the gap if you reach it.
|
||||
|
||||
---
|
||||
|
||||
## Source of truth hierarchy
|
||||
|
||||
When sources conflict, resolve in this order:
|
||||
1. This handover document (rationale and decisions)
|
||||
2. The relevant spec file (implementation detail)
|
||||
3. data-model.md (schema is authoritative)
|
||||
4. architecture.md (system structure)
|
||||
|
||||
Do not use legacy/ code as a source of truth for anything.
|
||||
The abandoned scaffolding for the original design still exists under `/app` — it is
|
||||
not part of the running system.
|
||||
|
||||
Reference in New Issue
Block a user