learning-platform/docs/data-model.md

# Data model: Respellion Learning Platform

## Overview

All structured data lives in **PocketBase** (SQLite). There is **no vector store**
— retrieval is computed at runtime with a local TF-IDF index over `topics`
(`src/lib/retrieval.js`).

Schema is defined by JS migrations in `pb_migrations/` (applied automatically by
the PocketBase binary) and mirrored for local bootstrap in
`scripts/setup-pb-collections.mjs`. The data-access layer is `src/lib/db.js`.

All collections use PocketBase's auto `id`, plus `created` / `updated` autodate
fields unless noted otherwise.

---

## PocketBase collections

### `topics`
Knowledge graph nodes. Created during ingestion, enriched for curriculum.

| Field | Type | Notes |
|---|---|---|
| id | text | kebab-case slug (e.g. `holacratic-roles`) |
| label | text | display name |
| type | text | `concept` · `role` · `process` (ingestion); `fact` is excluded from learning |
| description | text | 1–2 sentence summary |
| learning_relevance | text | `core` · `standard` · `peripheral` · `exclude` |
| relevance_locked | bool | if true, re-ingestion will not overwrite `learning_relevance` |
| theme | text | subject grouping (used by curriculum generation) |
| complexity_weight | number | 1–5 (curriculum ordering) |
| difficulty | text | `introductory` · `intermediate` · `advanced` |

Topics with `type='fact'` or `learning_relevance='exclude'` are filtered out of
learning, micro-learning, curriculum, and test selection.

---

### `relations`
Knowledge graph edges between topics.

| Field | Type | Notes |
|---|---|---|
| source | text | topic id |
| target | text | topic id |
| type | text | `related_to` · `depends_on` · `part_of` · `executed_by` |

Edges are de-duplicated on the `(source, target, type)` tuple.

---

### `content`
On-demand long-form learning content, one record per topic.

| Field | Type | Notes |
|---|---|---|
| topic_id | text | topic this content belongs to |
| data | json | merged object — only generated types are present |

`data` shape (each key generated independently and shallow-merged):

```json
{
  "article":     { "title", "intro", "sections": [{ "heading", "body" }], "keyTakeaways": [] },
  "slides":      [ { "title", "bullets": [], "speakerNote" } ],
  "infographic": { "headline", "tagline", "stats": [{ "value", "label", "icon" }],
                   "steps": [{ "number", "title", "description", "icon" }], "quote", "colorTheme" }
}
```

> There is no `podcast` key. The podcast type was removed.

---

### `micro_learnings`
Generated micro-learning artifacts. One record per topic per type.

| Field | Type | Notes |
|---|---|---|
| topic_id | relation → `topics` | cascade delete |
| type | select | `concept_explainer` · `scenario_quiz` · `flashcard_set` |
| content | json | structured output, schema varies per type |
| status | select | `draft` · `published` (only `published` is visible to employees) |

**Content JSON per type:**

```json
// concept_explainer
{ "sections": [ { "title": "string", "content": "string (HTML: <p>, <ul>, <li>, <strong>)" } ] }   // ≥3 sections

// scenario_quiz
{ "scenario": "string",
  "options": [ { "text": "string", "isCorrect": true, "explanation": "string" } ] }                 // 3–4 options, exactly 1 correct

// flashcard_set
{ "cards": [ { "front": "string", "back": "string" } ] }                                            // 5–10 cards
```

> A former `reflection_prompt` type was dropped and is no longer generated.

---

### `micro_learning_completions`
Append-only completion events. Never updated or deleted.

| Field | Type | Notes |
|---|---|---|
| team_member_id | relation → `team_members` | the employee |
| micro_learning_id | relation → `micro_learnings` | the artifact completed |
| topic_id | relation → `topics` | denormalized topic |
| type | text | type at time of completion |
| session_week | number | the user's **absolute** curriculum week (week 1 = day they enrolled) |

The 26-week slot and cycle are derived from `session_week`; there is no stored
`cycle` field.

---

### `curriculum_versions`
Versioned 26-week schedules. New version on each (re)generation.

| Field | Type | Notes |
|---|---|---|
| version_number | number | increments per generation |
| status | text | `draft` · `active` · `superseded` (exactly one `active`) |
| generation_reason | text | why this version was created |
| confirmed_by | text | admin id who activated it |
| confirmed_at | text | ISO datetime |
| schedule | json | array of 26 week objects (below) |
| coverage_stats | json | `{ themes_kb, themes_scheduled, topics_kb, topics_scheduled }` |

**`schedule[]` week object:**

```json
{ "week_number": 1,              // 1..26
  "theme": "string",
  "topic_ids": ["topic-id"],     // 1+ topic ids
  "estimated_duration": 30,      // 15..45 minutes
  "week_rationale": "string" }
```

---

### `team_members`
Registered users with PIN auth. This is the auth + employee record.

| Field | Type | Notes |
|---|---|---|
| name | text | display name |
| pin | text | login PIN |
| role | text | `admin` or empty/`employee` |
| curriculum_started_at | date | timestamp the user enrolled (week 1 anchor); empty until enrolled |
| enrollment_status | text | `not_started` · `active` |

A user is gated through the `/onboarding` screen until `enrollment_status='active'`
(admins are exempt when heading to the admin panel).

---

### `sources`
Uploaded source documents and their extraction status.

| Field | Type | Notes |
|---|---|---|
| name | text | original filename |
| status | text | `processing` · `completed` · `failed` · `cancelled` |
| error | text | failure message, if any |
| progress | json | `{ current, total, message }` during chunked extraction |

---

### `leaderboard`
Points ledger, one row per user.

| Field | Type | Notes |
|---|---|---|
| user_id | text | team member id |
| name | text | display name |
| points | number | cumulative (+2 per correct quiz answer) |
| tests_completed | number | count of completed tests |
| learnings_completed | number | reserved counter |

Admins are filtered out of the public leaderboard at render time.

---

### `settings`
App-wide key/value store.

| Field | Type | Notes |
|---|---|---|
| key | text | setting key |
| value | text | stringified value |

---

### `llm_calls`
Best-effort telemetry for every Anthropic call (written by `callLLM`).

| Field | Type | Notes |
|---|---|---|
| task | text | logging label (e.g. `learning.article`, `chat.r42`) |
| model | text | resolved model string |
| tier | text | `fast` · `standard` · `reasoning` |
| duration_ms | number | wall-clock |
| input_tokens / output_tokens | number | usage |
| cache_read_tokens / cache_create_tokens | number | prompt-cache usage |
| stop_reason | text | `end_turn` · `tool_use` · `max_tokens` |
| ok | bool | success flag |
| error_msg | text | error, if any |

---

### `onboarding_overviews`
Cached, breadth-first per-theme overview for the onboarding track (issue #30).
One row per theme, shared across all users. Generated lazily on first open.

| Field | Type | Notes |
|---|---|---|
| theme | text | canonical theme name (from `topics.theme`); **unique** |
| content | json | validated `onboardingOverviewSchema` payload (title, what_it_is, why_it_matters, key_points, topics_covered) |
| topics_fingerprint | text | stable hash of the theme's sorted topic ids; a mismatch triggers regeneration |

Unique index on `(theme)`. Rules: authenticated-only (`@request.auth.id != ""`).

---

### `onboarding_completions`
Per-user, per-theme completion marker for the onboarding track (issue #30).

| Field | Type | Notes |
|---|---|---|
| team_member_id | text | the employee — **plain text**, not a relation (admins can delete members; orphan rows are harmless) |
| theme | text | the theme marked complete |

Unique index on `(team_member_id, theme)` → idempotent. Rules: authenticated-only.
Completion is tracked per **theme**, not per day; the 5 "days" are a read-time
presentation grouping, so re-chunking never loses progress.

---

## Dropped / legacy collections

These existed in earlier iterations and have been removed. Their `db.js` helpers
remain as deprecated no-op stubs — do not build on them:

`quiz_banks`, `quiz_results`, `quiz_cache`, `learn_progress`, and the v1
`curriculum` collection.

---

## Client-side storage (not PocketBase)

`localStorage` is used only for admin/browser-local state:

| Key | Purpose |
|---|---|
| `admin:model:{fast,standard,reasoning}` | per-tier model overrides (legacy `admin:model`) |
| `admin:use_simulation` | stub LLM responses instead of calling Anthropic |
| `kb:suggestions` | R42 graph-delta suggestion queue (managed via `kbStore`) |
| `quiz:active:{userId}` | mid-quiz flag (hides R42) |
| `chat:thread:{userId}` | R42 conversation, capped at 50 messages |

`sessionStorage.respellion_session` holds the logged-in team member id.

---

## Retrieval (no vector DB)

R42 context is built by `src/lib/retrieval.js`:

```
buildIndex(topics)         → TF-IDF index over (label + description), cached by array ref
retrieveTopK(index, q, k)  → top-K topics, score = Σ (1 + log(tf)) · log((N+1)/(df+1))
```

`src/components/chat/rag.js` combines top-K results with verbatim topic mentions,
filters relations to the retrieved set, and injects limited deep content for
explicitly named topics.