The weekly quiz path made up to six sequential LLM calls (one per topic)
with three retries each, blowing past the 30s budget. Worse, the
quiz_banks collection has been dropped, so getQuizBank/setQuizBank are
no-ops and every generated batch was thrown away — the assembled quiz
ended up empty and surfaced as "Could not assemble enough questions."
Replace the per-topic loop with a single batched call on the fast tier
that emits all five questions in one round-trip, using the review topics
as prompt context instead of separate calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Created multiple hot-update JavaScript files for app layout and webpack.
- Each file includes a warning about the use of "eval-source-map" for development purposes.
- Added source mapping for CSS files in the app layout.
- Generated corresponding hot-update JSON files to manage module updates.
- Added a new "status" field to the themes collection with options: draft, published, and rejected.
- Updated the migration script to include the new field and its options.
- Created a new ingestion migration script to ensure the "status" field includes "rejected" as an option if not already present.
- Added multiple hot-update files for webpack to support the new changes in the frontend.
- Introduced new page component for library topics with type checks.
- Added migration scripts to update access rules for various collections including badges, curriculum versions, and themes.
- Implemented PocketBase integration for managing collection access rules dynamically.
- Ensured proper type validation for page props and metadata generation functions.
- Created type definitions for `auth`, `layout`, and `page` components to ensure type safety and consistency.
- Implemented checks for entry validity and prop types using utility types.
- Added a `package.json` file to specify module type for TypeScript compatibility.
- Introduced gamification service spec detailing responsibilities, API surface, XP calculation, levels, streaks, badges, milestone cards, and heatmap data.
- Added generation service spec outlining the process for generating micro learning content, including API endpoints, AI call configuration, prompt strategies, and error handling.
- Created R42 chat service spec covering chatbot interactions, retrieval pipeline, prompt construction, response generation, and stateless design principles.
- Created handover document outlining design decisions and application functionality.
- Developed implementation plan detailing phased approach for service development.
- Specified ingestion service responsibilities, API surface, and processing pipeline.
Truncates sources, curriculum, content, quiz banks/results/cache, topics
and relations in dependency order so AI-generated state can be wiped
between smoke runs without leaving dangling references. Handbook sync
state is cleared by default (otherwise re-sync is a no-op); user
progress and leaderboard are opt-in. Team members, settings, and LLM
telemetry are preserved.
UI lives in Admin → Settings → Danger Zone and requires typing RESET
before the button enables. Per-collection deletion counts are reported.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Handbook sync ran files sequentially under a 5 req/min limiter with a
hardcoded 60s LLM timeout, causing long syncs and AbortError timeouts on
large files. Now: limiter at 20 req/min, files processed with concurrency
4, handbook extraction timeout raised to 180s, and near-empty files skip
the LLM call.
callLLM gains a timeoutMs option; passing a signal no longer silently
disables the per-request timeout.
llm_calls telemetry self-disables after the first 404 so deploys without
the migration applied don't spam the console.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add dependency-free TF-IDF retrieval (src/lib/retrieval.js) with NL+EN
stopwords and a WeakMap-cached index.
- Rewrite buildKbContext to ship the top-K relevant topics + verbatim-
mentioned ids only, filter relations to the included set, and append a
[kb_hash: <8 hex>] suffix so the ephemeral prompt cache busts when the
graph changes. Returns { context, retrievedTopics, allTopics }.
- Add LOOKUP_TOPIC_TOOL and drive useChat through callLLM directly with a
multi-hop tool_result loop capped at 3 hops; preserve Anthropic-provided
tool_use ids through callLLM so the loop can echo correct tool_use_id.
- Truncate R42 history to the last 12 turns and prepend a single
"(earlier conversation truncated)" assistant message.
- Set R42 chat defaults: temperature 0.3, maxTokens 2048.
- Add pb_migrations/1780500002_created_llm_calls.js (the best-effort
logger in callLLM was already wired) and a new Admin → Diagnostics
view showing the last 100 calls with token usage, cache-hit rate, and
USD cost from a local Anthropic price table.
- Finalize AI_PIPELINE_HARDENING_PLAN.md: mark Phases 1–5 shipped and
Phase 6 (eval harness) explicitly out of scope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- src/lib/random.js: Fisher–Yates shuffle/sample/pickInt; replace every
biased .sort(() => 0.5 - Math.random()) site in testService.
- testService: debias correctIndex via prompt + runtime re-roll (up to 2x
when one position holds >50%); quality gate rejecting <4 distinct
options, banned filler ("all of the above" etc) and explanations
shorter than 20 chars; dedup new questions against the existing bank
via normalised question text.
- Quiz schema/tool/prompt require difficulty ('easy'|'medium'|'hard');
db.getQuizBank defaults legacy records to 'medium' on read.
- learningService.generateCustomTopic: kebab-case slug ID from the
polished label with collision suffixes; default learning_relevance
'standard' when the model omits it.
- Tests for random helpers, dedup/quality-gate behaviour and the
extended quiz schema.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a topic's quiz bank is empty (or below the requested count), we
previously seeded it with a fresh batch of 10 questions. That meant the
first weekly quiz for any new topic triggered a 10-question LLM call —
heavy for what's ultimately a 1-question sample for review topics, and
overkill for the typical 5-question primary topic.
- forceGenerateTopicQuestions default count: 10 → 5
- getOrGenerateTopicQuestions seed amount: 10 → 5
- TestManager "Generate" defaults + empty-state button copy: 10 → 5
- QUIZ_SYSTEM difficulty hint: rewritten for a 5-question batch (2 easy
/ 2 medium / 1 hard) with explicit "scale proportionally for larger
batches" so admins can still generate 10+ via TestManager when they
want more depth.
Tests 61/61 pass, lint clean (0 errors), build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace stateless one-shot extraction with a stateful, paced, cancellable
pipeline. Six subtasks:
- 3.1 Sentence-aware chunking with 800-char overlap (was paragraph-only
at 4000 chars). Hard-split fallback for runaway sentences.
- 3.2 Stateful extraction: chunks 2+ receive an "already-extracted topic
IDs" hint capped at 200 IDs, so the model reuses IDs instead of
inventing variants like software-developer vs software-engineer.
- 3.3 Token-bucket limiter in llmRetry.js (extractionLimiter, 5 req/min).
callLLM awaits the limiter before fetch; 429+Retry-After calls
pauseUntil. Replaces hard setTimeout(12000) and setTimeout(15000).
- 3.4 relevance_locked column on topics — admin edits to relevance are
sticky across re-extraction. Migration + merge respects the flag +
unlock checkbox in KnowledgeGraph edit form.
- 3.5 Unify relation vocabulary — handbook prompt no longer mentions
legacy "executes"; one-shot migration rewrites existing executes rows
to executed_by with source/target swapped.
- 3.6 Cancellation — Cancel button on UploadZone wired to an
AbortController threaded into callLLM; aborted runs persist status =
"cancelled" rather than "failed".
Tests: 16 new unit tests for chunkText, buildKnownIdsHint, and
createLimiter. All 61 tests pass, 0 lint errors, build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Anthropic has deprecated the temperature parameter for their reasoning
models (claude-opus-4-7). This was causing a 400 error when analyzeGraph
called callLLM with tier: 'reasoning'.
Solution: conditionally exclude temperature from the request body when
tier === 'reasoning'. Fast and standard tiers retain their temperature
parameter.
This unblocks the "Analyse and Optimize" button in the Knowledge Graph
admin panel post-Phase-2 deployment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
