Opening an onboarding theme could fail with "LLM output failed schema
validation": the fast-tier model occasionally emits the key_points and
topics_covered tool fields as JSON-encoded strings instead of real
arrays. Same failure class the quiz schema already guards against with
z.preprocess.
- onboardingOverviewSchema: parse "[...]" strings back to arrays before
validation; split a bullet/newline string of key points as a fallback;
keep the first 5 key points on overage instead of failing the user.
Genuinely bad output (too few points, non-JSON strings, wrong types)
still fails validation.
- emit_onboarding_overview tool: descriptions now state the fields must
be JSON arrays, never strings (prompt-side nudge).
- 5 regression tests reproducing the exact #32 payload shapes
(npm test 139/139).
Closes#32
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- src/lib/random.js: Fisher–Yates shuffle/sample/pickInt; replace every
biased .sort(() => 0.5 - Math.random()) site in testService.
- testService: debias correctIndex via prompt + runtime re-roll (up to 2x
when one position holds >50%); quality gate rejecting <4 distinct
options, banned filler ("all of the above" etc) and explanations
shorter than 20 chars; dedup new questions against the existing bank
via normalised question text.
- Quiz schema/tool/prompt require difficulty ('easy'|'medium'|'hard');
db.getQuizBank defaults legacy records to 'medium' on read.
- learningService.generateCustomTopic: kebab-case slug ID from the
polished label with collision suffixes; default learning_relevance
'standard' when the model omits it.
- Tests for random helpers, dedup/quality-gate behaviour and the
extended quiz schema.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements phase 1 of AI_PIPELINE_HARDENING_PLAN.md. Every Anthropic call
now goes through one module that owns retry, timeout, abort, structured-
output parsing, schema validation, and best-effort call telemetry.
* src/lib/llm.js — single callLLM entry point. Resolves model per tier
(fast / standard / reasoning) with admin:model legacy fallback for the
standard tier; 60s default timeout via AbortController; balanced-brace
JSON extraction; LLMHttpError, LLMTruncatedError, LLMOutputError, and
LLMValidationError surface clearly distinct failure modes.
* src/lib/llmRetry.js — exponential backoff with full jitter, retries
only on transient HTTP statuses, honours Retry-After up to 60s, never
retries on AbortError.
* src/lib/llmSchemas.js — Zod schemas for every structured task plus
normalizeHandbookResult (collapses legacy "executes" relations into
the canonical "executed_by" vocabulary).
* src/lib/api.js — thin shim over callLLM so existing callers (extraction
pipeline, learning, quiz, R42, knowledge graph) keep working unchanged.
* src/lib/__tests__/ — 32 Vitest cases covering parse paths, error
surfaces, simulation mode, model resolution, and schema validation.
* src/pages/Admin/index.jsx — three model inputs (fast / standard /
reasoning) replacing the single legacy field; legacy value falls back
for the standard tier so existing overrides survive.
Adds Zod and Vitest, plus an "npm run test" script.
Also cleans up the pre-existing repo-wide ESLint failures so phase 1's
"npm run lint passes" acceptance criterion can be checked: drops unused
React imports across the JSX tree (React 19 JSX runtime auto-imports),
attaches cause to rethrown errors in the service modules, ignores
pb_migrations in the ESLint config (PocketBase JSVM globals), and
removes one dead handleCreateCustom function in Leren.jsx. A real
behaviour bug surfaced in Testen.jsx — the quiz timer captured a stale
finishQuiz via setInterval closure; now updated via finishQuizRef so the
timer always invokes the latest callback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
