fix: coerce stringified arrays in onboarding overview output (#32)
All checks were successful
On Pull Request to Main / test (pull_request) Successful in 41s
On Pull Request to Main / publish (pull_request) Successful in 1m15s
On Pull Request to Main / deploy-dev (pull_request) Successful in 3m17s

Opening an onboarding theme could fail with "LLM output failed schema
validation": the fast-tier model occasionally emits the key_points and
topics_covered tool fields as JSON-encoded strings instead of real
arrays. Same failure class the quiz schema already guards against with
z.preprocess.

- onboardingOverviewSchema: parse "[...]" strings back to arrays before
  validation; split a bullet/newline string of key points as a fallback;
  keep the first 5 key points on overage instead of failing the user.
  Genuinely bad output (too few points, non-JSON strings, wrong types)
  still fails validation.
- emit_onboarding_overview tool: descriptions now state the fields must
  be JSON arrays, never strings (prompt-side nudge).
- 5 regression tests reproducing the exact #32 payload shapes
  (npm test 139/139).

Closes #32

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
RaymondVerhoef
2026-07-13 09:38:46 +02:00
parent b1d3686d35
commit 182fb2357f
3 changed files with 89 additions and 6 deletions

View File

@@ -470,11 +470,11 @@ export const EMIT_ONBOARDING_OVERVIEW_TOOL = {
items: { type: 'string' },
minItems: 3,
maxItems: 5,
description: '35 short, concrete takeaways a newcomer should remember about this theme.',
description: '35 short, concrete takeaways a newcomer should remember about this theme. Must be a JSON array of strings — never a single string.',
},
topics_covered: {
type: 'array',
description: 'The topics that make up this theme. Reuse the exact topic_id you were given so the UI can link back.',
description: 'The topics that make up this theme. Reuse the exact topic_id you were given so the UI can link back. Must be a JSON array of objects — never a string.',
items: {
type: 'object',
properties: {