feat: phase 3 of AI pipeline hardening — extraction quality
All checks were successful
On Pull Request to Main / test (pull_request) Successful in 31s
On Pull Request to Main / publish (pull_request) Successful in 1m1s
On Pull Request to Main / deploy-dev (pull_request) Successful in 1m32s

Replace stateless one-shot extraction with a stateful, paced, cancellable
pipeline. Six subtasks:

- 3.1 Sentence-aware chunking with 800-char overlap (was paragraph-only
  at 4000 chars). Hard-split fallback for runaway sentences.
- 3.2 Stateful extraction: chunks 2+ receive an "already-extracted topic
  IDs" hint capped at 200 IDs, so the model reuses IDs instead of
  inventing variants like software-developer vs software-engineer.
- 3.3 Token-bucket limiter in llmRetry.js (extractionLimiter, 5 req/min).
  callLLM awaits the limiter before fetch; 429+Retry-After calls
  pauseUntil. Replaces hard setTimeout(12000) and setTimeout(15000).
- 3.4 relevance_locked column on topics — admin edits to relevance are
  sticky across re-extraction. Migration + merge respects the flag +
  unlock checkbox in KnowledgeGraph edit form.
- 3.5 Unify relation vocabulary — handbook prompt no longer mentions
  legacy "executes"; one-shot migration rewrites existing executes rows
  to executed_by with source/target swapped.
- 3.6 Cancellation — Cancel button on UploadZone wired to an
  AbortController threaded into callLLM; aborted runs persist status =
  "cancelled" rather than "failed".

Tests: 16 new unit tests for chunkText, buildKnownIdsHint, and
createLimiter. All 61 tests pass, 0 lint errors, build clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
RaymondVerhoef
2026-05-20 17:56:45 +02:00
parent 9771928926
commit aeb197d5f4
10 changed files with 509 additions and 48 deletions

View File

@@ -180,10 +180,17 @@ const KnowledgeGraph = () => {
};
const saveEdit = async () => {
await db.upsertTopic({ ...selectedNode, ...editData });
const updated = topics.map(t => t.id === selectedNode.id ? { ...t, ...editData } : t);
// If the admin touched learning_relevance, lock it so re-extraction won't overwrite the choice.
// But an explicit relevance_locked in editData (the unlock checkbox) always wins.
const relevanceChanged =
editData.learning_relevance !== undefined &&
editData.learning_relevance !== selectedNode.learning_relevance;
const next = { ...editData };
if (relevanceChanged && next.relevance_locked === undefined) next.relevance_locked = true;
await db.upsertTopic({ ...selectedNode, ...next });
const updated = topics.map(t => t.id === selectedNode.id ? { ...t, ...next } : t);
setTopics(updated);
setSelectedNode({ ...selectedNode, ...editData });
setSelectedNode({ ...selectedNode, ...next });
setIsEditing(false);
};
@@ -265,22 +272,11 @@ const KnowledgeGraph = () => {
setSyncProgress(`Processing ${count} of ${filesToProcess.length}: ${file.name}...`);
try {
const rawContent = await getFileContent('respellion', 'employee-handbook', file.path);
// Pacing is handled centrally by extractionLimiter inside analyzeHandbookDelta.
await analyzeHandbookDelta(rawContent, file.path);
await db.updateHandbookSyncState(file.path, file.sha);
// To respect Anthropic's 5 requests per minute rate limit on this tier,
// we pause for 15 seconds before processing the next file.
if (count < filesToProcess.length) {
setSyncProgress(`Waiting 15s to avoid rate limits... (${count}/${filesToProcess.length})`);
await new Promise(resolve => setTimeout(resolve, 15000));
}
} catch (err) {
console.error('Failed to process file:', file.path, err);
// We continue processing other files even if one fails, but still wait to avoid further rate limits
if (count < filesToProcess.length) {
setSyncProgress(`Error on ${file.name}. Waiting 15s... (${count}/${filesToProcess.length})`);
await new Promise(resolve => setTimeout(resolve, 15000));
}
}
}
setSyncProgress('Sync Complete! Click "Analyze & Optimize Graph" above to clean up and merge.');
@@ -369,7 +365,7 @@ ${JSON.stringify({ topics: compactTopics, relations: compactRelations })}`;
if (actions.relevanceUpdates && Array.isArray(actions.relevanceUpdates)) {
for (const update of actions.relevanceUpdates) {
const topicIndex = updatedTopics.findIndex(t => t.id === update.id);
if (topicIndex !== -1) {
if (topicIndex !== -1 && !updatedTopics[topicIndex].relevance_locked) {
updatedTopics[topicIndex] = { ...updatedTopics[topicIndex], learning_relevance: update.learning_relevance };
}
}
@@ -532,6 +528,17 @@ ${JSON.stringify({ topics: compactTopics, relations: compactRelations })}`;
<option value="peripheral">Peripheral</option>
<option value="exclude">Exclude</option>
</select>
{selectedNode.relevance_locked && (
<label className="flex items-center gap-2 text-xs text-fg-muted mt-2 cursor-pointer">
<input
type="checkbox"
checked={editData.relevance_locked !== false}
onChange={e => setEditData({...editData, relevance_locked: e.target.checked})}
className="rounded bg-bg-warm border-transparent focus:ring-0 text-teal"
/>
Locked re-extraction will not change this
</label>
)}
</div>
<div>
<label className="text-xs text-fg-muted uppercase tracking-wider mb-1 block">Description</label>
@@ -554,9 +561,12 @@ ${JSON.stringify({ topics: compactTopics, relations: compactRelations })}`;
</div>
<div>
<p className="text-xs text-fg-muted uppercase tracking-wider mb-1">Type & Relevance</p>
<div className="flex gap-2">
<div className="flex gap-2 flex-wrap">
<span className="inline-block px-2 py-1 bg-bg-warm rounded-[var(--r-pill)] text-xs font-mono">{selectedNode.type}</span>
<span className="inline-block px-2 py-1 bg-bg-warm rounded-[var(--r-pill)] text-xs font-mono opacity-80">{selectedNode.learning_relevance || 'standard'}</span>
{selectedNode.relevance_locked && (
<span className="inline-block px-2 py-1 bg-teal/10 text-teal rounded-[var(--r-pill)] text-xs font-mono" title="Re-extraction will not change relevance for this topic.">locked</span>
)}
</div>
</div>
<div>