AI Extract — Structured Extraction

• *rea:*Intelligence
• *ath:*services/ai/extract
• *ind:*Schemadriven structured extraction (JSONoutofcontent), NER, KV pairs
• *tatus:*v0.0.1 — sector bootstrapping (20260509)

Role in the stack

extract consolidates a capability that was duplicated across the stack: every consumer needing JSONoutoftext (rag, recsys, halfadozen product backends parsing forms/invoices) had its own homegrown prompt + retry + repair loop, all subtly different and all subtly broken. This sector centralizes the capability — JSON Schema validation serverside, principled retrywith-correction (LLM sees its own validator error and fixes it), batch processing, cached repeat extractions.

It is the Koder analog of OpenAI Structured Outputs, Instructor (Python), and Marvin. It pairs with vision/ (visual document → text+layout pre-step) and is the canonical ingest path for rag/ and recsys/.

Boundary vs neighbors

• services/ai/vision is the visual prestep for PDF/image content; extract consumes its layoutaware output.
• services/ai/rag and services/ai/recsys are the highest-volume consumers — both will migrate from bespoke loops.
• services/ai/gateway provides the LLM with structuredoutput / functioncalling.
• services/ai/cache provides repeat-extraction caching.
• services/ai/prompt owns extraction prompt templates.

Features (v1 target)

• Schemadriven extraction (JSON Schema draft 202012)
• Bounded retrywithcorrection (default 3 attempts, hard cap 5)
• NER with canonical Koder taxonomy (16 types incl. CPFCNPJCEP)
• KV pair extraction (forms, receipts) — layout-aware when input has bounding boxes
• Async batch jobs with concurrency control + progress streaming (SSE)
• Repeat-extraction cache keyed by (schema, content, model, instructions)
• Pydanticshorthand acceptor (autotranslated to JSON Schema)
• SDK helpers (Go + Dart) for downstream consumers

Primary couplings

RFC and bootstrap

• RFC: extract-RFC-001-foundations.kmd — *ccepted*20260509
• Bootstrap ticket: services/ai/backlog/done/132-extract-bootstrap.md
• Implementation tickets: services/ai/extract/backlog/pending/{001..005}

Selfhostedfirst analysis (5 gates)