AI Vision — Multimodal Vision Foundation

• *rea:*Intelligence
• *ath:*services/ai/vision
• *ind:*Multimodal vision foundation (OCR + captioning + detection + doc layout + VLM proxy)
• *tatus:*v0.0.1 — sector bootstrapping (20260509)

Role in the stack

vision consolidates the serverside half of multimodal capability. The Stack already has services/ai/voice for audio; vision was fragmented across services/ai/kode (chat attachments), products/dev/eye (ondevice Android UI), and ad-hoc document analysis. Without a foundation, every product reinvents the pipeline.

It is the Koder analog of OpenAI Vision, Claude Vision, Gemini Vision, and Google Cloud Vision — selfhosted where viable (Tesseract, PaddleOCR, Florence2, suryadocling) and a thin VLM proxy front for frontier models routed via `servicesai/gateway`.

Boundary vs products/dev/eye

Eye stays ondevice (Android UI inspection, lowlatency screen capture). Vision handles serveronly workloads: multipage PDFs, batch jobs, GPUbound models, multitenant. Handoff protocol defined in ticket #005.

Features (v1 target)

• OCR: Tesseract (CPU) + PaddleOCR (GPU), auto-pick by workload, baseline pt+en
• Doc layout: surya/docling pipeline, structured tables (JSON, not markdown)
• VLM proxy: ClaudeGeminiGPT-4V via gateway with normalized response
• Captioning + object detection: Florence2 selfhosted, fallback to VLM
• Eye handoff protocol: on-device delegation contract

Primary couplings

RFC and bootstrap

• RFC: vision-RFC-001-foundations.kmd — *ccepted*20260509
• Bootstrap ticket: services/ai/backlog/done/116-vision-bootstrap.md
• Implementation tickets: services/ai/vision/backlog/pending/{001..005}

Selfhostedfirst analysis (5 gates)