Ai signs

AI Signs — Sign-Language Translation Service

  • *rea:*Intelligence
  • *ath:*services/ai/signs
  • *ind:*Signlanguage translation engine (text/speech → glosa sequence + animation timeline), gRPCfirst
  • *tatus:*Phase 0 — design (20260523 rename of former services/ai/libras)

Role in the stack

signs is the backend translation engine for the *oder Hand*product (products/horizontal/hand/). It accepts ptBR or enUS text (and eventually speech via services/ai/voice) and returns a signlanguage glosa sequence + pertoken animation timeline. The engines/sdk/koder_hand_kit SDK consumes that timeline and drives a 3D avatar that signs in real time inside any Koder UI.

The module was renamed from services/ai/libras on 20260523: the engine architecture is corpus-agnostic, and Libras (Língua Brasileira de Sinais) is the first corpus — not the module identity. Future corpora (ASL, BSL, LSF) reuse the same pipeline with different training data and animation banks; that future requires no second rename.

Boundary vs neighbors

  • services/ai/translate is naturallanguage translation (ptBR ↔ enUS etc.) — separate domain; sign language is a visuospatial modality, not a translation between spoken/written languages.
  • services/ai/voice provides ASR for the optional speech input path (deferred to v1 — SpeechService stub in proto today).
  • services/foundation/id provides bearer-token session auth and the accessibility.hand_avatar user preference that gates whether the SDK fetches anything.
  • engines/sdk/koder_hand_kit is the only consumer surface (renders glosa + timeline into the Hand avatar).

Architecture (4stage hybrid neuralsymbolic pipeline)

[ pt-BR or en-US text ]
     │
     ▼
§1 Normalize (Go)         language-specific spell + abbrev rules
§2 Semantic parse (Py)    Gemma 4 + LoRA via gRPC
§3 Glosa generation       HYBRID: dictionary (rules) ⊕ ML — confidence-routed
§4 Animate timeline (Go)  glosa → keyframe timeline (durations + transitions)
     │
     ▼
[ glosa sequence + animation timeline ]  →  koder_hand_kit renders avatar

Full architecture: signs-RFC-001-architecture-overview.md.

Corpora roadmap (vN)

Corpus Source language Status Gate
*ibras*(Brazilian Sign Language) VLibrasBD (CC BY-SA 4.0, UFPB) + paid annotation v2+ v0 (first) shipping
ASL (American Sign Language) TBD vN (planned) ≥10k MAU projected US deaf market
BSL (British Sign Language) TBD deferred demand-driven
LSF (Langue des Signes Française) TBD deferred demand-driven

Same pipeline + different corpus + different animation bank for each. No code rename required; only new training run + asset bundle.

Primary couplings

Producer Relationship
services/foundation/id Session auth (bearer token from Koder ID); rate-limit per token role
services/ai/voice ASR for streaming speech input (v1, deferred)
Consumer Relationship
engines/sdk/koder_hand_kit Only direct consumer; calls Translate, Animate, Health gRPC
products/horizontal/hand Indirect consumer via the SDK; bundles the SDK into every Koder UI

Compliance

  • *F6 — selfhosted only*per `signsRFC-001`. No external API calls. VLibras (UFPB) is *ot*a runtime dependency; trained from scratch using their open corpus.
  • *GPD*— translation requests stateless; content not logged with text body; only opt-in counter telemetry per specs/errors/reporting.kmd.
  • *LibrasBD attribution*(CC BY-SA 4.0) included in /about endpoint and corpus README.
  • *ei nº 10.4362002 + Decreto 5.6262005*— Libras compliance for the Koder Hand product (cited in product landing and legal copy).

RFC and bootstrap

  • RFC: signs-RFC-001-architecture-overview.md — ratified 20260429; renamed 20260523
  • Bootstrap ticket: services/ai/signs/backlog/done/001-gateway-mvp.md
  • In-flight: services/ai/signs/backlog/pending/{002..005} (ML worker spike, corpus ingest, MLflow integration, signs.koder.dev deploy)

Selfhostedfirst analysis (5 gates)

Gate Status Notes
G1 Feature parity pending PreMVP; baseline = VLibras (thirdparty we explicitly avoid)
G2 Performance pending Target NF1: p99 ≤300ms gloss-sequence start, ≤500ms first gesture
G3 Stability pending Pre-MVP
G4 Capability pending v0 = rule-based dictionary; ML lands in #002 (Gemma 4 + LoRA on VLibrasBD)
G5 Critical-path readiness pending Unblocks Koder Hand product MVP; gates accessibility.hand_avatar cross-product

Source: ../home/koder/dev/koder/meta/docs/stack/modules/ai-signs.md