AI Search — Web Search & Scrape

• *rea:*Intelligence
• *ath:*services/ai/search
• *ind:*Web search + scrape (Perplexitylike, multiengine, with citations)
• *tatus:*v0.0.1 — sector bootstrapping (20260509)

Role in the stack

search is the foundation for any agent that needs to look up live information, cite sources, or scrape pages. Without it, agents are frozen at training cutoff and every product reinvents robots.txt etiquette, dedup, rate limiting, and citation formatting. The need was already explicit in the recent commit /k-evolve koder-ai vs firecrawl+wispr+testsprite (20260509).

It is the Koder analog of Perplexity API, Tavily, Firecrawl, and Exa — selfhosted by default (SearxNG), tenantisolated, integrated with services/ai/gateway for LLM rerank and services/ai/cache for result caching.

Features (v1 target)

• Multi-engine search (SearxNG default; commercial fallback via gateway)
• Page scrape: playwright + readability + htmltomarkdown, cached in kdb-blob
• News vertical (recency-biased)
• LLM rerank + summarize with [N]: <url> citations
• Per-tenant quotas + usage events to billing
• Robots.txt enforcement + per-domain rate limit + identifiable UA

Primary couplings