Dev eye
Koder Eye — dev/eye
Ondevice UI analysis bridge for Android. Replaces most screenshottoLLM roundtrips (1 500 image tokens each) with a local JSON API (200 tokens) that agents query via adb forward. From v0.2 agents can also inject gestures, making Eye a full inspectanddrive bridge.
Role in the stack
| Area | Sector | Consumers |
|---|---|---|
| Developer Platform | — (new sector) | Kode, Kortex, Claude Code, ad-hoc test scripts |
Sits alongside dev/bridge and dev/kterm as a devicefacing developer tool. Unlike those, Eye runs *n the device*— it is the ondevice half of a split inspector.
Primary couplings
| Module | Nature |
|---|---|
| None (Android platform only) | v0.2 has no dependency on other Koder modules |
dev/store |
*ublished*at hub.koder.dev/apps/koder-eye (v0.2.1) |
ai/kode, ai/kortex |
consumers of the HTTP API — planned in RFC-001 §8 |
Interfaces
Local HTTP on 127.0.0.1:9876 (reach via adb forward tcp:9876 tcp:9876). All routes except /healthz require Authorization: Bearer <token>.
Observation
| Endpoint | Returns |
|---|---|
GET /healthz |
service status + granted permissions (no auth) |
GET /screen |
full UI tree as JSON + screen_hash |
GET /screen/text |
flat strings, reading order |
GET /screen/buttons |
clickable elements (label from child walk for Compose) |
GET /screen/diff?since=<hash> |
{added, removed, modified} vs a prior snapshot |
GET /windows |
all visible windows (id, layer, type, focused, bounds) |
GET /windows/<id>/screen |
UI tree for a specific window (dialogs, overlays) |
GET /events?since=<n>[&wait=<ms>] |
AccessibilityEvent stream; suspends up to 60 s |
GET /events/stream |
SSE long-lived stream; 15 s keepalive heartbeat |
Capture & OCR
| Endpoint | Body | Returns |
|---|---|---|
POST /capture |
{to?: filename} |
PNG path, dimensions, sha1 |
POST /ocr |
{region?: "x,y,w,h"} |
[{text, bounds}] — ML Kit on-device |
POST /query |
— | 501 (Gemma Nano / AICore — ticket EYE-006) |
Injection (requires toggle ON in app)
| Endpoint | Body | Action |
|---|---|---|
POST /tap |
{x,y} or {view_id} |
Tap gesture (50 ms) |
POST /scroll |
{x,y,dx,dy} or {direction} |
Fling gesture (300 ms) |
POST /type |
{text, view_id?} |
ACTIONSETTEXT on node |
POST /back |
— | GLOBALACTIONBACK |
POST /home |
— | GLOBALACTIONHOME |
POST /recents |
— | GLOBALACTIONRECENTS |
Rate limiting
Tokenbucket perpath rate limiter (RateLimiter.kt):
| Path prefix | Limit |
|---|---|
/screen, /capture, /ocr |
5 req/s |
| All others | 20 req/s |
Exceeded requests get 429 Too Many Requests with Retry-After header and error ID EYE-RL-429-001.
Advanced config (in-app)
EyeConfig exposes four parameters via SharedPreferences, editable in the app's "Advanced" section:
| Setting | Default | Range |
|---|---|---|
| Port | 9876 | 1024–65535 |
| Tree depth | 32 | 4–128 |
| Text length | 512 | 64–4096 |
| Event cap | 1024 | 256–8192 |
EventLog.resize() is called live when event cap changes.
Host-side CLI: keye
products/dev/eye/cli/keye — single-file Python 3 script, zero dependencies.
keye healthz | text | buttons | screen | diff <hash>
keye text --since-last # diff vs cached last screen hash (TTL 5 min)
keye windows | capture [file.png] | ocr [--region x,y,w,h]
keye watch [--since N] # SSE event stream
keye tap <x> <y> | tap --id <view_id>
keye scroll --dir down | type [--id <view_id>] <text>
keye back | home | recents
keye setup # adb forward + smoke test
keye fanout <command> # run read command on all ADB-connected devices
keye token [--set <token>]Token saved to ~/.config/koder-eye/token. Screen hash cache at ~/.cache/koder-eye/<serial>/last-hash. Coloured table output by default; --json for raw.
Status
*0.2.1 (20260423)*— verified on Galaxy S26 Ultra (Android 16).
| Component | Status |
|---|---|
| Android app builds (AGP 8.11.1, Kotlin 2.2.20, compileSdk 36) | ✅ |
| Bearer token auth (EncryptedSharedPreferences, Keystore AES |
✅ |
| Release signing (Android Keystore RSA-2048, APK v2 scheme) | ✅ |
| AccessibilityService captures UI tree | ✅ |
| Foreground service hosts Ktor CIO on localhost | ✅ |
/screen, /screen/text, /screen/buttons |
✅ |
/screen/diff (LRU |
✅ |
/events with long-poll (?wait=) and /events/stream SSE |
✅ |
/capture (takeScreenshot API 30+) |
✅ |
/ocr ML Kit Latin |
✅ |
/windows + /windows/<id>/screen |
✅ |
| Injection: tap, scroll, type, back, home, recents | ✅ (gated by toggle) |
| Per-path rate limiting (token bucket, 5/20 rps) | ✅ |
| Advanced config UI (port, tree depth, text len, event cap) | ✅ |
Host CLI keye with fanout + snapshot cache |
✅ |
| JVM unit tests (EventLog, RateLimiter, EyeConfig) | ✅ |
| Instrumented tests (androidTest, UiAutomation fixture, 4 integration tests) | ✅ |
| Theme toggle (LightDark per `specsthemeslight-dark.kmd`, sunmoon icon top |
✅ |
| MVP acceptance checklist (RFC-001 §10) | ✅ |
| Final 3D icon (almond sclera, teal iris, catchlight — adaptive + legacy mipmap) | ✅ |
Landing page at eye.koder.dev |
✅ |
| Koder Hub listing + APK published | ✅ hub.koder.dev/apps/koder-eye |
| iOS sibling | ⏳ RFC-002 |
Design references
- RFC
001 — Ondevice UI analysis bridge for AI agents meta/docs/stack/specs/errors/user-facing-messages.kmd— allEYE-*error IDs