RFC005 — Catalog adapter: bridging kdbplanner to kdb-record

1. Summary

The kdb-planner operates on decoded Vec<Value> rows via the RowSource / MutableRowSource traits, backed today by InMemoryTables. The kdb-record layer persists rows in a KvCluster (sled/TiKV) with schema-aware Protobuf encoding, fingerprint verification, and a four-table catalog in TENANT_SYSTEM.

This RFC defines the *dapter crate*(kdb-adapter) that bridges the two: it implements RowSource and MutableRowSource for Record, resolves table schemas from the catalog at query time, and converts between the planner's Value types and the record layer's Protobuf wire format.

2. Goals

1. *RowSource for RecordAdapter** — the planner's execute()` can run

   SQL against real persisted data without changes to the planner itself.

2. *chema resolution from catalog*— SELECT * FROM users resolves

   users via Catalog::lookup_table(tenant, "users") instead of requiring the caller to supply a TableSchema.

3. *alue conversion*— bidirectional mapping between kdb_planner::Value

   and kdb_record::ColumnValue (the Protobuf-decoded form).

4. *enant scoping*— every adapter instance is bound to a single

   Tenant, enforced at construction time. Cross-tenant queries are not supported in v0.1.

3. Non-goals (v0.1)

• DDL execution (CREATE TABLE, ALTER TABLE, DROP TABLE) — the adapter

  reads the catalog; schema mutations go through the Catalog API directly.

• Cross-tenant joins or federated queries.
• Query plan caching or prepared statement storage (see #093).
• Cost-based optimization using catalog statistics.

4. Architecture

┌──────────────────┐
│   kdb-gateway    │  HTTP POST /v1/sql  or  gRPC ExecuteSQL
│                  │
│  ┌────────────┐  │
│  │ kdb-adapter│  │  implements RowSource + MutableRowSource
│  │            │  │  wraps kdb-record::Record + kdb-record::Catalog
│  └──────┬─────┘  │
│         │        │
│  ┌──────▼─────┐  │
│  │ kdb-planner│  │  build() + execute() — unchanged
│  └────────────┘  │
└──────────────────┘
         │
    ┌────▼────┐
    │kdb-record│  KvCluster (sled/TiKV)
    └─────────┘

4.1 RecordAdapter struct

pub struct RecordAdapter<K: KvCluster> {
    record: Record<K>,
    catalog: Catalog<K>,
    tenant: Tenant,
}

4.2 RowSource implementation

impl<K: KvCluster> RowSource for RecordAdapter<K> {
    fn scan(&self, table: &str) -> ExecResult<Vec<Row>> {
        // 1. lookup_table(tenant, table) → (table_id, schema, fingerprint)
        // 2. record.scan(tenant, table_id) → Vec<RowBytes>
        // 3. decode each row: ColumnValue → Value
        // 4. return Vec<Row>
    }
}

4.3 MutableRowSource implementation

impl<K: KvCluster> MutableRowSource for RecordAdapter<K> {
    fn scan(&self, table: &str) -> ExecResult<Vec<Row>> { ... }

    fn replace_all(&mut self, table: &str, rows: Vec<Row>) -> ExecResult<()> {
        // 1. lookup schema
        // 2. delete_range for existing rows
        // 3. encode + put each new row
    }

    fn append(&mut self, table: &str, rows: Vec<Row>) -> ExecResult<()> {
        // 1. lookup schema
        // 2. encode + put each row with auto-generated key
    }
}

4.4 Value conversion

4.5 Schema resolution for the planner

The adapter exposes a method to build a kdb_planner::TableSchema from the catalog's kdb_record::Schema:

impl<K: KvCluster> RecordAdapter<K> {
    pub fn resolve_schema(&self, table: &str) -> Result<TableSchema, Error> {
        let (_id, schema, _fp) = self.catalog
            .lookup_table(self.tenant, table)?
            .ok_or(Error::TableNotFound(table.into()))?;
        convert_schema(&schema)
    }
}

This replaces the current requirement for callers to supply schema in the HTTP request body. The gateway will:

1. Try adapter.resolve_schema(table) from the catalog.
2. Fall back to request-supplied schema (for stateless/test mode).

5. Sync vs Async

The kdb-record layer is async (KvCluster uses async fn). The planner is sync. Two approaches:

*ption A: tokio block_on wrapper*— the adapter calls tokio::runtime::Handle::current().block_on(async { ... }) inside each RowSource method. Simple, but blocks a tokio thread during I/O.

*ption B: async RowSource*— refactor the planner's RowSource trait to be async (async fn scan). Requires changing every executor function to be async. More invasive but cleaner long-term.

*ecision: Option A for v0.1.*The v0.1 workload is single-tenant, low-concurrency. The planner executes in <1ms for typical queries; the I/O wait is dominated by the substrate. A spawn_blocking wrapper in the gateway keeps the async reactor unblocked. Option B is tracked for v0.2 when the executor needs streaming/pipeline execution.

6. Tickets unblocked

7. Implementation plan

1. *reate crate kdb-adapter*— deps: kdbplanner, kdbrecord, tokio.
2. *alue conversion module*— bidirectional planner::Value ↔ record::ColumnValue.
3. *ecordAdapter struct*— new(record, catalog, tenant).
4. *owSource impl*— scan with decode.
5. *utableRowSource impl*— replace_all, append with encode.
6. *chema resolution*— resolve_schema() + build_catalog() for planner.
7. *ire into gateway*— --mode real flag switches from InMemoryTables to RecordAdapter.
8. *ntegration tests*— full SQL → record → verify round-trip.

8. Open questions

1. *rimary key extraction*— the planner doesn't know about primary

   keys. For INSERT, the adapter needs to derive the KvCluster key from the row's PK columns. Should this be in the adapter or exposed as a planner concept?

2. *ransaction boundaries*— should each SQL statement be a single

   KvCluster transaction? Or should the adapter expose begincommitrollback for multi-statement transactions?

3. *ow limits*— scan() today returns all rows. For large tables,

   we need pushdown of LIMIT/WHERE to the substrate. This is a v0.2 optimization (predicate pushdown).