Protocol First

mandatory

Sub-policy de `reuse-first` para wire formats e protocol clients cross-cutting (koder_ipc, koder_chat, koder_store, observability envelopes, kpkg, federation protocol). Herda decision tree, protocolo pré-Write e promotion pipeline da meta; acrescenta apenas as regras categoricamente distintas: RFC ratificado antes do mainline, version negotiation no envelope, BC por N versões, schema evolution (add OK; rename/remove = major), cross-implementation tests.

Policy — Protocol First

Subpolicy of [`reusefirst`](../reuse-first.kmd). Inherits the decision tree, the pre-Write protocol and the promotion pipeline. Do not redeclare them here. This file contributes *nly*the rules that are categorically distinct for wire formats and the protocol clients that speak them.

Scope

Applies to any code in:

  • `enginessdkkoder_ipc

proto/* — observability envelopes (logs / traces / metrics).

  • products/dev/hub{proto,kpkg}/** — Hub federation protocol and .kpkg package format.
  • servicesengine/proto/** — servicetier RPC contracts. Koder services default to protogRPC over RESTJSON; the canonical pattern is services/foundation/id/engine/proto/koder/id/{oauth,identity,session,auth,admin,keys,invites,saml}/v1/ — one `<subservice>.proto defining the request/response messages plus an optional sub-servicestorage.proto for persistence-layer types, with option gopackage = "koder.devservicepkgprotosub-service/v1;sub-servicev1"`.
  • **/wire*.proto — any package that defines a cross-process schema.

A protocol client is in scope on *wo axes simultaneously* this subpolicy (wire format) and `runtimelib-first.kmd` (lib mechanics). Where the two contracts apply, both must be satisfied.

Out of scope: inprocess function calls, intramodule data structures, internal serialisation that never crosses a process boundary.

Categorical rules

1. Ratified RFC before mainline

A wire format or protocol does not ship from a single PR. It lands as RFC first.

  • New wire format → new RFC under meta/docs/stack/rfcs/<area>-RFC-NNN-<slug>.md with status Accepted before any code under lib/wire/ is merged.
  • Significant evolution of an existing wire format (new RPC, new message, new envelope field with semantic impact) → either an RFC amendment or a new RFC depending on scope. Flag-level changes to existing messages do not need an RFC; field additions to existing messages might (see §3 schema evolution).
  • The RFC carries the design rationale, the wirelevel encoding, the versionnegotiation strategy, the BC commitment, and the test matrix.

2. Version negotiation in the envelope

Every protocol envelope MUST carry an explicit version. Patterns by transport:

  • *RPC / protobuf* a version field in the request envelope, AND service/method names that include the major version (HubV1.GetPackage vs HubV2.GetPackage). This is the protobufcanonical way and survives reverseproxy renaming.
  • *TTP/JSON* Accept-Version: <semver-range> request header; server responds with X-Version: <chosen>.
  • *ustom binary* first 2 bytes of every frame encode (major, minor).
  • *pkg* the kpkg.toml declares format-version = "1.0".

Server and client announce their supported version range and pick the highest mutually supported. A version mismatch produces a structured error with the supported ranges, not a generic 400/500.

3. Schema evolution rules

For every message / record / table:

Change Required version bump
Add an optional field none (forward-compatible)
Add a required field major (consumers without it break)
Rename a field major
Remove a field major (after deprecation cycle in §4)
Change a field's type major
Change a field's semantics (without renaming) major; *nti-pattern* prefer rename
Change default value minor (document in CHANGELOG; observable behaviour change)
Add a new RPC / endpoint minor
Remove an RPC / endpoint major (after deprecation cycle in §4)

The rule of thumb: *dding optionality is non-breaking; subtracting or restricting is breaking*

4. Backwards compatibility for N versions

The wire-format contract: *lient at version N MUST interoperate with server at version N − k*for k = 0..2 (default N=2 for production protocols). A protocol with stricter or looser BC declares the value of N in its RFC.

Operationally:

  • A breaking change at version N lands behind a flag for at least one minor cycle before becoming default.
  • The server answers an old client with a structured protocol-version-too-old error including the minimum supported version, never with a generic failure.
  • Deprecated symbols / fields stay in the schema for the BC window; they may be marked deprecated = true but remain serialised and accepted.

5. Cross-implementation tests

Where multiple implementations of the same protocol exist (typical for Koder protocols: Go server + JS browser client + Python script + Rust embedded client), a cross-implementation test suite is mandatory:

  • *oundtrip* each pair (implA → impl-B) exchanges a representative message set; both sides decode without error.
  • *ompatibility matrix* client at v(N2) ↔ server at vN; client at vN ↔ server at v(N2). Both work for the BC window.
  • *uzzing* feed each implementation a corpus of malformed and adversarial inputs; expected output is a structured error, never a panic.

The test suite lives in a dedicated repo path (e.g. engines/sdk/koder_ipc/tests/cross-impl/) and runs in CI on changes to any implementation.

6. Wireformat antipatterns

The following are forbidden; reviewers reject PRs that introduce them:

  • *ntyped envelopes*(e.g. Map<String, Any> as the top-level message). Always typed.
  • *tring-encoded enums*without a registered enum type. Use protobuf enums, TS literal unions, Rust enum.
  • *erver-side boolean flags that change wire shape*depending on user role. Roles authorise; they don't reshape the wire.
  • *nband schema*(sending the schema with each message). Schema is outof-band, versioned, and discoverable.
  • *ptionaltyped fields with implicit defaults that vary by language*(e.g. JS undefined vs Go zerovalue vs Rust Option::None interpreted differently). Defaults are explicit per the schema.

7. Self-describing artefacts

Every wireformat release ships with a machinereadable schema artefact:

  • protobuf .proto files in a versioned directory.
  • JSON Schema for HTTP/JSON protocols.
  • Rust / Go / TS / Python bindings autogenerated from the schema (see `buildtoolingfirst.kmd` §4 selfstamping for the binding-generation contract).

A protocol client's runtime version metadata includes the schema version it was generated against.

8. Promotion criterion (overrides the meta default)

For wire formats, the meta's "≥ 3 consumers" rule changes to:

  • * protocol becomes shared (lifted to engines/sdk/) on its first cross-process consumer.*Reason: a wire format defined inside one product becomes a de facto contract the moment a second process speaks it; the cost of moving it later is much higher than UI widget migration.
  • The accompanying RFC (per §1 above) is the formalisation step.

Audit

./protocol-first-audit.sh (initial in Phase 2; substantive in Phase 4):

  1. *tructural (Phase 2):*validate inherits_from; verify referenced specssibling exist; verify each `enginessdkprotocol has a proto or wire` directory and an RFC link in its README.
  2. *ehavioural (Phase 4):*
    • Walk all .proto files; flag missing version fields in top-level request/response messages.
    • Walk all lib/wire/ directories; flag any Map<String, Any> / Object / interface{} envelope.
    • Parse RFCs; assert every RFC labelled protocol has a § "BC commitment" with an explicit value of N.
    • Run the cross-implementation test suite if present; CI gate fails the build if absent.

Cross-references

Source: ../home/koder/dev/koder/meta/docs/stack/policies/reuse/protocol-first.kmd