RFC-002 — Federation Protocol

*uthor:*Koder Engineering *ate:*20260415 *tatus:*Draft *odule:*productsdevhub

Table of Contents

1. Summary
2. Problem Statement
3. Goals
4. Non-Goals
5. Roles and Terminology
6. Design
   • 6.1 Federation Certificate
   • 6.2 DNS Discovery
   • 6.3 Metadata Sync — Listings API
   • 6.4 Binary Routing — Pull-Through Cache
   • 6.5 Trust Model — Mutual TLS
   • 6.6 Propagation Scope
   • 6.7 Developer Control
   • 6.8 Mirror Registration Flow
   • 6.9 MultiCluster Peerto-Peer Sync
   • 6.10 Rate Limiting and Anti-Abuse
   • 6.11 Certificate Revocation
7. API Reference
8. Security Considerations
9. Failure Modes and Mitigations
10. Observability
11. Roadmap
12. Open Questions
13. Alternatives Considered
14. References

1. Summary

Koder Hub (hub.koder.dev) operates as a centralized registry. This design limits enterprise adoption: organizations want to install apps from the public Koder catalog inside their private network, without exposing endpoints or allowing arbitrary outbound traffic from developer workstations. This RFC defines a *ederation Protocol*that allows authorized *irror*nodes to synchronize app listings and binaries from an *rigin*node, serve them locally, and (in peer mode) participate in a distributed store network.

2. Problem Statement

2.1 The Walled Garden Problem

Currently, to install any Koder app, a workstation must have direct connectivity to hub.koder.dev. For organizations with:

• *trict egress policies*— only whitelisted outbound domains allowed.
• *ir-gapped environments*— no internet access at all.
• *ompliance requirements*— all software must be sourced from an internal repository.

…Koder Hub is not usable today.

2.2 Bandwidth Amplification

In organizations with hundreds or thousands of developer workstations, every update causes each machine to independently download from hub.koder.dev. A 35 MB update for 500 workstations generates 17.5 GB of egress from the Origin, much of which is identical bytes.

A federated mirror would serve the download locally after the first pull, collapsing 500 identical requests to Origin into 1.

2.3 Multi-Team Publishing

Organizations running their own Koder environments (e.g., Crescer, Vivver) want to publish internal apps that are visible to their users but not to the general public. Today, this requires a separate isolated Store instance with no connection to the public catalog.

Federation allows an organization to run one mirror that serves:

• Internal private apps (hosted only on the mirror).
• Public Koder apps (pulled through from Origin).

3. Goals

• G1: Allow enterprises to run a local Mirror that serves the public Koder app catalog.
• G2: Reduce Origin egress by serving cached binaries locally.
• G3: Publishers retain full control over which apps can be federated.
• G4: End users on a Mirror get identical metadata and binaries to Origin (no tampering possible).
• G5: Mirror registration and operation require no custom code — only configuration.
• G6: Compatible with RFC-001 delta delivery — mirrors can serve cached delta artifacts.

4. Non-Goals

• Allowing mirrors to publish apps to Origin (publishing is Origin-only in Phase 1–2).
• Federated user authentication (auth is handled by Koder ID; this RFC covers only app distribution).
• Automatic discovery of mirrors by clients (clients are configured with their mirror URL explicitly).
• Offline-first operation where mirrors function without any Origin connectivity (Phase 4 consideration).

5. Roles and Terminology

6. Design

6.1 Federation Certificate

Every Mirror must hold a *ederation Certificate*issued by the Koder CA:

Subject: CN=org.example.com, O=Acme Corp, OU=KoderMirror
Issuer: CN=Koder Federation CA
Extensions:
  KoderFederationNode: mirror          # OID 1.3.6.1.4.1.99999.1.1
  KoderMirrorDomain: org.example.com   # OID 1.3.6.1.4.1.99999.1.2
  KoderMirrorID: <uuid>                # OID 1.3.6.1.4.1.99999.1.3

The certificate:

• Is valid for 1 year and must be renewed via the Mirror dashboard on hub.koder.dev.
• Is presented by the Mirror in every outbound request to Origin (mutual TLS).
• Is presented by Origin to the Mirror to prove it is talking to the real Origin.
• Revocation is handled by OCSP stapling; mirrors check OCSP at startup and cache the response for 24 hours.

6.2 DNS Discovery

Clients and other nodes discover a Mirror's federation endpoint via a DNS TXT record:

_koder-federation.org.example.com TXT "v=kf1 endpoint=https://store.org.example.com/api/v1/federation node=mirror id=<uuid>"

*ields:*

Clients configured with mirror=org.example.com resolve the TXT record to find the federation endpoint. If the TXT record is absent, the mirror is considered unreachable.

6.3 Metadata Sync — Listings API

The Mirror polls Origin periodically to stay up to date with new app listings and version updates.

*ndpoint (Origin serves, Mirror calls):*

GET /api/v1/federation/listings?since=<RFC3339_TIMESTAMP>
Authorization: (mTLS — certificate identity used, no bearer token)

*esponse:*

{
  "generated_at": "2026-04-15T12:00:00Z",
  "next_since": "2026-04-15T12:05:00Z",
  "listings": [
    {
      "slug": "koder-jet",
      "name": "Koder Jet",
      "publisher": "koder",
      "visibility": "public",
      "federation_allowed": true,
      "versions": [
        {
          "version": "1.4.2",
          "sha256": "a3f9e1...c2b4",
          "size_bytes": 34816000,
          "published_at": "2026-04-15T10:00:00Z",
          "platforms": ["linux-amd64", "linux-arm64", "darwin-arm64", "windows-amd64"],
          "signature": "<publisher_sig_base64url>"
        }
      ]
    }
  ],
  "yanked": [
    { "slug": "some-app", "version": "1.0.0", "reason": "security" }
  ]
}

*olling interval:*Mirror polls every 5 minutes. The since parameter is set to the next_since value from the previous response, enabling incremental sync.

*irror behavior on receiving listings:*

1. Upsert listing metadata into local database.
2. Process yanked entries: mark versions as unavailable, schedule binary GC.
3. Do *ot*immediately download binaries — binaries are pulled lazily on first client request (Section 6.4).

*rigin enforcement:*

• Listings with federation_allowed: false are excluded from the response entirely.
• Listings with visibility: private are excluded regardless of federation_allowed.

6.4 Binary Routing — Pull-Through Cache

Binaries are *ot*proactively pushed to mirrors. Instead, a Mirror pulls a binary from Origin on the first client request for that content hash.

*lient request to Mirror:*

GET /api/v1/apps/{slug}/download?version=X

*irror decision tree:*

1. Lookup listing for {slug}@{version} in local DB.
   → Not found: return 404.
   → Found but federation_allowed=false: return 403.

2. Check local object store for sha256 of version X.
   → Found: serve directly (cache hit).
   → Not found: pull from Origin.

3. Pull from Origin:
   GET https://hub.koder.dev/api/v1/apps/{slug}/download?version=X
   (mTLS with Federation Certificate)

4. Stream response to client while writing to local object store.
   → Compute SHA256 of received bytes.
   → If SHA256 != manifest sha256: abort stream, discard, return 502.
   → On success: mark binary as cached.

5. Subsequent requests for same sha256: served from local cache.

*elta artifact pull-through:*

If the client requests a delta (RFC-001):

GET /api/v1/apps/{slug}/download?version=X&base=SHA256_BASE
Accept: application/x-bsdiff

The Mirror checks its local delta artifact cache first. On miss, it pulls the delta from Origin using the same pull-through mechanism. Delta artifacts are identified by their content hash and cached independently.

*ache eviction:*

• Mirror operators configure maximum cache size (default: 50 GB).
• LRU eviction removes leastrecentlyserved binaries when the limit is reached.
• Yanked versions are evicted immediately.

6.5 Trust Model — Mutual TLS

All communication between federation nodes uses mutual TLS (mTLS):

Mirror → Origin:
  Mirror presents: Federation Certificate (from Koder CA)
  Origin presents: TLS cert for hub.koder.dev (standard CA)
  Origin validates: Mirror cert chain to Koder CA, KoderFederationNode=mirror extension

Origin → Mirror (webhooks, Phase 2):
  Origin presents: Federation Certificate (Origin's own cert, KoderFederationNode=origin)
  Mirror validates: Origin cert chain to Koder CA, KoderFederationNode=origin extension

*ertificate pinning:*Mirrors pin the Koder CA certificate fingerprint in their configuration (koder.toml). This prevents a compromised intermediate CA from issuing a rogue Koder CA impersonation.

[federation]
origin_url = "https://hub.koder.dev"
koder_ca_fingerprint = "sha256:<hex>"
cert_path = "/etc/koder-hub/federation.crt"
key_path = "/etc/koder-hub/federation.key"

6.6 Propagation Scope

Federation propagates only apps that satisfy *ll*of the following conditions:

Apps failing any condition are silently excluded from federation responses. The Mirror never learns that such apps exist.

*efault values for new apps:*

Publishers must explicitly opt in to federation for each app. This is a conservative default — no accidental leakage.

6.7 Developer Control

Publishers can control federation behavior per app via the manifest:

{
  "slug": "myapp",
  "visibility": "public",
  "federation_allowed": true,
  "federation_deny_list": ["org.competitor.com"],
  "federation_allow_list": []
}

*federation_allowed: false`*— App is never federated to any mirror.

*federationdenylist** — List of mirror domain names excluded from receiving this app. Origin filters these from federation responses based on the mirror's KoderMirrorDomain` certificate extension.

*federationallowlist`*— If non-empty, only mirrors in this list may receive the app. Useful for restricted enterprise distribution.

*I:*Publishers manage federation settings from the Store dashboard or via the koder-hub-cli publish --federation-allowed flag.

6.8 Mirror Registration Flow

1. *dmin registers mirror*at hub.koder.dev/federation/register:
   • Input: organization name, mirror domain, contact email.
   • Origin validates domain ownership via DNS TXT challenge.
   • Origin issues a Federation Certificate and returns it.

2. *irror operator installs certificate:*

   koder-hub config set federation.cert_path /etc/koder-hub/federation.crt
   koder-hub config set federation.key_path  /etc/koder-hub/federation.key
   koder-hub federation start

3. *irror begins polling*Origin's /federation/listings endpoint.

4. *lients configured*to use the mirror:

   # /etc/koder/store.toml
   [store]
   mirror = "https://store.org.example.com"
   fallback_to_origin = true   # if mirror doesn't have the binary, try Origin

6.9 MultiCluster Peerto-Peer Sync

In Phase 3, two Store nodes at equal authority can sync bidirectionally. Use case: two Koder deployments in different geographic regions, each with their own publisher community, sharing apps across both.

*ifferences from Mirror:*

*ync protocol (peers):*

Each peer maintains a vector clock of listing updates. On connect:

1. Exchange clocks: POST /api/v1/federation/sync { my_clock: {...} }.
2. Receive { missing_from_you: [...listings...] }.
3. Apply received listings.
4. Send own listings that the peer is missing.

Conflict resolution: lastwriterwins by published_at timestamp. Yanks always win over published state for the same (slug, version) pair.

6.10 Rate Limiting and Anti-Abuse

*n Origin's federation endpoint:*

/api/v1/federation/listings: 12 requests/hour per mirror (aligned with 5-min poll)
/api/v1/appsdownload (via federation): 1000 requests/hour per mirror

Limits are enforced by mirror identity (from the mTLS client certificate). Exceeding limits returns 429 Too Many Requests with Retry-After header.

*buse detection:*

• Mirror requesting unusually large listings bursts (> 10× normal rate) triggers an alert.
• Binary pull-through exceeding 500 GB/day triggers rate limiting and a notification to the mirror operator and Koder ops.

*irror blacklist:*

Origin maintains an internal blacklist of mirror IDs. Blacklisted mirrors receive 403 Forbidden on all federation endpoints. Certificate revocation (Section 6.11) is the formal mechanism; the blacklist serves as an emergency kill switch.

6.11 Certificate Revocation

When a Mirror is found to be abusive or its private key is compromised:

1. Koder ops marks the Mirror as revoked in the CA database.
2. OCSP responder immediately returns revoked for that certificate.
3. Mirror's current connections are reset.
4. Mirror's entries are removed from Origin's known-mirrors registry.
5. Clients configured with that mirror receive a 410 Gone response with a message directing them to reconfigure.

Mirrors check OCSP status at startup. If the OCSP response is revoked, the Mirror disables its federation client and logs a critical alert.

7. API Reference

7.1 Origin Federation Endpoints

7.2 Mirror Endpoints (exposed to peers and Origin)

7.3 Listings Delta Response Schema

GET /api/v1/federation/listings?since=2026-04-15T12:00:00Z

200 OK
Content-Type: application/json

{
  "generated_at":  string (RFC3339),
  "next_since":    string (RFC3339),
  "listings": [
    {
      "slug":                string,
      "name":                string,
      "publisher":           string,
      "visibility":          "public",
      "federation_allowed":  bool,
      "categories":          string[],
      "versions": [
        {
          "version":       string (semver),
          "sha256":        string (hex),
          "size_bytes":    integer,
          "published_at":  string (RFC3339),
          "platforms":     string[],
          "signature":     string (base64url),
          "changelog":     string (optional)
        }
      ]
    }
  ],
  "yanked": [
    {
      "slug":    string,
      "version": string,
      "reason":  string
    }
  ]
}

8. Security Considerations

8.1 Binary Integrity at Mirrors

Mirrors are a potential vector for serving tampered binaries. Mitigations:

• All binary SHA256 hashes are included in the manifest, which is signed by the publisher using their private key.
• Clients always verify sha256(downloaded_bytes) == manifest.sha256 before installation.
• The publisher signature is verified against the publisher's public key, which is fetched from Origin (not the Mirror). A compromised Mirror cannot forge a valid publisher signature.

8.2 Mirror Impersonation

An attacker could stand up a fake store claiming to be a legitimate mirror. Mitigations:

• Clients configure their mirror URL explicitly; they do not auto-discover mirrors.
• mTLS client certificate validates the Mirror's identity against Koder CA.
• DNS TXT record must match the certificate's KoderMirrorDomain extension.

8.3 Metadata Tampering by Mirror

A Mirror could theoretically alter listing metadata before serving it to clients (e.g., removing a security-relevant changelog entry). Mitigations:

• Critical metadata fields (SHA256, version string, publisher) are included in the publisher-signed manifest.
• Clients that verify the manifest signature detect any modification.
• Display-only fields (name, description, screenshots) are not signed; trust is delegated to the mirror for these.

8.4 Private App Leakage

Private apps (visibility: private) must never appear in federation responses. Mitigations:

• Origin's federation endpoint applies a server-side filter; private apps are excluded at query time, not filtered by the client.
• Integration test suite includes a "private app not in federation" assertion run on every CI build.

8.5 Replay Attacks on Listings

An attacker with access to a past federation response could replay old listings to roll back apps to vulnerable versions. Mitigations:

• Mirrors reject any listing update where published_at is older than their current record for the same (slug, version).
• Yanked versions, once applied, cannot be un-yanked by a replayed old listing.

9. Failure Modes and Mitigations

10. Observability

*etrics (Prometheus):*

store_federation_listings_sync_duration_seconds{mirror_id, status}
store_federation_listings_synced_total{mirror_id}
store_federation_binary_pull_duration_seconds{mirror_id, app, status}
store_federation_cache_hit_ratio{mirror_id}
store_federation_cache_size_bytes{mirror_id}
store_federation_mirrors_active_total
store_federation_cert_expiry_seconds{mirror_id}

*og events:*

• federation.mirror.registered — new mirror registered.
• federation.sync.completed — listings sync cycle finished, N listings updated.
• federation.sync.failed — sync error with details.
• federation.binary.pull — binary pulled from Origin, cached.
• federation.binary.cache_hit — binary served from local cache.
• federation.cert.expiry_warning — cert expires in < 14 days.
• federation.cert.revoked — cert revoked, federation client disabled.

*ashboard:*Koder ops dashboard shows per-mirror sync lag, cache hit rate, egress saved, and certificate status.

11. Roadmap

Phase 1 — Metadata Sync API (Weeks 1–4)

• GET /api/v1/federation/listings?since= endpoint on Origin.
• Mirror polling worker (5-minute interval, incremental sync).
• Local listing database on Mirror.
• Federation Certificate issuance via Mirror registration flow.
• mTLS setup on Origin and Mirror.
• Integration tests: listings sync, yank propagation, private app exclusion.

Phase 2 — Binary Pull-Through (Weeks 5–8)

• Mirror binary cache (object store on mirror host).
• Pull-through proxy on Mirror download endpoint.
• Delta artifact pullthrough (RFC001 compatibility).
• Cache eviction policy (LRU, yank-triggered).
• fallback_to_origin client configuration.
• Endtoend test: client on isolated network installs app via mirror.

Phase 3 — Bidirectional Peer Clusters (Weeks 9–16)

• Peer role and certificate extension.
• Vector clock sync protocol.
• Conflict resolution for concurrent publishes.
• DNS TXT node=peer discovery.
• Multiregion deployment test: two Originequivalent nodes in different DCs.

Phase 4 — Third-Party Mirrors

• Self-service mirror registration (no Koder ops intervention).
• Public mirror registry at hub.koder.dev/mirrors.
• Mirror health monitoring by Origin.
• Community mirror policy (SLAs, abuse reporting).
• Documentation for mirror operators.

12. Open Questions

1. *ush vs poll for listings sync:*The current design uses pollbased sync. A webhook/push model (Origin notifies mirrors on new publish) would reduce sync lag from up to 5 minutes to nearzero. Tradeoff: Origin must maintain a list of all mirror endpoints and retry failed deliveries. Start with poll; add push as an optimization in Phase 2.

2. *irrorlocal publishing:*Should a Mirror be able to host apps that are only available locally (not synced from Origin)? This is needed for internal enterprise apps. Phase 2 will allow mirrors to have a localonly app registry that is separate from the synced Origin listings. This RFC covers the sync protocol only.

3. *inary prewarming:*Should mirrors optionally prepull all binaries for synced apps (instead of lazy pullthrough)? Useful for airgapped environments. Opt-in configuration flag federation.prewarm = true planned for Phase 2.

4. *ultiOrigin:*What if an enterprise wants to federate from multiple Origins (Koder public + their own internal Origin)? Phase 3 peer protocol partially addresses this. Explicit multiorigin config is deferred to Phase 4.

5. *isting pagination:*For large catalogs, the ?since= endpoint may return large payloads. Add cursor-based pagination (?cursor=<opaque>) in Phase 1 if the Koder catalog exceeds 1000 apps, otherwise defer.

13. Alternatives Considered

13.1 Full OCI/Container Registry Protocol

The OCI Distribution Spec (used by Docker Hub, GitHub Container Registry) provides a wellestablished protocol for contentaddressed binary distribution with pull-through mirroring. However:

• It is designed for container images, not arbitrary app packages. Wrapping .kpkg in OCI layers adds unnecessary complexity.
• OCI does not have a native concept of app metadata (descriptions, screenshots, publisher identity).
• Implementing a full OCI-compatible registry requires significant infrastructure divergence from our existing Store backend.

A purpose-built protocol (this RFC) gives us precise control over the data model and trust hierarchy.

13.2 S3/Object Storage Direct Access

Give mirror operators read-only credentials to the Origin object store (S3 or compatible). Mirrors sync binaries directly from the bucket.

• Pros: No custom protocol; leverages mature S3 replication.
• Cons: Exposes all binaries including those not federation_allowed. Cannot enforce per-app federation rules. No metadata layer. Couples mirror operators to Origin's storage backend.

Rejected for security and architecture reasons.

13.3 CDN-Based Distribution

Use a global CDN (Cloudflare, Fastly) instead of operator-run mirrors.

• Pros: Zero operator burden; global edge presence.
• Cons: Does not solve the airgapped / egresspolicy enterprise use case. CDN nodes are not trusted by enterprise security policies. Cannot host internal-only apps.

CDN is complementary to federation, not a replacement. Phase 3 edge nodes may use CDN for public apps; enterprise mirrors are still needed for private/internal scenarios.

13.4 APT/YUM Repository Protocol

Standard Linux package manager protocols are familiar to ops teams and have wide tooling support.

• Pros: Existing mirror infrastructure (apt-mirror, reposync).
• Cons: Designed for OS packages, not crossplatform app bundles. No concept of publisher identity or perapp federation control. Diverges from the .kpkg format (spec kpkg/format.kmd).

14. References

• RFC001 — Delta Delivery (productsdevhubdocsrfcs/RFC001deltadelivery.md)
• specs/kpkg/format.kmd — Koder Package format specification
• OCI Distribution Specification — https:/ithub.comopencontainersdistribution-spec
• SWIM protocol — Das et al., "SWIM: Scalable Weaklyconsistent Infectionstyle Process Group Membership Protocol," DSN 2002
• Mutual TLS (RFC 8446, TLS 1.3, Section 4.3.2)
• OCSP Stapling (RFC 6961)
• Ticket #015 — Federation Protocol Implementation (productsdevhubbacklogpending/015federationprotocol.md)