Iron Curtain — Design Documentation

P2P Distribution (BitTorrent/WebTorrent)

Wire protocol specification: For the byte-level BT wire protocol, piece picker algorithm, choking strategy, authenticated announce, WebRTC signaling, icpkg binary header, and DHT design, see research/p2p-engine-protocol-design.md.

P2P piece mapping: The complete BitTorrent piece mapping for .icpkg packages — piece size, chunking, manifest-only fetch, CAS/BT interaction — is specified in research/p2p-engine-protocol-design.md § 10.

The cost problem: A popular 500MB mod downloaded 10,000 times generates 5TB of egress. At CDN rates ($0.01–0.09/GB), that’s $50–450/month — per mod. For a community project sustained by donations, centralized hosting is financially unsustainable at scale. A BitTorrent tracker VPS costs $5–20/month regardless of popularity.

The solution: Workshop distribution uses the BitTorrent protocol for large packages, with HTTP as both a concurrent transport (via BEP 17/19 web seeding) and a last-resort fallback. When web seed URLs are present in torrent metadata, HTTP mirrors participate simultaneously alongside BT peers in the piece scheduler — downloads aggregate bandwidth from both transports. The Workshop server acts as both metadata registry (SQLite, lightweight) and BitTorrent tracker (peer coordination, lightweight). See D049-web-seeding.md for the full web seeding design.

How it works:

┌─────────────┐     1. Search/browse     ┌──────────────────┐
│  ic CLI /    │ ───────────────────────► │  Workshop Server │
│  In-Game     │ ◄─────────────────────── │  (metadata +     │
│  Browser     │  2. manifest.yaml +      │   tracker)       │
│              │     torrent info         │                  │
│              │                          └──────────────────┘
│              │     3. P2P download
│              │ ◄──────────────────────► Other players (peers/seeds)
│              │     (BitTorrent protocol)
│              │
│              │     4. Web seeding (BEP 17/19 concurrent HTTP) + fallback
│              │ ◄─────────────────────── Workshop server / mirrors / seed box
└─────────────┘     5. Verify SHA-256

1. Publish: ic mod publish uploads .icpkg to Workshop server. Server computes SHA-256, generates torrent metadata (info hash), starts seeding the package alongside any initial seed infrastructure.
2. Browse/Search: Workshop server handles all metadata queries (search, dependency resolution, ratings) via the existing SQLite + FTS5 design. Lightweight.
3. Install: ic mod install fetches the manifest from the server, then downloads the .icpkg via BitTorrent + HTTP concurrently (when web seed URLs are present in torrent metadata). If no BT peers are available and no web seeds exist, falls back to HTTP direct download as a last resort.
4. Seed: Players who have downloaded a package automatically seed it to others (opt-out in settings). The more popular a resource, the faster it downloads — the opposite of CDN economics where popularity means higher cost.
5. Verify: SHA-256 checksum validation on the complete package, regardless of download method. BitTorrent’s built-in piece-level hashing provides additional integrity during transfer.

WebTorrent for browser builds (WASM): Standard BitTorrent uses TCP/UDP, which browsers can’t access. WebTorrent extends the BitTorrent protocol over WebRTC, enabling browser-to-browser P2P. The Workshop server includes a WebTorrent tracker endpoint. Desktop clients and browser clients can interoperate — desktop seeds serve browser peers and vice versa through hybrid WebSocket/WebRTC bridges. HTTP fallback is mandatory: if WebTorrent signaling fails (signaling server down, WebRTC blocked), the client must fall back to direct HTTP download without user intervention. Multiple signaling servers are maintained for redundancy. Signaling servers only facilitate WebRTC negotiation — they never see package content, so even a compromised signaling server cannot serve tampered data (SHA-256 verification catches that).

Tracker authentication & token rotation: P2P tracker access uses per-session tokens tied to client authentication (Workshop credentials or anonymous session token), not static URL secrets. Tokens rotate every release cycle. Even unauthorized peers joining a swarm cannot serve corrupt data (SHA-256 + piece hashing), but token rotation limits unauthorized swarm observation and bandwidth waste. See 06-SECURITY.md for the broader security model.

Transport strategy by package size:

Thresholds are configurable in settings.toml. Players on connections where BitTorrent is throttled or blocked can force HTTP-only mode.

D069 setup/maintenance wizard transport policy: The installation/setup wizard (D069) and its maintenance flows reuse the same transport stack with stricter UX-oriented defaults:

• Initial setup downloads use user-requested priority (not background) and surface source indicators (P2P / HTTP) in progress UI.
• Small setup assets/config packages (including player-config profiles, small language packs, and tiny metadata-driven fixes) should default to HTTP direct per the size strategy above to avoid P2P startup overhead.
• Large optional media packs (cutscenes, HD assets) use BT + HTTP concurrent download (when web seed URLs exist in torrent metadata), with HTTP-only as last resort. The wizard must explain this transparently (“faster from peers when available”).
• Offline-first behavior: if no network is available, the setup wizard completes local-only steps and defers downloadable packs instead of failing the entire flow.

D069 repair/verify mapping: The maintenance wizard’s Repair & Verify actions map directly to D049 primitives:

• Verify installed packages → re-check .icpkg/blob hashes against manifests and registry metadata
• Repair package content → re-fetch missing/corrupt blobs/packages (HTTP or P2P based on size/policy)
• Rebuild indexes/metadata → rebuild local package/cache indexes from installed manifests + blob store
• Reclaim space → run GC over unreferenced blobs/package references (same CAS cleanup model)

Repair/verify is an IC-side content/setup operation. Store-platform binary verification (Steam/GOG) remains a separate platform responsibility and is only linked/guided from the wizard.

Auto-download on lobby join (D030 interaction): When joining a lobby with missing resources, the client downloads via BT + HTTP concurrently (lobby peers are high-priority BT sources since they already have the content). If web seeds exist, HTTP mirrors contribute bandwidth immediately alongside lobby peers. If no BT peers or web seeds are available, the client uses HTTP direct download as a last resort. The lobby UI shows download progress with source indicators (P2P/HTTP). See D052 § “In-Lobby P2P Resource Sharing” for the detailed lobby protocol, including host-as-tracker, verification against Workshop index, and security constraints.

Gaming industry precedent:

• Blizzard (WoW, StarCraft 2, Diablo 3): Used a custom P2P downloader (“Blizzard Downloader”, later integrated into Battle.net) for game patches and updates from 2004–2016. Saved millions in CDN costs for multi-GB patches distributed to millions of players.
• Wargaming (World of Tanks): Used P2P distribution for game updates.
• Linux distributions: Ubuntu, Fedora, Arch all offer torrent downloads for ISOs — the standard solution for distributing large files from community infrastructure.
• Steam Workshop: Steam subsidizes centralized hosting from game sales revenue. We don’t have that luxury — P2P is the community-sustainable alternative.

Competitive landscape — game mod platforms:

IC’s Workshop exists in a space with several established modding platforms. None offer the combination of P2P distribution, federation, self-hosting, and in-engine integration that IC targets.

Competitive landscape — P2P + Registry infrastructure:

The game mod platforms above are all centralized. A separate set of projects tackle P2P distribution at the infrastructure level, but none target game modding specifically. See research/p2p-federated-registry-analysis.md for a comprehensive standalone analysis of this space and its applicability beyond IC.

What’s novel about IC’s combination: No existing system — modding platform or infrastructure — combines (1) federated registry with repository types, (2) P2P distribution via BitTorrent/WebTorrent, (3) zero-infrastructure git-hosted bootstrap, (4) browser-compatible P2P via WebTorrent, (5) in-engine integration with lobby auto-download, and (6) fully open-source with self-hosting as a first-class use case. The closest architectural comparison is mod.io (embeddable SDK approach, in-game integration) but mod.io is a proprietary centralized SaaS — no P2P, no federation, no self-hosting. The closest distribution comparison is Uber Kraken (P2P registry) but it has no modding features. Each piece has strong precedent; the combination is new. The Workshop architecture is game-agnostic and could serve as a standalone platform — see the research analysis for exploration of this possibility.

Seeding infrastructure:

The Workshop doesn’t rely solely on player altruism for seeding:

• Workshop seed server: A dedicated seed box (modest: a VPS with good upload bandwidth) that permanently seeds all Workshop content. This ensures new/unpopular packages are always downloadable even with zero player peers. Cost: ~$20-50/month for a VPS with 1TB+ storage and unmetered bandwidth.
• Community seed volunteers: Players who opt in to extended seeding (beyond just while the game is running). Similar to how Linux mirror operators volunteer bandwidth. Could be incentivized with Workshop badges/reputation (D036/D037).
• Mirror servers (federation): Community-hosted Workshop servers (D030 federation) also seed the content they host. Regional community servers naturally become regional seeds.
• Lobby-optimized seeding: When a lobby host has required mods, the game client prioritizes seeding to joining players who are downloading. The “auto-download on lobby join” flow aggregates bandwidth from lobby peers (highest priority) + wider swarm + HTTP web seeds concurrently, with HTTP-only as last resort.

Privacy and security:

• IP visibility: Standard BitTorrent exposes peer IP addresses. This is the same exposure as any multiplayer game (players already see each other’s IPs or relay IPs). For privacy-sensitive users, HTTP-only mode avoids P2P IP exposure.
• Content integrity: SHA-256 verification on complete packages catches any tampering. BitTorrent’s piece-level hashing catches corruption during transfer. Double-verified.
• No metadata leakage: The tracker only knows which peers have which packages (by info hash). It doesn’t inspect content. Package contents are just game assets — sprites, audio, maps.
• ISP throttling mitigation: BitTorrent traffic can be throttled by ISPs. Mitigations: protocol encryption (standard in modern BT clients), WebSocket transport (looks like web traffic), and HTTP fallback as ultimate escape. Settings allow forcing HTTP-only mode.
• Resource exhaustion: Rate-limited seeding (configurable upload cap in settings). Players control how much bandwidth they donate. Default: 1MB/s upload, adjustable to 0 (leech-only, no seeding — discouraged but available).

P2P protocol design details:

The Workshop’s P2P engine is informed by production experience from Uber Kraken (Apache 2.0, 6.6k★) and Dragonfly (Apache 2.0, CNCF Graduated). Kraken distributes 1M+ container images/day across 15K+ hosts using a custom BitTorrent-inspired protocol; Dragonfly uses centralized evaluator-based scheduling at Alibaba scale. IC adapts Kraken’s connection management and Dragonfly’s scoring insights for internet-scale game mod distribution. See research/p2p-federated-registry-analysis.md for full architectural analyses of both systems.

Cross-pollination with IC netcode and community infrastructure. The Workshop P2P engine and IC’s netcode infrastructure (relay server, tracking server — 03-NETCODE.md) share deep structural parallels: federation, heartbeat/TTL, rate control, connection state machines, observability, deployment model. Patterns flow both directions — netcode’s three-layer rate control and token-based liveness improve Workshop; Workshop’s EWMA scoring and multi-dimensional peer evaluation improve relay server quality tracking. A full cross-pollination analysis (including shared infrastructure opportunities: unified server binary, federation library, auth/identity layer) is in research/p2p-federated-registry-analysis.md § “Netcode ↔ Workshop Cross-Pollination.” Additional cross-pollination with D052/D053 (community servers, player profiles, trust-based filtering) is catalogued in D052 § “Cross-Pollination” — highlights include: two-key architecture for index signing and publisher identity, trust-based source filtering, server-side validation as a shared invariant, and trust-verified peer selection scoring.

Peer selection policy (tracker-side): The tracker returns a sorted peer list on each announce response. The sorting policy is pluggable — inspired by Kraken’s assignmentPolicy interface pattern. IC’s default policy prioritizes:

1. Seeders (completed packages — highest priority, like Kraken’s completeness policy)
2. Lobby peers (peers in the same multiplayer lobby — guaranteed to have the content, lowest latency)
3. Geographically close peers (same region/ASN — reduces cross-continent transfers)
4. High-completion peers (more pieces available — better utilization of each connection)
5. Random (fallback for ties — prevents herding)

Peer handout limit: 30 peers per announce response (Kraken uses 50, but IC has fewer total peers per package). Community-hosted trackers can implement custom policies via the server config.

Planned evolution — weighted multi-dimensional scoring (Phase 5+): Dragonfly’s evaluator demonstrates that combining capacity, locality, and node type into a weighted score produces better peer selection than linear priority tiers. IC’s Phase 5+ peer selection evolves to a weighted scoring model informed by Dragonfly’s approach:

PeerScore = Capacity(0.4) + Locality(0.3) + SeedStatus(0.2) + LobbyContext(0.1)

• Capacity (weight 0.4): Spare bandwidth reported in announce (1 - upload_bw_used / upload_bw_max). Peers with more headroom score higher. Inspired by Dragonfly’s LoadQuality metric (which sub-decomposes into peak bandwidth, sustained load, and concurrency). IC uses a single utilization ratio — simpler, captures the same core insight.
• Locality (weight 0.3): Hierarchical location matching. Clients self-report location as continent|country|region|city (4-level, pipe-delimited — adapted from Dragonfly’s 5-level country|province|city|zone|cluster). Score = matched_prefix_elements / 4. Two peers in the same city score 0.75; same country but different region: 0.5; same continent: 0.25.
• SeedStatus (weight 0.2): Seed box = 1.0, completed seeder = 0.7, uploading leecher = 0.3. Inspired by Dragonfly’s HostType score (seed peers = 1.0, normal = 0.5).
• LobbyContext (weight 0.1): Same lobby = 1.0, same game session = 0.5, no context = 0. IC-specific — Dragonfly has no equivalent (no lobby concept).

The initial 5-tier priority system (above) ships first and is adequate for community scale. Weighted scoring is additive — the same pluggable policy interface supports both approaches. Community servers can configure their own weights or contribute custom scoring policies.

Bucket-based scheduling (Phase 5+): The weighted scoring formula above is applied within pre-sorted bucket leaves, not flat across all peers. The embedded tracker organizes its peer table into a PeerBucketTree indexed by RegionKey × SeedStatus × TransportType. On each announce response, the tracker walks buckets from closest-matching outward, applying weighted scoring within each leaf to produce the final peer list. This reduces per-announce work from O(n) to O(k) where k is the bucket size, and naturally produces locality-optimized peer sets without requiring the scoring formula to carry the full locality signal. Connection pool bucketing (per-transport guaranteed slot minimums) prevents cross-transport starvation when TCP, uTP, and WebRTC peers coexist. Content popularity bucketing (Hot/Warm/Cold/Frozen tiers via EWMA) steers seed box bandwidth toward under-served swarms. Full design: research/p2p-distribute-crate-design.md § 2.8.

Piece request strategy (client-side): The engine uses rarest-first piece selection by default — a priority queue sorted by fewest peers having each piece. This is standard BitTorrent behavior, well-validated for internet conditions. Kraken also implements this as rarestFirstPolicy.

• Pipeline limit: 3 concurrent piece requests per peer (matches Kraken’s default). Prevents overwhelming slow peers.
• Piece request timeout: 8s base + 6s per MB of piece size (more generous than Kraken’s 4s+4s/MB, compensating for residential internet variance).
• Endgame mode: When remaining pieces ≤ 5, the engine sends duplicate piece requests to multiple peers. This prevents the “last piece stall” — a well-known BitTorrent problem where the final piece’s sole holder is slow. Kraken implements this as EndgameThreshold — it’s essential.

Connection state machine (client-side):

pending ──connect──► active ──timeout/error──► blacklisted
   ▲                    │                          │
   │                    │                          │
   └──────────── cooldown (5min) ◄─────────────────┘

• MaxConnectionsPerPackage: 8 (lower than Kraken’s 10 — residential connections have less bandwidth to share)
• Blacklisting: peers that produce zero useful throughput over 30 seconds are temporarily blacklisted (5-minute cooldown). Catches both dead peers and ISP-throttled connections.
• Sybil resistance: Maximum 3 peers per /24 subnet in a single swarm. Prefer peers from diverse autonomous systems (ASNs) when possible. Sybil attacks can waste bandwidth but cannot serve corrupt data (SHA-256 integrity), so the risk ceiling is low.
• Statistical degradation detection (Phase 5+): Inspired by Dragonfly’s IsBadParent algorithm — track per-peer piece transfer times. Peers whose last transfer exceeds max(3 × mean, 2 × p95) of observed transfer times are demoted in scoring (not hard-blacklisted — they may recover). For sparse data (< 50 samples per peer), fall back to the simpler “20× mean” ratio check. Hard blacklist remains only for zero-throughput (complete failure). This catches degrading peers before they fail completely.
• Connections have TTL — idle connections are closed after 60 seconds to free resources.

Announce cycle (client → tracker): Clients announce to the tracker every 30 seconds (Kraken uses 3s for datacenter — far too aggressive for internet). The tracker can dynamically adjust: faster intervals (10s) during active downloads, slower (60s) when seeding idle content. Max interval cap (120s) prevents unbounded growth. Announce payload includes: PeerID, package info hash, bitfield (what pieces the client has), upload/download speed.

Size-based piece length: Different package sizes use different piece lengths to balance metadata overhead against download granularity (inspired by Kraken’s PieceLengths config):

| Package Size | Piece Length | Rationale |