Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Coverage Mapping: Design Features → Tests

Design FeaturePrimary Test TierVerification Method
Deterministic sim (Invariant #1)T1 + T2 + T3Hash comparison across runs/platforms
Pluggable network model (Invariant #2)T2Integration tests with mock network
Tiered modding (Invariant #3)T1 + T3Sandbox smoke + escape vector suite
Fog-authoritative serverT2 + T3Anti-cheat detection + desync injection
Ed25519 session authT2Protocol handshake verification (challenge-response, nonce freshness, session establishment)
Workshop package integrityT2 + T4Sign/verify chain + ecosystem lifecycle
RTL/BiDi text handlingT3QA corpus regression suite
Display name validation (V46)T3UTS #39 confusable corpus
Key rotation (V47)T2 + T4T2: rotation sequence monotonicity, cooldown enforcement, emergency recovery (at M5). T4: full lifecycle exercise (at M9)
Anti-cheat behavioral detectionT3 + T4Labeled replay corpus evaluation
Desync classification (V55)T4Injected bug vs cheat classification
Performance budgetsT3criterion benchmarks with regression gates
Save/load integrityT3Round-trip hash comparison
Path security (strict-path)T1 + T3Unit tests + fuzz testing
WASM inter-module isolation (V50)T3 (planned)Basic cross-module probe attempts → all blocked (M6); expanded cross-module integration coverage including shared-host-resource leaks and memory growth fuzzing (M7–M8) — see api-misuse-patterns.md gap table for the M7 integration scenario
P2P replay attestation (V53)T2 + T4T2: basic signed-hash exchange smoke test (at M4). T4: full multi-peer verification exercise (at M9)
Campaign completionT4Automated playthrough
Visual UI consistencyT4Pixel-diff regression
Sub-tick ordering fairness (D008)T2 + T3Simultaneous-order scenarios; timestamp tiebreak verification
Order validation completeness (D012)T1 + T3Exhaustive order-type × rejection-category matrix; proptest
Merkle tree desync localizationT2 + T3Inject divergence → verify O(log N) leaf identification
Snapshot reconnection (D007)T2 + T4Disconnect/reconnect/hash-match; corruption/stale rejection
Workshop dependency resolution (D030)T1 + T3Transitive, diamond, circular, and conflict dependency graphs
Campaign DAG validation (D021)T1 + T3Cycle/reachability/dangling-ref rejection at construction
Campaign roster carryover (D021)T2 + T4Surviving units + veterancy persist across mission transitions
Mod profile fingerprint stability (D062)T2 + T3Serialize/deserialize/recompute identity; ordering independence
WASM memory growth defense (V50)T3Adversarial memory.grow → denied; host stable
WASM float rejection in simT3Module attempts float write to sim → rejected
Pathfinding LOD + multi-layer (D013)T2 + T3Path correctness across LOD transitions; benchmark vs budget
Balance preset inheritance (D019)T1 + T2 + T3Chain resolution, cycle rejection, multiplayer hash match
Weather determinism (D022)T2 + T3Schedule sync + surface state match across instances
AI behavior determinism (D041)T2 + T3Same seed → identical build order; cross-platform hash match
Command permission enforcement (D058)T1 + T2Privileged command rejection; cvar bounds clamping
Rate limiting (D007/V17)T2 + T3Exceed OrderBudget → excess dropped; budget recovery timing
LLM content validation (D016)T3 + T4Objective reachability; trigger syntax; unit-type existence
Relay time-authority (D007)T2 + T3Timestamp envelope clamping; listen-server parity
SCR sequence enforcement (D052)T2 + T4Monotonic sequence; key rotation grace period; emergency revocation
Cross-engine map exchange (D011)T2 + T3OpenRA .oramap round-trip; out-of-bounds rejection
Conflict resolution ordering (D062)T2 + T3Explicit priority determinism; all clients agree on resolved values
Chat scope enforcementT1 + T2Team message routed only to team; all-chat routed to all; scope conversion requires explicit call
Theme loading + switching (D032)T2 + T4Theme YAML schema validation; mid-gameplay switch produces no visual corruption; missing asset fallback
AI personality application (D043)T2 + T3PersonalityId resolves to valid preset; undefined personality rejected; AI behavior matches declared profile

Release Criteria

A release candidate is shippable when:

  1. All Tier 1–3 tests pass on the release branch
  2. Latest Tier 4 run has no blockers (within the past 7 days)
  3. Performance benchmarks show no regressions vs the previous release
  4. Fuzz testing has run ≥1M iterations per target with no new crashes
  5. Anti-cheat false-positive rate meets V54 thresholds on the labeled corpus
  6. Cross-platform determinism verified (Linux ↔ Windows ↔ macOS)
  7. Manual review gates pass for any exit criteria flagged as non-automatable (per testing-strategy.md Principle 2). Both CI and manual gates must pass — neither alone is sufficient.

Phase Rollout

PhaseTesting Scope Added
M0–M1Tier 1 pipeline, determinism harness, strict-path tests, clippy/fmt gates
M2Tier 2 pipeline, replay round-trip, ic-cnc-content fuzz targets, Merkle tree unit tests, order validation matrix
M3Performance benchmark suite (incl. pathfinding LOD, spatial hash, flowfield, stagger schedule, ECS cache benchmarks), zero-alloc assertions, save/load tests, display name validation (V46 — UTS #39 confusable corpus, aligned with M3.SEC.DISPLAY_NAME_VALIDATION cluster)
M4Network protocol tests, desync injection, Lua sandbox escape suite, sub-tick fairness scenarios, relay timestamp clamping, reconnection snapshot verification, order rate limiting, P2P replay attestation smoke tests (V53 — basic signed-hash exchange, aligned with M4.SEC.P2P_REPLAY_ATTESTATION; full multi-peer exercise deferred to T4 at M9)
M5Anti-cheat calibration corpus, false-positive evaluation, ranked tests, SCR sequence enforcement, command permission tests, cvar bounds tests, AI determinism (cross-platform), key rotation protocol tests (V47 — rotation sequence, cooldown, emergency recovery, aligned with M5.SEC.KEY_ROTATION_AND_REVOCATION; full lifecycle exercise deferred to T4 at M9)
M6RTL/BiDi QA corpus regression, WASM inter-module isolation (V50 — basic cross-module probe attempts, aligned with M6.SEC.WASM_INTERMODULE_ISOLATION cluster), visual regression
M7–M8Workshop ecosystem tests (dependency cycle detection, version immutability), WASM escape vectors, expanded cross-module isolation integration (shared-host-resource leak scenarios beyond M6’s basic probes), WASM memory growth fuzzing, mod profile fingerprint stability, balance preset validation, weather determinism, D059 RTL chat/marker text safety tests, p2p-distribute fuzz suite (bencode/wire/metadata ≥1M iterations each), P2P interop tests (tracker announce/scrape, DHT routing, piece verification, multi-peer swarm), P2P profile switching (embedded→desktop→seedbox), ic-server Workshop seeder integration smoke tests
M9Full Tier 4 weekly suite, release criteria enforcement, campaign DAG validation, roster carryover tests, LLM content validation
M10–M11Campaign playthrough automation, extended fuzz campaigns, cross-engine map exchange, full WASM memory growth fuzzing