every.channel/evolution/proposals/ECP-0028-testing-and-coverage.md

ECP-0028: test strategy + coverage gates

Status: Draft

Problem / context

every.channel is security and integrity sensitive. Before we scale ingestion and relaying, we need a test suite that makes protocol regressions obvious and makes junk injection difficult to accidentally enable.

We also want coverage as a forcing function to keep logic in testable libraries, not buried in binaries or UI glue.

Decision

Adopt a layered test strategy and a coverage gate:

• Unit tests for pure logic (hashing, manifests, proofs, encryption, quotas, URL parsing).
• Integration tests for cross-crate invariants (manifest <-> object meta compatibility, Merkle membership validation, catalog entry semantics).
• End to end tests for “single node publish + subscribe” flows, with deterministic fixtures and explicit opt-in for tests that require external dependencies (ffmpeg headers, OS devices).

Coverage goals are defined per layer:

• ec-core, ec-crypto, ec-moq, ec-iroh, ec-linux-iptv: target 100% line coverage (excluding third_party).
• Node runner + Tauri backend: target high coverage for shared logic modules; binary-only glue may be excluded, but the glue must be minimal by policy.
• UI: target behavioral tests for state transitions (not full DOM snapshots).

Tooling

• Use cargo llvm-cov for coverage measurement.
• Coverage is run per-crate and per-workspace in the nix dev shell so ac-ffmpeg can find ffmpeg headers.

Test matrix

Unit tests (must be deterministic)

• ec-core
  • Manifest ID determinism and change sensitivity.
  • Merkle root correctness.
  • Merkle proof generation + verification (including tamper detection).
• ec-crypto
  • Stream key/nonce derivation determinism.
  • Encrypt/decrypt roundtrip and mismatch failures.
  • Manifest signature sign/verify and allowlist behavior.
• ec-moq
  • Object frame encode/decode roundtrip.
  • Manifest frame encode/decode roundtrip.
  • File relay sanitization stability.
• ec-iroh
  • Token bucket throttling/refill behavior (no sleeps).
• ec-linux-iptv
  • channels.conf parsing (unique/sorted, ignore comments).
  • Default tune command construction.

Integration tests (cross-crate invariants)

• Manifest root validates object chunk_hash either by direct hash list or by Merkle proof.
• Encrypted objects preserve integrity checks (hash is over plaintext).
• Catalog entries carry manifest summaries consistently.

End to end tests (opt-in / platform dependent)

• moq publish/subscribe loopback tests with epoch manifests:
  • publish N chunks, subscribe and verify acceptance/rejection paths.
• Linux DVB discovery tests:
  • run only when /dev/dvb exists; otherwise skip.

Policy implications

• New protocol logic should land in libraries, not binaries.
• Any feature that changes integrity behavior (hashing, proofs, signing, validation, quotas) must add tests proving:
  • positive path (accept)
  • negative path (reject)
  • no panics on malformed inputs

Alternatives considered

• “Only E2E tests”: rejected (slow and flaky).
• “Only unit tests”: rejected (cross-crate breakages are likely).

Rollout / teardown

1. Add unit tests to core crates until 100% coverage is achievable.
2. Refactor binary logic into testable modules when coverage shows dead zones.
3. Add integration tests for manifest and object invariants.
4. Add opt-in E2E tests for ffmpeg and device-bound pipelines.

Teardown: if llvm-cov becomes too costly locally, keep tests and make coverage gates advisory, but retain per-crate coverage reports.