Advance forge rollout, Ethereum rails, and NBC sources

This commit is contained in:
every.channel 2026-04-01 15:58:49 -07:00
parent be26313225
commit 7d84510eac
No known key found for this signature in database
88 changed files with 11230 additions and 302 deletions

View file

@ -2,6 +2,8 @@
Status: Implemented
Note: Persistent declarative host operation is specified in ECP-0083.
## Context
Runner netboot artifacts now publish from CI, but there is no repository-native operating path for fleet provisioning on common prosumer networks (for example Unifi VLANs).
@ -17,6 +19,12 @@ Unifi DHCP can expose next-server/bootfile settings, but iPXE chainloading often
2. Keep Unifi DHCP as the IP authority; use ProxyDHCP only to supply bootfile logic.
3. Document a concrete NUC rollout sequence for same-VLAN provisioning.
4. Keep dependencies minimal (`curl`, `tar`, `python3`, `dnsmasq`) and avoid requiring image flashing workflows.
5. Support an optional UniFi-only mode by providing an embedded-script iPXE build path (`ec-ipxe.efi`) so clients can chainload without DHCP conditional logic.
6. Verify release artifact integrity during staging when `SHA256SUMS.txt` is published.
7. Harden serving/staging defaults:
- default to local iPXE artifacts (remote iPXE download requires explicit opt-in),
- support optional chain token protection for `netboot.ipxe`,
- support HTTP CIDR allowlists for artifact serving.
## Alternatives considered

View file

@ -0,0 +1,42 @@
# ECP-0083: Declarative Netboot Service Module
Status: Implemented
## Context
ECP-0082 added script-driven netboot staging and serving for UniFi/ProxyDHCP fleets. That path works, but it is still operator-session driven (`tmux`, manual env vars, manual restart order), which is fragile for sustained fleet bring-up.
The constitution favors explicit, reviewable infrastructure definitions. Netboot delivery should be operated as a normal NixOS service with stable options, systemd lifecycle, and auditable host config.
## Decision
1. Add a reusable NixOS module at `nix/modules/ec-netboot.nix` exported as `nixosModules.ec-netboot`.
2. Define a first-class `services.every-channel.netboot` option tree for:
- UniFi-only mode (default, no ProxyDHCP),
- optional ProxyDHCP mode,
- release source pinning (host/repo/tag/local tarball/token file),
- iPXE strategy (embedded build, local file, or explicit remote download),
- security controls (chain token file, HTTP CIDR allowlist).
3. Run persistent systemd units:
- `every-channel-netboot-ipxe` (oneshot, optional embedded EFI build),
- `every-channel-netboot-stage` (oneshot artifact staging),
- `every-channel-netboot` (long-running HTTP+TFTP service).
4. Add tmpfiles and firewall wiring in-module so host configs remain concise and reversible.
5. Keep existing scripts as execution primitives to avoid duplicate logic and preserve local/manual fallback operations.
## Alternatives considered
- Keep scripts only. Rejected because startup order, secret injection, and restart behavior remain ad-hoc.
- Implement host-specific module logic only in `key.store`. Rejected because this behavior is core `every.channel` netboot operations and should be reusable across hosts.
- Replace scripts with a brand new daemon immediately. Rejected to keep rollout incremental and avoid avoidable regressions.
## Rollout / teardown plan
- Rollout:
- import `every-channel.nixosModules.ec-netboot` on the boot host,
- set `services.every-channel.netboot.*` options,
- activate and verify `every-channel-netboot-stage` then `every-channel-netboot`.
- Teardown:
- disable `services.every-channel.netboot.enable`,
- remove host option stanza,
- fall back to manual script operation from `docs/NUC_UNIFI_NETBOOT.md` if needed.

View file

@ -0,0 +1,35 @@
# ECP-0084: Sovereign `ecp-forge` Host Deploy from every.channel
Status: Implemented
## Context
`git.every.channel` (Hetzner 300TB host) has been operated from external infra repos. That creates coupling and weakens operational independence for every.channel infrastructure changes, especially netboot/PXE and archive workflows.
The constitutional direction is explicit repository ownership over its infrastructure path. every.channel should be able to deploy its own forge host from this repository, with age/agenix material stored here.
## Decision
1. Add a sovereign `nixosConfigurations.ecp-forge` target to this repository.
2. Keep the forge role (`services.forgejo`, `services.caddy`) and archive role (`services.every-channel.ec-node`) in that host target.
3. Enable persistent netboot from this repository using `services.every-channel.netboot`, with local sovereign tarball staging as the default source path.
4. Keep UniFi-only mode as default (`proxyDhcp.enable = false`) to avoid cross-domain DHCP coupling.
5. Store host-consumed runtime secrets in this repository (`secrets/*.age`) and decrypt on-host via `agenix`.
6. Deploy directly from this repository to `git.every.channel`.
## Alternatives considered
- Continue deploying `git.every.channel` from shared infra repos. Rejected due ownership/coupling drift.
- Keep runtime-only netboot scripts on host. Rejected because boot resilience should survive reboot and config rebuilds.
- Move to ProxyDHCP-first by default. Rejected for now to keep DHCP authority in UniFi.
## Rollout / teardown plan
- Rollout:
- build/evaluate `.#nixosConfigurations.ecp-forge`,
- deploy from every.channel to `git.every.channel`,
- verify `every-channel-netboot-stage` and `every-channel-netboot`.
- Teardown:
- disable `services.every-channel.netboot.enable` in `nix/nixos/ecp-forge.nix`,
- redeploy,
- fall back to manual script flow (`docs/NUC_UNIFI_NETBOOT.md`) if required.

View file

@ -0,0 +1,35 @@
# ECP-0085: Enable Archive Auto-Worker on Sovereign `ecp-forge`
Status: Implemented
## Context
`ecp-forge` currently runs Forgejo and netboot services from `every.channel`, but archival ingestion is not active. Historical archive/manifests exist on host storage, yet no `every-channel-wt-archive*` unit is running, so stream persistence stalls.
The constitutional direction favors an operational archive network. The sovereign forge host should continuously ingest public streams into durable storage.
## Decision
1. Enable `services.every-channel.ec-node` on `ecp-forge`.
2. Run archive ingestion mode (`archive.enable = true`) with no broadcast publish role.
3. Use:
- `archive.outputDir = "/tank/every-channel/archive"` (large durable ZFS storage),
- `archive.manifestDir = "/var/lib/every-channel/manifests"` (existing manifest path continuity).
4. Keep `archive.serve.enable = false` for now; this change is ingest-only.
## Alternatives considered
- Keep archive workers manual/ephemeral. Rejected because ingestion must survive reboot/redeploy.
- Move manifests to `/tank` in this change. Deferred to avoid migration risk while restoring ingestion first.
- Enable replay serving (`wt-archive-serve`) immediately. Rejected for scope control; ingest health first.
## Rollout / teardown plan
- Rollout:
- deploy `.#nixosConfigurations.ecp-forge`,
- verify `every-channel-wt-archive-auto` is active,
- verify journal shows polling/worker spawn and manifests update.
- Teardown:
- set `services.every-channel.ec-node.archive.enable = false`,
- deploy,
- preserve existing on-disk archive/manifests data.

View file

@ -0,0 +1,35 @@
# ECP-0086: Web Watcher Jitter Budget Override
Status: Implemented
## Problem
Browser live playback on `every.channel` stays connected but still cuts in and out. A Chrome browser repro against `https://every.channel/watch?url=https%3A%2F%2Fcdn.moq.dev%2Fanon&name=la-nbc` showed no remounts or request failures, but the `@moq/watch@0.2.0` player emitted continuous `skipping slow group` warnings plus frequent small seek corrections.
## Constraints
- Keep the existing `@moq/watch` integration and WebTransport-only path.
- Avoid forking upstream player code for a site-level playback tuning change.
- Keep the fix reversible and local to the web app wiring.
## Decision
Set the web watcher's `jitter` attribute to `750` milliseconds in `apps/web/app.js` before connect.
Browser evidence for the same live stream showed:
- default `jitter=100`: `2762` warnings over the sample, including `324` seek corrections, final `readyState=1`
- forced `jitter=750`: `2482` warnings over the sample, including `75` seek corrections, final `readyState=4`
This does not eliminate upstream `skipping slow group` churn entirely, but it substantially reduces the seek-thrash that most directly surfaces as visible cut-in/cut-out playback.
## Alternatives Considered
- Leave the default `100ms` jitter budget. Rejected because the browser repro showed sustained seek churn under live playback.
- Raise jitter further (for example `1500ms`). Rejected for now because it reduced some warnings but regressed playback advancement more than `750ms` in the same sample.
- Fork `@moq/watch`. Rejected because the exposed `jitter` control was sufficient for a first mitigation.
## Rollout / Teardown
- Deploy the `jitter=750` override and validate live watch continuity in Chrome against public relay streams.
- If the extra latency is unacceptable or upstream player behavior changes, remove the override and fall back to the library default.

View file

@ -0,0 +1,40 @@
# ECP-0087: LAN-Capable iPXE/QEMU VM Module
Status: Implemented
## Context
`every.channel` already has reusable runner and netboot modules, but hosts that want a continuously-running iPXE test VM still have to carry ad hoc QEMU glue. That makes boot-path verification and downstream reuse harder, and it prevents hosts from declaratively opting into a LAN-visible guest for tuner discovery.
## Decision
1. Add `nixosModules.ec-ipxe-qemu` at `nix/modules/ec-ipxe-qemu.nix`.
2. Add a dedicated publisher guest module/output pair:
- `nixosModules.ec-publisher-guest`
- `nixosConfigurations.ec-publisher-x86_64`
- `nixosConfigurations.ec-publisher-x86_64-netboot`
This keeps the VM path explicitly publisher-oriented while leaving host-specific tuner/broadcast choices to downstream configs.
3. Define `services.every-channel.ipxe-qemu.*` options for:
- persistent qcow2/state directory handling,
- user-mode iPXE boot networking that chains to a configurable internet boot URL,
- optional second NIC via `macvtap` for non-disruptive LAN presence,
- guest sizing and raw QEMU argument overrides.
4. Run the guest as a persistent systemd service (`every-channel-ipxe-qemu`) with restart-on-exit semantics so host restarts or config switches naturally refresh the in-memory booted VM.
5. Enable one conservative instance on `ecp-forge` using the user-mode boot path only, so the module is exercised in-repo without assuming a local tuner LAN on the forge host.
## Alternatives Considered
- Keep host-specific shell glue outside `every.channel`. Rejected because downstream hosts cannot reuse or review the boot path as a first-class module.
- Require a Linux bridge on the host for LAN access. Rejected for now because it is more disruptive than `macvtap` and unnecessary for an initial deployment.
- Boot only with QEMU user networking. Rejected because tuner discovery needs a real LAN attachment on some hosts.
## Rollout / Teardown
- Rollout:
- import `nixosModules.ec-ipxe-qemu`,
- enable `services.every-channel.ipxe-qemu`,
- set `lan.enable = true` plus `lan.macvtap.interface` on hosts that need LAN discovery.
- Teardown:
- disable `services.every-channel.ipxe-qemu.enable`,
- remove host options,
- fall back to ad hoc QEMU or direct host publishers if needed.

View file

@ -0,0 +1,30 @@
# ECP-0088: Public RPC/NFS Hardening for `ecp-forge`
Status: Implemented
## Context
`ecp-forge` exports `/tank` over NFS for private consumers, but the host firewall also exposed `rpcbind` (`111/tcp,udp`) and NFS (`2049/tcp,udp`) on the public Hetzner address. CERT-Bund flagged the host because public `rpcbind` allowed internet enumeration of registered RPC services.
The current exports already constrain clients to private address space, so the exposure is a firewall boundary issue rather than a requirement for public access.
## Decision
1. Remove public firewall allowances for `111/tcp,udp` and `2049/tcp,udp` on `ecp-forge`.
2. Keep NFS enabled for trusted/private paths, including the existing `tailscale0` trusted interface and private-source exports.
3. Treat public RPC/NFS exposure on forge hosts as an anti-pattern unless a later ECP explicitly justifies it.
## Alternatives considered
- Disable NFS entirely. Rejected because `/tank` export remains useful for private consumers.
- Keep public ports open and rely only on `/etc/exports` CIDR restrictions. Rejected because `rpcbind` enumeration is itself enough to trigger abuse notifications and increases attack surface.
- Add bespoke public-interface firewall exceptions per private CIDR. Rejected because `ecp-forge` already has a trusted overlay path and does not need public-interface exposure for NFS.
## Rollout / teardown plan
- Rollout:
- evaluate `.#nixosConfigurations.ecp-forge`,
- deploy `ecp-forge`,
- verify `rpcbind`/NFS are no longer reachable on the public IP.
- Teardown:
- restore the public firewall allowances only with a replacement ECP that documents the requirement and compensating controls.

View file

@ -0,0 +1,61 @@
# ECP-0089: Broadcast-Scoped Discovery Identity For Gossip And DHT
Status: Implemented
## Decision
When `ec-node` needs to synthesize a default `stream_id` for publish/announce flows, it will prefer a broadcast-scoped identifier derived from channel metadata over a source-scoped identifier.
Current rule:
- If the selected channel yields a usable broadcast identity, generate `ec/stream/v1/broadcast/...`.
- Otherwise keep the existing source-scoped fallback `ec/stream/v1/source/...`.
- Explicit `--stream-id` continues to override both behaviors.
For HDHomeRun lineup sources, the usable broadcast identity is assembled from the channel record when present:
- `standard = atsc`
- `program_number` from lineup `ProgramNumber`/`ProgramID` when available
- `virtual_channel` from `GuideNumber`
- `callsign` from typed lineup metadata or channel name fallback
- optional `region` / `frequency` hints when present
## Motivation
Today identical OTA channels announced from different ingest nodes often diverge because the default identity is source-scoped (`hdhr`, device id, local channel selection). That defeats gossip/DHT dedupe and makes the network treat equivalent broadcasts as unrelated streams.
We already have a long-term architectural direction in ECP-0004: broadcast identity is the primary convergence key and source identity is a fallback. This change applies that rule to the current discovery boundary instead of waiting for deeper ingest metadata work.
## Scope
In scope:
- Channel-to-broadcast identity helper in `ec-core`
- Typed lineup metadata extraction in `ec-hdhomerun`
- Default `stream_id` synthesis updates in `ec-node`
- Unit coverage for identity derivation and lineup parsing
- Operator docs describing the precedence rules
Out of scope:
- Deep MPEG-TS/PSIP extraction of TSID/program data from live transport packets
- Automatic migration of already-running source-scoped publishers
- Cryptographic anti-spoofing or admission control for gossip/DHT announcements
## Constraints
- Must remain additive and reversible
- Must not break explicit operator-provided `--stream-id`
- Must keep non-broadcast-aware sources working unchanged
## Alternatives considered
- Keep source-scoped defaults until full PSIP parsing exists. Rejected because it preserves a known convergence flaw in the current network behavior.
- Force all operators to manually specify canonical `--stream-id`. Rejected because it is error-prone and defeats zero-config dedupe.
- Collapse purely onto channel number. Rejected because channel number alone is too weak without additional hints.
## Rollout / Reversibility
- Additive change: only synthesized defaults change, not explicit IDs.
- Roll back by restoring source-scoped default synthesis.
- Existing automation can pin old behavior with explicit `--stream-id` if needed during transition.

View file

@ -0,0 +1,52 @@
# ECP-0090: PAT-Derived Broadcast Identity Promotion
Status: Implemented
## Decision
Extend default discovery identity synthesis to probe early MPEG-TS packets for Program Association Table (PAT) data and use that to strengthen broadcast-scoped stream IDs.
Current rule:
- If source/channel metadata already yields a usable broadcast identity, use it.
- If transport probing finds PAT data, fill missing `transport_stream_id` and `program_number`.
- If PAT shows exactly one non-zero program, that program is eligible for broadcast-scoped convergence.
- If PAT is ambiguous (multiple non-zero programs), do not guess; keep `program_number` unset and fall back to source scope when needed.
- If the probe sees ATSC PSIP traffic, label the standard as `atsc`; otherwise use a generic `mpegts` fallback when PAT is the only signal.
## Motivation
ECP-0089 fixed the first duplication bug by preferring channel metadata over source-local IDs. That still left a hole:
- raw TS sources without lineup metadata could not converge,
- channel metadata could be incomplete or inconsistent,
- and weak identity inputs risked accidental convergence on the wrong broadcast.
PAT-derived identity gives us a stronger on-wire fingerprint without waiting for full PSIP/VCT parsing.
## Scope
In scope:
- PAT parsing in `ec-ts`
- bounded transport probing for `tsid` / single-program extraction
- source integration for raw TS inputs and HDHomeRun enrichment
- tests that enforce "single program promotes, ambiguous stream does not"
Out of scope:
- full ATSC VCT/MGT descriptor parsing
- DVB service table parsing
- cryptographic origin authentication for discovery announcements
## Alternatives considered
- Keep trusting channel metadata only. Rejected because it leaves raw TS paths weak and incomplete.
- Always promote from PAT even with multiple programs. Rejected because it guesses the channel and increases junk-collision risk.
- Require manual `--stream-id` for TS inputs. Rejected because it pushes canonicalization burden onto operators.
## Rollout / Reversibility
- Additive and reversible: explicit `--stream-id` still overrides everything.
- Roll back by removing PAT probing and returning to metadata-only promotion.
- Future PSIP/VCT work can refine the broadcast identity without changing the override surface.

View file

@ -0,0 +1,67 @@
# ECP-0091: Full ATSC PSIP Parsing and Real-Sample Validation
Status: Implemented
## Decision
Extend `ec-ts` from PAT-only plus partial PSIP awareness into table-level ATSC PSIP parsing for:
- `MGT`
- `TVCT` / `CVCT`
- `STT`
- `RRT`
- `EIT`
- `ETT`
Additional rules:
- `EIT` and `ETT` are parsed on their MGT-assigned PIDs, not only on base PID `0x1FFB`.
- Raw descriptor bytes are preserved in parsed tables where we do not yet expose typed descriptor
structs.
- ATSC STT is converted to Unix time using GPS epoch semantics instead of the earlier placeholder
arithmetic.
- Real-data checks are added as ignored tests using external ATSC captures rather than vendoring
large TS fixtures into the repo.
## Motivation
ECP-0089 and ECP-0090 fixed the first discovery-identity hole by preferring broadcast metadata and
then strengthening it with PAT probing. That still left two gaps:
- PSIP coverage was incomplete, so the parser could detect “this is ATSC” without understanding the
rest of the broadcast metadata surface.
- Validation was synthetic-only, which is not enough for transport tables that are notoriously easy
to parse incorrectly while still passing fixture tests.
We need both: fuller PSIP coverage and a rerunnable path against known real captures.
## Scope
In scope:
- full table parsing for `MGT`, `TVCT/CVCT`, `STT`, `RRT`, `EIT`, and `ETT`
- multiple-string structure parsing with conservative text decoding for common uncompressed modes
- non-base PID handling for `EIT` / `ETT`
- corrected STT to Unix-time conversion
- ignored real-sample tests against `tsduck-test` ATSC captures
Out of scope:
- semantic parsing of all ATSC descriptor payloads
- DVB/ISDB parity in this change
- using `RRT`, `EIT`, or `ETT` directly in the default discovery key
## Alternatives considered
- Keep partial PSIP parsing and rely on PAT + lineup hints only. Rejected because it leaves too much
unverified broadcast metadata on the floor.
- Vendor large real TS captures into the repo. Rejected because it bloats the tree and makes review
worse.
- Parse `EIT` / `ETT` only on `0x1FFB`. Rejected because live ATSC streams carry them on the PIDs
advertised by `MGT`.
## Rollout / Reversibility
- Additive for existing callers: explicit `--stream-id` still overrides discovery identity.
- If needed, roll back by reverting the new table parsers while keeping PAT-based identity promotion.
- The ignored real-sample tests remain opt-in and do not make normal CI depend on external downloads.

View file

@ -0,0 +1,158 @@
# ECP-0092: Ethereum rails and private settlement for every.channel
Status: Draft
## Problem / context
`every.channel` already has useful anti-junk and manifest integrity primitives, but they stop at
repo-local formats: custom Merkle trees, JSON body hashes, and Ed25519 signatures. That is enough
for local validation, not for the broader direction now requested:
- stream identity, broadcast de-duplication, manifests, and transport announcements should all have
Ethereum-compatible representations,
- private-chain settlement should reuse existing Ethereum tooling instead of inventing a custom
ledger,
- iroh should remain the transport substrate,
- storage on chain must stay compact, and
- `ecp-forge` should become the high-bandwidth head node for sequencing, archival, and bootstrap.
This supersedes the specific ECP-0022 alternative that rejected “global consensus / blockchain for
manifests” as overkill. The new direction is not a public general-purpose chain. It is a private,
Ethereum-compatible settlement rail for every.channel commitments.
## Decision
Adopt a dual-plane architecture:
- `iroh` remains the transport, discovery, and media movement layer.
- Local protocol integrity remains first-class:
- `manifest_id` stays BLAKE3 over the manifest body,
- object membership proofs stay `merkle+blake3`,
- Ed25519 manifest signatures remain supported.
- Ethereum-compatible commitments become an additive settlement/mirroring layer for stream
identity, transport announcements, and manifests.
- Ethereum-native signatures are added alongside local signatures:
- manifests may carry secp256k1 EIP-712 signatures over `ManifestBody`,
- no existing local signature path is removed as part of this proposal.
- On-chain state remains compact:
- manifests store only stream/epoch ids plus commitment hashes,
- control announcements store only stream ids plus announcement commitments,
- full media bytes, manifests, PSIP tables, and chunk inventories remain off-chain.
- The chain is private and purpose-built for every.channel use cases:
- no arbitrary application compute in the near term,
- no fraud-proof system in the near term,
- permissionless submission as far as operationally possible,
- pseudonymous participation preferred over account-heavy identity.
## Details
### Canonical Ethereum representations
Introduce Solidity-compatible struct mirrors for the protocol surfaces we actually ship:
- `BroadcastId`
- `SourceId`
- `StreamKey`
- `StreamDescriptor`
- `StreamControlAnnouncement`
- `ChunkId`
- `ManifestVariant`
- `ManifestBody`
- `ManifestSignature`
- `Manifest`
These are encoded via Solidity ABI rules and hashed with `keccak256`. They mirror the shipped
protocol types; they do not replace the local wire format or local hashing rules.
### Manifest commitments
Publisher-generated manifests may carry a set of Ethereum commitments as first-class fields:
- `manifest-data-merkle-keccak256-v1`
- Keccak Merkle root over ordered chunk hashes, or over ordered variant roots for multi-variant
manifests.
- `manifest-body-abi-keccak256-v1`
- `keccak256(abi.encode(EthManifestBody))`
- `manifest-envelope-abi-keccak256-v1`
- `keccak256(abi.encode(EthManifest))`
`manifest_id` and `merkle_root` remain the local BLAKE3-backed protocol fields. Ethereum
commitments are explicit, separately named, and never inferred from those local field names alone.
### Signature rails
Manifest signatures become dual-rail:
- local signatures continue to sign the local BLAKE3 `manifest_id` with Ed25519,
- Ethereum-native signatures sign the typed `EthManifestBody` with secp256k1 EIP-712,
- both signature families may appear on the same manifest.
The Ethereum signature target is `ManifestBody`, not the full manifest envelope, so signatures do
not become self-referential when more signatures are appended.
### Stream and control commitments
Stream descriptors and control announcements may also carry Ethereum-compatible commitments:
- `stream-id-keccak256-v1`
- `stream-descriptor-abi-keccak256-v1`
- `control-announcement-abi-keccak256-v1`
This ties broadcast-scoped discovery identity to the same settlement vocabulary used by manifests
without forcing transport-time consumers to abandon the local vocabulary they already use.
### Solidity rail contract
Add a minimal contract that stores only the latest compact pointers:
- latest manifest pointer per `stream_id + epoch_id`
- latest transport announcement pointer per `stream_id`
- event emissions for historical indexing
The contract must stay event-heavy and storage-light. Full manifests stay in iroh / relay / archive
storage.
### `ecp-forge` role
`ecp-forge` becomes the maximal head node for this network:
- private-chain bootstrap and packaging,
- archival indexer for settlement events,
- high-bandwidth manifest and control announcer,
- optional sequencer / execution-client host for the private network.
This role is operational, not constitutional. Other nodes must still be able to validate, mirror,
and submit commitments without going through a closed control plane.
## Alternatives considered
- Keep custom BLAKE3-only manifests forever. Rejected because it strands every.channel outside the
tooling and audit surface of the Ethereum ecosystem.
- Replace BLAKE3 and Ed25519 immediately. Rejected because it weakens the local protocols existing
integrity path before the private rail is operational and discards useful cryptographic
properties for no near-term operational win.
- Build a custom blockchain. Rejected because existing Rust + Solidity stacks are already good
enough and reduce implementation risk.
- Use public Ethereum mainnet / arbitrary public L2s directly. Rejected for now because cost,
privacy, and operational control do not fit the current network shape.
## Rollout / teardown plan
1. Add `ec-eth` with canonical Solidity-compatible representations and Keccak commitments.
2. Preserve the local BLAKE3/Ed25519 rail as the default transport integrity path.
3. Add optional secp256k1 EIP-712 manifest-body signatures alongside Ed25519 signatures.
4. Add the minimal Solidity rail contract for compact settlement pointers.
5. Teach publishers/indexers to mirror manifests and announcements onto the private chain.
6. Stand up the private Ethereum-compatible network on `ecp-forge` and companion nodes.
Teardown:
- back out the Ethereum rail changes explicitly in code; the preserved local rail remains as the
fallback integrity path.
## Open questions
- Which execution stack should back the private network first: a slim Reth-based node, or a more
minimal Rust EVM deployment?
- When the private rail becomes live, do we require both signature families for publisher policy, or
is one valid rail-specific signature enough for a given transport or settlement action?

View file

@ -0,0 +1,146 @@
# ECP-0093: `ecp-forge` OP Stack Sepolia testnet and observation consensus rail
Status: Draft
## Problem / context
`every.channel` now has additive Ethereum-compatible commitments and signatures, but it does not yet
have:
- a concrete chain-operator path for bringing up a real OP Stack testnet on `ecp-forge`,
- a repo-owned bootstrap workflow that matches the current official OP Stack deployment guidance,
- an application-level consensus surface for "reality-derived" blocks where witnesses attest to the
same observed broadcast epoch, or
- a reproducible smoke path that pushes real every.channel archive data through the rail.
The current `EveryChannelRail` contract is useful as a compact pointer store, but it does not
capture the consensus shape requested by the founder: nodes independently discover an observation
from reality, then witnesses sign off on that shared observation hash.
## Decision
Adopt a two-part rollout:
1. Stand up an OP Stack Sepolia-anchored testnet on `ecp-forge` using the official OP Stack stack:
`op-deployer`, `op-geth`, `op-node`, `op-batcher`, `op-proposer`, `op-challenger`, and
optional `op-dispute-mon`.
2. Introduce an application-level observation consensus rail for every.channel:
- an `ObservationHeader` is derived from real stream epoch data,
- witnesses attest to the derived observation hash,
- the observation is finalized when quorum is reached,
- full media bytes remain off-chain.
This keeps Ethereum as the ordering/finality substrate while making the application-level "block"
come from reality, not from arbitrary contract writes.
## Details
### `ecp-forge` OP Stack deployment
- Use the official OP Stack deployment topology described in the current Optimism chain-operator
tutorial:
- Sepolia L1 contracts via `op-deployer`,
- `op-geth` + `op-node` sequencer,
- `op-batcher`,
- `op-proposer`,
- `op-challenger`,
- optional `op-dispute-mon`.
- Host these components on `ecp-forge` via repo-owned NixOS services and pinned official container
images.
- Keep L2 RPC surfaces private by default; expose P2P only as required for testnet participation.
- Default L1 RPC / beacon endpoints may use public Sepolia providers, but deployment and operation
require a repo-managed Sepolia private key secret with sufficient ETH.
- `op-challenger` and `op-dispute-mon` require a repo-managed Cannon absolute prestate artifact.
If the prestate is absent, the core rollup may still run, but the dispute path is intentionally
gated off instead of starting with incomplete configuration.
### Observation consensus
Introduce a new application-level surface alongside `EveryChannelRail`:
- `ObservationHeader`
- `streamHash`
- `epochHash`
- `parentObservationHash`
- `dataRoot`
- `locatorHash`
- `observedUnixMs`
- `sequence`
- `ObservationHash`
- `keccak256(abi.encode(ObservationHeader))`
- `ObservationSlot`
- `keccak256(abi.encode(streamHash, epochHash))`
Witnesses attest to an `ObservationHash`, not directly to arbitrary payloads. This lets the chain
finalize the winning observation for a `(stream, epoch)` slot without storing the full manifest,
chunk list, PSIP data, or media bytes.
### Witness model
For the first testnet tranche:
- witness membership is registry-backed,
- quorum is fixed and explicit,
- no staking or slashing is required,
- sending an attestation transaction from a witness address is sufficient "sign off".
Off-chain EIP-712 witness signatures may be added later for relayed submission, but they are not a
prerequisite for the first live testnet.
### Real-data validation
Use real every.channel archive data already present on `ecp-forge` as the smoke source:
- derive observation headers from actual archive JSONL entries,
- submit them to a local Anvil deployment of the observation ledger,
- prove quorum/finalization against production-shaped data before attempting Sepolia rollout.
## Components
- `contracts/EveryChannelRail.sol`
- compact manifest / transport pointer store (existing rail)
- `contracts/EveryChannelWitnessRegistry.sol`
- witness membership for observation consensus
- `contracts/EveryChannelObservationLedger.sol`
- candidate observation storage, attestation tracking, and finalization
- `scripts/op-stack/download-op-deployer.sh`
- pinned `op-deployer` fetch helper
- `scripts/op-stack/setup-rollup.sh`
- repo-owned OP Stack bootstrap/config generation
- `scripts/op-stack/anvil-reality-smoke.sh`
- local smoke against Anvil using real `ecp-forge` archive data
- `nix/modules/ec-op-stack.nix`
- NixOS service/module surface for `ecp-forge`
## Alternatives considered
- Use Base Sepolia directly as the only test environment. Rejected because it validates contract
compatibility, but not operation of an every.channel-owned chain.
- Wait for a future custom consensus implementation before any chain bring-up. Rejected because it
delays operator learning and hides infrastructure problems until late.
- Put media bytes or full manifests on chain. Rejected because it is operationally wasteful and not
required for observation finalization.
## Rollout / teardown plan
1. Add the observation-ledger contracts and Foundry tests.
2. Add Anvil + real-archive smoke scripts.
3. Add an `ec-op-stack` Nix module and `ecp-forge` integration points.
4. Wire secrets for a Sepolia deployment key on `ecp-forge`.
5. Bring up the OP Stack services on `ecp-forge`.
6. Deploy observation-ledger contracts onto the new chain.
7. Submit real archive-derived observations and verify finalization.
Teardown:
- disable `services.every-channel.op-stack`,
- stop and remove the OP Stack containers/state directories,
- preserve the local Anvil smoke path and the additive Ethereum rail code.
## Open questions
- Should `ecp-forge` be the only sequencer for the first tranche, or should a second verifier-only
peer be provisioned immediately?
- When we move beyond a registry-backed witness set, do we want bonds, reputation, or both?
- Should finalized observation slots bridge back into the existing manifest/transport pointer
contract automatically, or remain a separate ledger surface for the first rollout?

View file

@ -0,0 +1,60 @@
# ECP-0094: NBC Browser-Backed Source with Adobe Auth
Status: Proposed
## Decision
Add `nbc.com/watch/...` as a manual source in the Tauri app by treating it as a browser-backed
capture source instead of a direct manifest source.
The desktop app will:
- recognize NBC watch URLs as a distinct `nbc` source kind
- launch Chrome with a persistent profile so Adobe Pass / MVPD state can survive across runs
- wait for the Adobe/NBC entitlement sequence seen in `intake/`
- capture the rendered video element from the authenticated browser tab and feed those frames into
the existing ffmpeg CMAF ladder path
## Motivation
The intake traces show NBC live playback is not exposed as a plain live HLS or clear DASH input:
- page metadata comes from `friendship.nbc.com`
- entitlement is gated by Adobe Pass on `sp.auth.adobe.com`
- authorization is validated through NBC's `tokenverifier`
- playout resolves to MediaTailor DASH plus Widevine license requests
That means the existing source model cannot ingest NBC by simply resolving a URL for ffmpeg. We
need a source path that can execute the browser login/auth/player flow first.
## Scope
In scope:
- `nbc.com/watch/...` detection in the Tauri add-stream flow
- persistent manual-source storage for the new kind
- Chrome launch/profile handling for NBC playback
- readiness checks tied to the Adobe/session/authorize/token-verifier path observed in the traces
- browser-frame capture into the existing live transcode/publish pipeline
Out of scope:
- generic DRM extraction or CDM key handling
- unattended credential entry for every MVPD provider
- CLI ingest support in `ec-node`
- perfect parity with native audio capture in the first cut
## Alternatives Considered
- Treat NBC as a normal HLS/DASH source. Rejected because the trace shows DRM-gated DASH that
ffmpeg cannot ingest directly.
- Re-implement the full Adobe + MVPD HTTP flow in-process. Rejected for the first cut because the
real browser/player state is already required for playout and provider flows vary.
- Leave NBC unsupported. Rejected because the traces are sufficient to define a pragmatic desktop
path now.
## Rollout / Reversibility
- The change is additive and isolated to the desktop app's manual-source flow.
- Roll back by removing the `nbc` source kind and browser capture path.
- Existing HDHomeRun, HLS, Linux DVB, and yt-dlp flows remain unchanged.

View file

@ -0,0 +1,60 @@
# ECP-0095: Native macOS Webview First for NBC Auth and Playback
Status: Proposed
## Decision
On macOS, the Tauri app should attempt NBC playback through an in-app `WebviewWindow` backed by
`WKWebView` before launching an external browser.
The native path will:
- create a dedicated NBC auth/player window inside the app with a persistent webview data
directory
- keep Adobe Pass / MVPD interaction inside the native app window, including popup sign-in flows
- use native `WKWebView` JavaScript evaluation and snapshot APIs to drive readiness checks and
capture rendered frames for ffmpeg
The existing Chrome-backed path remains as a fallback when native playback cannot become ready.
## Motivation
The first NBC implementation proved that browser-backed capture is the right source model, but it
still pushes authentication into an external Chrome session. For the desktop app, that is a worse
operator experience than a native in-app window and makes session management less coherent.
Tauri 2 already exposes the platform webview and, on macOS, gives access to the underlying
`WKWebView`. That is enough to move the login and capture loop in-process without redesigning the
rest of the CMAF ladder or MoQ publish flow.
## Scope
In scope:
- macOS-only `WKWebView` session creation for NBC sources
- popup/new-window handling for MVPD login flows
- native JavaScript probing plus native snapshot capture for the existing frame pipeline
- automatic fallback to the existing Chrome path when native playback fails to get ready
Out of scope:
- removing the Chrome fallback entirely
- guaranteeing DRM parity with Chrome across every NBC playback variant
- audio capture changes beyond the current video-first path
## Alternatives Considered
- Keep the external Chrome-only path. Rejected because the native app can host the auth/player
surface directly on macOS.
- Replace the entire source implementation with custom CEF/Chromium embedding. Rejected because it
is materially heavier than using Tauri's existing native webview APIs.
- Use an in-app webview only for login, then continue playback in Chrome. Rejected for the first
pass because it still splits operator state across two browser stacks.
## Rollout / Reversibility
- The change is additive and scoped to NBC handling in the Tauri app on macOS.
- If native playback proves unreliable for a given stream/auth path, the app falls back to the
existing Chrome implementation automatically.
- Roll back by removing the native webview path and retaining the Chrome-backed implementation from
ECP-0094.

View file

@ -0,0 +1,53 @@
# ECP-0096: Public NBC Guide Before Auth
Status: Proposed
## Decision
Add a public NBC guide feed to the Tauri app so browseable NBC linear channels appear before any
Adobe or MVPD authentication happens.
The desktop app will:
- fetch the unauthenticated `friendship.nbc.com` live guide shelf seen in `intake/`
- materialize those rows as normal `StreamDescriptor` entries in the existing channel list
- route each discovered row to a stable `nbc.com/live?brand=...` URL so selecting a card enters
the existing NBC auth and playback path only when the user chooses to watch
## Motivation
The NBC capture already proved that playback is auth-gated, but the same trace also contains a
public guide layer with channel titles, current programs, branding, and stream access names. That
is enough to support browse-before-auth behavior without exposing entitlement or playout tokens.
This keeps discovery public while preserving the existing browser-backed auth boundary from
ECP-0094 and ECP-0095.
## Scope
In scope:
- unauthenticated NBC public guide fetches for channel browsing
- conversion of live guide rows into `nbc` stream descriptors and sources
- Tauri UI updates needed to surface `nbc` rows in the channel list
Out of scope:
- reverse-engineering or exposing entitlement responses
- direct manifest playback without the existing auth flow
- a full public sports-event catalog beyond the live channel guide in this cut
## Alternatives Considered
- Keep NBC URL-only. Rejected because it hides browseable public metadata that the trace already
exposes.
- Expose MediaTailor or token-verifier outputs in discovery. Rejected because those belong to the
authenticated playback path.
- Build a separate NBC-only discovery UI. Rejected because the existing channel list can host the
guide with less surface area.
## Rollout / Reversibility
- The change is additive to the desktop app and only affects discovery presentation.
- Roll back by removing the public guide fetch and the synthesized NBC discovery entries.
- Existing NBC auth and playback handling remains the same once a user chooses to watch.

View file

@ -0,0 +1,51 @@
# ECP-0097: Tauri Desktop Bundle Boot via Relative Assets and Custom Protocol
Status: Proposed
## Decision
Make the Tauri desktop frontend boot from bundled assets reliably by:
- building the Tauri UI with relative Trunk asset URLs
- exposing the standard `custom-protocol` cargo feature so direct Cargo runs can load bundled
assets instead of the dev server URL
- showing a minimal boot-status overlay until the Dioxus app mounts, so startup failures are no
longer silent blank windows
## Motivation
The desktop shell was starting as a blank window even though the frontend bundle itself rendered
correctly over HTTP. Two separate issues caused that failure mode:
- the Trunk bundle used root-absolute asset paths, which are fragile for desktop asset loading
- `cargo run` without the Tauri custom-protocol feature followed the dev-server path instead of the
bundled `dist/` assets
This made local desktop verification ambiguous and hid useful error information from operators.
## Scope
In scope:
- Tauri UI bundling for desktop
- direct Cargo run behavior for the desktop app
- a minimal visible frontend boot diagnostic
Out of scope:
- redesigning the viewer UI
- changing the web deployment path for non-Tauri surfaces
- full developer tooling around webview devtools
## Alternatives Considered
- Keep root-absolute Trunk paths and require `cargo tauri dev` only. Rejected because desktop
bundle verification remains fragile and blank-window failures remain opaque.
- Add only the boot overlay. Rejected because diagnostics help, but the bundle/load path still
needs correction.
## Rollout / Reversibility
- The change is local to the Tauri desktop app.
- Roll back by restoring the prior Trunk public URL and removing the package feature / boot
overlay.

View file

@ -0,0 +1,57 @@
# ECP-0098: NBC Native Auth Completion and Explicit Guide Program Labels
Status: Proposed
## Decision
Tighten the macOS-native NBC flow so Adobe Pass background-login completion in popup windows is
treated as a successful auth handoff, and expose explicit current-program metadata for NBC public
guide rows so the desktop channel list can render live program info directly.
The desktop app will:
- treat Adobe's `completeBackgroundLogin.html` popup landing as a native auth completion signal
- refocus and reload the main NBC player webview once that signal arrives so playback readiness can
continue in the same in-app session
- reload the main NBC player webview once when it is already back on a fully loaded NBC watch
surface but still has not materialized a video element
- persist the current live program as first-class guide metadata instead of relying only on the
generic `number` field
## Motivation
The first native `WKWebView` pass gets the user into Verizon login, but the Adobe handoff can end
inside a short-lived popup instead of automatically returning control to the main player window.
Without explicit completion handling, the native flow appears to stall after successful login.
Separately, NBC public discovery already carries current-program titles, but the desktop list was
rendering only the generic `number` slot. That makes guide rows look incomplete even when the
metadata is already available.
## Scope
In scope:
- native popup completion handling for Adobe background login on macOS
- readiness-loop updates needed to continue playback after popup completion or an already-authenticated return to the main watch surface
- explicit current-program metadata and guide-row subtitle rendering for NBC discovery rows
Out of scope:
- a broader redesign of the channel list for all source types
- removal of the Chrome NBC fallback path
- changes to NBC entitlement semantics beyond surfacing existing discovery metadata
## Alternatives Considered
- Keep waiting on the main player page only. Rejected because Adobe can finish inside a popup and
leave the main page idle.
- Continue storing NBC program info only in `StreamDescriptor.number`. Rejected because it obscures
intent and makes guide rendering brittle.
## Rollout / Reversibility
- The auth-completion change is additive to the native macOS NBC path and falls back to Chrome if
playback still does not become ready.
- The guide-label change is additive metadata plus UI presentation.
- Roll back by removing the popup completion hook and the explicit program-label rendering.

View file

@ -0,0 +1,50 @@
# ECP-0099: NBC Hidden Native Worker Mode Instead of True Headless Chrome
Status: Proposed
## Decision
Do not target true Chrome headless mode for NBC playback in `every.channel`.
Instead, add a macOS hidden native-worker mode for the existing `WKWebView` path so NBC playback
can run without a visible browser window when the session is already authenticated.
The desktop app will:
- allow the native NBC webview windows to start hidden behind `EVERY_CHANNEL_NBC_HIDE_WINDOWS=1`
- surface those hidden windows only when the navigation flow reaches an interactive auth page
- refuse the visible Chrome fallback when hidden native mode is explicitly requested
## Motivation
On this Mac, local Google Chrome in true headless mode does not expose EME / `requestMediaKeySystemAccess`,
which makes Widevine-backed NBC playback non-viable in a real headless browser process.
That means the practical path to future unattended playback is not “headless Chrome,” but an
invisible native browser surface with a warm authenticated session and explicit surfacing only
when auth has expired.
## Scope
In scope:
- macOS hidden-window behavior for the existing NBC `WKWebView` path
- auth-page detection that reveals the hidden window when user interaction is required
- blocking visible Chrome fallback when hidden mode is requested
Out of scope:
- automating Verizon credential entry in this cut
- making Chrome headless support Widevine / EME
- guaranteeing unattended runs when the MVPD session has expired
## Alternatives Considered
- Use true Chrome headless. Rejected because local testing showed no EME path.
- Keep only the visible native/Chrome flows. Rejected because a warm session should be able to run
without putting browser chrome on screen.
## Rollout / Reversibility
- The change is opt-in behind `EVERY_CHANNEL_NBC_HIDE_WINDOWS=1`.
- Roll back by removing the hidden-window flag and retaining the visible native flow from ECP-0098.

View file

@ -0,0 +1,51 @@
# ECP-0100: NBC Auth Bootstrap Command for Hidden Worker Sessions
Status: Proposed
## Decision
Add an explicit `bootstrap_nbc_auth` command to the Tauri app so operators can warm an NBC /
Adobe session ahead of hidden playback runs.
The command will:
- accept either an existing NBC stream id or a raw `nbc.com/watch/...` or `nbc.com/live?...` URL
- run the macOS native NBC path in hidden mode by default
- surface the auth window only when MVPD interaction is actually required
- close the bootstrap window after the session is ready so later hidden playback can reuse the
persisted session state
## Motivation
Hidden worker mode makes unattended playback plausible only when the session is already warm.
Without an explicit bootstrap surface, the operator has to guess which playback attempt should be
used to re-authenticate the session.
An explicit bootstrap command gives `every.channel` a clean operator primitive: warm auth now,
then let later hidden playback reuse that session without additional visible browser chrome.
## Scope
In scope:
- a Tauri backend command for NBC auth bootstrap
- a small operator affordance in the desktop UI to invoke that command
- reusing the same persisted native webview data directory as hidden playback
Out of scope:
- a full standalone CLI lifecycle for bootstrap without the Tauri app runtime
- storing Verizon credentials in the repo
- automating expired-auth recovery beyond surfacing the interactive window when needed
## Alternatives Considered
- Reuse ordinary playback attempts as implicit bootstrap. Rejected because it obscures operator
intent and complicates hidden-run automation.
- Build a separate bootstrap-only app. Rejected because the existing Tauri runtime already hosts
the relevant webview/session state.
## Rollout / Reversibility
- The command is additive.
- Roll back by removing the command and UI affordance while retaining ECP-0099 hidden native mode.