every.channel: sanitized baseline

evolution/proposals/ECP-0023-hls-ingest.md

@@ -0,0 +1,54 @@
+# ECP-0023: HLS ingest pipeline
+
+Status: Draft
+
+## Problem / context
+
+Some broadcasters and community relays expose HLS playlists directly. If two peers ingest the same HLS origin and the bytes are identical, we can dedupe and swarm immediately. We need a first-class ingest path for HLS that plugs into the deterministic chunking pipeline.
+
+## Decision
+
+Add HLS as a supported ingest source. The ingest path:
+
+- Uses ffmpeg/libav demuxing (no custom parsers).
+- Normalizes input into our deterministic chunker.
+- Treats identical origin bytes as a fast path for dedupe; if bytes vary, re-encode deterministically.
+
+## Details
+
+### Ingest modes
+
+- **Pass-through**: If the HLS origin serves MPEG-TS segments that are already deterministic and stable, chunk the byte stream as-is.
+- **Normalize**: If the origin uses fMP4 or inconsistent segmenting, remux via ffmpeg into a stable TS stream before chunking.
+- **Deterministic re-encode**: If byte identity isn’t stable (e.g. ad insertion, timestamps), re-encode with the deterministic profile.
+
+Initial implementation note: use ffmpeg (CLI) to output a continuous MPEG-TS stream which is then chunked by the ac-ffmpeg path. This keeps parsing inside ffmpeg and avoids custom HLS logic.
+
+### Encryption
+
+- If the HLS stream is encrypted with clear AES-128 and a key is provided, the ingest pipeline can decrypt and then re-encrypt using our stream encryption (stream id + chunk index + optional network secret).
+- DRM (Widevine/PlayReady) is out of scope for now; ingest will skip or mark as protected.
+- Our chunk encryption remains independent of HLS origin encryption.
+
+### Identifiers
+
+- HLS sources map to a `StreamKey` with `source.kind = "hls"` and `source.channel = <url-or-name>`.
+- `StreamId` is derived as usual from `StreamKey`.
+
+## Alternatives considered
+
+- Directly serving HLS playlists to peers: rejected (pushes HLS semantics to every client).
+- Custom HLS parser: rejected (ffmpeg already solves it).
+
+## Rollout / teardown plan
+
+1. Add `HlsSource` to `ec-node` ingest CLI and `ec-core` source typing.
+2. Add ffmpeg/libav HLS ingest path to `ec-chopper`.
+3. Wire HLS ingest into MoQ publisher and manifest pipeline.
+
+Teardown: disable HLS ingest and fall back to HDHR/IPTV sources.
+
+## Open questions
+
+- Which HLS modes should default to remux vs re-encode?
+- Should HLS playlist metadata (program info) be carried into stream metadata?