every.channel/evolution/proposals/ECP-0091-full-atsc-psip-parsing-and-real-sample-validation.md

67 lines
2.5 KiB
Markdown

# ECP-0091: Full ATSC PSIP Parsing and Real-Sample Validation
Status: Implemented
## Decision
Extend `ec-ts` from PAT-only plus partial PSIP awareness into table-level ATSC PSIP parsing for:
- `MGT`
- `TVCT` / `CVCT`
- `STT`
- `RRT`
- `EIT`
- `ETT`
Additional rules:
- `EIT` and `ETT` are parsed on their MGT-assigned PIDs, not only on base PID `0x1FFB`.
- Raw descriptor bytes are preserved in parsed tables where we do not yet expose typed descriptor
structs.
- ATSC STT is converted to Unix time using GPS epoch semantics instead of the earlier placeholder
arithmetic.
- Real-data checks are added as ignored tests using external ATSC captures rather than vendoring
large TS fixtures into the repo.
## Motivation
ECP-0089 and ECP-0090 fixed the first discovery-identity hole by preferring broadcast metadata and
then strengthening it with PAT probing. That still left two gaps:
- PSIP coverage was incomplete, so the parser could detect “this is ATSC” without understanding the
rest of the broadcast metadata surface.
- Validation was synthetic-only, which is not enough for transport tables that are notoriously easy
to parse incorrectly while still passing fixture tests.
We need both: fuller PSIP coverage and a rerunnable path against known real captures.
## Scope
In scope:
- full table parsing for `MGT`, `TVCT/CVCT`, `STT`, `RRT`, `EIT`, and `ETT`
- multiple-string structure parsing with conservative text decoding for common uncompressed modes
- non-base PID handling for `EIT` / `ETT`
- corrected STT to Unix-time conversion
- ignored real-sample tests against `tsduck-test` ATSC captures
Out of scope:
- semantic parsing of all ATSC descriptor payloads
- DVB/ISDB parity in this change
- using `RRT`, `EIT`, or `ETT` directly in the default discovery key
## Alternatives considered
- Keep partial PSIP parsing and rely on PAT + lineup hints only. Rejected because it leaves too much
unverified broadcast metadata on the floor.
- Vendor large real TS captures into the repo. Rejected because it bloats the tree and makes review
worse.
- Parse `EIT` / `ETT` only on `0x1FFB`. Rejected because live ATSC streams carry them on the PIDs
advertised by `MGT`.
## Rollout / Reversibility
- Additive for existing callers: explicit `--stream-id` still overrides discovery identity.
- If needed, roll back by reverting the new table parsers while keeping PAT-based identity promotion.
- The ignored real-sample tests remain opt-in and do not make normal CI depend on external downloads.