every.channel/evolution/proposals/ECP-0091-full-atsc-psip-parsing-and-real-sample-validation.md

ECP-0091: Full ATSC PSIP Parsing and Real-Sample Validation

Status: Implemented

Decision

Extend ec-ts from PAT-only plus partial PSIP awareness into table-level ATSC PSIP parsing for:

• MGT
• TVCT / CVCT
• STT
• RRT
• EIT
• ETT

Additional rules:

• EIT and ETT are parsed on their MGT-assigned PIDs, not only on base PID 0x1FFB.
• Raw descriptor bytes are preserved in parsed tables where we do not yet expose typed descriptor structs.
• ATSC STT is converted to Unix time using GPS epoch semantics instead of the earlier placeholder arithmetic.
• Real-data checks are added as ignored tests using external ATSC captures rather than vendoring large TS fixtures into the repo.

Motivation

ECP-0089 and ECP-0090 fixed the first discovery-identity hole by preferring broadcast metadata and then strengthening it with PAT probing. That still left two gaps:

• PSIP coverage was incomplete, so the parser could detect “this is ATSC” without understanding the rest of the broadcast metadata surface.
• Validation was synthetic-only, which is not enough for transport tables that are notoriously easy to parse incorrectly while still passing fixture tests.

We need both: fuller PSIP coverage and a rerunnable path against known real captures.

Scope

In scope:

• full table parsing for MGT, TVCT/CVCT, STT, RRT, EIT, and ETT
• multiple-string structure parsing with conservative text decoding for common uncompressed modes
• non-base PID handling for EIT / ETT
• corrected STT to Unix-time conversion
• ignored real-sample tests against tsduck-test ATSC captures

Out of scope:

• semantic parsing of all ATSC descriptor payloads
• DVB/ISDB parity in this change
• using RRT, EIT, or ETT directly in the default discovery key

Alternatives considered

• Keep partial PSIP parsing and rely on PAT + lineup hints only. Rejected because it leaves too much unverified broadcast metadata on the floor.
• Vendor large real TS captures into the repo. Rejected because it bloats the tree and makes review worse.
• Parse EIT / ETT only on 0x1FFB. Rejected because live ATSC streams carry them on the PIDs advertised by MGT.

Rollout / Reversibility

• Additive for existing callers: explicit --stream-id still overrides discovery identity.
• If needed, roll back by reverting the new table parsers while keeping PAT-based identity promotion.
• The ignored real-sample tests remain opt-in and do not make normal CI depend on external downloads.