Skip to content

Conversation

@cfsmp3
Copy link
Contributor

@cfsmp3 cfsmp3 commented Dec 8, 2025

Fix CEA-708 Decoder State Persistence (Issue #1499)

This PR fully implements the fix for issue #1499. The Rust CEA-708 decoder was creating a new Dtvcc struct on every call to ccxr_process_cc_data(), causing all state to be reset and breaking stateful caption processing.

Based on #1618, which doesn't merge cleanly and contains implementations that differ from current code.

Changes by Phase

Phase 1: Rust Core

  • Added DtvccRust struct in decoder/mod.rs that owns its decoder state
  • Added CCX_DTVCC_MAX_SERVICES constant (63)
  • Added FFI functions in lib.rs:
    • ccxr_dtvcc_init(): Create persistent context
    • ccxr_dtvcc_free(): Free context and all owned memory
    • ccxr_dtvcc_set_encoder(): Set encoder (not available at init)
    • ccxr_dtvcc_process_data(): Process CC data on persistent context
    • ccxr_flush_active_decoders(): Flush all active decoders
    • ccxr_dtvcc_is_active(): Check if context is active
  • Used heap allocation for large structs to avoid stack overflow
  • Added unit tests for DtvccRust

Phase 2: C Headers

  • Added void *dtvcc_rust field to lib_cc_decode struct in ccx_decoders_structs.h
  • Added extern declarations in ccx_dtvcc.h for init/free/process_data/is_active
  • Added extern declaration in lib_ccx.h for ccxr_dtvcc_set_encoder
  • Added extern declaration in ccx_decoders_common.h for ccxr_flush_active_decoders

Phase 3: C Implementation

  • Modified ccx_decoders_common.c:
    • init_cc_decode(): Use ccxr_dtvcc_init() when Rust enabled
    • dinit_cc_decode(): Use ccxr_dtvcc_free() when Rust enabled
    • flush_cc_decode(): Use ccxr_flush_active_decoders() when Rust enabled
  • Modified general_loop.c: Set encoder via ccxr_dtvcc_set_encoder() at 3 locations
  • Modified mp4.c: Use ccxr_dtvcc_set_encoder() and ccxr_dtvcc_process_data()
  • All changes guarded with #ifndef DISABLE_RUST

Phase 4: Bug Fix & Testing

  • Fixed ccxr_process_cc_data() to use persistent DtvccRust from dec_ctx.dtvcc_rust instead of creating new Dtvcc from dec_ctx.dtvcc (which is NULL when Rust enabled)
  • Added do_cb_dtvcc_rust() function for processing with DtvccRust

Testing

Automated Tests

  • All 269 Rust unit tests pass
  • Cargo clippy passes with no errors
  • CI builds pass on Linux, Mac, Windows

Manual Testing

  • Tested with CEA-708 transport stream file (ANDE.ts):
    • ~10 minute file with 25,598 DTVCC packets
    • Successfully extracted 21KB of captions
    • No crashes, proper SRT output with timestamps
  • Tested with CEA-708 program stream file:
    • Processed without crashes
    • Verified state persistence across calls

Files Modified

File Changes
src/rust/src/decoder/mod.rs Added DtvccRust struct and methods
src/rust/src/lib.rs Added FFI functions, fixed ccxr_process_cc_data
src/lib_ccx/ccx_decoders_structs.h Added dtvcc_rust field
src/lib_ccx/ccx_dtvcc.h Added extern declarations
src/lib_ccx/lib_ccx.h Added ccxr_dtvcc_set_encoder declaration
src/lib_ccx/ccx_decoders_common.h Added ccxr_flush_active_decoders declaration
src/lib_ccx/ccx_decoders_common.c Use Rust init/free/flush
src/lib_ccx/general_loop.c Set encoder via Rust FFI
src/lib_ccx/mp4.c Use Rust FFI for encoder and processing

Fixes: #1499

const CCX_DTVCC_MAX_COLUMNS: u8 = 32 * 2;

/// Maximum number of CEA-708 services
pub const CCX_DTVCC_MAX_SERVICES: usize = 63;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have this: pub const DTVCC_MAX_SERVICES: usize = 63;
CCExtractor/ccextractor/src/rust/lib_ccxr/src/common/constants.rs

@cfsmp3 cfsmp3 force-pushed the fix/1499-dtvcc-persistent-state branch from 18e2e32 to eb241d2 Compare December 12, 2025 16:39
cfsmp3 added a commit to cfsmp3/ccextractor that referenced this pull request Dec 12, 2025
The cb_708 counter was being incremented twice for each CEA-708 data block:
1. In do_cb_dtvcc_rust() in Rust (src/rust/src/lib.rs)
2. In do_cb() in C (src/lib_ccx/ccx_decoders_common.c)

Since FTS calculation uses cb_708 (fts = fts_now + fts_global + cb_708 * 1001 / 30),
the double-increment caused timestamps to advance ~2x as fast as expected,
resulting in incorrect milliseconds in start timestamps.

This fix removes the increment from the Rust code since the C code already
handles it in do_cb().

Fixes timestamp issues reported in PR CCExtractor#1782 tests where start times like
00:00:20,688 were incorrectly output as 00:00:20,737.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
cfsmp3 added a commit to cfsmp3/ccextractor that referenced this pull request Dec 13, 2025
The cb_708 counter was being incremented twice for each CEA-708 data block:
1. In do_cb_dtvcc_rust() in Rust (src/rust/src/lib.rs)
2. In do_cb() in C (src/lib_ccx/ccx_decoders_common.c)

Since FTS calculation uses cb_708 (fts = fts_now + fts_global + cb_708 * 1001 / 30),
the double-increment caused timestamps to advance ~2x as fast as expected,
resulting in incorrect milliseconds in start timestamps.

This fix removes the increment from the Rust code since the C code already
handles it in do_cb().

Fixes timestamp issues reported in PR CCExtractor#1782 tests where start times like
00:00:20,688 were incorrectly output as 00:00:20,737.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@cfsmp3 cfsmp3 force-pushed the fix/1499-dtvcc-persistent-state branch from abcfafe to 8ed6cc3 Compare December 13, 2025 01:38
cfsmp3 added a commit to cfsmp3/ccextractor that referenced this pull request Dec 13, 2025
The cb_708 counter was being incremented twice for each CEA-708 data block:
1. In do_cb_dtvcc_rust() in Rust (src/rust/src/lib.rs)
2. In do_cb() in C (src/lib_ccx/ccx_decoders_common.c)

Since FTS calculation uses cb_708 (fts = fts_now + fts_global + cb_708 * 1001 / 30),
the double-increment caused timestamps to advance ~2x as fast as expected,
resulting in incorrect milliseconds in start timestamps.

This fix removes the increment from the Rust code since the C code already
handles it in do_cb().

Fixes timestamp issues reported in PR CCExtractor#1782 tests where start times like
00:00:20,688 were incorrectly output as 00:00:20,737.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@cfsmp3 cfsmp3 force-pushed the fix/1499-dtvcc-persistent-state branch from 8ed6cc3 to 4ff33fc Compare December 13, 2025 05:34
@ccextractor-bot
Copy link
Collaborator

CCExtractor CI platform finished running the test files on linux. Below is a summary of the test results, when compared to test for commit 77e1dff...:
Report Name Tests Passed
Broken 12/13
CEA-708 14/14
DVB 7/7
DVD 3/3
DVR-MS 2/2
General 22/27
Hardsubx 1/1
Hauppage 3/3
MP4 3/3
NoCC 10/10
Options 86/86
Teletext 21/21
WTV 13/13
XDS 32/34

NOTE: The following tests have been failing on the master branch as well as the PR:

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --autoprogram --out=srt --latin1 1d9731bd80..., Last passed: Never
  • ccextractor --out=srt --latin1 --autoprogram 73d9313d64..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 8e8229b88b..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 7236304cfc..., Last passed: Never
  • ccextractor --out=ttxt --latin1 001dd8cdf7..., Last passed: Never
  • ccextractor --out=srt --latin1 06b3a9237d..., Last passed: Never
  • ccextractor --out=srt --latin1 83f8cceb74..., Last passed: Never
  • ccextractor --out=srt --latin1 611b4a9235..., Last passed: Never
  • ccextractor --out=srt --latin1 b46e9e8e3f..., Last passed: Never
  • ccextractor --out=srt --latin1 89e417e622..., Last passed: Never
  • ccextractor --out=srt --latin1 d59eadc4ed..., Last passed: Never
  • ccextractor --out=srt --latin1 4d4e938ef6..., Last passed: Never
  • ccextractor --service 1 --out=txt --no-bom --no-rollup ea83ff7bcb..., Last passed: Never
  • ccextractor --service 1 --out=txt f17524b53f..., Last passed: Never
  • ccextractor --service 1 --out=txt da904de35d..., Last passed: Never
  • ccextractor --service 1 --out=txt 80848c45f8..., Last passed: Never
  • ccextractor --service 1 --out=txt --no-bom --no-rollup b5d6aad89f..., Last passed: Never
  • ccextractor --service 1[EUC-KR] --out=txt --no-rollup b5d6aad89f..., Last passed: Never
  • ccextractor --service 1 --out=srt da904de35d..., Last passed: Never
  • ccextractor --service 1 --out=sami da904de35d..., Last passed: Never
  • ccextractor --service 1 --out=ttxt da904de35d..., Last passed: Never
  • ccextractor --service 1[EUC-KR] b5d6aad89f..., Last passed: Never
  • ccextractor --service 1[EUC-KR] --no-rollup b5d6aad89f..., Last passed: Never
  • ccextractor --service all da904de35d..., Last passed: Never
  • ccextractor --service all[EUC-KR] b5d6aad89f..., Last passed: Never
  • ccextractor --service 1,2[UTF-8],3[EUC-KR],54 --out=txt da904de35d..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 1020459a86..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 f1422b8bfe..., Last passed: Never
  • ccextractor --datapid 5603 --autoprogram --out=srt --latin1 --teletext 85c7fc1ad7..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 d41b53b504..., Last passed: Never
  • ccextractor --stdout --quiet --no-fontcolor 79a51f3500..., Last passed: Never
  • ccextractor --stdout --quiet --no-fontcolor 767b546f96..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 5ae2007a79..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 1e44efd810..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 add511677c..., Last passed: Never
  • ccextractor --wtvconvertfix --autoprogram --out=srt --latin1 acf871cbfd..., Last passed: Never
  • ccextractor --wtvconvertfix --autoprogram --out=srt --latin1 5cbb21adb6..., Last passed: Never
  • ccextractor --out=srt --latin1 --autoprogram 56c9f34548..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 e9b9008fdf..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 c032183ef0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 27e46255f0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 d037c7509e..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 1974a299f0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 132d7df7e9..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla c41f73056a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 70000200c0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 6dc772d881..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla adce82fd39..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 15feae9133..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 95dd33c6f1..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 01509e4d27..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla ab9cf8cfad..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 15feae9133..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --output-field 2 5d3a29f9f8..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --output-field 2 c41f73056a..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --sentencecap c032183ef0..., Last passed: Never
  • ccextractor --autoprogram --out=bin --latin1 c032183ef0..., Last passed: Never
  • ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b..., Last passed: Never
  • ccextractor --hardsubx 1a0302f7fd..., Last passed: Never
  • ccextractor --hauppauge --autoprogram --out=srt --latin1 a03b5b2a56..., Last passed: Never
  • ccextractor --autoprogram --out=srt --hauppauge --latin1 553d78e755..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --hauppauge --ucla --latin1 553d78e755..., Last passed: Never
  • ccextractor --in=mp4 --out=srt --latin1 b2771c84c2..., Last passed: Never
  • ccextractor --in=mp4 --out=srt --latin1 5df914ce77..., Last passed: Never
  • ccextractor --autoprogram --out=srt --bom --latin1 8849331dda..., Last passed: Never
  • ccextractor --mp4vidtrack --autoprogram --out=ttxt --latin1 adc0a818c3..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 08bdf0e2c1..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 bee139671a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 3842d00925..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 80af83c038..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 837b02f722..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 41dab6b2a7..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 4b117b4d66..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 26ee6add4d..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds fca0dce412..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --output-field 1 a65d39ccb3..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --output-field 2 a65d39ccb3..., Last passed: Never
  • ccextractor --autoprogram c83f765c66..., Last passed: Never
  • ccextractor --service 1 c83f765c66..., Last passed: Never
  • ccextractor --in=ts c83f765c66..., Last passed: Never
  • ccextractor --out=srt c83f765c66..., Last passed: Never
  • ccextractor --out=sami c83f765c66..., Last passed: Never
  • ccextractor --out=dvdraw c83f765c66..., Last passed: Never
  • ccextractor --out=txt c83f765c66..., Last passed: Never
  • ccextractor --out=ttxt c83f765c66..., Last passed: Never
  • ccextractor --out=smptett c83f765c66..., Last passed: Never
  • ccextractor --out=spupng c83f765c66..., Last passed: Never
  • ccextractor --goptime c83f765c66..., Last passed: Never
  • ccextractor --no-goptime c83f765c66..., Last passed: Never
  • ccextractor --fixpadding c83f765c66..., Last passed: Never
  • ccextractor --90090 c83f765c66..., Last passed: Never
  • ccextractor --myth c83f765c66..., Last passed: Never
  • ccextractor --program-number 1 c83f765c66..., Last passed: Never
  • ccextractor --datapid 256 c83f765c66..., Last passed: Never
  • ccextractor --datastreamtype 2 c83f765c66..., Last passed: Never
  • ccextractor --datastreamtype 2 --streamtype 2 c83f765c66..., Last passed: Never
  • ccextractor --no-autotimeref c83f765c66..., Last passed: Never
  • ccextractor --bom c83f765c66..., Last passed: Never
  • ccextractor --no-bom c83f765c66..., Last passed: Never
  • ccextractor --unicode c83f765c66..., Last passed: Never
  • ccextractor --utf8 c83f765c66..., Last passed: Never
  • ccextractor --latin1 c83f765c66..., Last passed: Never
  • ccextractor --no-fontcolor c83f765c66..., Last passed: Never
  • ccextractor --no-typesetting c83f765c66..., Last passed: Never
  • ccextractor --trim c83f765c66..., Last passed: Never
  • ccextractor --sentencecap c83f765c66..., Last passed: Never
  • ccextractor --capfile /repository/Dictionary/MattS_dictionary.txt c83f765c66..., Last passed: Never
  • ccextractor --unixts 5 --out=txt c83f765c66..., Last passed: Never
  • ccextractor --out=txt --datets c83f765c66..., Last passed: Never
  • ccextractor --out=txt --sects c83f765c66..., Last passed: Never
  • ccextractor --out=txt --lf c83f765c66..., Last passed: Never
  • ccextractor --autodash --trim c83f765c66..., Last passed: Never
  • ccextractor --bufferinput c83f765c66..., Last passed: Never
  • ccextractor --no-bufferinput c83f765c66..., Last passed: Never
  • ccextractor --buffersize 1M c83f765c66..., Last passed: Never
  • ccextractor --dru c83f765c66..., Last passed: Never
  • ccextractor --no-rollup c83f765c66..., Last passed: Never
  • ccextractor --ru1 c83f765c66..., Last passed: Never
  • ccextractor --ru2 c83f765c66..., Last passed: Never
  • ccextractor --ru3 c83f765c66..., Last passed: Never
  • ccextractor --delay 200 c83f765c66..., Last passed: Never
  • ccextractor --startat 4 --endat 7 c83f765c66..., Last passed: Never
  • ccextractor --no-codec dvbsub c83f765c66..., Last passed: Never
  • ccextractor --debug --out=srt c83f765c66..., Last passed: Never
  • ccextractor --608 --out=srt c83f765c66..., Last passed: Never
  • ccextractor --708 --out=srt c83f765c66..., Last passed: Never
  • ccextractor --goppts --out=srt c83f765c66..., Last passed: Never
  • ccextractor --xdsdebug --out=srt c83f765c66..., Last passed: Never
  • ccextractor --vides --out=srt c83f765c66..., Last passed: Never
  • ccextractor --cbraw --out=srt c83f765c66..., Last passed: Never
  • ccextractor --no-sync --out=srt c83f765c66..., Last passed: Never
  • ccextractor --fullbin --out=srt c83f765c66..., Last passed: Never
  • ccextractor --parsedebug --out=srt c83f765c66..., Last passed: Never
  • ccextractor --parsePAT --out=srt c83f765c66..., Last passed: Never
  • ccextractor --parsePMT --out=srt c83f765c66..., Last passed: Never
  • ccextractor --investigate-packets --out=srt c83f765c66..., Last passed: Never
  • ccextractor --in=ps e9b9008fdf..., Last passed: Never
  • ccextractor --in=es dc7169d7c4..., Last passed: Never
  • ccextractor --in=asf 6395b281ad..., Last passed: Never
  • ccextractor --in=wtv b46e9e8e3f..., Last passed: Never
  • ccextractor --in=bin 988d4e8bba..., Last passed: Never
  • ccextractor --in=raw fb79021542..., Last passed: Never
  • ccextractor --in=mp4 b2771c84c2..., Last passed: Never
  • ccextractor --mp4vidtrack 5df914ce77..., Last passed: Never
  • ccextractor --wtvconvertfix acf871cbfd..., Last passed: Never
  • ccextractor --wtvmpeg2 10f0f77cf4..., Last passed: Never
  • ccextractor --hauppauge d6df1b227a..., Last passed: Never
  • ccextractor --codec dvbsub --out=spupng 85271be4d2..., Last passed: Never
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --endcreditstext "CCextractor Ends crdit Testing" addf5e2fc9..., Last passed: Never
  • ccextractor --endcreditsforatleast 3 --endcreditstext "CCextractor Ends crdit Testing" addf5e2fc9..., Last passed: Never
  • ccextractor --endcreditsforatmost 2 --endcreditstext "CCextractor Ends crdit Testing" addf5e2fc9..., Last passed: Never
  • ccextractor --tpage 801 4e56e88ba4..., Last passed: Never
  • ccextractor --tverbose 4e56e88ba4..., Last passed: Never
  • ccextractor --teletext 4e56e88ba4..., Last passed: Never
  • ccextractor --out=txt --ucla c83f765c66..., Last passed: Never
  • ccextractor --xmltv=3 --out=null 96efd279cf..., Last passed: Never
  • ccextractor --datapid 2310 --autoprogram --out=srt --latin1 e639e54550..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 4e56e88ba4..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 c0d2fba8c0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 006fdc391a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 e92a1d4d2a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 b37ce60eb9..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 7e4ebf7fd7..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 9256a60e4b..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 27d7a43dd6..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 297a44921a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 efbe129086..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 eae0077731..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 e2e2b501e0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 8c1615c1a8..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 c6407fb294..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --datets dcada745de..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --tpage 398 5d5838bde9..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --tpage 299 44c45593fb..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --tpage 299 b8c55aa2e9..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --teletext --tpage 398 3b276ad8bf..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 b236a0590b..., Last passed: Never
  • ccextractor --out=srt --latin1 f23a544ba8..., Last passed: Never
  • ccextractor --out=srt --latin1 97cc394d87..., Last passed: Never
  • ccextractor --out=srt --latin1 10f0f77cf4..., Last passed: Never
  • ccextractor --out=srt --latin1 df3b4d62d3..., Last passed: Never
  • ccextractor --out=srt --latin1 d7e7dbdf68..., Last passed: Never
  • ccextractor --out=srt --latin1 76734ac4a7..., Last passed: Never
  • ccextractor --out=srt --latin1 c791382c94..., Last passed: Never
  • ccextractor --out=srt --latin1 f673b2f916..., Last passed: Never
  • ccextractor --out=srt --latin1 da75bdee47..., Last passed: Never
  • ccextractor --out=srt --latin1 bd6f33a669..., Last passed: Never
  • ccextractor --out=srt --latin1 0e5e6b26be..., Last passed: Never
  • ccextractor --out=srt --latin1 a226cc302d..., Last passed: Never
  • ccextractor --out=srt --latin1 ae6327683e..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 725a49f871..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla d037c7509e..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla d037c7509e..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla e274a73653..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla 85058ad37e..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds b22260d065..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --xds --ucla c813e713a0..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla c813e713a0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 27fab4dbb6..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 27fab4dbb6..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds bbd5bb52fc..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla bbd5bb52fc..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds b992e0cccb..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds d0291cdcf6..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 7d2730d38e..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 7d2730d38e..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds c8dc039a88..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla c8dc039a88..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 53339f3455..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 53339f3455..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 83b03036a2..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 83b03036a2..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 7d3f25c32c..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 7d3f25c32c..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds f41d4c29a1..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla f41d4c29a1..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 88cd42b89a..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 88cd42b89a..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --output-field 2 --ucla 88cd42b89a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 7f41299cc7..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 7f41299cc7..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 0069dffd21..., Last passed: Never

All tests passing on the master branch were passed completely.

Check the result page for more info.

cfsmp3 and others added 9 commits December 14, 2025 11:19
…se 1)

This is Phase 1 of the fix for issue CCExtractor#1499. It adds the Rust-side
infrastructure for a persistent CEA-708 decoder context without
modifying any C code, ensuring backward compatibility.

Problem:
The current Rust CEA-708 decoder creates a new Dtvcc struct on every
call to ccxr_process_cc_data(), causing all state to be reset. This
breaks stateful caption processing.

Solution:
Add a new DtvccRust struct that:
- Owns its decoder state (rather than borrowing from C)
- Persists across processing calls
- Is managed via FFI functions callable from C

Changes:
- Add DtvccRust struct in decoder/mod.rs with owned decoders
- Add CCX_DTVCC_MAX_SERVICES constant (63)
- Add FFI functions in lib.rs:
  - ccxr_dtvcc_init(): Create persistent context
  - ccxr_dtvcc_free(): Free context and all owned memory
  - ccxr_dtvcc_set_encoder(): Set encoder (not available at init)
  - ccxr_dtvcc_process_data(): Process CC data
  - ccxr_flush_active_decoders(): Flush all active decoders
  - ccxr_dtvcc_is_active(): Check if context is active
- Add unit tests for DtvccRust
- Use heap allocation for large structs to avoid stack overflow

The existing Dtvcc struct and ccxr_process_cc_data() remain unchanged
for backward compatibility. Phase 2-3 will add C header declarations
and modify C code to use the new functions.

Fixes: CCExtractor#1499 (partial)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove duplicate CCX_DTVCC_MAX_SERVICES constant from decoder/mod.rs
- Import existing DTVCC_MAX_SERVICES from lib_ccxr::common
- Fix clippy uninlined_format_args warnings in avc/core.rs and decoder/mod.rs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add void *dtvcc_rust field to lib_cc_decode struct
- Declare ccxr_dtvcc_init, ccxr_dtvcc_free, ccxr_dtvcc_process_data in ccx_dtvcc.h
- Declare ccxr_dtvcc_set_encoder in lib_ccx.h
- Declare ccxr_flush_active_decoders in ccx_decoders_common.h
- All declarations guarded with #ifndef DISABLE_RUST
- Update implementation plan to mark Phase 2 complete

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- init_cc_decode(): Initialize dtvcc_rust via ccxr_dtvcc_init()
- dinit_cc_decode(): Free dtvcc_rust via ccxr_dtvcc_free()
- flush_cc_decode(): Flush via ccxr_flush_active_decoders()
- general_loop.c: Set encoder via ccxr_dtvcc_set_encoder() (3 locations)
- mp4.c: Use ccxr_dtvcc_set_encoder() and ccxr_dtvcc_process_data()
- Add ccxr_dtvcc_is_active() declaration to ccx_dtvcc.h
- Fix clippy warnings in tv_screen.rs (unused assignments)
- All changes guarded with #ifndef DISABLE_RUST
- Update implementation plan to mark Phase 3 complete

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove extra space before comment in ccx_decoders_common.c
- Fix comment indentation in mp4.c

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The ccxr_process_cc_data function was still accessing dec_ctx.dtvcc
(which is NULL when Rust is enabled), causing a null pointer panic.

Changed to use dec_ctx.dtvcc_rust (the persistent DtvccRust context)
instead, which fixes the crash when processing CEA-708 data.

Added do_cb_dtvcc_rust() function that works with DtvccRust instead
of the old Dtvcc struct.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move PLAN_PR1618_REIMPLEMENTATION.md to local plans/ folder
- Add plans/ to .gitignore to keep plans local

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The cb_708 counter was being incremented twice for each CEA-708 data block:
1. In do_cb_dtvcc_rust() in Rust (src/rust/src/lib.rs)
2. In do_cb() in C (src/lib_ccx/ccx_decoders_common.c)

Since FTS calculation uses cb_708 (fts = fts_now + fts_global + cb_708 * 1001 / 30),
the double-increment caused timestamps to advance ~2x as fast as expected,
resulting in incorrect milliseconds in start timestamps.

This fix removes the increment from the Rust code since the C code already
handles it in do_cb().

Fixes timestamp issues reported in PR CCExtractor#1782 tests where start times like
00:00:20,688 were incorrectly output as 00:00:20,737.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@cfsmp3 cfsmp3 force-pushed the fix/1499-dtvcc-persistent-state branch from 4ff33fc to cf74762 Compare December 14, 2025 10:21
@ccextractor-bot
Copy link
Collaborator

CCExtractor CI platform finished running the test files on windows. Below is a summary of the test results, when compared to test for commit 134cd75...:
Report Name Tests Passed
Broken 4/13
CEA-708 14/14
DVB 6/7
DVD 0/3
DVR-MS 2/2
General 2/27
Hardsubx 1/1
Hauppage 0/3
MP4 3/3
NoCC 10/10
Options 72/86
Teletext 21/21
WTV 0/13
XDS 21/34

Your PR breaks these cases:

NOTE: The following tests have been failing on the master branch as well as the PR:

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --autoprogram --out=srt --latin1 f1422b8bfe..., Last passed: Never
  • ccextractor --datapid 5603 --autoprogram --out=srt --latin1 --teletext 85c7fc1ad7..., Last passed: Never
  • ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b..., Last passed: Never
  • ccextractor --hardsubx 1a0302f7fd..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --output-field 1 a65d39ccb3..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --output-field 2 a65d39ccb3..., Last passed: Never
  • ccextractor --autoprogram c83f765c66..., Last passed: Never
  • ccextractor --in=ts c83f765c66..., Last passed: Never
  • ccextractor --out=srt c83f765c66..., Last passed: Never
  • ccextractor --out=sami c83f765c66..., Last passed: Never
  • ccextractor --out=txt c83f765c66..., Last passed: Never
  • ccextractor --out=smptett c83f765c66..., Last passed: Never
  • ccextractor --goptime c83f765c66..., Last passed: Never
  • ccextractor --no-goptime c83f765c66..., Last passed: Never
  • ccextractor --fixpadding c83f765c66..., Last passed: Never
  • ccextractor --90090 c83f765c66..., Last passed: Never
  • ccextractor --program-number 1 c83f765c66..., Last passed: Never
  • ccextractor --datapid 256 c83f765c66..., Last passed: Never
  • ccextractor --datastreamtype 2 c83f765c66..., Last passed: Never
  • ccextractor --datastreamtype 2 --streamtype 2 c83f765c66..., Last passed: Never
  • ccextractor --no-autotimeref c83f765c66..., Last passed: Never
  • ccextractor --bom c83f765c66..., Last passed: Never
  • ccextractor --no-bom c83f765c66..., Last passed: Never
  • ccextractor --unicode c83f765c66..., Last passed: Never
  • ccextractor --utf8 c83f765c66..., Last passed: Never
  • ccextractor --latin1 c83f765c66..., Last passed: Never
  • ccextractor --no-fontcolor c83f765c66..., Last passed: Never
  • ccextractor --no-typesetting c83f765c66..., Last passed: Never
  • ccextractor --trim c83f765c66..., Last passed: Never
  • ccextractor --sentencecap c83f765c66..., Last passed: Never
  • ccextractor --capfile /repository/Dictionary/MattS_dictionary.txt c83f765c66..., Last passed: Never
  • ccextractor --unixts 5 --out=txt c83f765c66..., Last passed: Never
  • ccextractor --out=txt --datets c83f765c66..., Last passed: Never
  • ccextractor --out=txt --sects c83f765c66..., Last passed: Never
  • ccextractor --out=txt --lf c83f765c66..., Last passed: Never
  • ccextractor --autodash --trim c83f765c66..., Last passed: Never
  • ccextractor --bufferinput c83f765c66..., Last passed: Never
  • ccextractor --no-bufferinput c83f765c66..., Last passed: Never
  • ccextractor --buffersize 1M c83f765c66..., Last passed: Never
  • ccextractor --dru c83f765c66..., Last passed: Never
  • ccextractor --no-rollup c83f765c66..., Last passed: Never
  • ccextractor --ru1 c83f765c66..., Last passed: Never
  • ccextractor --ru2 c83f765c66..., Last passed: Never
  • ccextractor --ru3 c83f765c66..., Last passed: Never
  • ccextractor --delay 200 c83f765c66..., Last passed: Never
  • ccextractor --startat 4 --endat 7 c83f765c66..., Last passed: Never
  • ccextractor --no-codec dvbsub c83f765c66..., Last passed: Never
  • ccextractor --debug --out=srt c83f765c66..., Last passed: Never
  • ccextractor --608 --out=srt c83f765c66..., Last passed: Never
  • ccextractor --708 --out=srt c83f765c66..., Last passed: Never
  • ccextractor --goppts --out=srt c83f765c66..., Last passed: Never
  • ccextractor --xdsdebug --out=srt c83f765c66..., Last passed: Never
  • ccextractor --vides --out=srt c83f765c66..., Last passed: Never
  • ccextractor --cbraw --out=srt c83f765c66..., Last passed: Never
  • ccextractor --no-sync --out=srt c83f765c66..., Last passed: Never
  • ccextractor --fullbin --out=srt c83f765c66..., Last passed: Never
  • ccextractor --parsedebug --out=srt c83f765c66..., Last passed: Never
  • ccextractor --parsePAT --out=srt c83f765c66..., Last passed: Never
  • ccextractor --investigate-packets --out=srt c83f765c66..., Last passed: Never
  • ccextractor --in=ps e9b9008fdf..., Last passed: Never
  • ccextractor --in=asf 6395b281ad..., Last passed: Never
  • ccextractor --in=mp4 b2771c84c2..., Last passed: Never
  • ccextractor --wtvconvertfix acf871cbfd..., Last passed: Never
  • ccextractor --hauppauge d6df1b227a..., Last passed: Never
  • ccextractor --out=txt --ucla c83f765c66..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 c0d2fba8c0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 006fdc391a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 e92a1d4d2a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 7e4ebf7fd7..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 9256a60e4b..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 27d7a43dd6..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 297a44921a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 efbe129086..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 eae0077731..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 e2e2b501e0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 c6407fb294..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --datets dcada745de..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --tpage 398 5d5838bde9..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --teletext --tpage 398 3b276ad8bf..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla d037c7509e..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla d037c7509e..., Last passed: Never
  • ccextractor --autoprogram --out=smptett --latin1 --ucla e274a73653..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla b22260d065..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla c813e713a0..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla c8dc039a88..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 88cd42b89a..., Last passed: Never

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Initialize data structures correctly for the rust CEA-708 decoder

3 participants