feat: add mock VLM dataset and Gemma4 pretokenize support#1682

Merged

akoumpa merged 16 commits intomainfrom

huiyingl/vlm-mock-dataset

Apr 16, 2026

Commits on Apr 5, 2026

feat: add mock VLM dataset and Gemma4 pretokenize support

HuiyingLi
and
claude
committed
fix: rename test_mock.py to test_mock_vlm.py to avoid pytest collection conflict

HuiyingLi
and
claude
committed
fix: handle 1D tensor truncation and add truncate mode test

HuiyingLi
and
claude
committed
fix: correct mock patch paths for local imports in truncate test

HuiyingLi
and
claude
committed
fix: remove unreliable label-content assertion from truncate test

HuiyingLi
and
claude
committed
fix: remove PreTokenizedDatasetWrapper truncate test

HuiyingLi
and
claude
committed
fix: switch Gemma4 configs to eager attention and right-padding

HuiyingLi
and
claude
committed

Commits on Apr 8, 2026

fix: remove activation_checkpointing from mock configs to match originals

HuiyingLi
and
claude
committed

Commits on Apr 11, 2026

Commits on Apr 16, 2026