fix: pass gts argument in _dump_generations call in _train_step

[yunkai-yyk]

Summary

Fixes the TypeError caused by missing gts argument when calling _dump_generations in _train_step with rollout_data_dir enabled.

Closes #492

Root Cause

_dump_generations requires gts as a positional argument, but the call site in _train_step did not pass it, resulting in a TypeError at runtime.

Changes

Extract ground_truth from each sample's non_tensor_batch and pass it as gts to _dump_generations:

sample_gts = [item.non_tensor_batch.get("reward_model", {}).get("ground_truth", None) for item in batch]
self._dump_generations(
    inputs=inputs,
    outputs=outputs,
    scores=scores,
    gts=sample_gts,
    reward_extra_infos_dict=reward_extra_infos_dict,
    dump_path=rollout_data_dir,
)

ground_truth is retrieved from reward_model in non_tensor_batch if available, otherwise falls back to None.

Testing

• Verified the fix resolves the TypeError when rollout_data_dir is set in trainer config
• Tested with cases where ground_truth is present and absent in non_tensor_batch

Environment

• Version: v0.3.0
• File: agentlightning/verl/trainer.py
• Class: RayPPOTrainer

[ultmaster]

I guess it doesn't harm. Let's run the tests.

[ultmaster]

/ci

[github-actions]

🚀 CI Watcher for correlation id-3976202555-mm5qiqg2 triggered by comment 3976202555
🏃‍♀️ Tracking 4 workflow run(s):

• 🔴 Calc-X - PR #493 - ci-calc-x - id-3976202555-mm5qiqg2 — completed/failure
• 🔴 RAG - PR #493 - ci-rag - id-3976202555-mm5qiqg2 — completed/failure
• 🟢 ChartQA - PR #493 - ci-chartqa - id-3976202555-mm5qiqg2 — completed/success
• 🔴 Spider - PR #493 - ci-spider - id-3976202555-mm5qiqg2 — completed/failure

✅ All runs completed.