[Fix](nvbug6304585): specdec README online base-model example should use Instruct model

[h-guo18]

What does this PR do?

Type of change: Bug fix

The Training Draft Model with Online/Offline Base Model examples in
examples/speculative_decoding/README.md used meta-llama/Llama-3.2-1B, a
base / pretrained checkpoint that ships no chat template. The online EAGLE3
flow tokenizes conversations through tokenizer.apply_chat_template(...), so
the data collator fails fast at startup:

ValueError: No valid chat template!

This PR:

• Switches both README example commands (online and offline) to
  meta-llama/Llama-3.2-1B-Instruct, which carries a chat template.
• Makes the collator error message in
  modelopt/torch/utils/plugins/transformers_dataset.py actionable — it now
  explains the cause (base checkpoints have no chat template) and points users
  at an Instruct model or a custom chat_template.

Usage

./launch_train.sh \
    --config ../../modelopt_recipes/general/speculative_decoding/eagle3.yaml \
    model.model_name_or_path=meta-llama/Llama-3.2-1B-Instruct \
    data.data_path=input_conversations/train.jsonl \
    training.output_dir=ckpts/llama-3.2-1b-online

Testing

• Reproduced the original No valid chat template! failure with the base
  Llama-3.2-1B and confirmed the Instruct variant carries a chat template.
• Verified the new error message renders correctly when a template is missing.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ✅
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: N/A
• Did you update Changelog?: N/A
• Did you get Claude approval on this PR?: ❌

Additional Information

Fixes nvbug 6304585: https://nvbugspro.nvidia.com/bug/6304585

Summary by CodeRabbit

Release Notes

• Documentation

  • Updated speculative decoding example training commands to reference the Llama-3.2-1B-Instruct model.

• Bug Fixes

  • Enhanced error message when chat template configuration is missing, providing actionable guidance for resolution.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a19eff1e-c47d-4a22-b838-12c2487e230d

📥 Commits

Reviewing files that changed from the base of the PR and between f8b0754 and 3188a2c.

📒 Files selected for processing (2)

• examples/speculative_decoding/README.md
• modelopt/torch/utils/plugins/transformers_dataset.py

---

📝 Walkthrough

Walkthrough

Two small coordinated fixes: the speculative decoding README replaces the base Llama-3.2-1B model identifier with Llama-3.2-1B-Instruct in both online and offline training command examples, and LanguageDataCollator.__init__ replaces a terse ValueError with a detailed message explaining why base checkpoints lack a chat template and how to fix it.

Changes

Instruct Model Alignment

Layer / File(s)	Summary
Improved chat_template error guidance modelopt/torch/utils/plugins/transformers_dataset.py	The ValueError raised when tokenizer.chat_template is None now explains that base/pretrained checkpoints typically lack a chat template and instructs users to switch to an instruction-tuned model or pass a custom chat_template.
README examples updated to Instruct model examples/speculative_decoding/README.md	Both the online base model and dumped hidden states training command examples replace meta-llama/Llama-3.2-1B with meta-llama/Llama-3.2-1B-Instruct for the model.model_name_or_path override.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Suggested reviewers

• ChenhanYu
• Fridah-nv

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: updating the speculative decoding README to use the Instruct model variant instead of the base model.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns	✅ Passed	PR contains no security anti-patterns. Only changes are: README.md model reference updates and an improved error message in transformers_dataset.py with no security implications.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch haoguo/specdec-readme-chat-template

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[h-guo18]

/claude review

[claude]

Claude review passed — no blocking issues found. LGTM

Findings: CRITICAL: 0, IMPORTANT: 0, SUGGESTION: 0

Reviewed both changed files end-to-end:

• examples/speculative_decoding/README.md — Online and offline example commands now use meta-llama/Llama-3.2-1B-Instruct, consistent with the other 3 references to the model already in the README (lines 64, 276, 307). Correctly addresses the No valid chat template! startup failure since the base Llama-3.2-1B checkpoint ships no chat template.

• modelopt/torch/utils/plugins/transformers_dataset.py — The expanded error message is actionable and, importantly, both remediation paths it cites are real: data.chat_template=<path> is threaded into the collator via eagle_utils.py (data_args.chat_template → LanguageDataCollator(chat_template=...)), and chat_template is a genuine LanguageDataCollator.__init__ parameter. No misleading guidance. The string change is non-breaking (no callers pattern-match on the message).

No algorithm, mode/state, export, backward-compat, or performance concerns — the change touches only documentation text and an error string. Risk level: minimal.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-1755/
Built to branch gh-pages at 2026-06-16 19:25 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.53%. Comparing base (1cccf66) to head (3188a2c).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1755       +/-   ##
===========================================
+ Coverage   58.45%   76.53%   +18.07%     
===========================================
  Files         510      511        +1     
  Lines       56274    56342       +68     
===========================================
+ Hits        32896    43120    +10224     
+ Misses      23378    13222    -10156     

Flag	Coverage Δ
examples	41.80% <ø> (+19.36%)	⬆️
gpu	57.77% <ø> (+37.17%)	⬆️
regression	14.70% <ø> (+0.06%)	⬆️
unit	54.35% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.