Skip to content

Remove examples/diffusers/eval example#1694

Open
jingyu-ml wants to merge 1 commit into
mainfrom
remove-diffusers-eval-example
Open

Remove examples/diffusers/eval example#1694
jingyu-ml wants to merge 1 commit into
mainfrom
remove-diffusers-eval-example

Conversation

@jingyu-ml

@jingyu-ml jingyu-ml commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: Documentation / example removal

Removes the unmaintained examples/diffusers/eval/ image-quality evaluation example (ImageReward / CLIP-IQA / CLIP metrics), which is no longer maintained. Also cleans up the now-dangling references to it in examples/diffusers/README.md:

  • the intro sentence clause about evaluation scripts,
  • the table-of-contents "Evaluate Accuracy" row,
  • the per-subsection requirements.txt note (example updated from eval to fastgen, distillation),
  • the entire "Evaluate Accuracy" section.

Scope is intentionally limited to the diffusers example. The unrelated examples/llm_ptq "Evaluate Accuracy" section is not touched.

Usage

N/A — this removes example scripts; there are no library API changes.

Testing

N/A — deletion of example scripts plus documentation cleanup. Verified that examples/diffusers/README.md has no remaining references to the deleted eval/ directory, no broken section anchors, no trailing whitespace, and the final newline is intact.

Before your PR is "Ready for review"

  • Is this change backward compatible?: ✅ — only removes example scripts under examples/diffusers/eval/; no public modelopt API is affected.
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A — no code copied; this removes the example's own requirements.txt.
  • Did you write any new necessary tests?: N/A — removal only.
  • Did you update Changelog?: N/A — example/doc removal, not a library feature or API change.
  • Did you get Claude approval on this PR?: N/A — trivial example/doc removal.

Additional Information

Deletes examples/diffusers/eval/ (5 files) and trims its references from the diffusers README.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Documentation

    • Updated Diffusers Model Optimizations documentation to remove evaluation-related references.
  • Chores

    • Removed evaluation scripts and metric computation functionality from Diffusers examples.

The diffusers image-quality eval scripts (ImageReward/CLIP-IQA/CLIP) are no longer maintained. Delete examples/diffusers/eval/ and drop the now-dangling "Evaluate Accuracy" references from examples/diffusers/README.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jingyu Xin <jingyux@nvidia.com>
@jingyu-ml jingyu-ml requested review from a team as code owners June 11, 2026 23:01
@jingyu-ml jingyu-ml requested a review from kevalmorabia97 June 11, 2026 23:01
@copy-pr-bot

copy-pr-bot Bot commented Jun 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2ff75991-575a-4504-8748-cfcf4d8a03d5

📥 Commits

Reviewing files that changed from the base of the PR and between dd49a46 and f352498.

📒 Files selected for processing (6)
  • examples/diffusers/README.md
  • examples/diffusers/eval/main.py
  • examples/diffusers/eval/metrics/imagereward.py
  • examples/diffusers/eval/metrics/multimodal.py
  • examples/diffusers/eval/requirements.txt
  • examples/diffusers/eval/utils.py
💤 Files with no reviewable changes (5)
  • examples/diffusers/eval/utils.py
  • examples/diffusers/eval/metrics/imagereward.py
  • examples/diffusers/eval/main.py
  • examples/diffusers/eval/metrics/multimodal.py
  • examples/diffusers/eval/requirements.txt

📝 Walkthrough

Walkthrough

The Diffusers example documentation is updated to remove evaluation feature references. The README intro sentence, navigation table, and installation guidance are modified to eliminate mentions of evaluation scripts and metrics. The full "Evaluate Accuracy" section is then deleted entirely from the document.

Changes

Diffusers evaluation documentation removal

Layer / File(s) Summary
Documentation navigation and guidance updates
examples/diffusers/README.md
Top-level description removes the clause about providing evaluation scripts; the "Evaluate Accuracy" link is removed from the centered navigation table; and the Local Installation note is updated to reference subsections like fastgen and distillation instead of eval.
Evaluate Accuracy section deletion
examples/diffusers/README.md
The entire "## Evaluate Accuracy" section—including requirements installation instructions, JSON input format specification, evaluation command usage, and sample results table—is deleted from the document.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Remove examples/diffusers/eval example' directly and accurately describes the primary change: deletion of the examples/diffusers/eval directory and related references.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed Diff vs origin/main for PR changes shows only README edits and deletions under examples/diffusers/eval; searched diff contains none of torch.load(weights_only=False), numpy.load(allow_pickle=True),...

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch remove-diffusers-eval-example

Comment @coderabbitai help to get the list of available commands and usage tips.

@jingyu-ml jingyu-ml self-assigned this Jun 11, 2026
@jingyu-ml jingyu-ml added the cherry-pick-0.45.0 After code freeze, cherry-pick to release branch for next rc (bulk update). Only for bug fixes / doc label Jun 11, 2026
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.24%. Comparing base (dd49a46) to head (f352498).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1694      +/-   ##
==========================================
+ Coverage   67.72%   68.24%   +0.52%     
==========================================
  Files         511      511              
  Lines       56168    56168              
==========================================
+ Hits        38037    38333     +296     
+ Misses      18131    17835     -296     
Flag Coverage Δ
examples 41.81% <ø> (+0.50%) ⬆️
unit 54.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jingyu-ml

Copy link
Copy Markdown
Contributor Author

/ok to test f352498

@kevalmorabia97

Copy link
Copy Markdown
Collaborator

@jingyu-ml please update Changelog to mention removal of this example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-pick-0.45.0 After code freeze, cherry-pick to release branch for next rc (bulk update). Only for bug fixes / doc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants