Skip to content

(re)naming#72

Merged
haideraltahan merged 5 commits into
mainfrom
(Re)naming
May 26, 2026
Merged

(re)naming#72
haideraltahan merged 5 commits into
mainfrom
(Re)naming

Conversation

@haideraltahan
Copy link
Copy Markdown
Collaborator

(Re)naming: Implementing proposed issue #61

oellmoellm-eval, schedule-evalschedule, collect-resultscollect

The package is currently named oellm (installed from the oellm-cli repo), and the binary is also oellm. Neither name communicates the tool's purpose. This PR renames:

  • package oellmoellm-eval
  • binary entrypoint oellmoellm-eval
  • subcommand schedule-evalschedule (avoids the redundant "eval eval")
  • subcommand collect-resultscollect

The benefit is a name that expresses intent and is less confusing, especially when the tool is shared with other projects.

Before / After

# Before
oellm schedule-eval --models "..." --task_groups "open-sci-0.01"
oellm collect-results --results_dir /path/to/output

# After
oellm-eval schedule --models "..." --task_groups "open-sci-0.01"
oellm-eval collect --results_dir /path/to/output

Copy link
Copy Markdown
Collaborator

@geoalgo geoalgo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for doing this! I have one comment about not changing the path, the rest looks good to go.

Comment thread README.md
- **Schedule evaluations** on multiple models and tasks: `oellm schedule-eval`
- **Collect results** and check for missing evaluations: `oellm collect-results`
- **Schedule evaluations** on multiple models and tasks: `oellm-eval schedule`
- **Collect results** and check for missing evaluations: `oellm-eval collect`
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit but we could leave collect-results as its orthogonal to the PR goal (that being said, collect is fine with me too)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this change, simplifies the commands, but no strong preference, I'm keeping it unless you prefer otherwise

Comment thread README.md Outdated
```

Results are written to `./oellm-output/<timestamp>/results/`.
Results are written to `./oellm-eval-output/<timestamp>/results/`.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok this one is actually going to be quite annoying. We are relying on the path logic for many of our tooling (database, visualisations, etc). Can we keep the path to oellm-output?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah for sure, can revert!

@haideraltahan
Copy link
Copy Markdown
Collaborator Author

@geoalgo PR ready for final review

@haideraltahan haideraltahan merged commit 1ed3e21 into main May 26, 2026
3 checks passed
@haideraltahan haideraltahan deleted the (Re)naming branch May 26, 2026 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants