(re)naming#72
Conversation
geoalgo
left a comment
There was a problem hiding this comment.
LGTM, thanks for doing this! I have one comment about not changing the path, the rest looks good to go.
| - **Schedule evaluations** on multiple models and tasks: `oellm schedule-eval` | ||
| - **Collect results** and check for missing evaluations: `oellm collect-results` | ||
| - **Schedule evaluations** on multiple models and tasks: `oellm-eval schedule` | ||
| - **Collect results** and check for missing evaluations: `oellm-eval collect` |
There was a problem hiding this comment.
Nit but we could leave collect-results as its orthogonal to the PR goal (that being said, collect is fine with me too)
There was a problem hiding this comment.
I like this change, simplifies the commands, but no strong preference, I'm keeping it unless you prefer otherwise
| ``` | ||
|
|
||
| Results are written to `./oellm-output/<timestamp>/results/`. | ||
| Results are written to `./oellm-eval-output/<timestamp>/results/`. |
There was a problem hiding this comment.
Ok this one is actually going to be quite annoying. We are relying on the path logic for many of our tooling (database, visualisations, etc). Can we keep the path to oellm-output?
There was a problem hiding this comment.
Yeah for sure, can revert!
|
@geoalgo PR ready for final review |
(Re)naming: Implementing proposed issue #61
oellm→oellm-eval,schedule-eval→schedule,collect-results→collectThe package is currently named
oellm(installed from theoellm-clirepo), and the binary is alsooellm. Neither name communicates the tool's purpose. This PR renames:oellm→oellm-evaloellm→oellm-evalschedule-eval→schedule(avoids the redundant "eval eval")collect-results→collectThe benefit is a name that expresses intent and is less confusing, especially when the tool is shared with other projects.
Before / After