Skip to content

include scenarios from HuggingFace#77

Merged
jdsheehan merged 2 commits intomainfrom
issue73
Oct 10, 2025
Merged

include scenarios from HuggingFace#77
jdsheehan merged 2 commits intomainfrom
issue73

Conversation

@jdsheehan
Copy link
Collaborator

@jdsheehan jdsheehan commented Oct 8, 2025

Copy link
Member

@bradleyjeck bradleyjeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest having a different ScenarioSet for the groupings:

  • IoT
  • Work order
  • TSFM
  • FMSR
  • end-to-end

note: scenarios with no type are handled by aob.py

issue #73

Signed-off-by: John D Sheehan <john.d.sheehan@ie.ibm.com>
@jdsheehan
Copy link
Collaborator Author

looking at the scenario file on hf (https://huggingface.co/datasets/ibm-research/AssetOpsBench/blob/main/data/scenarios/all_utterance.jsonl) the types break down as follows,

{'no type': 20, 'IoT': 19, 'TSFM': 23, 'Workorder': 36, 'blank': 42}

aob/aob.py will handle no type and blank. aob_iot, aob_tsfm, and aob_workorder will handle iot, tsfm, and workorder

@jdsheehan jdsheehan merged commit 1576b3d into main Oct 10, 2025
1 check passed
@jdsheehan jdsheehan deleted the issue73 branch February 4, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants