-
Notifications
You must be signed in to change notification settings - Fork 5
Open
Labels
Description
Bug Description
When running a PreferenceDataset.generate()
Problem 1
Inconsistent expected number of rows printed vs logged.
Expected 78 preference pairs
2025-11-20 11:46:47 | INFO | datafast.datasets:generate - Starting PreferenceDataset.generate() | Expected rows: 234
Problem 2
When a validation error is thrown during llm.generate() the PreferenceDataset generation loop is broken. We need to catch, log, and gracefully continue