Skip to content

[BUG] - PreferenceDataset #136

@patrickfleith

Description

@patrickfleith

Bug Description

When running a PreferenceDataset.generate()

Problem 1
Inconsistent expected number of rows printed vs logged.

Expected 78 preference pairs
2025-11-20 11:46:47 | INFO     | datafast.datasets:generate - Starting PreferenceDataset.generate() | Expected rows: 234

Problem 2
When a validation error is thrown during llm.generate() the PreferenceDataset generation loop is broken. We need to catch, log, and gracefully continue

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions