Skip to content

feat(config): add checkpoint_on_improvement option to save only on new best#461

Open
mvanhorn wants to merge 1 commit intoalgorithmicsuperintelligence:mainfrom
mvanhorn:feat/434-checkpoint-on-improvement
Open

feat(config): add checkpoint_on_improvement option to save only on new best#461
mvanhorn wants to merge 1 commit intoalgorithmicsuperintelligence:mainfrom
mvanhorn:feat/434-checkpoint-on-improvement

Conversation

@mvanhorn
Copy link
Copy Markdown

@mvanhorn mvanhorn commented May 8, 2026

Summary

Optimization runs that span many iterations without improvement currently produce checkpoints anyway, since checkpoint_interval fires on a fixed cadence. This PR adds an opt-in checkpoint_on_improvement config flag that triggers the checkpoint callback when a new best program is found, in addition to the existing interval-based trigger.

Why this matters

  • Issue: #434 - "Add option to checkpoint only when improvement is detected" (label: enhancement, help wanted). Reporter notes that interval-only checkpointing wastes I/O when slow-progress runs save the same best program over and over.

Changes

  • openevolve/config.py:407 adds checkpoint_on_improvement: bool = False next to checkpoint_interval. Default preserves existing behavior.
  • openevolve/process_parallel.py:387 wires the new field into the worker config dict so worker processes see it.
  • openevolve/process_parallel.py:670 splits the trigger into interval_hit and improvement_hit and fires the checkpoint callback when either is true. Logs distinguish "Checkpoint interval reached" vs "Checkpointing new best solution".
  • tests/test_process_parallel.py:156 adds test_checkpoint_on_improvement_only_fires_for_new_best. Sets checkpoint_on_improvement=True and checkpoint_interval=10000 so only the improvement path can fire, then drives one new-best iteration and one not-best iteration. Asserts checkpoint_calls == [1].

Testing

Full test suite passes locally on Python 3.10 via uv:

Ran 371 tests in 66.808s
OK (skipped=1)

Default checkpoint_on_improvement=False keeps existing runs identical; the new path only activates when users opt in.

Fixes #434

AI was used for assistance.

…w best

Adds a new config flag, checkpoint_on_improvement, that triggers a
checkpoint callback whenever a new-best program is found, in addition
to the existing checkpoint_interval gate. Default False preserves
existing behavior.

When the optimization is making slow progress, checkpoint_interval
saves work that the next interval would overwrite without any new
best. This option lets users say 'only checkpoint when there's
actually something new to save.'

Wires the flag through the worker config dict (process_parallel.py:386)
and adds a unit test that verifies the callback fires for a new-best
run and not for a not-best run when checkpoint_interval is set high.

Fixes algorithmicsuperintelligence#434
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add option to checkpoint only when improvement is detected

2 participants