Skip to content

Commit 9700afd

Browse files
update TODO
1 parent dd34cee commit 9700afd

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

TODO.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@ Instead of asking the users to specify the evaluation budgets, installed package
3838

3939
All configs within the code are dicts. This can get quite confusing and hard to read. We should implement dataclasses with these configs and defaults. This is conceptually easy, but would require a major refactor of the code.
4040

41+
### Better dependency handling for problems
42+
43+
Each benchmark problem should have its own dependencies, and they should be installed whenever the problem is first run. Currently, we bundle all dependencies into CodeEvolve itself, which isn't great.
44+
4145
## Unit tests
4246

4347
We currently only have a really simple test suite for the SEARCH/REPLACE operator. We need to vastly increase this: from basic tests of our classes, to more complicated simulated runs with the MOCK LLM setting.

0 commit comments

Comments
 (0)