Use evaluation objects in place of files#195
Use evaluation objects in place of files#195tonyxty wants to merge 6 commits intoalgorithmicsuperintelligence:mainfrom
Conversation
|
@codelion What do you think? |
|
Can you update from main, maybe we should add an example that shows how to use it as a library with this evaluation object instead of file. Otherwise looks good. |
|
Added a variant of the online judge example. |
This example still seems to use |
|
It still has But the point is not merely combining files - in this example, it makes it possible for the user to pick the problem to solve at runtime, e.g., from the command line. To implement such functionality without evaluation objects, the program will have to modify the |
|
I guess a slightly more general way of explaining is that evaluation objects can have parameters stored in them, be it the problem to solve, timeout value, or something else. Meanwhile, if one wants to communicate a parameter to a evaluator file, one has to either modify its source - and it can easily get messy when there are multiple parameters; or have the evaluator module read from yet another config file, which adds unnecessary complexity. |
This PR is part of #191.
Add a
evaluation_objectparameter which, when present, will be used instead ofevaluation_filefor evaluation tasks.Example usage: in the online judge example, it would be easier to add a command line argument that enables the user to select their own favorite problem.
Note: if the
evaluation_objectmodifies itself in the evaluation functions, such modifications are not guaranteed to be visible in later calls due to the multi-process nature of the evaluation process.