`Compute Worker` Use docker-py instead of the current subprocess way of doing things (for podman and docker)#2065

Open

ObadaS wants to merge 39 commits intodevelopfrom

compute_workerDockerwithPython

Commits on Dec 23, 2025

Using docker-py and podman-py instead of the current subprocess way of doing things
Obada Haddad
committed
Removed useless line
Obada Haddad
committed
Started work on Podman; re-added log stream to codabench instance
Obada Haddad
committed
Corrected typo in variable name; small changes for image pulls for podman
Obada Haddad
committed
remove the podman package and use only docker API calls for both podman and docker containers
Obada Haddad
committed
add better progress bar when downloading and some debug logger output
Obada Haddad
committed
add github workflow
Obada Haddad
committed
workflow fixes
Obada Haddad
committed
format with ruff; add cleanup
Obada Haddad
committed
format with ruff; fix websockets connection infinite waiting
Obada Haddad
committed
change Dockerfile.compute_worker to remove docker; make GPU selection compatible with podman and docker
Obada Haddad
committed
fix a typo
Obada Haddad
committed
ruff formatting; update documentation
Obada Haddad
committed
delete useless workflow
Obada Haddad
committed
update documentation, remove useless files, add docker in pyproject for pytests
Obada Haddad
committed
try to fix pytest errors
Obada Haddad
committed
fix typo
Obada Haddad
committed
remove unused folder for podman
Obada Haddad
committed
add more logs for the compute worker; update documentation
Obada Haddad
committed
fix poetry lock after rebase
Obada Haddad
committed
rebase + remove failing tests. We can test this in e2e tests
Obada Haddad
committed
fix flake8 errors
Obada Haddad
committed
fix poetry lock after rebase
Obada Haddad
committed
remove poetry at the end to fix vulnerabilities and make image lighter
Obada Haddad
committed
remove docker and rich from main pyproject.toml
Obada Haddad
committed
fix missing f
Obada Haddad
committed
add timeout for websocket connections
Obada Haddad
committed
add more information about which container engine we are using and if GPU will be used
Obada Haddad
committed
tentative fix for send_detailed_results websocket connection attempt
Obada Haddad
committed
make error reporting better; handle websocket conenctions failures better
Obada Haddad
committed
fix hostname inside celery worker to use container hostname
Obada Haddad
committed
fix compute worker startup in compute worker service since we use entrypoint in Dockerfile.compute worker
Obada Haddad
committed
add workflow to build and push compute worker image automatically on file changes for every branch
Obada Haddad
committed
change Dockerfile.compute_worker and test workflow
Obada Haddad
committed
fix missing checkout in new workflows
Obada Haddad
committed
fix missing needs: to make the push job wait for the build to finish first
Obada Haddad
committed
further fixes in workflow
Obada Haddad
committed
rebase branch and update documentation about the new docker image workflow
Obada Haddad
committed

Commits on Dec 31, 2025

update documentation to be more clear
Obada Haddad
committed