Skip to content

Get parallelism working when building wheels #1033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Strilanc wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Get parallelism working when building wheels #1033

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
5add95b
Drop prebuilt wheels for python 3.8 and 3.9
Strilanc Jan 20, 2026
53b00fb
getconf _NPROCESSORS_ONLN
Strilanc Jan 20, 2026
b6717ff
try a custom build
Strilanc Jan 20, 2026
9be5701
moved
Strilanc Jan 20, 2026
ea2304e
respect wheel directory
Strilanc Jan 20, 2026
956447f
return
Strilanc Jan 20, 2026
d07a10a
Try forcing path
Strilanc Jan 20, 2026
0d5df11
prepool
Strilanc Jan 20, 2026
328e607
initializer
Strilanc Jan 20, 2026
3fa7141
tuple
Strilanc Jan 20, 2026
7c67b5f
flatter
Strilanc Jan 20, 2026
3086f1a
typo
Strilanc Jan 20, 2026
73dfd4f
try windows
Strilanc Jan 20, 2026
7e33f04
.......windows why
Strilanc Jan 20, 2026
08d149b
darwin flags
Strilanc Jan 20, 2026
e64002e
colon
Strilanc Jan 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 7 additions & 43 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,13 @@ jobs:
fail-fast: false
matrix:
os_dist: [
{os: ubuntu-24.04, dist: cp38-manylinux_x86_64},
{os: ubuntu-24.04, dist: cp39-manylinux_x86_64},
{os: ubuntu-24.04, dist: cp310-manylinux_x86_64},
{os: ubuntu-24.04, dist: cp311-manylinux_x86_64},
{os: ubuntu-24.04, dist: cp312-manylinux_x86_64},
{os: ubuntu-24.04, dist: cp313-manylinux_x86_64},
{os: ubuntu-24.04, dist: cp314-manylinux_x86_64},

# cp38-manylinux_i686 disabled because pandas isn't prebuilt and takes 20 minutes to build.
# {os: ubuntu-latest, dist: cp38-manylinux_i686},
# cp39-manylinux_i686 disabled because pandas isn't prebuilt and takes 20 minutes to build.
# {os: ubuntu-latest, dist: cp39-manylinux_i686},
# cp310-manylinux_i686 disabled because scipy isn't prebuilt and fails to build.
# manylinux_i686 disabled because scipy isn't prebuilt and fails to build.
#
# The actual error seen in github actions:
#
Expand All @@ -59,11 +53,7 @@ jobs:
# libraries found. To build Scipy from sources, BLAS & LAPACK
# libraries need to be installed.
#
# {os: ubuntu-latest, dist: pp37-manylinux_x86_64},
# {os: ubuntu-latest, dist: pp38-manylinux_x86_64},
# {os: ubuntu-latest, dist: pp39-manylinux_x86_64},
# {os: ubuntu-latest, dist: pp37-manylinux_i686},
# {os: ubuntu-latest, dist: pp38-manylinux_i686},
# {os: ubuntu-latest, dist: pp39-manylinux_i686},

# musllinux builds disabled because scipy isn't prebuilt and fails to build.
Expand All @@ -74,27 +64,15 @@ jobs:
# libraries found. To build Scipy from sources, BLAS & LAPACK
# libraries need to be installed.
#
# {os: ubuntu-latest, dist: cp36-musllinux_x86_64},
# {os: ubuntu-latest, dist: cp37-musllinux_x86_64},
# {os: ubuntu-latest, dist: cp38-musllinux_x86_64},
# {os: ubuntu-latest, dist: cp39-musllinux_x86_64},
# {os: ubuntu-latest, dist: cp310-musllinux_x86_64},
# {os: ubuntu-latest, dist: cp36-musllinux_i686},
# {os: ubuntu-latest, dist: cp37-musllinux_i686},
# {os: ubuntu-latest, dist: cp38-musllinux_i686},
# {os: ubuntu-latest, dist: cp39-musllinux_i686},
# {os: ubuntu-latest, dist: cp310-musllinux_i686},

{os: macos-14, dist: cp38-macosx_x86_64, macosarch: x86_64},
{os: macos-14, dist: cp39-macosx_x86_64, macosarch: x86_64},
{os: macos-14, dist: cp310-macosx_x86_64, macosarch: x86_64},
{os: macos-14, dist: cp311-macosx_x86_64, macosarch: x86_64},
{os: macos-14, dist: cp312-macosx_x86_64, macosarch: x86_64},
{os: macos-14, dist: cp313-macosx_x86_64, macosarch: x86_64},
{os: macos-14, dist: cp314-macosx_x86_64, macosarch: x86_64},

{os: macos-14, dist: cp38-macosx_arm64, macosarch: arm64},
{os: macos-14, dist: cp39-macosx_arm64, macosarch: arm64},
{os: macos-14, dist: cp310-macosx_arm64, macosarch: arm64},
{os: macos-14, dist: cp311-macosx_arm64, macosarch: arm64},
{os: macos-14, dist: cp312-macosx_arm64, macosarch: arm64},
Expand All @@ -112,28 +90,14 @@ jobs:
# BLAS and LAPACK by setting the environment variables
# NPY_BLAS_ORDER="" and NPY_LAPACK_ORDER="" before building NumPy.
#
# {os: macOS-10.15, dist: pp37-macosx_x86_64},
# {os: macOS-10.15, dist: pp38-macosx_x86_64},
# {os: macOS-10.15, dist: pp39-macosx_x86_64},

{os: windows-2025, dist: cp38-win_amd64},
{os: windows-2025, dist: cp39-win_amd64},
{os: windows-2025, dist: cp310-win_amd64},
{os: windows-2025, dist: cp311-win_amd64},
{os: windows-2025, dist: cp312-win_amd64},
{os: windows-2025, dist: cp313-win_amd64},
{os: windows-2025, dist: cp314-win_amd64},

# cp38-win32 and cp39-win32 disabled because scipy fails to build.
#
# The actual error seen in github actions:
#
# Need python for 64-bit, but found 32-bit
# ..\..\meson.build:82:0: ERROR: Python dependency not found
#
#{os: windows-2025, dist: cp38-win32},
#{os: windows-2025, dist: cp39-win32},

# cp310-win32 disabled because numpy isn't prebuilt and fails to build.
#
# The actual error seen in github actions:
Expand Down Expand Up @@ -254,7 +218,7 @@ jobs:
steps:
- uses: actions/checkout@50fbc622fc4ef5163becd7fab6573eac35f8462e # v1
- run: cmake .
- run: make stim -j 2
- run: make stim -j $(getconf _NPROCESSORS_ONLN)
- run: echo -e "H 0 \n CNOT 0 1 \n M 0 1" | out/stim --sample
build_bazel:
runs-on: ubuntu-24.04
Expand Down Expand Up @@ -324,7 +288,7 @@ jobs:
steps:
- uses: actions/checkout@50fbc622fc4ef5163becd7fab6573eac35f8462e # v1
- run: cmake .
- run: make libstim -j 2
- run: make libstim -j $(getconf _NPROCESSORS_ONLN)
- run: echo -e '#include "stim.h"\nint main(int argc,const char **argv) {return !stim::find_bool_argument("test", argc, argv);}' > test.cc
- run: g++ -std=c++20 test.cc out/libstim.a -I src
- run: ./a.out test
Expand All @@ -334,7 +298,7 @@ jobs:
- uses: actions/checkout@50fbc622fc4ef5163becd7fab6573eac35f8462e # v1
- run: mkdir install_dir
- run: cmake . -DCMAKE_INSTALL_PREFIX=install_dir
- run: make -j 2
- run: make -j $(getconf _NPROCESSORS_ONLN)
- run: make install
- run: echo -e '#include "stim.h"\nint main(int argc,const char **argv) {return !stim::find_bool_argument("test", argc, argv);}' > test.cc
- run: g++ -std=c++20 test.cc install_dir/lib/libstim.a -I install_dir/include
Expand All @@ -356,7 +320,7 @@ jobs:
steps:
- uses: actions/checkout@50fbc622fc4ef5163becd7fab6573eac35f8462e # v1
- run: cmake . -DSIMD_WIDTH=${{ matrix.simd_width }}
- run: make stim_perf -j 2
- run: make stim_perf -j $(getconf _NPROCESSORS_ONLN)
- run: out/stim_perf
test:
runs-on: ubuntu-24.04
Expand All @@ -373,7 +337,7 @@ jobs:
make
sudo make install
- run: cmake . -DSIMD_WIDTH=${{ matrix.simd_width }}
- run: make stim_test -j 2
- run: make stim_test -j $(getconf _NPROCESSORS_ONLN)
- run: out/stim_test
test_o3:
runs-on: ubuntu-24.04
Expand All @@ -387,7 +351,7 @@ jobs:
make
sudo make install
- run: cmake . -DSIMD_WIDTH=256
- run: make stim_test_o3 -j 2
- run: make stim_test_o3 -j $(getconf _NPROCESSORS_ONLN)
- run: out/stim_test_o3
test_generated_docs_are_fresh:
runs-on: ubuntu-24.04
Expand Down
2 changes: 1 addition & 1 deletion dev/clean_build_files.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
set -e

#########################################################################
# Deletes files created by cmake, python setup.py, and other build steps.
# Deletes files created by cmake, pip install ., and other build steps.
#########################################################################

# Get to this script's git repo root.
Expand Down
4 changes: 2 additions & 2 deletions dev/overwrite_dev_versions_with_date.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def main():
# Generate dev version starting from major.minor version.
# (Requires the existing version to have a 'dev' suffix.)
# (Uses the timestamp of the HEAD commit, to ensure consistency when run multiple times.)
with open('setup.py') as f:
with open('glue/python/src/stim_custom_setup/__init__.py') as f:
maj_min_version_line, = [line for line in f.read().splitlines() if re.match("^__version__ = '[^']+'", line)]
maj_version, min_version, patch = maj_min_version_line.split()[-1].strip("'").split('.')
if 'dev' not in patch:
Expand All @@ -33,7 +33,7 @@ def main():

# Overwrite existing versions.
package_setup_files = [
"setup.py",
"glue/python/src/stim_custom_setup/__init__.py",
"glue/cirq/setup.py",
"glue/cirq/stimcirq/__init__.py",
"glue/zx/stimzx/__init__.py",
Expand Down
Loading
Loading