Skip to content

CI: fan out Azure pipelines on Python version (parallel matrix) - speedup x5 #5013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
GuillemSeCa wants to merge 1 commit into
base: develop
Choose a base branch
from
+46 −34
Draft

CI: fan out Azure pipelines on Python version (parallel matrix) - speedup x5 #5013

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 23 additions & 17 deletions etc/ci/azure-posix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,29 +11,35 @@ jobs:
pool:
vmImage: ${{ parameters.image_name }}

# Fan out on both Python version AND test suite so each
# (python_version, test_suite) combination runs as its own parallel
# job on its own agent VM. Previously python_versions were iterated
# as sequential steps inside a single job, so a failure on the first
# Python version skipped all subsequent ones.
strategy:
matrix:
${{ each tsuite in parameters.test_suites }}:
${{ tsuite.key }}:
test_suite_label: ${{ tsuite.key }}
test_suite: ${{ tsuite.value }}
${{ each pyver in parameters.python_versions }}:
${{ each tsuite in parameters.test_suites }}:
py${{ replace(pyver, '.', '_') }}_${{ tsuite.key }}:
python_version: ${{ pyver }}
test_suite_label: ${{ tsuite.key }}
test_suite: ${{ tsuite.value }}

steps:
- checkout: self
fetchDepth: 10

- ${{ each pyver in parameters.python_versions }}:
- task: UsePythonVersion@0
inputs:
versionSpec: '${{ pyver }}'
architecture: '${{ parameters.python_architecture }}'
displayName: '${{ pyver }} - Install Python'
- task: UsePythonVersion@0
inputs:
versionSpec: '$(python_version)'
architecture: '${{ parameters.python_architecture }}'
displayName: 'Install Python $(python_version)'

- script: |
python${{ pyver }} --version
echo "python${{ pyver }}" > PYTHON_EXECUTABLE
./configure --clean && ./configure --dev
displayName: '${{ pyver }} - Configure'
- script: |
python$(python_version) --version
echo "python$(python_version)" > PYTHON_EXECUTABLE
./configure --clean && ./configure --dev
displayName: 'Configure with Python $(python_version)'

- script: $(test_suite)
displayName: '${{ pyver }} - $(test_suite_label) on ${{ parameters.job_name }}'
- script: $(test_suite)
displayName: '$(test_suite_label) on ${{ parameters.job_name }} with Python $(python_version)'
40 changes: 23 additions & 17 deletions etc/ci/azure-win.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,29 +11,35 @@ jobs:
pool:
vmImage: ${{ parameters.image_name }}

# Fan out on both Python version AND test suite so each
# (python_version, test_suite) combination runs as its own parallel
# job on its own agent VM. Previously python_versions were iterated
# as sequential steps inside a single job, so a failure on the first
# Python version skipped all subsequent ones.
strategy:
matrix:
${{ each tsuite in parameters.test_suites }}:
${{ tsuite.key }}:
test_suite_label: ${{ tsuite.key }}
test_suite: ${{ tsuite.value }}
${{ each pyver in parameters.python_versions }}:
${{ each tsuite in parameters.test_suites }}:
py${{ replace(pyver, '.', '_') }}_${{ tsuite.key }}:
python_version: ${{ pyver }}
test_suite_label: ${{ tsuite.key }}
test_suite: ${{ tsuite.value }}

steps:
- checkout: self
fetchDepth: 10

- ${{ each pyver in parameters.python_versions }}:
- task: UsePythonVersion@0
inputs:
versionSpec: '${{ pyver }}'
architecture: '${{ parameters.python_architecture }}'
displayName: '${{ pyver }} - Install Python'
- task: UsePythonVersion@0
inputs:
versionSpec: '$(python_version)'
architecture: '${{ parameters.python_architecture }}'
displayName: 'Install Python $(python_version)'

- script: |
python --version
echo | set /p=python> PYTHON_EXECUTABLE
configure --clean && configure --dev
displayName: '${{ pyver }} - Configure'
- script: |
python --version
echo | set /p=python> PYTHON_EXECUTABLE
configure --clean && configure --dev
displayName: 'Configure with Python $(python_version)'

- script: $(test_suite)
displayName: '${{ pyver }} - $(test_suite_label) on ${{ parameters.job_name }}'
- script: $(test_suite)
displayName: '$(test_suite_label) on ${{ parameters.job_name }} with Python $(python_version)'
Loading