ci: enable linux_amd64_musl extension build#87
Open
joseph-isaacs wants to merge 5 commits into
Open
Conversation
Remove linux_amd64_musl from exclude_archs now that vortex builds for x86_64-unknown-linux-musl in its own CI. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
linux_amd64_musl is marked opt_in in extension-ci-tools' distribution_matrix.json, so dropping it from exclude_archs alone does not build it — the matrix generator skips opt_in archs unless they are also listed in opt_in_archs. Add the opt_in_archs input so the musl extension build is actually generated and runs in CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
The linux_amd64_musl extension build links the vortex-duckdb staticlib into the loadable extension shared object. Rust's x86_64-unknown-linux-musl target defaults to a static CRT and non-PIC codegen, which fails at link time with relocation R_X86_64_32 against a shared object. Pass -Ctarget-feature=-crt-static and -Crelocation-model=pic for the crate, scoped to musl via the loader path so glibc/macOS builds are unaffected. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
The linux_amd64_musl build fails in custom-labels' build script: Unable to find libclang: "... could not be opened: Dynamic loading not supported" custom-labels (a mandatory dep of vortex-duckdb/vortex-io) runs bindgen, which dlopen()s libclang. In the Alpine/musl image the host is musl and Rust defaults crt-static=on, so the build-script executable is static and cannot dlopen. The prior -crt-static RUSTFLAGS (added via corrosion) only affects the target artifacts under --target, not host build scripts. Add a .cargo/config.toml disabling crt-static for the musl triple. Under --target, host build scripts read [target.<host-triple>].rustflags from config (via target-applies-to-host), so this makes them dynamically linked and able to dlopen libclang. Only consulted for musl builds; glibc/macOS/ Windows are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
The previous attempt set -crt-static/-relocation-model=pic via corrosion's
RUSTFLAGS env. That has two problems on the musl build:
- under --target, env RUSTFLAGS only applies to the target artifacts, so the
host build scripts (compiled for the musl host triple) stayed statically
linked. custom-labels' build script runs bindgen, which dlopen()s libclang,
and a static binary cannot dlopen -> "Dynamic loading not supported".
- when RUSTFLAGS env is set, cargo ignores [target.*].rustflags from config.
Move the flags into .cargo/config.toml [target.x86_64-unknown-linux-musl]
and drop the corrosion RUSTFLAGS. With no env RUSTFLAGS, the config table
applies to both the target crate (fixing the relocation/PIC error) and host
build scripts (target-applies-to-host), making the latter dynamically linked
so bindgen can dlopen libclang. Scoped to the musl triple; other platforms
unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Enables the
linux_amd64_muslbuild in CI so thevortexextension is built and tested against musl libc (Alpine/static).Changes
.github/workflows/MainDistributionPipeline.yml: Removelinux_amd64_muslfromexclude_archs. This turns on the musl target in the reusable_extension_distribution.ymlpipeline, which builds the extension inside an Alpine/musl container. Thevortex-duckdbRust crate is compiled (via corrosion) forx86_64-unknown-linux-musl.Verification
CI on this PR now includes a
linux_amd64_musljob. Since the musl/Alpine build can't be reproduced on the glibc Linux CI sandbox locally, the green CI run on this PR is the verification. I'll watch the PR and fix any musl-specific build failures that surface.https://claude.ai/code/session_01HJ7A8P4YMVgJ64bZyqpJHV
Generated by Claude Code