Skip to content

Fix mac-os build#487

Open
RamSaw wants to merge 29 commits intoOpenMined:devfrom
RamSaw:fix_mac_os_build
Open

Fix mac-os build#487
RamSaw wants to merge 29 commits intoOpenMined:devfrom
RamSaw:fix_mac_os_build

Conversation

@RamSaw
Copy link
Contributor

@RamSaw RamSaw commented Mar 20, 2026

Fix macOS Build: Resolving ABI Dynamic Lookup Collision

This pull request resolves the persistent ImportError / symbol not found failures that exclusively affected the macOS GitHub Actions CI runners. It ensures that the PyDP macOS python wheels reliably and natively invoke C++ bindings exactly as they do locally and on Linux/Windows target platforms.

The Root Cause

Through our debugging, we discovered a fatal collision caused by GitHub Actions' macos-latest environment. The previous CI workflow instructed macOS to eagerly install Protobuf system-wide via brew install protobuf.
However, Homebrew implicitly installs Abseil globally as dynamic libraries (.dylib).

When compiling PyDP normally, the native CMakeLists.txt uses FetchContent to safely download and build google_dp and abseil-cpp statically from scratch (.a archives). But due to Apple's linker behavior within Pybind11's flat namespace evaluation, Apple's ld aggressively encountered the globally injected Homebrew dynamic Abseil signatures on the host system, stripping the expected internal symbols from our extensions under the assumption that they would organically launch at runtime.
Naturally, because these libraries do explicitly not exist dynamically mapped into pip-installed wheels on user machines, it cleanly broke importing PyDP!

The Fix

The implementation zeroes out this "Bermuda Triangle" natively and elegantly without having to compromise PyDP CMake configurations:

  • Eradicate Host System Poisoning: Deleted brew install protobuf for macOS structurally from .github/workflows/versions.yml (and synchronized with publish.yml Attempt to fix publish #488 separately). Because PyDP utilizes FetchContent to fully build the components from exact source code securely, global host installations are obsolete and actively harmful to cross-compilation!
  • Target Scoping: Removed --plat-name macosx_10_14_x86_64 hardcoding, leveraging native GitHub Actions arm64 macos-latest (M1) architecture workflows naturally.
  • Modernization & Deprecation Fixes: Upgraded globally antiquated workflow actions (checkout@v4, setup-python@v5) to resolve Node.js 20 warnings. Replaced deprecating ::set-output syntax precisely with $GITHUB_OUTPUT across matrix targets.

Verification

The versions.yml pipeline reliably achieves SUCCESS natively across macOS runners!

dvadym
dvadym previously approved these changes Mar 20, 2026
RamSaw added 25 commits March 20, 2026 14:00
…hon wheels to prevent universal2 cross-compile linker breaks
…compilation wheel generation via _PYTHON_HOST_PLATFORM
…mbers to bypass Apple dead-code-stripping symbols that pybind11 delegates to runtime
…es and force -all_load into the target linker array itself
…quests to cut noise and focus on versions.yml matrix exclusively
…ope while safely sustaining standard target visibility vectors
…ation pipelines employing target_link_options strictly protecting weak-level abseil symbol evaluation across static archives
…le_archive weak linkage protections via native modern CMake generator expressions
…oying the global Homebrew abseil poison pill on macOS CI runners, and reverting aggressive dead-code stripping overriding force_load
…config by stripping entirely brew-installed protocol buffers avoiding ABI contamination
…ing compiler flag overrides, and fundamentally modernize GitHub Actions dependencies resolving all ecosystem warnings
… scrubbing without touching CMake linker, while preserving dependency updates
…nts gracefully, strictly targeting only Apple protobuf mitigation inside versions.yml and preserving minimal deprecation action bumps
…s.yml simplifications natively, and drop Python 3.10 from versions testing matrix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants