Skip to content

Upgrade Struct2Tensor to TensorFlow 2.21.0#59

Open
vkarampudi wants to merge 4 commits intogoogle:masterfrom
vkarampudi:testing1
Open

Upgrade Struct2Tensor to TensorFlow 2.21.0#59
vkarampudi wants to merge 4 commits intogoogle:masterfrom
vkarampudi:testing1

Conversation

@vkarampudi
Copy link
Copy Markdown
Collaborator

@vkarampudi vkarampudi commented Apr 10, 2026

This PR upgrades struct2tensor to support TensorFlow 2.21.0 and resolves various build system incompatibilities, dependency conflicts, and compilation errors encountered during the process.

Key Changes

  1. TensorFlow & Bazel Version Alignment

    • Updated .bazelversion to 7.7.0 to align with TensorFlow 2.21.0 requirements.
    • Updated WORKSPACE to fetch TensorFlow 2.21.0
  2. Dependency Management & Bzlmod

    • Disabled Bzlmod: Added common --noenable_bzlmod to .bazelrc. This was necessary to resolve severe version mismatch issues with Protobuf when using Bazel 7, ensuring we use the versions specified in WORKSPACE.
    • Updated PyArrow: Updated pyarrow dependency in setup.py to >=14 to resolve security vulnerabilities and ensure compatibility.
  3. Build System Workarounds (Dummy Repositories)

    • TensorFlow 2.21.0 introduced complex external dependency chains that caused circular dependencies or expected configurations (like CUDA/ROCm) that are not present in this project.
    • To bypass these, I created several "dummy" repositories in third_party/ and registered them in WORKSPACE:
      • rules_java: A minimal dummy repo to satisfy Java dependencies without pulling in heavy external rules.
      • local_config_cuda, local_config_rocm, local_config_tensorrt, local_config_sycl: Dummy repositories to satisfy TensorFlow's GPU configuration checks without requiring actual GPU setups.
  4. Compiler & Linker Fixes

    • Enforced C++17: Added flags to .bazelrc to ensure C++17 is used globally, as required by newer Abseil and TensorFlow headers.
    • Protobuf Patch: Added third_party/protobuf_retain_options.patch to remove the --retain_options flag from Protobuf generation rules, which was causing failures with the version of protoc used in Bazel 7.
    • Missing Includes: Added #include <cstdint> to several C++ files in struct2tensor/ops/ and struct2tensor/kernels/ to fix missing uintptr_t definitions when compiling with newer compilers (like gcc 15).
    • SFrame Flag (Attempted & Removed): I initially added --copt=-Wa,--gsframe=no to .bazelrc to fix a relocation error specific to gcc 15. However, this caused the PR build to fail because the PR runner's assembler did not support that flag. I have removed it for PR compatibility. If you build locally with gcc 15 and see relocation errors, you may need to pass it manually.
  5. Test Suite Execution

    • Sharding Check Disabled: Passed --noincompatible_check_sharding_support to bazel test to bypass sharding support checks that were failing in benchmark tests.

Verification

All tests passed successfully locally:

bazel test //... --noincompatible_check_sharding_support ...

@vkarampudi vkarampudi changed the title Testing1 Upgrade Struct2Tensor to TensorFlow 2.21.0 Apr 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant