Skip to content

Conversation

@gpshead
Copy link

@gpshead gpshead commented Dec 8, 2025

Backport acc8f74 to allow numpy 2.3.x

Amar3tto and others added 19 commits September 3, 2025 14:41
…pache#36044) (apache#36052)

* fix(Partition): support numpy integer types in partition validation

Update partition type checking to accept numpy integer types by using numbers.Integral
Add tests for numpy integer handling in partition functions

* convert to int

* fix lints
* update iceberg to 1.9.2

* cleanup

* move iceberg sql into a separate module

* check test java version too

* cleanup

* spotless

* deps

* add missing dep

* block java 8

* create examples/iceberg module

* dep fixes

* import fixes

* test fix

* update changes

* update changes

* include integration tests

* add examples:iceberg to javaPreCommit

* fix deps

* fix deps
…sdk worker running in docker env (apache#36085)

Cherrypick apache#36014: Fix jdbc logical type not found when python sdk worker running in docker env
…messages by making them lazy (apache#36084)

Cherrypick apache#36063: [Java] Fix expensive OneOfType precondition messages by making them lazy
* update dependencies due to CVE-2024-13009 and CVE-2025-24970

* update dependency due to transitive dependency with  CVE-2025-48734

* outstanding netty dependency.

* fix netty's CVE-2025-55163

* Revert "fix netty's CVE-2025-55163"

This reverts commit 874a77c.

* revert netty

Co-authored-by: Radosław Stankiewicz <radoslaws@google.com>
@gpshead
Copy link
Author

gpshead commented Dec 8, 2025

sorry, wrong base repo, my bad.

@gpshead gpshead closed this Dec 8, 2025
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @gpshead, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request primarily focuses on a significant dependency upgrade and refactoring effort across the Apache Beam project. The core objective is to enhance compatibility with newer versions of critical libraries like NumPy (now supporting up to 2.4.x) and Apache Iceberg (upgraded to 1.9.2). This involves updating numerous build configurations, adjusting dependency declarations, and restructuring modules, particularly for Iceberg-related components which now require Java 11+. The changes also include various other version bumps, minor bug fixes, and improvements across the Java, Python, Go, and TypeScript SDKs to ensure the project remains current and functional with its ecosystem.

Highlights

  • NumPy 2.3.x/2.4.x Compatibility: The Python SDK's dependency constraints for NumPy have been updated to allow versions up to 2.4.0, ensuring broader compatibility with newer NumPy releases.
  • Apache Iceberg Upgrade and Refactoring: The Apache Iceberg dependency has been upgraded to version 1.9.2. This upgrade necessitates Java 11+ for Iceberg-related modules and the IO expansion service, leading to significant restructuring of Iceberg examples and SQL integration into dedicated modules.
  • Core Dependency Version Bumps: Multiple core dependencies across the Java, Python, Go, and TypeScript SDKs have been updated, including Hadoop (3.4.1 to 3.4.2), Jetty (9.4.54 to 9.4.57), Netty (4.1.51 to 4.1.110), and internal Beam versions.
  • Python SDK Enhancements: The PartitionFn in the Python SDK now correctly handles NumPy integer types, and the detection logic for named tuples has been made more robust. Additionally, a temporary fix for JDBC logical types has been implemented.
  • Java SDK Improvements: Error messages for union types in the Java SDK have been improved for clarity, and the Java expansion service now explicitly includes and forces Avro 1.12.0 to resolve dependency conflicts.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/build_wheels_manual.yml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants