feat: migrate native-io implementation from C to Rust

[nodece]

Motivation

The existing native-io module is implemented in C and built using the NAR (Native ARchive) toolchain. This approach has several limitations:

• Cross-compilation across architectures (especially Linux amd64 and arm64) is complex and hard to maintain
• The build pipeline relies on legacy tooling (NAR) with limited flexibility

To improve maintainability and provide reliable multi-platform support, we migrate the native implementation and build system to a modern Rust-based toolchain.

Changes

1. Native implementation & build system

• Replace the C-based implementation with a Rust-based shared library

• Remove NAR-based build (nar packaging and plugin)

• Introduce Rust build via cargo and cargo-zigbuild

• Add Maven profiles to:

  • Cross-compile for:

    • Linux amd64 (glibc)
    • Linux arm64 (glibc)

  • Package both variants into a single JAR

• Add an optional pure Rust profile for local development (no cross-compilation)

2. CI/CD pipeline updates

• Update GitHub Actions workflows to:

  • Install Rust toolchain and Zig
  • Enable cross-compilation via cargo-zigbuild
  • Build and validate artifacts for both amd64 and arm64 targets

• Ensure consistent multi-architecture coverage across all CI jobs

3. Cleanup and dependency updates

• Remove NAR-specific configurations and dependencies
• Update RAT exclusions for Rust build artifacts
• Add test dependencies (e.g., JUnit) where needed

[StevenLuMT]

reopen to rerun CI

[Copilot]

Pull request overview

This PR migrates the native-io JNI implementation from the existing C-based build (nar + assembly packaging) to a Rust cdylib, and updates Java-side library loading/build automation to embed the produced shared libraries in the module JAR.

Changes:

• Replaced the C JNI implementation with a Rust JNI implementation (cdylib) and added a Rust build layout (Cargo files + build script).
• Updated Java JNI library loading to support explicit-path overrides and candidate-based loading from JAR resources.
• Switched native-io from nar packaging to jar packaging and introduced a cargo-zigbuild-driven Maven build flow; CI workflows were updated to install Rust targets.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
native-io/src/test/java/org/apache/bookkeeper/common/util/nativeio/NativeIOLibraryPathTest.java	Adds unit tests for library path/candidate resolution and override precedence.
native-io/src/main/native-io-jni/rust/src/lib.rs	New Rust JNI implementation of the native I/O operations.
native-io/src/main/native-io-jni/rust/Cargo.toml	Declares the Rust cdylib crate and dependencies.
native-io/src/main/native-io-jni/rust/Cargo.lock	Locks Rust dependency versions for reproducible builds.
native-io/src/main/native-io-jni/rust/build.rs	Adds macOS install_name adjustment for produced dylibs.
native-io/src/main/native-io-jni/cpp/native_io_jni.c	Removes the legacy C JNI implementation.
native-io/src/main/java/org/apache/bookkeeper/common/util/nativeio/NativeIOLibraryPath.java	Introduces candidate resolution + explicit override path logic for JNI library loading.
native-io/src/main/java/org/apache/bookkeeper/common/util/nativeio/NativeIOJni.java	Updates static initialization to load from explicit path or iterate candidate resources.
native-io/src/main/assembly/assembly.xml	Removes the prior assembly-based packaging approach for native artifacts.
native-io/pom.xml	Switches packaging to JAR and adds cargo zigbuild + resource embedding steps.
.github/workflows/windows-daily-build.yml	Adds Rust toolchain/targets setup prior to Maven build.
.github/workflows/java21-daily-build.yml	Adds Rust toolchain/targets setup prior to Maven build.
.github/workflows/codeql.yml	Adds Rust toolchain/targets setup prior to Maven build/analysis.
.github/workflows/bk-streamstorage-python.yml	Adds Rust toolchain/targets setup for Maven build dependency chain.
.github/workflows/bk-ci.yml	Adds Rust toolchain/targets setup across CI jobs that run Maven builds.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.