Search before asking
Motivation
paimon-rust currently supports file://, s3://, s3a://, and oss:// storage schemes via OpenDAL, but does not support hdfs://. Many production Paimon deployments use HDFS as the primary warehouse storage, making this a blocker for adoption.
Solution
The storage layer already has a clean pattern for adding new backends — Storage enum variant + config parser + feature flag (see storage-s3, storage-oss). OpenDAL provides two HDFS-compatible services that could be leveraged:
opendal::services::Webhdfs — WebHDFS REST API. No native library or JVM required.opendal::services::HdfsNative — Pure Rust HDFS client via hdfs-native. No JVM/libhdfs dependency. Built-in HA NameNode support.
Adding a storage-hdfs feature flag (similar to storage-s3 / storage-oss) with either backend would unblock HDFS users.
Anything else?
No response
Willingness to contribute
Search before asking
Motivation
paimon-rust currently supports file://, s3://, s3a://, and oss:// storage schemes via OpenDAL, but does not support hdfs://. Many production Paimon deployments use HDFS as the primary warehouse storage, making this a blocker for adoption.
Solution
The storage layer already has a clean pattern for adding new backends — Storage enum variant + config parser + feature flag (see storage-s3, storage-oss). OpenDAL provides two HDFS-compatible services that could be leveraged:
opendal::services::Webhdfs— WebHDFS REST API. No native library or JVM required.opendal::services::HdfsNative— Pure Rust HDFS client via hdfs-native. No JVM/libhdfs dependency. Built-in HA NameNode support.Adding a storage-hdfs feature flag (similar to storage-s3 / storage-oss) with either backend would unblock HDFS users.
Anything else?
No response
Willingness to contribute