Skip to content

[Feature] Support RangeBitmap File Index #146

@fafacao86

Description

@fafacao86

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

I've looked at the code, currently paimon-cpp does not have rangebitmap file index support.
The design doc: https://cwiki.apache.org/confluence/display/PAIMON/PIP-33%3A+Introduce+the+range-bitmap+file+index

Solution

Following the design doc and Java implementation, I've written a draft to support rangebitmap file index, but only support fixed length type Read and Write now.

Here is the PR of the draft #147, sorry for the huge PR, it's just for PoC purpose. Just want to make sure that I'm in the right direction, and the community is willing to take the contribution of rangebitmap file index. I'll refine the code, comment and test coverage. Break down the tasks for easier code review later if the community think it's ok or community might have some suggestions.

Few questions/concerns:

  1. For Literal key SerDe, I didn't found a common utils, so I followed bitmap file index's Value Reader/Writer, and implemented my own serialization utils under rangebitmap.
  2. For tests, I tested reading Java generated rangebitmap index file(with magic starter and some headers meta) using file index reader interface. But I couldn't find a way to test the complete Write path of file index. Does paimon-cpp currently support writing a complete file index?

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions