-
Notifications
You must be signed in to change notification settings - Fork 32
Description
Search before asking
- I searched in the issues and found nothing similar.
Motivation
I've looked at the code, currently paimon-cpp does not have rangebitmap file index support.
The design doc: https://cwiki.apache.org/confluence/display/PAIMON/PIP-33%3A+Introduce+the+range-bitmap+file+index
Solution
Following the design doc and Java implementation, I've written a draft to support rangebitmap file index, but only support fixed length type Read and Write now.
Here is the PR of the draft #147, sorry for the huge PR, it's just for PoC purpose. Just want to make sure that I'm in the right direction, and the community is willing to take the contribution of rangebitmap file index. I'll refine the code, comment and test coverage. Break down the tasks for easier code review later if the community think it's ok or community might have some suggestions.
Few questions/concerns:
- For Literal key SerDe, I didn't found a common utils, so I followed bitmap file index's Value Reader/Writer, and implemented my own serialization utils under rangebitmap.
- For tests, I tested reading Java generated rangebitmap index file(with magic starter and some headers meta) using file index reader interface. But I couldn't find a way to test the complete Write path of file index. Does paimon-cpp currently support writing a complete file index?
Anything else?
No response
Are you willing to submit a PR?
- I'm willing to submit a PR!