Skip to content

DRILL-8544: HDF5 Files Ignoring Config Options#3038

Merged
cgivre merged 4 commits intoapache:masterfrom
cgivre:hdf5_fix
Mar 9, 2026
Merged

DRILL-8544: HDF5 Files Ignoring Config Options#3038
cgivre merged 4 commits intoapache:masterfrom
cgivre:hdf5_fix

Conversation

@cgivre
Copy link
Contributor

@cgivre cgivre commented Mar 8, 2026

DRILL-8544: HDF5 Files Ignoring Config Options

Description

There is a minor bug in the HDF5 plugin where the plugin was ignoring the showPreview config variable. This PR fixes it.

Documentation

No user facing changes except bug fixed.

Testing

Ran existing unit tests and tested manually.

@cgivre cgivre self-assigned this Mar 8, 2026
@cgivre cgivre added bug minor-update backport-to-stable This bug fix is applicable to the latest stable release and should be considered for inclusion there labels Mar 8, 2026
@cgivre cgivre requested a review from pjfanning March 8, 2026 17:20
Copy link
Member

@pjfanning pjfanning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to test what the serialization and deserialization results in as unit tests?

@cgivre
Copy link
Contributor Author

cgivre commented Mar 8, 2026

Would it be possible to test what the serialization and deserialization results in as unit tests?

@pjfanning
There is a SerDe test for this plugin already. I'm not sure how to test the issue that I fixed, but I'd be happy to add a test.

@Test
public void testSerDe() throws Exception {
String sql = "SELECT COUNT(*) FROM dfs.`hdf5/dset.h5`";
String plan = queryBuilder().sql(sql).explainJson();
long cnt = queryBuilder().physical(plan).singletonLong();
assertEquals("Counts should match",1L, cnt);
}

@pjfanning
Copy link
Member

@cgivre I was thinking more of a test that:

  • create a Jackson ObjectMapper instance
  • call writeValueAsString with a representative instance of HDF5FormatConfig
  • check the JSON string returned includes the values that you want
  • call readVale on the ObjectMapper instance to see if the value round trips ok (ie the HDF5FormatConfig instance you get returned from readValue matches the HDF5FormatConfig instance that you started with)

When you start playing with Jackson annotations, that affects what the serialization and deserialization does and unfortunately, Jackson releases can change the behaviour when trying to fix bugs.

@cgivre
Copy link
Contributor Author

cgivre commented Mar 9, 2026

@cgivre I was thinking more of a test that:

  • create a Jackson ObjectMapper instance
  • call writeValueAsString with a representative instance of HDF5FormatConfig
  • check the JSON string returned includes the values that you want
  • call readVale on the ObjectMapper instance to see if the value round trips ok (ie the HDF5FormatConfig instance you get returned from readValue matches the HDF5FormatConfig instance that you started with)

When you start playing with Jackson annotations, that affects what the serialization and deserialization does and unfortunately, Jackson releases can change the behaviour when trying to fix bugs.

I added some new unit tests.

@cgivre
Copy link
Contributor Author

cgivre commented Mar 9, 2026

@pjfanning Would you mind please approving (assuming you are ok with the changes)?
Thx!

Copy link
Member

@pjfanning pjfanning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cgivre cgivre merged commit b38ba3e into apache:master Mar 9, 2026
6 checks passed
@cgivre cgivre deleted the hdf5_fix branch March 9, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-stable This bug fix is applicable to the latest stable release and should be considered for inclusion there bug minor-update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants