[python] Fix KeyError when reading manifest files without _VALUE_STATS_COLS field #6976
Conversation
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
I'm also curious why the none _VALUE_STATS_COLS is not written out when using Python to write?
Confused too, User says that table is written by older version pypaimon (0.3), I need to double check it, along with the IndexError. |
Not just old version, Python write append table with full metadata.stats-mode should produce none COLS too. |
| file_dict = records[0]['_FILE'] | ||
| try: | ||
| if file_dict['_VALUE_STATS_COLS'] is None: | ||
| pass |
There was a problem hiding this comment.
Add an assertion to show that None COLS works well with code if file_dict['_VALUE_STATS_COLS'] is None. The root cause of the KeyError is that very older version pypaimon (before 205.09.10) . And double checked user's test table is written before that by pypaimon.
In short, I suggest to discard this fix and ask user do not use the older table, as the root cause is bug during writing, what's your suggestion?
2 participants
…S_COLS field
Purpose
Linked issue: close #xxx
Tests
API and Format
Documentation