Commit 75ef45d
authored
Core: Fix bin packing when target file size is smaller than a row (#2844)
## What does this change do?
When `write.target-file-size-bytes` is smaller than a single row, bin
packing computed a 0 row chunk size and PyArrow raised a ValueError.
This change clamps the chunk size to at least 1, so writes still succeed
(one row per batch/file when needed).
## Why is this needed?
Fixes a crash when users set a small target file size and attempt to
write large records.
## How was this tested?
- make lint
- uv run python -m pytest tests/io/test_pyarrow.py -k
"bin_pack_arrow_table" -v
- make test (timed out at ~42%)
Closes #2795
Co-authored-by: Soham <010Soham@users.noreply.github.com>1 parent 6f8f57e commit 75ef45d
2 files changed
+7
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2681 | 2681 | | |
2682 | 2682 | | |
2683 | 2683 | | |
2684 | | - | |
| 2684 | + | |
2685 | 2685 | | |
2686 | 2686 | | |
2687 | 2687 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2248 | 2248 | | |
2249 | 2249 | | |
2250 | 2250 | | |
| 2251 | + | |
| 2252 | + | |
| 2253 | + | |
| 2254 | + | |
| 2255 | + | |
| 2256 | + | |
2251 | 2257 | | |
2252 | 2258 | | |
2253 | 2259 | | |
| |||
0 commit comments