0.59.4

Changes

🐛 Bug Fixes

• Published packages cannot depend on unpublished packages (#6592) @robert3005

0.59.3

Changes

🐛 Bug Fixes

• minor c++ duckdb integration changes (#6566) @myrrc
• Vortex crate doesn't unconditionally depend on vortex-cuda-macros (#6580) @robert3005

0.59.2

Changes

🧰 Maintenance

• Add ci check for java publish (#6578) @robert3005

0.59.1

Changes

🐛 Bug Fixes

• fix: CUDA scalar inplace (#6567) @0ax1

🧰 Maintenance

• chore: drop flaky duckdb http e2e test (#6570) @0ax1
• chore: make auto clang-format consistent (#6568) @0ax1

0.59.0

Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better vx-bench docs in README (#5900) @AdamGS

⚠️ Breaks

• Remove vortex-compute (#6561) @connortsui20
• Introduce VortexError::Other and simplify Display implementation (#6410) @AdamGS
• Move vortex-scalar into vortex-array (#6523) @connortsui20
• break[array]: lazy between (#6528) @joseph-isaacs
• break[array]: mask compute to lazy mask execution (#6485) @joseph-isaacs
• break[array]: lazy compare function (#6495) @joseph-isaacs
• Remove vortex-vector (#6529) @connortsui20
• break[array]: move like compute to lazy compute (#6497) @joseph-isaacs
• Pass buffers to VTable::deserialize (#6473) @connortsui20
• break[array]: move cast traits to exprs (#6487) @joseph-isaacs
• Revert #6363 (#6476) @connortsui20
• Use Scalar directly for constant array metadata inlining (#6439) @connortsui20
• Fix Scalar de/serialization with proto and outline nulls from ScalarValue (#6309) @connortsui20
• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• fallible scalar_at and validity (#6102) @joseph-isaacs
• Remove the python feature from vortex-error (#6064) @AdamGS
• break[*]: make array methods fallible (#6086) @joseph-isaacs
• chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
• Array VTable (#6000) @gatesn
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
• Remove EncodeVTable (#6036) @gatesn
• chore: fallible canonical (#6006) @joseph-isaacs
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs

✨ Features

• Add FileSystem trait and make DuckDB configurable (#6564) @gatesn
• Add scan concurrency config for datafusion source (#5945) @AdamGS
• feat: implement DuckDB filesystem integration for Vortex file handling (#6198) @iceboundrock
• vx convert to support cuda compatible vortex files (#6522) @onursatici
• cuda flat layout (#6521) @onursatici
• feat[xtask]: faster public-api (#6498) @joseph-isaacs
• Improve substrait support to include more types and expressions (#6484) @AdamGS
• feat[array]: refactor zip compute kernel to lazy zip via ScalarFn Array (#6461) @joseph-isaacs
• feat[gpu]: CUDA dynamic dispatch kernel (#6483) @0ax1
• feat[array]: lazy numeric (#6469) @joseph-isaacs
• feat[array]: move over to a expr based cast (#6404) @joseph-isaacs
• refactor[array]: move boolean ops to lazy binary expr (#6447) @joseph-isaacs
• Public api lockfiles (#6348) @gatesn
• feat[array]: refactor invert compute kernel to lazy not (#6446) @joseph-isaacs
• sqllogictest runner for DF and DuckDB on Vortex (#6273) @AdamGS
• feat[array]: move over fill_null to scalar_fn array making it lazy (#6406) @joseph-isaacs
• Upgrade Pco to 1.0.1 (#6354) @mwlon
• Add execution target that recursively canonicalises as well as canonical with canonical validity (#6380) @robert3005
• dict of strings kernel to support outlined views (#6401) @onursatici
• zstd buffers cuda kernel (#6394) @onursatici
• Make VortexSource::new public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• chore[cuda]: add date32 exporter test (#6345) @a10y
• feat: display tree with no metadata (#5898) @danking
• feat[cuda]: device exports for varbinview (#6338) @a10y
• vortex-metrics v2 (#6187) @AdamGS
• feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
• feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
• feat[gpu]: CUDA date_time_parts kernel (#6303) @0ax1
• feat: alternate for StructFields (#6276) @blaginin
• Add a canonical matcher (#6292) @gatesn
• Move over to array matchers (#6291) @gatesn
• feat[gpu]: cuda runend decoding (#6275) @0ax1
• feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
• feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
• feat[array]: validate array encodings on write (#6241) @joseph-isaacs
• feat[cuda]: SequenceArray (#6232) @a10y
• feat[cuda]: patches kernel (#6231) @a10y
• Pass limit through Python Vortex (#6239) @gatesn
• Ship Vortex CLI as part of Python package (#6233) @gatesn
• Feature: Add more Filter<_> implementations (#6173) @connortsui20
• Benchmarks website revamped (#6215) @palaska
• Dataset API (#5949) @gatesn
• feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
• feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
• feat[python]: support full read/write from object storage (#6022) @a10y
• feat[scan]: gpu scan (#6199) @joseph-isaacs
• feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
• feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
• Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
• feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
• Feature: Use array.filter() instead of filter(array) compute (#6146) @connortsui20
• Rewrite benchmark website (#6160) @palaska
• feat: stats pruning for LIKE filters (#6049) @a10y
• feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
• feat[cuda]: canonical to_host (#6130) @joseph-isaacs
• feat[cuda]: dict decimal take (#6129) @joseph-isaacs
• feat[gpu]: dict take values prim (#6101) @joseph-isaacs
• chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
• feat[gpu]: scalar encodings (#6109) @joseph-isaacs
• Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
• feat: nvcomp wrapper crate (#6108) @0ax1
• Allow configuring different readers for DF integration (#6027) @AdamGS
• feat: introduce CudaDeviceBuffer (#6060) @0ax1
• Array VTable (#6000) @gatesn
• feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
• Attempting to simplify the IO read traits (#5935) @AdamGS
• feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
• feat[array]: execute::<PrimitiveArray>() (#5954) @joseph-isaacs
• Spark 4 (#5944) @a10y
• feat: cuda gpu skeleton (#5936) @0ax1
• Add flag to run benchmarks under Samply (#5940) @gatesn
• Datafusion 52 upgrade (#5863) @AdamGS
• Support precision casting for DScalar (#5937) @AdamGS
• feat[array]: use execute in canonical (#5925) @joseph-isaacs
• feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs
• feature: TableStrategy (#5640) @a10y

🚀 Performance

• Faster primitive take (#6559) @connortsui20
• perf[vortex-btrblocks]: remove listview expansion ratio (#6530) @asubiotto
• VarBinBuilder: eagerly set IsSorted stat on offsets (#6452) @dimitarvdimitrov
• Implement ReduceNode for Array to not require creating new arc when returning children (#6443) @robert3005
• chore: clean cancelled read requests earlier in driver state transitions (#6382) @caicancai
• shared array to avoid duplicate execute (#6453) @onursatici
• vortex-btrblocks: write listviews when elements expand > 50% (#6393) @asubiotto
• Implement ReduceNode for ScalarFnArray to avoid boxing (#6435) @robert3005
• Improve visitor performance by reducing Vec and String allocations (#6337) @AdamGS
• lazy dict layout values eval (#6423) @onursatici
• cuda strategy to use zstd buffers for strings (#6403) @onursatici
• bitunpacking cuda kernels store output into shared memory before copying to main memory (#6384) @robert3005
• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to {reduce/execute}_parent (#6312) @joseph-isaacs
• perf[buf]: inline collect_bool (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
• Check read-only cache first (#6249) @gatesn
• Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
• Perf: Use optimized filter implementations is FilterArray::execute (#6189) @connortsui20
• coalesced request align all segments (#6158) @onursatici
• perf: CUDA async to host copy (#6085) @0ax1
• feat: make host to device copy async (#6075) @0ax1
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
• Only check for ConstantVTable (#6002) @gatesn
• do not validate is_sorted on runend (#6025) @onursatici
• perf: CUDA FoR loop unrolling (#6017) @0ax1
• feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
• Remove unnecessary work from ListArray::validate (#6004) @AdamGS
• scan: make LazyS...

0.59.0-rc.0

Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better vx-bench docs in README (#5900) @AdamGS

⚠️ Breaks

• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• fallible scalar_at and validity (#6102) @joseph-isaacs
• Remove the python feature from vortex-error (#6064) @AdamGS
• break[*]: make array methods fallible (#6086) @joseph-isaacs
• chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
• Array VTable (#6000) @gatesn
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
• Remove EncodeVTable (#6036) @gatesn
• chore: fallible canonical (#6006) @joseph-isaacs
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs

✨ Features

• Make VortexSource::new public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• chore[cuda]: add date32 exporter test (#6345) @a10y
• feat: display tree with no metadata (#5898) @danking
• feat[cuda]: device exports for varbinview (#6338) @a10y
• vortex-metrics v2 (#6187) @AdamGS
• feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
• feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
• feat[gpu]: CUDA date_time_parts kernel (#6303) @0ax1
• feat: alternate for StructFields (#6276) @blaginin
• Add a canonical matcher (#6292) @gatesn
• Move over to array matchers (#6291) @gatesn
• feat[gpu]: cuda runend decoding (#6275) @0ax1
• feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
• feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
• feat[array]: validate array encodings on write (#6241) @joseph-isaacs
• feat[cuda]: SequenceArray (#6232) @a10y
• feat[cuda]: patches kernel (#6231) @a10y
• Pass limit through Python Vortex (#6239) @gatesn
• Ship Vortex CLI as part of Python package (#6233) @gatesn
• Feature: Add more Filter<_> implementations (#6173) @connortsui20
• Benchmarks website revamped (#6215) @palaska
• Dataset API (#5949) @gatesn
• feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
• feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
• feat[python]: support full read/write from object storage (#6022) @a10y
• feat[scan]: gpu scan (#6199) @joseph-isaacs
• feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
• feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
• Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
• feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
• Feature: Use array.filter() instead of filter(array) compute (#6146) @connortsui20
• Rewrite benchmark website (#6160) @palaska
• feat: stats pruning for LIKE filters (#6049) @a10y
• feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
• feat[cuda]: canonical to_host (#6130) @joseph-isaacs
• feat[cuda]: dict decimal take (#6129) @joseph-isaacs
• feat[gpu]: dict take values prim (#6101) @joseph-isaacs
• chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
• feat[gpu]: scalar encodings (#6109) @joseph-isaacs
• Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
• feat: nvcomp wrapper crate (#6108) @0ax1
• Allow configuring different readers for DF integration (#6027) @AdamGS
• feat: introduce CudaDeviceBuffer (#6060) @0ax1
• Array VTable (#6000) @gatesn
• feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
• Attempting to simplify the IO read traits (#5935) @AdamGS
• feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
• feat[array]: execute::<PrimitiveArray>() (#5954) @joseph-isaacs
• Spark 4 (#5944) @a10y
• feat: cuda gpu skeleton (#5936) @0ax1
• Add flag to run benchmarks under Samply (#5940) @gatesn
• Datafusion 52 upgrade (#5863) @AdamGS
• Support precision casting for DScalar (#5937) @AdamGS
• feat[array]: use execute in canonical (#5925) @joseph-isaacs
• feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs
• feature: TableStrategy (#5640) @a10y

🚀 Performance

• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to {reduce/execute}_parent (#6312) @joseph-isaacs
• perf[buf]: inline collect_bool (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
• Check read-only cache first (#6249) @gatesn
• Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
• Perf: Use optimized filter implementations is FilterArray::execute (#6189) @connortsui20
• coalesced request align all segments (#6158) @onursatici
• perf: CUDA async to host copy (#6085) @0ax1
• feat: make host to device copy async (#6075) @0ax1
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
• Only check for ConstantVTable (#6002) @gatesn
• do not validate is_sorted on runend (#6025) @onursatici
• perf: CUDA FoR loop unrolling (#6017) @0ax1
• feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
• Remove unnecessary work from ListArray::validate (#6004) @AdamGS
• scan: make LazyScanStream initialization non-blocking (#5906) @godnight10061
• perf[sequence]: array constant compare execution kernel (#5795) @joseph-isaacs
• perf: reduce time_elapsed_opening overhead (#5879) @godnight10061

🐛 Bug Fixes

89 changes

• Measure output bytes in cuda bitunpack benchmarks (#6395) @robert3005
• fix[array]: decimal fill_null larger than buffer (#6388) @joseph-isaacs
• fix[array]: dict value nullable values (#6387) @joseph-isaacs
• fix[fuzz]: coverage report (#6375) @joseph-isaacs
• join ops before in format (#6370) @blaginin
• Include config.js in benchmark website docker image (#6379) @palaska
• Exporting struct array to arrow respects vortex nullabilty (#6365) @robert3005
• Fix null take on structs (#6364) @blaginin
• varbinview on device build skips validation (#6358) @onursatici
• fix: vortex-layout doc tests require zstd (#6179) @danking
• DF: Only calculate the size of the metadata cache entry once (#6342) @AdamGS
• bump time version to clear CI (#6335) @onursatici
• All-invalid compressor (#6330) @gatesn
• fix: Empty select (#6314) @gatesn
• Avoid some potential stack overflow cases with deep expression trees (#6321) @AdamGS
• shared array in dict layout (#6305) @onursatici
• skip sparse array validation for device resident arrays (#6315) @onursatici
• varbin to be constructed from buffer handle directly (#6317) @onursatici
• Do not use display_tree in error message (#6251) @gatesn
• Minor memory-related improvements in vortex-array and vortex-duckdb (#6299) @AdamGS
• use is_host to recursively check residency to skip validation (#6297) @onursatici
• Add slice kernel for BitPackedArray execute_parent (#6283) @onursatici
• select preserve validity of fields (#6260) @connortsui20
• fsst array: AllTrue and AllFalse are handled by filter fn (#6272) @blaginin
• fix: separate ptx build profiles (#6252) @0ax1
• fix[cuda]: Fix gpu_scan test (#6208) @a10y
• [benchmark-website] Fix mobile charts, better colors, better tooltip (#6226) @palaska
• fix: exclude flaky benchmarks from codspeed (#6220) @0ax1
• fix: skip buffer validation if on device (#6219) @0ax1
• fix: simd build (#6217) @0ax1
• chore[cuda]: fix windows (#6206) @joseph-isaacs
• Fix: dont do lazy filters on write (#6194) @connortsui20
• fix: evaluate OR for validity (#6184) @blaginin
• fix: cuda ptx folder (#6170) @joseph-isaacs
• Fix: Comment on a failed benchmark (#6181) @connortsui20
• fix[alp]: cast didn't cast patches (#6174) @joseph-isaacs
• fix[fuzz]: fix gpu fuzzer cfg flags (#6165) @joseph-isaacs
• fix[gpu]: synchronous copy to host (#6157) @0ax1
• fix[fuzz]: single sesssion for the fuzzer (#6155) @joseph-isaacs
• fix: sample multiples of 1024 (#6151) @danking
• fix: only link nvcomp for linux (#6144) @0ax1
• fix: fastlanes canonicalize benchmarks (#6141) @0ax1
• fix[fsst]: execute_parent empty validity non empty filter (#6133) @joseph-isaacs
• fix[vortex-array]: fix filter of sliced list (#6113) @asubiotto
• fix[list]: filter list execute parent (#6119) @blaginin
• fix[dtype]: fix typo in conversion from dtype to decimaltype (#6121) @joseph-isaacs
• fix[bitpacking]: filter (#6120) @joseph-isaacs
• fix[vortex-...

0.58.0

Changes

✨ Features

• Feature: Make vortex-tui a library (#5779) @connortsui20

🚀 Performance

• End DataFusion stream early if file can be pruned away by dynamic expr (#5874) @AdamGS

🧰 Maintenance

• chore: add overflow tests for unsigned types in BigCast (#5866) @caicancai

0.57.2

Changes

🐛 Bug Fixes

• fix: Move from macos-13 runners to macos-15-intel runners (#5875) @robert3005

0.57.1

Changes

🐛 Bug Fixes

• fix: Correctly pin maturin to 1.10.2 (#5873) @robert3005

0.57.0

Changes

⚠️ Breaks

• Fix BoolArray::new to take BitBuffer (#5690) @gatesn
• Lazy array expression evaluation (#5625) @gatesn
• Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
• Move stats into expr (#5582) @gatesn
• Combine Array VTable with Encoding (#5462) @gatesn
• Consolidate into stat_expression function (#5366) @gatesn
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs

✨ Features

• feat: teach TemporalMetadata about its relationship to DTypes (#5864) @danking
• Pluggable expression conversion for DF (#5853) @AdamGS
• Introduce extendable access plan for DF scans (#5855) @AdamGS
• Upgrade DF to 51 (#5189) @AdamGS
• Benchmarks re-org (#5676) @AdamGS
• Push down cast (#5803) @gatesn
• ListView to List (#5802) @gatesn
• feat[array]: stats + validity in display array (#5799) @joseph-isaacs
• deterministic encoding ordering (#5770) @onursatici
• feat[fixed-size-list]: impl repeat and slice (#5751) @joseph-isaacs
• feat[compute]: compare decimal and varbin (#5757) @joseph-isaacs
• feat[datafusion]: Support DataFusion PartitionedFile extensions for external index (#5556) @haohuaijin
• FSST Filter Kernel (#5755) @gatesn
• perf[ci]: stopped instances (#5753) @joseph-isaacs
• Feature: makes the decompressor and compressor methods public (#5752) @connortsui20
• Feature: add fused bitpack filter kernel (#5741) @connortsui20
• Execute parent (#5736) @gatesn
• feat[array]: duckdb executor (#5737) @joseph-isaacs
• Arrow Executor (#5647) @gatesn
• Feature: add primitive casting (#5718) @connortsui20
• Feature: add some Cast infrastructure (#5708) @connortsui20
• feat[array]: dynamic expr execute (#5714) @joseph-isaacs
• Feature: add append_array_as_list methods to the list arrays (#5707) @connortsui20
• Feature: implement DictArray batched execute (#5686) @connortsui20
• feat[array]: list contains scalar (#5713) @joseph-isaacs
• feat[vector]: Eq (#5681) @joseph-isaacs
• Feature: add zero and null constructors for scalars (#5705) @connortsui20
• Feature: batch execute for DeltaArray (#5667) @connortsui20
• feat[vector]: slice varbinview (#5697) @joseph-isaacs
• Feature: add take for the view vectors (#5682) @connortsui20
• Feature: Implement take on more vectors + stubs (#5653) @connortsui20
• feature: add __version__ attribute to Python package (#5641) @d33bs
• Feature: add take for primitive and bool vectors (#5540) @connortsui20
• vx layout display (#5650) @joseph-isaacs
• Feature: added properties to VortexScanner (#5500) @sherlockbeard
• Add --version arg to vx (#5646) @palaska
• Feature: add total read size counter (#5600) @harryscholes
• feat[scalar_fn]: binary ops (#5617) @joseph-isaacs
• Execute ScalarFn arrays (#5611) @gatesn
• FilterArray (#5610) @gatesn
• Add GetItemFn (#5609) @gatesn
• Add MaskFn (#5606) @gatesn
• feat[scalar_fn]: is_null (#5601) @joseph-isaacs
• Scalar functions (#5561) @gatesn
• feat[buffer]: use BufferHandle in more places (#5567) @joseph-isaacs
• feat: alp decompress to vector (#5548) @0ax1
• feat[gpu]: use BufferHandle to the array vtable build (#5565) @joseph-isaacs
• feat[vortex-expr]: is_fallible analysis (#5546) @joseph-isaacs
• Feature: add take compute for slices (#5537) @connortsui20
• feat[vortex-expr]: is_fallible expression analysis (#5531) @joseph-isaacs
• feat[vortex-expr]: add a is_null_sensitive expression analysis (#5528) @joseph-isaacs
• Feature: add a take compute trait (#5503) @connortsui20
• Feature: add impl owned filter impls (#5493) @connortsui20
• Feature: add filter impl for FixedSizeListVector (#5494) @connortsui20
• Vector cast (#5461) @gatesn
• Scalar arithmetic (#5460) @gatesn
• feat[cli]: add scroll for tree encoding view (#5497) @a10y
• Revert "Revert "chore[ci]: add windows test to CI (#5439)"" (#5486) @joseph-isaacs
• More vector scalars (#5459) @gatesn
• feature: Support std_file on windows as it's used by cxx bindings (#5450) @robert3005
• Feature: implement bitpacking batch execute (#5446) @connortsui20
• feat[vortex-array]: added debug to expr/array reduce rules (#5442) @joseph-isaacs
• feat: publish vortex-spark-all assembly JAR (#5435) @a10y
• feat[vortex-array] add an array session rule registry (#5414) @joseph-isaacs
• Feature: add spare_capacity_mut to PVectorMut (#5287) @connortsui20
• feat: pipeline-pcodec (#5415) @0ax1
• Add VectorOps::clear (#5409) @gatesn
• feat[vortex-array]: expr array that represents lazy computation (#5400) @joseph-isaacs
• feat[vortex-expr]: reduce_parent has typed parent (#5398) @joseph-isaacs
• feat: cast expression (#5389) @danking
• feat: PyExpr::evaluate (#5388) @danking
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs
• Faster Pipelines (#5352) @gatesn
• feat[vortex-expr]: better debug print (#5374) @joseph-isaacs

🚀 Performance

• perf: list contains scalar scalar (#5792) @0ax1
• Upgrade DF to 51 (#5189) @AdamGS
• performance[vortex-array]: don't call is_valid to count bytes in varbinview (#5814) @asubiotto
• perf[array]: extension pushdown filter + constant (#5794) @joseph-isaacs
• perf[expr]: between not fallible (#5798) @joseph-isaacs
• perf[dtp]: push down compare over dtp constant second subsecond (#5793) @joseph-isaacs
• perf[date-time-parts]: compressor handle constant and pushdown rules (#5778) @joseph-isaacs
• perf[duckdb]: dont push down pack & dedup filters for ddb (#5773) @joseph-isaacs
• ci: use windows-x64 pool for Windows tests (#5756) @joseph-isaacs
• feat[duckdb]: optimize array (#5738) @joseph-isaacs
• Perf: optimize take_scalar (#5723) @connortsui20
• Make sure all encodings are enabled for DF (#5720) @AdamGS
• Perf: Optimized bool take (#5701) @connortsui20
• Run optimizer on root array only on (#5710) @gatesn
• perf[array]: Constant kernel filter pushdown (#5698) @joseph-isaacs
• perf[array]: with_children vtable function (#5694) @joseph-isaacs
• perf[ci]: speed up ci MORE (#5576) @joseph-isaacs
• perf(ci): skip rust-cache on runs-on runners with S3 sccache (#5570) @joseph-isaacs
• chunked array zip (#5530) @onursatici
• varbinview zip kernel (#5526) @onursatici
• chore[vortex-dict]: numeric pushdown & clean up the dict ctor (#5438) @joseph-isaacs
• Handwritten pipeline benchmark and analysis (#5408) @connortsui20
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• perf[dict]: all_values_referenced to allow pushdown optimisation. (#5416) @joseph-isaacs
• perf[vortex-expr]: replace can stop looking at subtree if it finds a match (#5405) @joseph-isaacs
• actually make ZCTL (#5402) @a10y
• fix: performance regression for filtering ListView (#5390) @a10y
• fix[vortex-dict]: faster min_max for dict array (#5288) @joseph-isaacs
• fix: yo dawg, I heard you like inlining (#5363) @a10y

🐛 Bug Fixes

81 changes

• fix: enable all features for vortex-mask docsrs and enable serde-derive when serde is enabled (#5871) @robert3005
• fix[array]: correct the handling of to_arrow execution and struct casting (#5867) @joseph-isaacs
• fix[ci]: pin maturin due to fix upstream error (#5868) @joseph-isaacs
• fix: int compressor mul overflow bp (#5854) @joseph-isaacs
• Fix: IPC AsyncMessageReader does not handle partial reads (#5758) @connortsui20
• Fix: incorrect child vector size in duckdb list array import (#5740) @connortsui20
• Fix: bring back lance compression benchmark (#5840) @connortsui20
• Fix: don't fail on missing lance for datafusion (#5841) @connortsui20
• Fix: CI sql script (#5838) @connortsui20
• Fix: CI SQL Remote (#5836) @connortsui20
• Fix: finally fix CI (#5835) @connortsui20
• Fix: actually actually fix CI (#5830) @connortsui20
• Fix: benchmark name for bench.yml (#5828) @connortsui20
• Fix: remove lance feature from vortex-bench CI (#5823) @connortsui20
• Fix: decimal cast error message (#5821) @connortsui20
• Fix Arrow list export (#5804) @gatesn
• Fix: support widening of ListViewVector offsets and sizes (#5796) @connortsui20
• perf[layout]: apply then filter (#5788) @joseph-isaacs
• perf[date-time-parts]: compressor handle constant and pushdown rules p2 (#5786) @joseph-isaacs
• fix: disable noisy btrblocks micro-benchmarks on codspeed (#5782) @0ax1
• Fix arrow conversion (#5769) @gatesn
• fix: downcast in list new_vector_exporter (#5768) @0ax1
• fix[duckdb]: no to_struct in exporter (#5750) @joseph-isaacs
• Fix: remove memory limit on fuzzers (#5763) @connortsui20
• Fix: ListArray take incorrect validity (#5762) @connortsui20
• fix: list contains (#5748) @0ax1
• fix[layout]: soundness note on values eval (#5747) @joseph-isaacs
• fix: only validate non-null decimals (#5745) @0ax1
• fix: zstd frame step width for alignment (#5730) @0ax1
• fix[duckdb]: remove validity mask from vector export (#5739) @joseph-isaacs
• fix: Buffer slice end does not need to be aligned (#5734) @0ax1
• Fix Arrow struct export (#5731) @gatesn
• Disable GHArchive benchmarks to unblock perf reference in CI (#5729) @AdamGS
• feat[array]: fixup operator stuff (#5711) @joseph-isaacs
• Reduce optimizer logging to debug (#5702) @gatesn
• fix: list take offset type (#5679) @0ax1
• fix: make error type explicit (#5696) @0ax1
• Avoid taking write lock on VortexSession::get (#5688) @gatesn
• Fix: claude action concurrency group (#5680) @connortsui20
• Temp fix: Make take bound by Copy instead of NativePType (#5666) @connortsui20
• Fix: add RUSTFLAGS="--cfg vortex_nightly" to all nightly tests (#5655) @connortsui20
• Fix: portable SIMD take implementation (#5654) @connortsui20
• Fix session get-or-default (#5662) @gatesn
• Include zstd encoding in IO fuzzer (#5661) @AdamGS
• Feature: added properties to VortexScanner (#5500) @sherlockbeard
• fix[vortex-array]: fix take on varbinviews with NULL indices...