Releases: vortex-data/vortex
Releases · vortex-data/vortex
0.59.4
Immutable
release. Only release title and notes can be modified.
0.59.3
Immutable
release. Only release title and notes can be modified.
Changes
🐛 Bug Fixes
- minor c++ duckdb integration changes (#6566) @myrrc
- Vortex crate doesn't unconditionally depend on vortex-cuda-macros (#6580) @robert3005
0.59.2
Immutable
release. Only release title and notes can be modified.
0.59.1
0.59.0
Immutable
release. Only release title and notes can be modified.
Changes
- docs: tidy and unify Python quickstart examples (#5972) @caicancai
- Better
vx-benchdocs in README (#5900) @AdamGS
⚠️ Breaks
- Remove
vortex-compute(#6561) @connortsui20 - Introduce
VortexError::Otherand simplifyDisplayimplementation (#6410) @AdamGS - Move
vortex-scalarintovortex-array(#6523) @connortsui20 - break[array]: lazy between (#6528) @joseph-isaacs
- break[array]: mask compute to lazy mask execution (#6485) @joseph-isaacs
- break[array]: lazy compare function (#6495) @joseph-isaacs
- Remove
vortex-vector(#6529) @connortsui20 - break[array]: move like compute to lazy compute (#6497) @joseph-isaacs
- Pass buffers to
VTable::deserialize(#6473) @connortsui20 - break[array]: move cast traits to exprs (#6487) @joseph-isaacs
- Revert #6363 (#6476) @connortsui20
- Use
Scalardirectly for constant array metadata inlining (#6439) @connortsui20 - Fix
Scalarde/serialization with proto and outline nulls fromScalarValue(#6309) @connortsui20 - chore[array]: inline take compute function (#6376) @joseph-isaacs
- Add Array::deserialize arguments (#6340) @gatesn
- Update FSST to store components of compressed codes (#6325) @gatesn
- Array Execution (#6307) @gatesn
- Execute parent to return ArrayRef (#6257) @gatesn
- Extension DType vtables (#6081) @gatesn
- break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
- Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
- fallible scalar_at and validity (#6102) @joseph-isaacs
- Remove the python feature from vortex-error (#6064) @AdamGS
- break[*]: make array methods fallible (#6086) @joseph-isaacs
- chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
- Array VTable (#6000) @gatesn
- Remove duplicate ValidityVTable logic (#5975) @gatesn
- chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
- Remove EncodeVTable (#6036) @gatesn
- chore: fallible canonical (#6006) @joseph-isaacs
- feat[array]: use execute to return
Canonicalarray notVector(#5895) @joseph-isaacs
✨ Features
- Add FileSystem trait and make DuckDB configurable (#6564) @gatesn
- Add scan concurrency config for datafusion source (#5945) @AdamGS
- feat: implement DuckDB filesystem integration for Vortex file handling (#6198) @iceboundrock
- vx convert to support cuda compatible vortex files (#6522) @onursatici
- cuda flat layout (#6521) @onursatici
- feat[xtask]: faster public-api (#6498) @joseph-isaacs
- Improve substrait support to include more types and expressions (#6484) @AdamGS
- feat[array]: refactor
zipcompute kernel to lazyzipvia ScalarFn Array (#6461) @joseph-isaacs - feat[gpu]: CUDA dynamic dispatch kernel (#6483) @0ax1
- feat[array]: lazy numeric (#6469) @joseph-isaacs
- feat[array]: move over to a expr based cast (#6404) @joseph-isaacs
- refactor[array]: move boolean ops to lazy binary expr (#6447) @joseph-isaacs
- Public api lockfiles (#6348) @gatesn
- feat[array]: refactor invert compute kernel to lazy
not(#6446) @joseph-isaacs - sqllogictest runner for DF and DuckDB on Vortex (#6273) @AdamGS
- feat[array]: move over fill_null to scalar_fn array making it lazy (#6406) @joseph-isaacs
- Upgrade Pco to 1.0.1 (#6354) @mwlon
- Add execution target that recursively canonicalises as well as canonical with canonical validity (#6380) @robert3005
- dict of strings kernel to support outlined views (#6401) @onursatici
- zstd buffers cuda kernel (#6394) @onursatici
- Make
VortexSource::newpublic (#6392) @AdamGS - Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
- feat[expr]: new fill null expr (#6385) @joseph-isaacs
- zstd buffers array (#6362) @onursatici
- gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
- cuda dictionary of vbv decompression (#6361) @0ax1
- CLI flag to enable tracing for benchmarks (#6350) @gatesn
- chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
- chore[cuda]: add date32 exporter test (#6345) @a10y
- feat: display tree with no metadata (#5898) @danking
- feat[cuda]: device exports for varbinview (#6338) @a10y
- vortex-metrics v2 (#6187) @AdamGS
- feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
- feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
- feat[gpu]: CUDA
date_time_partskernel (#6303) @0ax1 - feat:
alternateforStructFields(#6276) @blaginin - Add a canonical matcher (#6292) @gatesn
- Move over to array matchers (#6291) @gatesn
- feat[gpu]: cuda runend decoding (#6275) @0ax1
- feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
- feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
- feat[array]: validate array encodings on write (#6241) @joseph-isaacs
- feat[cuda]: SequenceArray (#6232) @a10y
- feat[cuda]: patches kernel (#6231) @a10y
- Pass limit through Python Vortex (#6239) @gatesn
- Ship Vortex CLI as part of Python package (#6233) @gatesn
- Feature: Add more
Filter<_>implementations (#6173) @connortsui20 - Benchmarks website revamped (#6215) @palaska
- Dataset API (#5949) @gatesn
- feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
- feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
- feat[python]: support full read/write from object storage (#6022) @a10y
- feat[scan]: gpu scan (#6199) @joseph-isaacs
- feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
- feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
- Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
- feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
- Feature: Use
array.filter()instead offilter(array)compute (#6146) @connortsui20 - Rewrite benchmark website (#6160) @palaska
- feat: stats pruning for LIKE filters (#6049) @a10y
- feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
- feat[cuda]: canonical
to_host(#6130) @joseph-isaacs - feat[cuda]: dict decimal take (#6129) @joseph-isaacs
- feat[gpu]: dict take values prim (#6101) @joseph-isaacs
- chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
- feat[gpu]: scalar encodings (#6109) @joseph-isaacs
- Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
- feat:
nvcompwrapper crate (#6108) @0ax1 - Allow configuring different readers for DF integration (#6027) @AdamGS
- feat: introduce
CudaDeviceBuffer(#6060) @0ax1 - Array VTable (#6000) @gatesn
- feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
- Attempting to simplify the IO read traits (#5935) @AdamGS
- feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
- feat[array]:
execute::<PrimitiveArray>()(#5954) @joseph-isaacs - Spark 4 (#5944) @a10y
- feat: cuda gpu skeleton (#5936) @0ax1
- Add flag to run benchmarks under Samply (#5940) @gatesn
- Datafusion 52 upgrade (#5863) @AdamGS
- Support precision casting for
DScalar(#5937) @AdamGS - feat[array]: use execute in canonical (#5925) @joseph-isaacs
- feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
- feat[array]: use execute to return
Canonicalarray notVector(#5895) @joseph-isaacs - feature: TableStrategy (#5640) @a10y
🚀 Performance
- Faster primitive
take(#6559) @connortsui20 - perf[vortex-btrblocks]: remove listview expansion ratio (#6530) @asubiotto
- VarBinBuilder: eagerly set IsSorted stat on offsets (#6452) @dimitarvdimitrov
- Implement ReduceNode for Array to not require creating new arc when returning children (#6443) @robert3005
- chore: clean cancelled read requests earlier in driver state transitions (#6382) @caicancai
- shared array to avoid duplicate execute (#6453) @onursatici
- vortex-btrblocks: write listviews when elements expand > 50% (#6393) @asubiotto
- Implement ReduceNode for ScalarFnArray to avoid boxing (#6435) @robert3005
- Improve visitor performance by reducing
Vecand String allocations (#6337) @AdamGS - lazy dict layout values eval (#6423) @onursatici
- cuda strategy to use zstd buffers for strings (#6403) @onursatici
- bitunpacking cuda kernels store output into shared memory before copying to main memory (#6384) @robert3005
- chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
- constant array metadata for small scalars (#6363) @onursatici
- chore[array]: move over filtering to
{reduce/execute}_parent(#6312) @joseph-isaacs - perf[buf]: inline
collect_bool(#6313) @joseph-isaacs - perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
- chore: list contains over scalars (#6242) @blaginin
- perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
- performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
- perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
- Check read-only cache first (#6249) @gatesn
- Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
- Perf: Use optimized filter implementations is
FilterArray::execute(#6189) @connortsui20 - coalesced request align all segments (#6158) @onursatici
- perf: CUDA async to host copy (#6085) @0ax1
- feat: make host to device copy async (#6075) @0ax1
- Remove duplicate ValidityVTable logic (#5975) @gatesn
- performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
- Only check for ConstantVTable (#6002) @gatesn
- do not validate is_sorted on runend (#6025) @onursatici
- perf: CUDA FoR loop unrolling (#6017) @0ax1
- feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
- Remove unnecessary work from
ListArray::validate(#6004) @AdamGS - scan: make LazyS...
0.59.0-rc.0
Immutable
release. Only release title and notes can be modified.
Changes
- docs: tidy and unify Python quickstart examples (#5972) @caicancai
- Better
vx-benchdocs in README (#5900) @AdamGS
⚠️ Breaks
- chore[array]: inline take compute function (#6376) @joseph-isaacs
- Add Array::deserialize arguments (#6340) @gatesn
- Update FSST to store components of compressed codes (#6325) @gatesn
- Array Execution (#6307) @gatesn
- Execute parent to return ArrayRef (#6257) @gatesn
- Extension DType vtables (#6081) @gatesn
- break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
- Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
- fallible scalar_at and validity (#6102) @joseph-isaacs
- Remove the python feature from vortex-error (#6064) @AdamGS
- break[*]: make array methods fallible (#6086) @joseph-isaacs
- chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
- Array VTable (#6000) @gatesn
- Remove duplicate ValidityVTable logic (#5975) @gatesn
- chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
- Remove EncodeVTable (#6036) @gatesn
- chore: fallible canonical (#6006) @joseph-isaacs
- feat[array]: use execute to return
Canonicalarray notVector(#5895) @joseph-isaacs
✨ Features
- Make
VortexSource::newpublic (#6392) @AdamGS - Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
- feat[expr]: new fill null expr (#6385) @joseph-isaacs
- zstd buffers array (#6362) @onursatici
- gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
- cuda dictionary of vbv decompression (#6361) @0ax1
- CLI flag to enable tracing for benchmarks (#6350) @gatesn
- chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
- chore[cuda]: add date32 exporter test (#6345) @a10y
- feat: display tree with no metadata (#5898) @danking
- feat[cuda]: device exports for varbinview (#6338) @a10y
- vortex-metrics v2 (#6187) @AdamGS
- feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
- feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
- feat[gpu]: CUDA
date_time_partskernel (#6303) @0ax1 - feat:
alternateforStructFields(#6276) @blaginin - Add a canonical matcher (#6292) @gatesn
- Move over to array matchers (#6291) @gatesn
- feat[gpu]: cuda runend decoding (#6275) @0ax1
- feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
- feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
- feat[array]: validate array encodings on write (#6241) @joseph-isaacs
- feat[cuda]: SequenceArray (#6232) @a10y
- feat[cuda]: patches kernel (#6231) @a10y
- Pass limit through Python Vortex (#6239) @gatesn
- Ship Vortex CLI as part of Python package (#6233) @gatesn
- Feature: Add more
Filter<_>implementations (#6173) @connortsui20 - Benchmarks website revamped (#6215) @palaska
- Dataset API (#5949) @gatesn
- feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
- feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
- feat[python]: support full read/write from object storage (#6022) @a10y
- feat[scan]: gpu scan (#6199) @joseph-isaacs
- feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
- feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
- Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
- feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
- Feature: Use
array.filter()instead offilter(array)compute (#6146) @connortsui20 - Rewrite benchmark website (#6160) @palaska
- feat: stats pruning for LIKE filters (#6049) @a10y
- feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
- feat[cuda]: canonical
to_host(#6130) @joseph-isaacs - feat[cuda]: dict decimal take (#6129) @joseph-isaacs
- feat[gpu]: dict take values prim (#6101) @joseph-isaacs
- chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
- feat[gpu]: scalar encodings (#6109) @joseph-isaacs
- Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
- feat:
nvcompwrapper crate (#6108) @0ax1 - Allow configuring different readers for DF integration (#6027) @AdamGS
- feat: introduce
CudaDeviceBuffer(#6060) @0ax1 - Array VTable (#6000) @gatesn
- feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
- Attempting to simplify the IO read traits (#5935) @AdamGS
- feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
- feat[array]:
execute::<PrimitiveArray>()(#5954) @joseph-isaacs - Spark 4 (#5944) @a10y
- feat: cuda gpu skeleton (#5936) @0ax1
- Add flag to run benchmarks under Samply (#5940) @gatesn
- Datafusion 52 upgrade (#5863) @AdamGS
- Support precision casting for
DScalar(#5937) @AdamGS - feat[array]: use execute in canonical (#5925) @joseph-isaacs
- feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
- feat[array]: use execute to return
Canonicalarray notVector(#5895) @joseph-isaacs - feature: TableStrategy (#5640) @a10y
🚀 Performance
- chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
- constant array metadata for small scalars (#6363) @onursatici
- chore[array]: move over filtering to
{reduce/execute}_parent(#6312) @joseph-isaacs - perf[buf]: inline
collect_bool(#6313) @joseph-isaacs - perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
- chore: list contains over scalars (#6242) @blaginin
- perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
- performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
- perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
- Check read-only cache first (#6249) @gatesn
- Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
- Perf: Use optimized filter implementations is
FilterArray::execute(#6189) @connortsui20 - coalesced request align all segments (#6158) @onursatici
- perf: CUDA async to host copy (#6085) @0ax1
- feat: make host to device copy async (#6075) @0ax1
- Remove duplicate ValidityVTable logic (#5975) @gatesn
- performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
- Only check for ConstantVTable (#6002) @gatesn
- do not validate is_sorted on runend (#6025) @onursatici
- perf: CUDA FoR loop unrolling (#6017) @0ax1
- feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
- Remove unnecessary work from
ListArray::validate(#6004) @AdamGS - scan: make LazyScanStream initialization non-blocking (#5906) @godnight10061
- perf[sequence]: array constant compare execution kernel (#5795) @joseph-isaacs
- perf: reduce time_elapsed_opening overhead (#5879) @godnight10061
🐛 Bug Fixes
89 changes
- Measure output bytes in cuda bitunpack benchmarks (#6395) @robert3005
- fix[array]: decimal fill_null larger than buffer (#6388) @joseph-isaacs
- fix[array]: dict value nullable values (#6387) @joseph-isaacs
- fix[fuzz]: coverage report (#6375) @joseph-isaacs
- join ops before in format (#6370) @blaginin
- Include config.js in benchmark website docker image (#6379) @palaska
- Exporting struct array to arrow respects vortex nullabilty (#6365) @robert3005
- Fix null take on structs (#6364) @blaginin
- varbinview on device build skips validation (#6358) @onursatici
- fix: vortex-layout doc tests require zstd (#6179) @danking
- DF: Only calculate the size of the metadata cache entry once (#6342) @AdamGS
- bump time version to clear CI (#6335) @onursatici
- All-invalid compressor (#6330) @gatesn
- fix: Empty select (#6314) @gatesn
- Avoid some potential stack overflow cases with deep expression trees (#6321) @AdamGS
- shared array in dict layout (#6305) @onursatici
- skip sparse array validation for device resident arrays (#6315) @onursatici
- varbin to be constructed from buffer handle directly (#6317) @onursatici
- Do not use display_tree in error message (#6251) @gatesn
- Minor memory-related improvements in vortex-array and vortex-duckdb (#6299) @AdamGS
- use is_host to recursively check residency to skip validation (#6297) @onursatici
- Add slice kernel for BitPackedArray execute_parent (#6283) @onursatici
selectpreserve validity of fields (#6260) @connortsui20- fsst array: AllTrue and AllFalse are handled by filter fn (#6272) @blaginin
- fix: separate ptx build profiles (#6252) @0ax1
- fix[cuda]: Fix gpu_scan test (#6208) @a10y
- [benchmark-website] Fix mobile charts, better colors, better tooltip (#6226) @palaska
- fix: exclude flaky benchmarks from codspeed (#6220) @0ax1
- fix: skip buffer validation if on device (#6219) @0ax1
- fix: simd build (#6217) @0ax1
- chore[cuda]: fix windows (#6206) @joseph-isaacs
- Fix: dont do lazy filters on write (#6194) @connortsui20
- fix: evaluate OR for validity (#6184) @blaginin
- fix: cuda ptx folder (#6170) @joseph-isaacs
- Fix: Comment on a failed benchmark (#6181) @connortsui20
- fix[alp]: cast didn't cast patches (#6174) @joseph-isaacs
- fix[fuzz]: fix gpu fuzzer cfg flags (#6165) @joseph-isaacs
- fix[gpu]: synchronous copy to host (#6157) @0ax1
- fix[fuzz]: single sesssion for the fuzzer (#6155) @joseph-isaacs
- fix: sample multiples of 1024 (#6151) @danking
- fix: only link nvcomp for linux (#6144) @0ax1
- fix: fastlanes canonicalize benchmarks (#6141) @0ax1
- fix[fsst]: execute_parent empty validity non empty filter (#6133) @joseph-isaacs
- fix[vortex-array]: fix filter of sliced list (#6113) @asubiotto
- fix[list]: filter list execute parent (#6119) @blaginin
- fix[dtype]: fix typo in conversion from
dtypetodecimaltype(#6121) @joseph-isaacs - fix[bitpacking]: filter (#6120) @joseph-isaacs
- fix[vortex-...
0.58.0
Changes
✨ Features
- Feature: Make
vortex-tuia library (#5779) @connortsui20
🚀 Performance
🧰 Maintenance
- chore: add overflow tests for unsigned types in BigCast (#5866) @caicancai
0.57.2
0.57.1
0.57.0
Changes
⚠️ Breaks
- Fix BoolArray::new to take BitBuffer (#5690) @gatesn
- Lazy array expression evaluation (#5625) @gatesn
- Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
- Move stats into expr (#5582) @gatesn
- Combine Array VTable with Encoding (#5462) @gatesn
- Consolidate into stat_expression function (#5366) @gatesn
- chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
- feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs
✨ Features
- feat: teach TemporalMetadata about its relationship to DTypes (#5864) @danking
- Pluggable expression conversion for DF (#5853) @AdamGS
- Introduce extendable access plan for DF scans (#5855) @AdamGS
- Upgrade DF to 51 (#5189) @AdamGS
- Benchmarks re-org (#5676) @AdamGS
- Push down cast (#5803) @gatesn
- ListView to List (#5802) @gatesn
- feat[array]: stats + validity in display array (#5799) @joseph-isaacs
- deterministic encoding ordering (#5770) @onursatici
- feat[fixed-size-list]: impl repeat and slice (#5751) @joseph-isaacs
- feat[compute]: compare decimal and varbin (#5757) @joseph-isaacs
- feat[datafusion]: Support DataFusion
PartitionedFileextensions for external index (#5556) @haohuaijin - FSST Filter Kernel (#5755) @gatesn
- perf[ci]: stopped instances (#5753) @joseph-isaacs
- Feature: makes the decompressor and compressor methods public (#5752) @connortsui20
- Feature: add fused bitpack filter kernel (#5741) @connortsui20
- Execute parent (#5736) @gatesn
- feat[array]: duckdb executor (#5737) @joseph-isaacs
- Arrow Executor (#5647) @gatesn
- Feature: add primitive casting (#5718) @connortsui20
- Feature: add some
Castinfrastructure (#5708) @connortsui20 - feat[array]: dynamic expr execute (#5714) @joseph-isaacs
- Feature: add
append_array_as_listmethods to the list arrays (#5707) @connortsui20 - Feature: implement
DictArraybatched execute (#5686) @connortsui20 - feat[array]: list contains scalar (#5713) @joseph-isaacs
- feat[vector]: Eq (#5681) @joseph-isaacs
- Feature: add zero and null constructors for scalars (#5705) @connortsui20
- Feature: batch execute for
DeltaArray(#5667) @connortsui20 - feat[vector]: slice varbinview (#5697) @joseph-isaacs
- Feature: add
takefor the view vectors (#5682) @connortsui20 - Feature: Implement
takeon more vectors + stubs (#5653) @connortsui20 - feature: add
__version__attribute to Python package (#5641) @d33bs - Feature: add
takefor primitive and bool vectors (#5540) @connortsui20 - vx layout display (#5650) @joseph-isaacs
- Feature: added properties to VortexScanner (#5500) @sherlockbeard
- Add --version arg to vx (#5646) @palaska
- Feature: add total read size counter (#5600) @harryscholes
- feat[scalar_fn]: binary ops (#5617) @joseph-isaacs
- Execute ScalarFn arrays (#5611) @gatesn
- FilterArray (#5610) @gatesn
- Add GetItemFn (#5609) @gatesn
- Add MaskFn (#5606) @gatesn
- feat[scalar_fn]:
is_null(#5601) @joseph-isaacs - Scalar functions (#5561) @gatesn
- feat[buffer]: use BufferHandle in more places (#5567) @joseph-isaacs
- feat: alp decompress to vector (#5548) @0ax1
- feat[gpu]: use BufferHandle to the array vtable build (#5565) @joseph-isaacs
- feat[vortex-expr]:
is_fallibleanalysis (#5546) @joseph-isaacs - Feature: add
takecompute for slices (#5537) @connortsui20 - feat[vortex-expr]:
is_fallibleexpression analysis (#5531) @joseph-isaacs - feat[vortex-expr]: add a
is_null_sensitiveexpression analysis (#5528) @joseph-isaacs - Feature: add a take compute trait (#5503) @connortsui20
- Feature: add impl owned filter impls (#5493) @connortsui20
- Feature: add filter impl for
FixedSizeListVector(#5494) @connortsui20 - Vector cast (#5461) @gatesn
- Scalar arithmetic (#5460) @gatesn
- feat[cli]: add scroll for tree encoding view (#5497) @a10y
- Revert "Revert "chore[ci]: add windows test to CI (#5439)"" (#5486) @joseph-isaacs
- More vector scalars (#5459) @gatesn
- feature: Support std_file on windows as it's used by cxx bindings (#5450) @robert3005
- Feature: implement bitpacking batch execute (#5446) @connortsui20
- feat[vortex-array]: added debug to expr/array reduce rules (#5442) @joseph-isaacs
- feat: publish vortex-spark-all assembly JAR (#5435) @a10y
- feat[vortex-array] add an array session rule registry (#5414) @joseph-isaacs
- Feature: add
spare_capacity_muttoPVectorMut(#5287) @connortsui20 - feat: pipeline-pcodec (#5415) @0ax1
- Add VectorOps::clear (#5409) @gatesn
- feat[vortex-array]: expr array that represents lazy computation (#5400) @joseph-isaacs
- feat[vortex-expr]:
reduce_parenthas typed parent (#5398) @joseph-isaacs - feat: cast expression (#5389) @danking
- feat: PyExpr::evaluate (#5388) @danking
- feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs
- Faster Pipelines (#5352) @gatesn
- feat[vortex-expr]: better debug print (#5374) @joseph-isaacs
🚀 Performance
- perf: list contains scalar scalar (#5792) @0ax1
- Upgrade DF to 51 (#5189) @AdamGS
- performance[vortex-array]: don't call is_valid to count bytes in varbinview (#5814) @asubiotto
- perf[array]: extension pushdown filter + constant (#5794) @joseph-isaacs
- perf[expr]: between not fallible (#5798) @joseph-isaacs
- perf[dtp]: push down compare over dtp constant second subsecond (#5793) @joseph-isaacs
- perf[date-time-parts]: compressor handle constant and pushdown rules (#5778) @joseph-isaacs
- perf[duckdb]: dont push down pack & dedup filters for ddb (#5773) @joseph-isaacs
- ci: use windows-x64 pool for Windows tests (#5756) @joseph-isaacs
- feat[duckdb]: optimize array (#5738) @joseph-isaacs
- Perf: optimize
take_scalar(#5723) @connortsui20 - Make sure all encodings are enabled for DF (#5720) @AdamGS
- Perf: Optimized bool take (#5701) @connortsui20
- Run optimizer on root array only on (#5710) @gatesn
- perf[array]: Constant kernel filter pushdown (#5698) @joseph-isaacs
- perf[array]: with_children vtable function (#5694) @joseph-isaacs
- perf[ci]: speed up ci MORE (#5576) @joseph-isaacs
- perf(ci): skip rust-cache on runs-on runners with S3 sccache (#5570) @joseph-isaacs
- chunked array zip (#5530) @onursatici
- varbinview zip kernel (#5526) @onursatici
- chore[vortex-dict]: numeric pushdown & clean up the dict ctor (#5438) @joseph-isaacs
- Handwritten pipeline benchmark and analysis (#5408) @connortsui20
- Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
- perf[dict]: all_values_referenced to allow pushdown optimisation. (#5416) @joseph-isaacs
- perf[vortex-expr]: replace can stop looking at subtree if it finds a match (#5405) @joseph-isaacs
- actually make ZCTL (#5402) @a10y
- fix: performance regression for filtering ListView (#5390) @a10y
- fix[vortex-dict]: faster min_max for dict array (#5288) @joseph-isaacs
- fix: yo dawg, I heard you like inlining (#5363) @a10y
🐛 Bug Fixes
81 changes
- fix: enable all features for vortex-mask docsrs and enable serde-derive when serde is enabled (#5871) @robert3005
- fix[array]: correct the handling of to_arrow execution and struct casting (#5867) @joseph-isaacs
- fix[ci]: pin maturin due to fix upstream error (#5868) @joseph-isaacs
- fix: int compressor mul overflow bp (#5854) @joseph-isaacs
- Fix: IPC
AsyncMessageReaderdoes not handle partial reads (#5758) @connortsui20 - Fix: incorrect child vector size in duckdb list array import (#5740) @connortsui20
- Fix: bring back lance compression benchmark (#5840) @connortsui20
- Fix: don't fail on missing lance for datafusion (#5841) @connortsui20
- Fix: CI sql script (#5838) @connortsui20
- Fix: CI SQL Remote (#5836) @connortsui20
- Fix: finally fix CI (#5835) @connortsui20
- Fix: actually actually fix CI (#5830) @connortsui20
- Fix: benchmark name for
bench.yml(#5828) @connortsui20 - Fix: remove lance feature from vortex-bench CI (#5823) @connortsui20
- Fix: decimal cast error message (#5821) @connortsui20
- Fix Arrow list export (#5804) @gatesn
- Fix: support widening of
ListViewVectoroffsets and sizes (#5796) @connortsui20 - perf[layout]: apply then filter (#5788) @joseph-isaacs
- perf[date-time-parts]: compressor handle constant and pushdown rules p2 (#5786) @joseph-isaacs
- fix: disable noisy btrblocks micro-benchmarks on codspeed (#5782) @0ax1
- Fix arrow conversion (#5769) @gatesn
- fix: downcast in list
new_vector_exporter(#5768) @0ax1 - fix[duckdb]: no
to_structin exporter (#5750) @joseph-isaacs - Fix: remove memory limit on fuzzers (#5763) @connortsui20
- Fix:
ListArraytakeincorrect validity (#5762) @connortsui20 - fix: list contains (#5748) @0ax1
- fix[layout]: soundness note on values eval (#5747) @joseph-isaacs
- fix: only validate non-null decimals (#5745) @0ax1
- fix: zstd frame step width for alignment (#5730) @0ax1
- fix[duckdb]: remove validity mask from vector export (#5739) @joseph-isaacs
- fix:
Buffersliceend does not need to be aligned (#5734) @0ax1 - Fix Arrow struct export (#5731) @gatesn
- Disable GHArchive benchmarks to unblock perf reference in CI (#5729) @AdamGS
- feat[array]: fixup operator stuff (#5711) @joseph-isaacs
- Reduce optimizer logging to debug (#5702) @gatesn
- fix: list take offset type (#5679) @0ax1
- fix: make error type explicit (#5696) @0ax1
- Avoid taking write lock on VortexSession::get (#5688) @gatesn
- Fix: claude action concurrency group (#5680) @connortsui20
- Temp fix: Make
takebound byCopyinstead ofNativePType(#5666) @connortsui20 - Fix: add
RUSTFLAGS="--cfg vortex_nightly"to all nightly tests (#5655) @connortsui20 - Fix: portable SIMD
takeimplementation (#5654) @connortsui20 - Fix session get-or-default (#5662) @gatesn
- Include zstd encoding in IO fuzzer (#5661) @AdamGS
- Feature: added properties to VortexScanner (#5500) @sherlockbeard
- fix[vortex-array]: fix take on varbinviews with NULL indices...