Skip to content

Commit 03df5f5

Browse files
committed
Add script to diff zjit stats from output csv
Example output: ================================================================================ ZJIT Stats Comparison ================================================================================ ruby-mdev (baseline): ruby 4.1.0dev (2026-01-30T17:29:15Z master 1298f9ac1a) +ZJIT dev +PRISM [arm64-darwin25] ruby-bdev: ruby 4.1.0dev (2026-01-30T17:29:15Z profile-block-arg 1298f9ac1a) +ZJIT dev +PRISM [arm64-darwin25] BENCHMARK TIMINGS (lower is better) -------------------------------------------------------------------------------- lobsters: ruby-mdev avg: 1.076s min: 0.944s ★ (baseline) ruby-bdev avg: 1.103s min: 0.967s +2.5% (slower) railsbench: ruby-mdev avg: 1.580s min: 1.532s ★ (baseline) ruby-bdev avg: 1.587s min: 1.539s +0.4% (slower) MEMORY USAGE -------------------------------------------------------------------------------- lobsters: ruby-mdev maxrss: 510.3MB zjit_mem: 65.9MB ruby-bdev maxrss: 510.6MB zjit_mem: 65.5MB railsbench: ruby-mdev maxrss: 200.2MB zjit_mem: 30.1MB ruby-bdev maxrss: 203.9MB zjit_mem: 29.8MB NOT INLINED C METHODS (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: #<Module:0x000000012a06a460>#object_id 241 → 0 ▼ -100.0% Kernel#instance_variable_defined? 416 → 8 ▼ -98.1% Numeric#nonzero? 9,789 → 1,556 ▼ -84.1% ... and 11 more railsbench: Array#hash 133 → 285 ▲ +114.3% #<Module:0x000000012804a500>#enum_for 180 → 0 ▼ -100.0% String#<=> 9,896 → 1,674 ▼ -83.1% ... and 10 more CALLS TO C FUNCTIONS FROM JIT CODE (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: #<Module:0x000000012a06a460>#object_id 241 → 0 ▼ -100.0% Kernel#instance_variable_defined? 416 → 8 ▼ -98.1% Class#new 561 → 22 ▼ -96.1% ... and 18 more railsbench: Array#hash 133 → 285 ▲ +114.3% #<Module:0x000000012804a500>#enum_for 180 → 0 ▼ -100.0% Numeric#nonzero? 9,896 → 1,674 ▼ -83.1% ... and 15 more NOT OPTIMIZED METHOD TYPES FOR SEND (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: optimized 0 → 4,591 ▲ new railsbench: optimized 19,471 → 49,608 ▲ +154.8% NOT OPTIMIZED METHOD TYPES FOR SUPER (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: attrset 2,642 → 1,866 ▼ -29.4% SEND FALLBACK REASONS (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: send_not_optimized_method_type 1,682 → 6,273 ▲ +272.9% one_or_more_complex_arg_pass 1,145,141 → 2,821,156 ▲ +146.4% send_no_profiles 2,068,333 → 196,661 ▼ -90.5% ... and 3 more railsbench: send_not_optimized_method_type 19,471 → 49,608 ▲ +154.8% one_or_more_complex_arg_pass 2,641,602 → 4,786,718 ▲ +81.2% send_no_profiles 4,046,018 → 1,069,740 ▼ -73.6% ... and 1 more SETIVAR FALLBACK REASONS (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: new_shape_needs_extension 946 → 170 ▼ -82.0% railsbench: new_shape_needs_extension 935 → 161 ▼ -82.8% GETBLOCKPARAMPROXY HANDLER (showing differences > 10.0%) -------------------------------------------------------------------------------- railsbench: no_profiles 1,146 → 350 ▼ -69.5% COMPLEX ARGUMENT-PARAMETER FEATURES (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: caller_blockarg 21,564 → 1,697,716 ▲+7772.9% caller_kwarg 23,322 → 135,011 ▲ +478.9% caller_splat 33,663 → 37,288 ▲ +10.8% railsbench: caller_kwarg 944 → 110,104 ▲+11563.6% caller_blockarg 184,024 → 2,329,272 ▲+1165.7% caller_splat 170,800 → 260,718 ▲ +52.6% SIDE EXIT REASONS (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: obj_to_string_fallback 1,948 → 443 ▼ -77.3% railsbench: obj_to_string_fallback 2,021 → 370 ▼ -81.7% NOT ANNOTATED C METHODS (showing differences > 10.0%) -------------------------------------------------------------------------------- lobsters: #<Module:0x000000012a06a460>#object_id 241 → 0 ▼ -100.0% Class#new 561 → 22 ▼ -96.1% String#<=> 9,789 → 1,556 ▼ -84.1% ... and 10 more railsbench: Array#hash 133 → 285 ▲ +114.3% #<Module:0x000000012804a500>#enum_for 180 → 0 ▼ -100.0% Numeric#nonzero? 9,896 → 1,674 ▼ -83.1% ... and 11 more
1 parent 5b8bc3a commit 03df5f5

File tree

1 file changed

+434
-0
lines changed

1 file changed

+434
-0
lines changed

0 commit comments

Comments
 (0)