Commit 03df5f5
committed
Add script to diff zjit stats from output csv
Example output:
================================================================================
ZJIT Stats Comparison
================================================================================
ruby-mdev (baseline):
ruby 4.1.0dev (2026-01-30T17:29:15Z master 1298f9ac1a) +ZJIT dev +PRISM [arm64-darwin25]
ruby-bdev:
ruby 4.1.0dev (2026-01-30T17:29:15Z profile-block-arg 1298f9ac1a) +ZJIT dev +PRISM [arm64-darwin25]
BENCHMARK TIMINGS (lower is better)
--------------------------------------------------------------------------------
lobsters:
ruby-mdev avg: 1.076s min: 0.944s ★ (baseline)
ruby-bdev avg: 1.103s min: 0.967s +2.5% (slower)
railsbench:
ruby-mdev avg: 1.580s min: 1.532s ★ (baseline)
ruby-bdev avg: 1.587s min: 1.539s +0.4% (slower)
MEMORY USAGE
--------------------------------------------------------------------------------
lobsters:
ruby-mdev maxrss: 510.3MB zjit_mem: 65.9MB
ruby-bdev maxrss: 510.6MB zjit_mem: 65.5MB
railsbench:
ruby-mdev maxrss: 200.2MB zjit_mem: 30.1MB
ruby-bdev maxrss: 203.9MB zjit_mem: 29.8MB
NOT INLINED C METHODS (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
#<Module:0x000000012a06a460>#object_id 241 → 0 ▼ -100.0%
Kernel#instance_variable_defined? 416 → 8 ▼ -98.1%
Numeric#nonzero? 9,789 → 1,556 ▼ -84.1%
... and 11 more
railsbench:
Array#hash 133 → 285 ▲ +114.3%
#<Module:0x000000012804a500>#enum_for 180 → 0 ▼ -100.0%
String#<=> 9,896 → 1,674 ▼ -83.1%
... and 10 more
CALLS TO C FUNCTIONS FROM JIT CODE (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
#<Module:0x000000012a06a460>#object_id 241 → 0 ▼ -100.0%
Kernel#instance_variable_defined? 416 → 8 ▼ -98.1%
Class#new 561 → 22 ▼ -96.1%
... and 18 more
railsbench:
Array#hash 133 → 285 ▲ +114.3%
#<Module:0x000000012804a500>#enum_for 180 → 0 ▼ -100.0%
Numeric#nonzero? 9,896 → 1,674 ▼ -83.1%
... and 15 more
NOT OPTIMIZED METHOD TYPES FOR SEND (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
optimized 0 → 4,591 ▲ new
railsbench:
optimized 19,471 → 49,608 ▲ +154.8%
NOT OPTIMIZED METHOD TYPES FOR SUPER (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
attrset 2,642 → 1,866 ▼ -29.4%
SEND FALLBACK REASONS (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
send_not_optimized_method_type 1,682 → 6,273 ▲ +272.9%
one_or_more_complex_arg_pass 1,145,141 → 2,821,156 ▲ +146.4%
send_no_profiles 2,068,333 → 196,661 ▼ -90.5%
... and 3 more
railsbench:
send_not_optimized_method_type 19,471 → 49,608 ▲ +154.8%
one_or_more_complex_arg_pass 2,641,602 → 4,786,718 ▲ +81.2%
send_no_profiles 4,046,018 → 1,069,740 ▼ -73.6%
... and 1 more
SETIVAR FALLBACK REASONS (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
new_shape_needs_extension 946 → 170 ▼ -82.0%
railsbench:
new_shape_needs_extension 935 → 161 ▼ -82.8%
GETBLOCKPARAMPROXY HANDLER (showing differences > 10.0%)
--------------------------------------------------------------------------------
railsbench:
no_profiles 1,146 → 350 ▼ -69.5%
COMPLEX ARGUMENT-PARAMETER FEATURES (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
caller_blockarg 21,564 → 1,697,716 ▲+7772.9%
caller_kwarg 23,322 → 135,011 ▲ +478.9%
caller_splat 33,663 → 37,288 ▲ +10.8%
railsbench:
caller_kwarg 944 → 110,104 ▲+11563.6%
caller_blockarg 184,024 → 2,329,272 ▲+1165.7%
caller_splat 170,800 → 260,718 ▲ +52.6%
SIDE EXIT REASONS (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
obj_to_string_fallback 1,948 → 443 ▼ -77.3%
railsbench:
obj_to_string_fallback 2,021 → 370 ▼ -81.7%
NOT ANNOTATED C METHODS (showing differences > 10.0%)
--------------------------------------------------------------------------------
lobsters:
#<Module:0x000000012a06a460>#object_id 241 → 0 ▼ -100.0%
Class#new 561 → 22 ▼ -96.1%
String#<=> 9,789 → 1,556 ▼ -84.1%
... and 10 more
railsbench:
Array#hash 133 → 285 ▲ +114.3%
#<Module:0x000000012804a500>#enum_for 180 → 0 ▼ -100.0%
Numeric#nonzero? 9,896 → 1,674 ▼ -83.1%
... and 11 more1 parent 5b8bc3a commit 03df5f5
1 file changed
+434
-0
lines changed
0 commit comments