python 3.14 compiled with zlib instead of zlib-ng -- 30% slower on windows for zlib #821
Description
Solution to issue cannot be found in the documentation.
- I checked the documentation.
Issue
Hello,
I am trying the new release of python 3.14.
This is a short script to compress a log file that's 58 MB.
Running on my laptop with turbo boost off to have stable results.
official python release on windows (you can download from python.org)
.\Python314\python.exe .\bench.py
Data size: 58.82 MB
compress: 0.618 seconds (95.13 MB/s)
decompress: 0.059 seconds (1003.92 MB/s)
crc32: 0.002 seconds (26193.28 MB/s)
python release on conda forge (newly created with conda create ... python=3.14)
.\myconda314\python.exe .\bench.py
Data size: 58.82 MB
compress: 1.056 seconds (55.72 MB/s)
decompress: 0.107 seconds (551.28 MB/s)
crc32: 0.022 seconds (2656.00 MB/s)
PROBLEM: conda python is 30%-50% slower than the official release of python, 1000% slower on crc 😢
REASON: With the official release of python 3.14 on Windows, the python interpreter has switched to zlib-ng as the compression library. This provides an enormous performance boost to compression/decompression/crc routines.
This affects all usage of compression including zlib, gzip, tarfile, pip install, etc...
python/cpython#91349
Conda is still using the old zlib library, that's slow and mostly unmaintained.
Do you think you people could update conda to build with zlib-ng in compatibility mode?
It's a drop in replacement for zlib.
Regards.
Installed packages
python=3.14 on windowsEnvironment info
windows laptop
bench.py
import zlib
import time
# Load data
with open('access.log', 'rb') as f:
data = f.read()
size_mb = len(data) / 1024 / 1024
print(f"Data size: {size_mb:.2f} MB\n")
# Compress
start = time.perf_counter()
compressed = zlib.compress(data)
compress_time = time.perf_counter() - start
print(f"compress: {compress_time:.3f} seconds ({size_mb/compress_time:.2f} MB/s)")
# Decompress
start = time.perf_counter()
decompressed = zlib.decompress(compressed)
decompress_time = time.perf_counter() - start
print(f"decompress: {decompress_time:.3f} seconds ({size_mb/decompress_time:.2f} MB/s)")
# CRC32
start = time.perf_counter()
checksum = zlib.crc32(data)
crc_time = time.perf_counter() - start
print(f"crc32: {crc_time:.3f} seconds ({size_mb/crc_time:.2f} MB/s)")