Skip to content

gh-149718: Aggregate same stack frames in Tachyon in some collectors#149719

Merged
pablogsal merged 7 commits into
python:mainfrom
maurycy:remote-debugger-batching
May 12, 2026
Merged

gh-149718: Aggregate same stack frames in Tachyon in some collectors#149719
pablogsal merged 7 commits into
python:mainfrom
maurycy:remote-debugger-batching

Conversation

@maurycy

@maurycy maurycy commented May 12, 2026

Copy link
Copy Markdown
Contributor

Please see gh-149718 for more details.

On a MacBook M4, using test_asyncio since it has longer stacks (without --async-aware):

Sample rate (samples/sec)

Format main (c6fd7de) remote-debugger-batching (2864c275847)
--pstats 31,161.59 103,657.01
--collapsed 57,302.45 102,350.22
--flamegraph 33,490.73 100,641.14
--gecko 31,346.37 98,031.88
--heatmap 14,960.64 97,607.28
--jsonl 24,554.56 98,018.93

Error rate (%)

Format main (c6fd7de) remote-debugger-batching (2864c275847)
--pstats 1.94 0.54
--collapsed 1.47 0.78
--flamegraph 1.77 0.57
--gecko 1.92 0.57
--heatmap 2.33 0.44
--jsonl 2.06 0.54

Missed samples (%)

Format main (c6fd7de) remote-debugger-batching (2864c275847)
--pstats 90.65 68.90
--collapsed 82.81 69.29
--flamegraph 89.95 69.81
--gecko 90.60 70.59
--heatmap 95.51 70.72
--jsonl 92.63 70.59

Bench

mkdir -p /tmp/bench_results

git checkout c6fd7de
sudo -n ./python.exe -m profiling.sampling run -r 300khz --pstats     -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_pstats.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --collapsed  -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_collapsed.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --flamegraph -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_flamegraph.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --gecko      -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_gecko.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --heatmap    -o /tmp/heatmap_main    -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_heatmap.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --jsonl      -o /dev/null            -m test test_asyncio 2>&1 | tee /tmp/bench_results/main_jsonl.txt

git checkout remote-debugger-batching
sudo -n ./python.exe -m profiling.sampling run -r 300khz --pstats     -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_pstats.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --collapsed  -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_collapsed.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --flamegraph -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_flamegraph.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --gecko      -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_gecko.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --heatmap    -o /tmp/heatmap_batching  -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_heatmap.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --jsonl      -o /dev/null              -m test test_asyncio 2>&1 | tee /tmp/bench_results/batching_jsonl.txt

Comment thread Lib/profiling/sampling/collector.py

@pablogsal pablogsal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fantastic! Thanks a lot @maurycy 🚀 Excellent job

@pablogsal pablogsal enabled auto-merge (squash) May 12, 2026 23:31
@pablogsal pablogsal added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 12, 2026
@pablogsal pablogsal merged commit 76f2285 into python:main May 12, 2026
58 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @maurycy for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@bedevere-app

bedevere-app Bot commented May 12, 2026

Copy link
Copy Markdown

GH-149747 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 12, 2026
@maurycy maurycy deleted the remote-debugger-batching branch May 12, 2026 23:47
pablogsal pushed a commit that referenced this pull request May 13, 2026
…ectors (GH-149719) (#149747)

gh-149718: Aggregate same stack frames in Tachyon in some collectors (GH-149719)
(cherry picked from commit 76f2285)

Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants