Skip to content

feat(prof): use a trampoline for FLF functions to intercept timings#3595

Open
bwoebi wants to merge 21 commits intomasterfrom
bob/prof-flf-test
Open

feat(prof): use a trampoline for FLF functions to intercept timings#3595
bwoebi wants to merge 21 commits intomasterfrom
bob/prof-flf-test

Conversation

@bwoebi
Copy link
Copy Markdown
Collaborator

@bwoebi bwoebi commented Jan 21, 2026

Frameless functions (FLF) do not check the EG(vm_interrupt) flag like it would happen usually on internal function returns. This means that from the point of view of the profiler, those functions do not exist, as the engine never checks the interrupt flag and as such never calls the profilers interrupt handler.

This PR adds a trampoline for aarch64 and x84_64 to all known frameless functions and a tail call to check the EG(vm_interrupt) flag and in case it is raised, call the interrupt handler.

One (acceptable) trade-off is that using these trampolines, libunwind in the crash tracker is not able to unwind passed the trampoline. Anyway, the impact is minimal:

  • Stack frames from the crash site up to the trampoline are still captured
  • Only frames above the trampoline (from caller to main) are lost
  • For most crashes, the relevant context is near the crash site, not at the program entry point

The profiler itself is unaffected of this, as we are unwinding the VM stack, following the execute_data linked list.

https://datadoghq.atlassian.net/browse/PROF-12085

@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 1976c3c to 222e3a4 Compare January 21, 2026 17:19
@datadog-datadog-prod-us1
Copy link
Copy Markdown

datadog-datadog-prod-us1 bot commented Jan 21, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 60.68% (+0.03%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ce9eaa3 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jan 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.48%. Comparing base (f1af9ca) to head (ce9eaa3).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3595      +/-   ##
==========================================
- Coverage   68.79%   67.48%   -1.31%     
==========================================
  Files         166      166              
  Lines       19015    19015              
  Branches     1792     1792              
==========================================
- Hits        13081    12832     -249     
- Misses       5121     5373     +252     
+ Partials      813      810       -3     
Flag Coverage Δ
helper-rust-integration 69.53% <ø> (-9.30%) ⬇️
helper-rust-unit 49.36% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 15 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1af9ca...ce9eaa3. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Collaborator

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this can run afoul of permissions somehow, since we're making executable code at runtime? I'm not well versed here, and yes, if the JIT is enabled you'd have to have that capability anyway, but I'm trying to understand implications of this WIP.

@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Jan 21, 2026

Benchmarks [ profiler ]

Benchmark execution time: 2026-03-27 18:32:25

Comparing candidate commit ce9eaa3 in PR branch bob/prof-flf-test with baseline commit f1af9ca in branch master.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 26 metrics, 8 unstable metrics.

scenario:php-profiler-timeline-memory-control

  • 🟥 cpu_user_time [+32.466ms; +37.703ms] or [+5.345%; +6.208%]
  • 🟥 execution_time [+36.226ms; +41.481ms] or [+5.720%; +6.550%]

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 222e3a4 to 0eb96ee Compare January 21, 2026 17:52
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from dc825a7 to c6bdefc Compare January 21, 2026 18:06
@bwoebi
Copy link
Copy Markdown
Collaborator Author

bwoebi commented Jan 21, 2026

@morrisonlevi dynasmrt takes care of setting RX permissions after compiling the code. As long as you're not running this under a hardened runtime (like app store apps or android (?)), there's no fundamental problem. Also I currently call assembler.finalize().unwrap() (which takes care of settign RX) - proper usage should just handle the potential error here and abort.

@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 95f422a to 7471599 Compare January 21, 2026 18:17
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 7471599 to 4358ab8 Compare January 21, 2026 18:24
dynasm!(assembler
; mov rax, QWORD original as i64
; call rax
; mov rax, QWORD interrupt_addr as i64
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume original is not returning anything, because you're writing over RAX.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct

#[cfg(target_arch = "aarch64")]
dynasm!(assembler
; mov x16, original as u64
; blr x16
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this overwrites x30/lr, aren't you gonna lose the original return location of the handler? so when br x16 returns, it goes back to calling interrupt_addr

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right. call on x86_64 pushes %eip to the stack, but blr doesn't.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed up, is it correct now?

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from a3c79dc to 078acef Compare January 21, 2026 19:34
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch 2 times, most recently from e4930fe to 2604b22 Compare January 21, 2026 19:35
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from 2604b22 to 6307860 Compare January 21, 2026 19:38
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from a246940 to bcebfb0 Compare January 21, 2026 20:37
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/prof-flf-test branch from b0995bc to d9d3f43 Compare January 22, 2026 13:45
@realFlowControl realFlowControl changed the title WIP: use a trampoline for FLF functions to intercept timings feat(prof): use a trampoline for FLF functions to intercept timings Feb 16, 2026
@realFlowControl realFlowControl marked this pull request as ready for review February 16, 2026 17:17
@realFlowControl realFlowControl requested review from a team as code owners February 16, 2026 17:17
Copy link
Copy Markdown
Member

@realFlowControl realFlowControl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bwoebi this is awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants