[DRAFT] NOT MERGING POC: Send JVM runtime metrics via OTLP using OTel-native naming by link04 · Pull Request #10985 · DataDog/dd-trace-java

link04 · 2026-03-27T16:13:57Z

Do not review — initial POC only.

Sends jvm.memory.used, jvm.gc.duration, jvm.thread.count, jvm.class.loaded, jvm.cpu.recent_utilization via OTLP. Related: DataDog/dd-trace-dotnet#8299

🤖 Generated with Claude Code

pr-commenter · 2026-03-27T16:56:50Z

Benchmarks

⚠️ Warning: Baseline build not found for merge-base commit. Comparing against the latest commit on master instead.

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	maximo/otlp-runtime-metrics-poc
git_commit_date	1778103449	1778108168
git_commit_sha	`2e6ce6c`	`5a31372`
release_version	1.63.0-SNAPSHOT~2e6ce6ce0c	1.62.0-SNAPSHOT~5a31372eb2

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1778110045	1778110045
ci_job_id	1663320276	1663320276
ci_pipeline_id	111888000	111888000
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-zdjy8du2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-zdjy8du2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 7 performance improvements and 2 performance regressions! Performance is the same for 48 metrics, 14 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:BytebuddyAgent	better [-28.732ms; -21.763ms] or [-3.468%; -2.627%]	803.320ms	828.567ms
scenario:startup:insecure-bank:tracing:Debugger	better [-4.743ms; -1.648ms] or [-7.613%; -2.644%]	59.114ms	62.309ms
scenario:startup:insecure-bank:tracing:Telemetry	better [-393.293µs; -196.736µs] or [-4.737%; -2.370%]	8.007ms	8.302ms
scenario:startup:petclinic:appsec:Telemetry	worse [+636.028µs; +1078.372µs] or [+8.135%; +13.792%]	8.676ms	7.819ms
scenario:startup:petclinic:appsec:Flare Poller	better [-5.718ms; -5.263ms] or [-62.773%; -57.771%]	3.619ms	9.110ms
scenario:startup:petclinic:iast:BytebuddyAgent	better [-47.321ms; -20.915ms] or [-5.674%; -2.508%]	799.854ms	833.972ms
scenario:startup:petclinic:profiling:Agent.start	better [-148.423ms; -115.258ms] or [-11.263%; -8.746%]	1.186s	1.318s
scenario:startup:petclinic:profiling:crashtracking	worse [+620.095µs; +655.049µs] or [+114.409%; +120.858%]	1179.571µs	541.999µs
scenario:startup:petclinic:profiling:Debugger	better [-4.977ms; -1.627ms] or [-7.300%; -2.386%]	64.874ms	68.175ms

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1066578
Total [baseline] (8.875 s) : 0, 8874523
Agent [candidate] (1.056 s) : 0, 1055849
Total [candidate] (8.858 s) : 0, 8858408
section iast
Agent [baseline] (1.251 s) : 0, 1251067
Total [baseline] (9.531 s) : 0, 9531313
Agent [candidate] (1.227 s) : 0, 1226568
Total [candidate] (9.585 s) : 0, 9585066

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.067 s	-
Agent	iast	1.251 s	184.489 ms (17.3%)
Total	tracing	8.875 s	-
Total	iast	9.531 s	656.79 ms (7.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	iast	1.227 s	170.719 ms (16.2%)
Total	tracing	8.858 s	-
Total	iast	9.585 s	726.658 ms (8.2%)

gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.253 ms) : 0, 1253
crashtracking [candidate] (1.238 ms) : 0, 1238
BytebuddyAgent [baseline] (636.521 ms) : 0, 636521
BytebuddyAgent [candidate] (632.716 ms) : 0, 632716
AgentMeter [baseline] (29.78 ms) : 0, 29780
AgentMeter [candidate] (29.334 ms) : 0, 29334
GlobalTracer [baseline] (247.612 ms) : 0, 247612
GlobalTracer [candidate] (248.568 ms) : 0, 248568
AppSec [baseline] (32.725 ms) : 0, 32725
AppSec [candidate] (32.059 ms) : 0, 32059
Debugger [baseline] (62.309 ms) : 0, 62309
Debugger [candidate] (59.114 ms) : 0, 59114
Remote Config [baseline] (604.935 µs) : 0, 605
Remote Config [candidate] (600.503 µs) : 0, 601
Telemetry [baseline] (8.302 ms) : 0, 8302
Telemetry [candidate] (8.007 ms) : 0, 8007
Flare Poller [baseline] (10.658 ms) : 0, 10658
Flare Poller [candidate] (8.144 ms) : 0, 8144
section iast
crashtracking [baseline] (1.255 ms) : 0, 1255
crashtracking [candidate] (1.242 ms) : 0, 1242
BytebuddyAgent [baseline] (828.567 ms) : 0, 828567
BytebuddyAgent [candidate] (803.32 ms) : 0, 803320
AgentMeter [baseline] (11.463 ms) : 0, 11463
AgentMeter [candidate] (11.421 ms) : 0, 11421
GlobalTracer [baseline] (237.275 ms) : 0, 237275
GlobalTracer [candidate] (239.62 ms) : 0, 239620
AppSec [baseline] (29.982 ms) : 0, 29982
AppSec [candidate] (32.795 ms) : 0, 32795
Debugger [baseline] (64.785 ms) : 0, 64785
Debugger [candidate] (58.882 ms) : 0, 58882
Remote Config [baseline] (538.249 µs) : 0, 538
Remote Config [candidate] (1.133 ms) : 0, 1133
Telemetry [baseline] (7.996 ms) : 0, 7996
Telemetry [candidate] (12.474 ms) : 0, 12474
Flare Poller [baseline] (3.425 ms) : 0, 3425
Flare Poller [candidate] (3.466 ms) : 0, 3466
IAST [baseline] (28.95 ms) : 0, 28950
IAST [candidate] (25.961 ms) : 0, 25961

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1076254
Total [baseline] (10.983 s) : 0, 10983203
Agent [candidate] (1.066 s) : 0, 1066244
Total [candidate] (11.096 s) : 0, 11095952
section appsec
Agent [baseline] (1.285 s) : 0, 1285037
Total [baseline] (11.104 s) : 0, 11103645
Agent [candidate] (1.251 s) : 0, 1250934
Total [candidate] (11.204 s) : 0, 11203995
section iast
Agent [baseline] (1.261 s) : 0, 1260761
Total [baseline] (11.176 s) : 0, 11175836
Agent [candidate] (1.223 s) : 0, 1222886
Total [candidate] (11.265 s) : 0, 11265051
section profiling
Agent [baseline] (1.318 s) : 0, 1317790
Total [baseline] (11.092 s) : 0, 11091747
Agent [candidate] (1.186 s) : 0, 1185949
Total [candidate] (11.043 s) : 0, 11042812

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.076 s	-
Agent	appsec	1.285 s	208.783 ms (19.4%)
Agent	iast	1.261 s	184.507 ms (17.1%)
Agent	profiling	1.318 s	241.536 ms (22.4%)
Total	tracing	10.983 s	-
Total	appsec	11.104 s	120.441 ms (1.1%)
Total	iast	11.176 s	192.632 ms (1.8%)
Total	profiling	11.092 s	108.544 ms (1.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.066 s	-
Agent	appsec	1.251 s	184.69 ms (17.3%)
Agent	iast	1.223 s	156.641 ms (14.7%)
Agent	profiling	1.186 s	119.705 ms (11.2%)
Total	tracing	11.096 s	-
Total	appsec	11.204 s	108.043 ms (1.0%)
Total	iast	11.265 s	169.098 ms (1.5%)
Total	profiling	11.043 s	-53.14 ms (-0.5%)

gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.247 ms) : 0, 1247
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (641.764 ms) : 0, 641764
BytebuddyAgent [candidate] (639.308 ms) : 0, 639308
AgentMeter [baseline] (30.101 ms) : 0, 30101
AgentMeter [candidate] (29.708 ms) : 0, 29708
GlobalTracer [baseline] (250.016 ms) : 0, 250016
GlobalTracer [candidate] (250.424 ms) : 0, 250424
AppSec [baseline] (33.005 ms) : 0, 33005
AppSec [candidate] (32.239 ms) : 0, 32239
Debugger [baseline] (63.326 ms) : 0, 63326
Debugger [candidate] (60.389 ms) : 0, 60389
Remote Config [baseline] (605.784 µs) : 0, 606
Remote Config [candidate] (595.033 µs) : 0, 595
Telemetry [baseline] (9.132 ms) : 0, 9132
Telemetry [candidate] (8.095 ms) : 0, 8095
Flare Poller [baseline] (10.009 ms) : 0, 10009
Flare Poller [candidate] (8.013 ms) : 0, 8013
section appsec
crashtracking [baseline] (1.254 ms) : 0, 1254
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (687.184 ms) : 0, 687184
BytebuddyAgent [candidate] (663.494 ms) : 0, 663494
AgentMeter [baseline] (12.262 ms) : 0, 12262
AgentMeter [candidate] (12.032 ms) : 0, 12032
GlobalTracer [baseline] (251.411 ms) : 0, 251411
GlobalTracer [candidate] (249.139 ms) : 0, 249139
AppSec [baseline] (186.504 ms) : 0, 186504
AppSec [candidate] (185.206 ms) : 0, 185206
Debugger [baseline] (66.511 ms) : 0, 66511
Debugger [candidate] (65.957 ms) : 0, 65957
Remote Config [baseline] (589.473 µs) : 0, 589
Remote Config [candidate] (603.173 µs) : 0, 603
Telemetry [baseline] (7.819 ms) : 0, 7819
Telemetry [candidate] (8.676 ms) : 0, 8676
Flare Poller [baseline] (9.11 ms) : 0, 9110
Flare Poller [candidate] (3.619 ms) : 0, 3619
IAST [baseline] (25.139 ms) : 0, 25139
IAST [candidate] (24.625 ms) : 0, 24625
section iast
crashtracking [baseline] (1.254 ms) : 0, 1254
crashtracking [candidate] (1.229 ms) : 0, 1229
BytebuddyAgent [baseline] (833.972 ms) : 0, 833972
BytebuddyAgent [candidate] (799.854 ms) : 0, 799854
AgentMeter [baseline] (11.559 ms) : 0, 11559
AgentMeter [candidate] (11.385 ms) : 0, 11385
GlobalTracer [baseline] (238.339 ms) : 0, 238339
GlobalTracer [candidate] (239.104 ms) : 0, 239104
AppSec [baseline] (33.14 ms) : 0, 33140
AppSec [candidate] (30.286 ms) : 0, 30286
Debugger [baseline] (65.791 ms) : 0, 65791
Debugger [candidate] (62.172 ms) : 0, 62172
Remote Config [baseline] (553.365 µs) : 0, 553
Remote Config [candidate] (1.685 ms) : 0, 1685
Telemetry [baseline] (8.016 ms) : 0, 8016
Telemetry [candidate] (11.843 ms) : 0, 11843
Flare Poller [baseline] (3.421 ms) : 0, 3421
Flare Poller [candidate] (3.465 ms) : 0, 3465
IAST [baseline] (27.661 ms) : 0, 27661
IAST [candidate] (25.792 ms) : 0, 25792
section profiling
ProfilingAgent [baseline] (93.904 ms) : 0, 93904
ProfilingAgent [candidate] (94.268 ms) : 0, 94268
crashtracking [baseline] (541.999 µs) : 0, 542
crashtracking [candidate] (1.18 ms) : 0, 1180
BytebuddyAgent [baseline] (692.968 ms) : 0, 692968
BytebuddyAgent [candidate] (692.148 ms) : 0, 692148
AgentMeter [baseline] (9.253 ms) : 0, 9253
AgentMeter [candidate] (9.154 ms) : 0, 9154
GlobalTracer [baseline] (209.945 ms) : 0, 209945
GlobalTracer [candidate] (207.41 ms) : 0, 207410
AppSec [baseline] (32.698 ms) : 0, 32698
AppSec [candidate] (32.663 ms) : 0, 32663
Debugger [baseline] (68.175 ms) : 0, 68175
Debugger [candidate] (64.874 ms) : 0, 64874
Remote Config [baseline] (580.504 µs) : 0, 581
Remote Config [candidate] (577.755 µs) : 0, 578
Telemetry [baseline] (8.2 ms) : 0, 8200
Telemetry [candidate] (8.673 ms) : 0, 8673
Flare Poller [baseline] (3.696 ms) : 0, 3696
Flare Poller [candidate] (3.568 ms) : 0, 3568
Profiling [baseline] (94.478 ms) : 0, 94478
Profiling [candidate] (94.847 ms) : 0, 94847

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	maximo/otlp-runtime-metrics-poc
git_commit_date	1778103449	1778108168
git_commit_sha	`2e6ce6c`	`5a31372`
release_version	1.63.0-SNAPSHOT~2e6ce6ce0c	1.62.0-SNAPSHOT~5a31372eb2

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1778110420	1778110420
ci_job_id	1663320279	1663320279
ci_pipeline_id	111888000	111888000
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-bd4qxeno 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-bd4qxeno 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 4 performance regressions! Performance is the same for 12 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load	worse [+153.957µs; +534.957µs] or [+2.926%; +10.168%]	worse [+375.117µs; +1274.013µs] or [+2.946%; +10.006%]	unstable [-128.778op/s; +29.216op/s] or [-16.510%; +3.746%]	5.606ms	13.557ms	730.219op/s	5.261ms	12.732ms	780.000op/s
scenario:load:petclinic:appsec:high_load	unsure [-747.044µs; -162.266µs] or [-3.980%; -0.864%]	better [-1.923ms; -0.696ms] or [-6.190%; -2.240%]	unstable [-18.093op/s; +30.905op/s] or [-7.431%; +12.694%]	18.316ms	29.755ms	249.875op/s	18.771ms	31.064ms	243.469op/s
scenario:load:petclinic:profiling:high_load	worse [+0.884ms; +1.759ms] or [+4.866%; +9.683%]	worse [+0.806ms; +2.061ms] or [+2.756%; +7.043%]	unstable [-38.555op/s; +10.493op/s] or [-15.328%; +4.172%]	19.486ms	30.696ms	237.500op/s	18.164ms	29.262ms	251.531op/s
scenario:load:petclinic:no_agent:high_load	better [-2.050ms; -0.619ms] or [-10.737%; -3.242%]	unstable [-2.940ms; +0.437ms] or [-9.481%; +1.408%]	unstable [-8.884op/s; +43.759op/s] or [-3.729%; +18.367%]	17.758ms	29.758ms	255.688op/s	19.092ms	31.010ms	238.250op/s
scenario:load:petclinic:code_origins:high_load	better [-683.774µs; -365.225µs] or [-3.843%; -2.052%]	unsure [-1303.064µs; -102.090µs] or [-4.495%; -0.352%]	unstable [-19.313op/s; +32.000op/s] or [-7.497%; +12.423%]	17.270ms	28.289ms	263.938op/s	17.794ms	28.991ms	257.594op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.285 ms) : 1272, 1298
.   : milestone, 1285,
iast (3.371 ms) : 3323, 3419
.   : milestone, 3371,
iast_FULL (5.928 ms) : 5868, 5989
.   : milestone, 5928,
iast_GLOBAL (3.69 ms) : 3628, 3753
.   : milestone, 3690,
profiling (2.544 ms) : 2515, 2572
.   : milestone, 2544,
tracing (1.937 ms) : 1921, 1953
.   : milestone, 1937,
section candidate
no_agent (1.288 ms) : 1275, 1300
.   : milestone, 1288,
iast (3.354 ms) : 3309, 3399
.   : milestone, 3354,
iast_FULL (6.338 ms) : 6271, 6405
.   : milestone, 6338,
iast_GLOBAL (3.588 ms) : 3540, 3637
.   : milestone, 3588,
profiling (2.355 ms) : 2332, 2377
.   : milestone, 2355,
tracing (1.917 ms) : 1902, 1933
.   : milestone, 1917,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.285 ms [1.272 ms, 1.298 ms]	-
iast	3.371 ms [3.323 ms, 3.419 ms]	2.086 ms (162.4%)
iast_FULL	5.928 ms [5.868 ms, 5.989 ms]	4.644 ms (361.4%)
iast_GLOBAL	3.69 ms [3.628 ms, 3.753 ms]	2.405 ms (187.2%)
profiling	2.544 ms [2.515 ms, 2.572 ms]	1.259 ms (98.0%)
tracing	1.937 ms [1.921 ms, 1.953 ms]	652.109 µs (50.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.288 ms [1.275 ms, 1.3 ms]	-
iast	3.354 ms [3.309 ms, 3.399 ms]	2.067 ms (160.5%)
iast_FULL	6.338 ms [6.271 ms, 6.405 ms]	5.05 ms (392.3%)
iast_GLOBAL	3.588 ms [3.54 ms, 3.637 ms]	2.301 ms (178.7%)
profiling	2.355 ms [2.332 ms, 2.377 ms]	1.067 ms (82.9%)
tracing	1.917 ms [1.902 ms, 1.933 ms]	629.898 µs (48.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.593 ms) : 19396, 19790
.   : milestone, 19593,
appsec (19.167 ms) : 18967, 19368
.   : milestone, 19167,
code_origins (18.111 ms) : 17932, 18291
.   : milestone, 18111,
iast (18.345 ms) : 18163, 18528
.   : milestone, 18345,
profiling (18.554 ms) : 18373, 18735
.   : milestone, 18554,
tracing (17.749 ms) : 17574, 17924
.   : milestone, 17749,
section candidate
no_agent (18.249 ms) : 18064, 18435
.   : milestone, 18249,
appsec (18.674 ms) : 18488, 18860
.   : milestone, 18674,
code_origins (17.675 ms) : 17502, 17847
.   : milestone, 17675,
iast (18.069 ms) : 17892, 18247
.   : milestone, 18069,
profiling (19.652 ms) : 19456, 19848
.   : milestone, 19652,
tracing (18.229 ms) : 18050, 18407
.   : milestone, 18229,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.593 ms [19.396 ms, 19.79 ms]	-
appsec	19.167 ms [18.967 ms, 19.368 ms]	-425.939 µs (-2.2%)
code_origins	18.111 ms [17.932 ms, 18.291 ms]	-1.482 ms (-7.6%)
iast	18.345 ms [18.163 ms, 18.528 ms]	-1.248 ms (-6.4%)
profiling	18.554 ms [18.373 ms, 18.735 ms]	-1.039 ms (-5.3%)
tracing	17.749 ms [17.574 ms, 17.924 ms]	-1.844 ms (-9.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.249 ms [18.064 ms, 18.435 ms]	-
appsec	18.674 ms [18.488 ms, 18.86 ms]	424.287 µs (2.3%)
code_origins	17.675 ms [17.502 ms, 17.847 ms]	-574.811 µs (-3.1%)
iast	18.069 ms [17.892 ms, 18.247 ms]	-180.21 µs (-1.0%)
profiling	19.652 ms [19.456 ms, 19.848 ms]	1.403 ms (7.7%)
tracing	18.229 ms [18.05 ms, 18.407 ms]	-20.985 µs (-0.1%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	maximo/otlp-runtime-metrics-poc
git_commit_date	1778103449	1778108168
git_commit_sha	`2e6ce6c`	`5a31372`
release_version	1.63.0-SNAPSHOT~2e6ce6ce0c	1.62.0-SNAPSHOT~5a31372eb2

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1778110475	1778110475
ci_job_id	1663320283	1663320283
ci_pipeline_id	111888000	111888000
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-zslqvuuj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-zslqvuuj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.307 s) : 15307000, 15307000
.   : milestone, 15307000,
appsec (14.977 s) : 14977000, 14977000
.   : milestone, 14977000,
iast (18.453 s) : 18453000, 18453000
.   : milestone, 18453000,
iast_GLOBAL (17.953 s) : 17953000, 17953000
.   : milestone, 17953000,
profiling (14.948 s) : 14948000, 14948000
.   : milestone, 14948000,
tracing (15.056 s) : 15056000, 15056000
.   : milestone, 15056000,
section candidate
no_agent (15.07 s) : 15070000, 15070000
.   : milestone, 15070000,
appsec (14.834 s) : 14834000, 14834000
.   : milestone, 14834000,
iast (18.182 s) : 18182000, 18182000
.   : milestone, 18182000,
iast_GLOBAL (18.1 s) : 18100000, 18100000
.   : milestone, 18100000,
profiling (14.864 s) : 14864000, 14864000
.   : milestone, 14864000,
tracing (14.92 s) : 14920000, 14920000
.   : milestone, 14920000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.307 s [15.307 s, 15.307 s]	-
appsec	14.977 s [14.977 s, 14.977 s]	-330.0 ms (-2.2%)
iast	18.453 s [18.453 s, 18.453 s]	3.146 s (20.6%)
iast_GLOBAL	17.953 s [17.953 s, 17.953 s]	2.646 s (17.3%)
profiling	14.948 s [14.948 s, 14.948 s]	-359.0 ms (-2.3%)
tracing	15.056 s [15.056 s, 15.056 s]	-251.0 ms (-1.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.07 s [15.07 s, 15.07 s]	-
appsec	14.834 s [14.834 s, 14.834 s]	-236.0 ms (-1.6%)
iast	18.182 s [18.182 s, 18.182 s]	3.112 s (20.7%)
iast_GLOBAL	18.1 s [18.1 s, 18.1 s]	3.03 s (20.1%)
profiling	14.864 s [14.864 s, 14.864 s]	-206.0 ms (-1.4%)
tracing	14.92 s [14.92 s, 14.92 s]	-150.0 ms (-1.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~5a31372eb2, baseline=1.63.0-SNAPSHOT~2e6ce6ce0c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.499 ms) : 1487, 1511
.   : milestone, 1499,
appsec (3.866 ms) : 3644, 4088
.   : milestone, 3866,
iast (2.302 ms) : 2232, 2372
.   : milestone, 2302,
iast_GLOBAL (2.337 ms) : 2267, 2407
.   : milestone, 2337,
profiling (2.116 ms) : 2062, 2171
.   : milestone, 2116,
tracing (2.098 ms) : 2044, 2151
.   : milestone, 2098,
section candidate
no_agent (1.501 ms) : 1489, 1513
.   : milestone, 1501,
appsec (3.854 ms) : 3630, 4078
.   : milestone, 3854,
iast (2.297 ms) : 2228, 2366
.   : milestone, 2297,
iast_GLOBAL (2.327 ms) : 2258, 2396
.   : milestone, 2327,
profiling (2.114 ms) : 2059, 2169
.   : milestone, 2114,
tracing (2.113 ms) : 2059, 2167
.   : milestone, 2113,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.499 ms [1.487 ms, 1.511 ms]	-
appsec	3.866 ms [3.644 ms, 4.088 ms]	2.367 ms (157.9%)
iast	2.302 ms [2.232 ms, 2.372 ms]	802.96 µs (53.6%)
iast_GLOBAL	2.337 ms [2.267 ms, 2.407 ms]	838.037 µs (55.9%)
profiling	2.116 ms [2.062 ms, 2.171 ms]	617.153 µs (41.2%)
tracing	2.098 ms [2.044 ms, 2.151 ms]	598.426 µs (39.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.501 ms [1.489 ms, 1.513 ms]	-
appsec	3.854 ms [3.63 ms, 4.078 ms]	2.353 ms (156.8%)
iast	2.297 ms [2.228 ms, 2.366 ms]	796.583 µs (53.1%)
iast_GLOBAL	2.327 ms [2.258 ms, 2.396 ms]	826.323 µs (55.1%)
profiling	2.114 ms [2.059 ms, 2.169 ms]	613.518 µs (40.9%)
tracing	2.113 ms [2.059 ms, 2.167 ms]	612.174 µs (40.8%)

New scenario OTLP_RUNTIME_METRICS that sets DD_METRICS_OTEL_ENABLED=true alongside DD_RUNTIME_METRICS_ENABLED=true. Tests verify OTel-native metric names (dotnet.*, jvm.*, go.*, v8js.*) appear in OTLP payloads and that DD-proprietary names (runtime.dotnet.*, runtime.go.*) do not. All languages marked as missing_feature in manifests until POC PRs are merged: - .NET: DataDog/dd-trace-dotnet#8299 - Go: DataDog/dd-trace-go#4611 - Node.js: DataDog/dd-trace-js#7869 - Java: DataDog/dd-trace-java#10985 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Adds jvm.memory.used, jvm.memory.committed, jvm.memory.limit, jvm.gc.duration, jvm.gc.count, jvm.thread.count, jvm.class.loaded, jvm.class.unloaded, jvm.cpu.recent_utilization, jvm.cpu.count as OTel instruments on the existing OTLP metrics pipeline. Includes jvm.memory.type attribute for heap/non_heap breakdown required by semantic-core equivalence mappings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…u.utilization, jvm.class.count Aligns with OTel JVM semantic conventions spreadsheet. Updates test to verify all 16 metrics are registered. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Call JvmOtlpRuntimeMetrics.start() from OpenTelemetryMetricsInstrumentation when both DD_METRICS_OTEL_ENABLED and DD_RUNTIME_METRICS_ENABLED are true. Matches .NET/Go/NodeJS config gating pattern. Also updates metrics to match OTel spec exactly (Option B): removes jvm.gc.duration/count, fixes types, adds missing metrics. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

OtlpDataPoint, OtlpLongPoint, OtlpDoublePoint, OtlpMetricVisitor, OtlpMetricsVisitor, OtlpScopedMetricsVisitor moved from datadog.trace.bootstrap.otel.metrics.data/export to datadog.trace.bootstrap.otlp.metrics after PR #11055 merge. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Assert exactly 18 metrics (was missing jvm.file_descriptor.count/limit) - Assert no DD-proprietary names present - Matches .NET PR #8457 test pattern Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Per OTel semconv, jvm.system.cpu.utilization, jvm.system.cpu.load_1m, jvm.file_descriptor.count and jvm.file_descriptor.limit are Opt-In: they MUST only be reported when the instrumentation is explicitly configured to do so. This POC currently has no DD_METRICS_OTEL_OPTIN_ENABLED gating, so emitting them would violate the spec. Removing the three we shipped (jvm.system.cpu.utilization, jvm.file_descriptor.count, jvm.file_descriptor.limit) brings the metric set down to the 15 Recommended JVM metrics. Adding opt-in support is tracked separately and will reintroduce these behind the env var. Reference: https://opentelemetry.io/docs/specs/semconv/general/metric-requirement-level/#opt-in

The previous wiring only fired when the application called OpenTelemetry.getMeterProvider() — which most JVM apps never do unless they themselves use the OTel API. Spring Boot, Akka, Vert.x and the system-tests weblogs don't, so JvmOtlpRuntimeMetrics.start() was never reached and zero jvm.* metrics were exported. Match the pattern Node and .NET use: start runtime metrics from the tracer's own init path (here, Agent.startJmx, the same place startJmxFetch lives). Reflective load through AGENT_CLASSLOADER mirrors startJmxFetch exactly. Gated on DD_RUNTIME_METRICS_ENABLED && OTel metrics enabled, both already set when the OTLP_RUNTIME_METRICS scenario runs. The OTLP exporter pipeline (OtlpMetricsService) is already started by CoreTracer when DD_METRICS_OTEL_ENABLED=true; this change only registers the JVM metric callbacks with OtelMeterProvider so the periodic export has something to collect.

Two changes: 1. Strip 4 more JVM metrics that aren't Recommended per OTel semconv: jvm.memory.init, jvm.buffer.memory.used, jvm.buffer.memory.limit, jvm.buffer.count. They live in jvm-metrics-experimental.yaml with stability: development. Same partitioning OTel Java upstream applies in JmxRuntimeMetricsFactory's emitExperimentalTelemetry branch: non-experimental subset is exactly the 11 we now ship. Will return behind a future DD_METRICS_OTEL_OPTIN_ENABLED flag together with the previously-dropped Opt-In metrics (jvm.system.cpu.*, jvm.file_descriptor.*). Reference: https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/ https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/JmxRuntimeMetricsFactory.java 2. Migrate JvmOtlpRuntimeMetricsTest from Spock/Groovy to JUnit 5 Java per the new-Groovy-file enforcement workflow. Same 4 test cases, identical assertion semantics. Note: the opentelemetry-1.47 module still has 2 other pre-existing .groovy test files, so the module is not added to .github/g2j-migrated-modules.txt yet.

Calling startOtlpRuntimeMetrics() from startJmx() inherits its 15-second jitter delay, which races with short-lived test scenarios — for slower weblogs (ratpack hit this), OtelMetricRegistry callbacks are registered after the OtlpMetricsService has already done its first scheduled flush, so the agent receives 0 jvm.* metrics for the test's wait window. The runtime metric callbacks don't actually need JMX initialized — they read ManagementFactory directly. Move the call into installDatadogTracer right after CoreTracer (and OtlpMetricsService) are up. Same effective gating (DD_RUNTIME_METRICS_ENABLED + DD_METRICS_OTEL_ENABLED), no delay.

Agent.installDatadogTracer now triggers runtime metric registration on agent boot (the canonical entry point), so the duplicate call from the OpenTelemetry.getMeterProvider advice is dead code. JvmOtlpRuntimeMetrics is already idempotent so removing the second invocation is a no-op behaviorally; this just reduces review surface.

dd-octo-sts · 2026-05-07T11:41:15Z

⚠️ Java Benchmark SLOs — Some SLO checks did not complete

Suite	Status
Startup	⚠️ (check incomplete)
Load	⚠️ (check incomplete)
DaCapo	⚠️ (check incomplete)

Commit: 67d91a9d · CI Pipeline · Benchmarking Platform UI

Mirrors OTel Java upstream's runtime-telemetry reflect-config.json: GraalVM AOT analysis can't see Class.forName(...) lookups through AGENT_CLASSLOADER, so the class gets dead-stripped from the native binary unless explicitly listed. Without this entry, the Agent.startOtlpRuntimeMetrics() reflection in installDatadogTracer fails on native-image with ClassNotFoundException and zero jvm.* metrics are emitted.

Ship the 4 metrics that are stability:development per OTel semconv but NOT marked Opt-In: jvm.memory.init, jvm.buffer.memory.used, jvm.buffer.memory.limit, jvm.buffer.count. Final set is 15 metrics — everything OTel ships except the 4 explicit Opt-In ones (jvm.system.cpu.*, jvm.file_descriptor.*).

link04 changed the title ~~POC: Send JVM runtime metrics via OTLP using OTel-native naming~~ [DRAFT] NOT MERGING POC: Send JVM runtime metrics via OTLP using OTel-native naming Mar 27, 2026

link04 force-pushed the maximo/otlp-runtime-metrics-poc branch from e819041 to 4d69327 Compare April 9, 2026 16:53

link04 mentioned this pull request Apr 9, 2026

Add OTLP runtime metrics system test DataDog/system-tests#6715

Merged

4 tasks

link04 mentioned this pull request Apr 22, 2026

[java@maximo/otlp-runtime-metrics-poc] Enable OTLP runtime metrics test for Java DataDog/system-tests#6800

Draft

3 tasks

link04 and others added 9 commits May 7, 2026 07:30

Add missing JVM metrics: jvm.memory.init, jvm.buffer.*, jvm.system.cp…

cc5fa97

…u.utilization, jvm.class.count Aligns with OTel JVM semantic conventions spreadsheet. Updates test to verify all 16 metrics are registered. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

link04 force-pushed the maximo/otlp-runtime-metrics-poc branch from 5a31372 to 67d91a9 Compare May 7, 2026 11:34

link04 added inst: opentelemetry OpenTelemetry instrumentation tag: ai generated Largely based on code generated by an AI or LLM labels May 7, 2026

link04 added 2 commits May 7, 2026 07:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DRAFT] NOT MERGING POC: Send JVM runtime metrics via OTLP using OTel-native naming#10985

[DRAFT] NOT MERGING POC: Send JVM runtime metrics via OTLP using OTel-native naming#10985
link04 wants to merge 12 commits intomasterfrom
maximo/otlp-runtime-metrics-poc

link04 commented Mar 27, 2026

Uh oh!

pr-commenter Bot commented Mar 27, 2026 •

edited

Loading

Uh oh!

dd-octo-sts Bot commented May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

link04 commented Mar 27, 2026

Uh oh!

pr-commenter Bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

dd-octo-sts Bot commented May 7, 2026

⚠️ Java Benchmark SLOs — Some SLO checks did not complete

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenter Bot commented Mar 27, 2026 •

edited

Loading