Improper test isolation for muzzle plugin tests by bric3 · Pull Request #10875 · DataDog/dd-trace-java

bric3 · 2026-03-17T15:14:48Z

What Does This Do

The tests are asserting the correct things and pass, however during some cleanup operations, the test kit dir could be deleted too early.

In particular the use of withTestKitDir was likely incorrect.

As the Javadoc says, having no value should prevent this behavior.

If no value has been specified when the build is initiated, a directory will be created within a temporary directory.

When executed from a Gradle Test task, the Test task's temporary directory is used (see Task.getTemporaryDir()).

From https://docs.gradle.org/8.14.4/javadoc/org/gradle/testkit/runner/GradleRunner.html#withTestKitDir(java.io.File) > If no value has been specified when the build is initiated, a directory will be created within a temporary directory. > > * When executed from a Gradle Test task, the Test task's temporary directory is used (see Task.getTemporaryDir()).

pr-commenter · 2026-03-17T15:56:42Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/improper-test-isolation
git_commit_date	1773751598	1773779280
git_commit_sha	`99d47ca`	`25009ba`
release_version	1.61.0-SNAPSHOT~99d47ca835	1.61.0-SNAPSHOT~25009ba8e9

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773781050	1773781050
ci_job_id	1515166801	1515166801
ci_pipeline_id	103135836	103135836
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ka9fqzh9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ka9fqzh9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 66 metrics, 5 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061158
Total [baseline] (11.06 s) : 0, 11059672
Agent [candidate] (1.06 s) : 0, 1059563
Total [candidate] (11.02 s) : 0, 11019985
section appsec
Agent [baseline] (1.246 s) : 0, 1246238
Total [baseline] (11.119 s) : 0, 11119259
Agent [candidate] (1.249 s) : 0, 1248755
Total [candidate] (11.133 s) : 0, 11133191
section iast
Agent [baseline] (1.225 s) : 0, 1225159
Total [baseline] (11.298 s) : 0, 11298458
Agent [candidate] (1.228 s) : 0, 1228064
Total [candidate] (11.348 s) : 0, 11347540
section profiling
Agent [baseline] (1.184 s) : 0, 1183533
Total [baseline] (11.064 s) : 0, 11064075
Agent [candidate] (1.182 s) : 0, 1181953
Total [candidate] (11.051 s) : 0, 11051465

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.061 s	-
Agent	appsec	1.246 s	185.08 ms (17.4%)
Agent	iast	1.225 s	164.001 ms (15.5%)
Agent	profiling	1.184 s	122.375 ms (11.5%)
Total	tracing	11.06 s	-
Total	appsec	11.119 s	59.587 ms (0.5%)
Total	iast	11.298 s	238.786 ms (2.2%)
Total	profiling	11.064 s	4.403 ms (0.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	appsec	1.249 s	189.192 ms (17.9%)
Agent	iast	1.228 s	168.501 ms (15.9%)
Agent	profiling	1.182 s	122.39 ms (11.6%)
Total	tracing	11.02 s	-
Total	appsec	11.133 s	113.206 ms (1.0%)
Total	iast	11.348 s	327.554 ms (3.0%)
Total	profiling	11.051 s	31.48 ms (0.3%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.215 ms) : 0, 1215
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (628.034 ms) : 0, 628034
BytebuddyAgent [candidate] (628.404 ms) : 0, 628404
AgentMeter [baseline] (29.078 ms) : 0, 29078
AgentMeter [candidate] (29.089 ms) : 0, 29089
GlobalTracer [baseline] (257.103 ms) : 0, 257103
GlobalTracer [candidate] (257.369 ms) : 0, 257369
AppSec [baseline] (31.743 ms) : 0, 31743
AppSec [candidate] (31.895 ms) : 0, 31895
Debugger [baseline] (60.373 ms) : 0, 60373
Debugger [candidate] (60.348 ms) : 0, 60348
Remote Config [baseline] (597.348 µs) : 0, 597
Remote Config [candidate] (598.627 µs) : 0, 599
Telemetry [baseline] (8.09 ms) : 0, 8090
Telemetry [candidate] (8.12 ms) : 0, 8120
Flare Poller [baseline] (8.837 ms) : 0, 8837
Flare Poller [candidate] (6.498 ms) : 0, 6498
section appsec
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (658.109 ms) : 0, 658109
BytebuddyAgent [candidate] (660.891 ms) : 0, 660891
AgentMeter [baseline] (11.993 ms) : 0, 11993
AgentMeter [candidate] (12.027 ms) : 0, 12027
GlobalTracer [baseline] (258.052 ms) : 0, 258052
GlobalTracer [candidate] (258.014 ms) : 0, 258014
IAST [baseline] (24.195 ms) : 0, 24195
IAST [candidate] (24.145 ms) : 0, 24145
AppSec [baseline] (177.76 ms) : 0, 177760
AppSec [candidate] (177.36 ms) : 0, 177360
Debugger [baseline] (66.156 ms) : 0, 66156
Debugger [candidate] (66.244 ms) : 0, 66244
Remote Config [baseline] (618.563 µs) : 0, 619
Remote Config [candidate] (621.474 µs) : 0, 621
Telemetry [baseline] (8.269 ms) : 0, 8269
Telemetry [candidate] (8.243 ms) : 0, 8243
Flare Poller [baseline] (3.614 ms) : 0, 3614
Flare Poller [candidate] (3.616 ms) : 0, 3616
section iast
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (794.296 ms) : 0, 794296
BytebuddyAgent [candidate] (796.187 ms) : 0, 796187
AgentMeter [baseline] (11.289 ms) : 0, 11289
AgentMeter [candidate] (11.342 ms) : 0, 11342
GlobalTracer [baseline] (246.926 ms) : 0, 246926
GlobalTracer [candidate] (247.541 ms) : 0, 247541
IAST [baseline] (25.245 ms) : 0, 25245
IAST [candidate] (25.315 ms) : 0, 25315
AppSec [baseline] (26.499 ms) : 0, 26499
AppSec [candidate] (26.48 ms) : 0, 26480
Debugger [baseline] (70.692 ms) : 0, 70692
Debugger [candidate] (70.77 ms) : 0, 70770
Remote Config [baseline] (529.965 µs) : 0, 530
Remote Config [candidate] (530.227 µs) : 0, 530
Telemetry [baseline] (9.17 ms) : 0, 9170
Telemetry [candidate] (9.251 ms) : 0, 9251
Flare Poller [baseline] (3.329 ms) : 0, 3329
Flare Poller [candidate] (3.368 ms) : 0, 3368
section profiling
crashtracking [baseline] (1.16 ms) : 0, 1160
crashtracking [candidate] (1.169 ms) : 0, 1169
BytebuddyAgent [baseline] (683.161 ms) : 0, 683161
BytebuddyAgent [candidate] (682.516 ms) : 0, 682516
AgentMeter [baseline] (8.65 ms) : 0, 8650
AgentMeter [candidate] (8.662 ms) : 0, 8662
GlobalTracer [baseline] (215.814 ms) : 0, 215814
GlobalTracer [candidate] (215.529 ms) : 0, 215529
AppSec [baseline] (32.268 ms) : 0, 32268
AppSec [candidate] (32.357 ms) : 0, 32357
Debugger [baseline] (65.305 ms) : 0, 65305
Debugger [candidate] (65.004 ms) : 0, 65004
Remote Config [baseline] (583.345 µs) : 0, 583
Remote Config [candidate] (579.538 µs) : 0, 580
Telemetry [baseline] (8.451 ms) : 0, 8451
Telemetry [candidate] (8.494 ms) : 0, 8494
Flare Poller [baseline] (3.505 ms) : 0, 3505
Flare Poller [candidate] (3.475 ms) : 0, 3475
ProfilingAgent [baseline] (93.883 ms) : 0, 93883
ProfilingAgent [candidate] (93.527 ms) : 0, 93527
Profiling [baseline] (94.459 ms) : 0, 94459
Profiling [candidate] (94.079 ms) : 0, 94079

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064921
Total [baseline] (8.863 s) : 0, 8862795
Agent [candidate] (1.069 s) : 0, 1069138
Total [candidate] (8.875 s) : 0, 8874765
section iast
Agent [baseline] (1.226 s) : 0, 1226475
Total [baseline] (9.52 s) : 0, 9519602
Agent [candidate] (1.232 s) : 0, 1231757
Total [candidate] (9.535 s) : 0, 9534674

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	iast	1.226 s	161.554 ms (15.2%)
Total	tracing	8.863 s	-
Total	iast	9.52 s	656.806 ms (7.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.069 s	-
Agent	iast	1.232 s	162.619 ms (15.2%)
Total	tracing	8.875 s	-
Total	iast	9.535 s	659.909 ms (7.4%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (633.108 ms) : 0, 633108
BytebuddyAgent [candidate] (632.831 ms) : 0, 632831
AgentMeter [baseline] (29.214 ms) : 0, 29214
AgentMeter [candidate] (29.172 ms) : 0, 29172
GlobalTracer [baseline] (258.339 ms) : 0, 258339
GlobalTracer [candidate] (258.842 ms) : 0, 258842
AppSec [baseline] (31.97 ms) : 0, 31970
AppSec [candidate] (31.919 ms) : 0, 31919
Debugger [baseline] (59.799 ms) : 0, 59799
Debugger [candidate] (59.928 ms) : 0, 59928
Remote Config [baseline] (597.184 µs) : 0, 597
Remote Config [candidate] (602.759 µs) : 0, 603
Telemetry [baseline] (8.138 ms) : 0, 8138
Telemetry [candidate] (8.126 ms) : 0, 8126
Flare Poller [baseline] (6.441 ms) : 0, 6441
Flare Poller [candidate] (10.308 ms) : 0, 10308
section iast
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (795.801 ms) : 0, 795801
BytebuddyAgent [candidate] (800.254 ms) : 0, 800254
AgentMeter [baseline] (11.369 ms) : 0, 11369
AgentMeter [candidate] (11.441 ms) : 0, 11441
GlobalTracer [baseline] (247.515 ms) : 0, 247515
GlobalTracer [candidate] (247.753 ms) : 0, 247753
IAST [baseline] (25.301 ms) : 0, 25301
IAST [candidate] (25.272 ms) : 0, 25272
AppSec [baseline] (26.569 ms) : 0, 26569
AppSec [candidate] (26.327 ms) : 0, 26327
Debugger [baseline] (68.36 ms) : 0, 68360
Debugger [candidate] (69.518 ms) : 0, 69518
Remote Config [baseline] (518.536 µs) : 0, 519
Remote Config [candidate] (528.163 µs) : 0, 528
Telemetry [baseline] (10.13 ms) : 0, 10130
Telemetry [candidate] (9.818 ms) : 0, 9818
Flare Poller [baseline] (3.635 ms) : 0, 3635
Flare Poller [candidate] (3.462 ms) : 0, 3462

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/improper-test-isolation
git_commit_date	1773751598	1773779280
git_commit_sha	`99d47ca`	`25009ba`
release_version	1.61.0-SNAPSHOT~99d47ca835	1.61.0-SNAPSHOT~25009ba8e9

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773781527	1773781527
ci_job_id	1515166803	1515166803
ci_pipeline_id	103135836	103135836
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-8gy9pa21 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-8gy9pa21 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	better [-147.836µs; -36.847µs] or [-8.572%; -2.137%]	unstable [-1020.607µs; -223.543µs] or [-19.500%; -4.271%]	unstable [-10.946op/s; +466.196op/s] or [-0.536%; +22.813%]	1.632ms	4.612ms	2271.188op/s	1.725ms	5.234ms	2043.562op/s
scenario:load:petclinic:profiling:high_load	better [-1.830ms; -0.864ms] or [-9.319%; -4.401%]	better [-2.854ms; -1.203ms] or [-9.086%; -3.830%]	unstable [-6.203op/s; +39.265op/s] or [-2.646%; +16.749%]	18.287ms	29.388ms	250.969op/s	19.634ms	31.416ms	234.438op/s
scenario:load:petclinic:no_agent:high_load	worse [+0.387ms; +2.098ms] or [+2.161%; +11.733%]	unstable [-571.905µs; +2542.129µs] or [-1.877%; +8.344%]	unstable [-38.361op/s; +11.611op/s] or [-15.193%; +4.599%]	19.127ms	31.452ms	239.125op/s	17.885ms	30.467ms	252.500op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.196 ms) : 1184, 1208
.   : milestone, 1196,
iast (3.13 ms) : 3088, 3171
.   : milestone, 3130,
iast_FULL (5.801 ms) : 5744, 5857
.   : milestone, 5801,
iast_GLOBAL (3.535 ms) : 3472, 3599
.   : milestone, 3535,
profiling (2.215 ms) : 2192, 2239
.   : milestone, 2215,
tracing (1.814 ms) : 1799, 1829
.   : milestone, 1814,
section candidate
no_agent (1.175 ms) : 1164, 1185
.   : milestone, 1175,
iast (3.175 ms) : 3133, 3217
.   : milestone, 3175,
iast_FULL (5.885 ms) : 5827, 5944
.   : milestone, 5885,
iast_GLOBAL (3.462 ms) : 3402, 3522
.   : milestone, 3462,
profiling (1.987 ms) : 1970, 2004
.   : milestone, 1987,
tracing (1.764 ms) : 1748, 1779
.   : milestone, 1764,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.196 ms [1.184 ms, 1.208 ms]	-
iast	3.13 ms [3.088 ms, 3.171 ms]	1.934 ms (161.7%)
iast_FULL	5.801 ms [5.744 ms, 5.857 ms]	4.605 ms (385.1%)
iast_GLOBAL	3.535 ms [3.472 ms, 3.599 ms]	2.34 ms (195.6%)
profiling	2.215 ms [2.192 ms, 2.239 ms]	1.02 ms (85.3%)
tracing	1.814 ms [1.799 ms, 1.829 ms]	618.184 µs (51.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.175 ms [1.164 ms, 1.185 ms]	-
iast	3.175 ms [3.133 ms, 3.217 ms]	2.0 ms (170.3%)
iast_FULL	5.885 ms [5.827 ms, 5.944 ms]	4.711 ms (401.0%)
iast_GLOBAL	3.462 ms [3.402 ms, 3.522 ms]	2.287 ms (194.7%)
profiling	1.987 ms [1.97 ms, 2.004 ms]	812.056 µs (69.1%)
tracing	1.764 ms [1.748 ms, 1.779 ms]	589.236 µs (50.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.482 ms) : 18288, 18676
.   : milestone, 18482,
appsec (18.557 ms) : 18368, 18746
.   : milestone, 18557,
code_origins (17.651 ms) : 17477, 17824
.   : milestone, 17651,
iast (17.805 ms) : 17631, 17980
.   : milestone, 17805,
profiling (19.912 ms) : 19706, 20119
.   : milestone, 19912,
tracing (17.735 ms) : 17558, 17912
.   : milestone, 17735,
section candidate
no_agent (19.522 ms) : 19320, 19725
.   : milestone, 19522,
appsec (18.932 ms) : 18739, 19125
.   : milestone, 18932,
code_origins (17.901 ms) : 17722, 18081
.   : milestone, 17901,
iast (17.687 ms) : 17514, 17860
.   : milestone, 17687,
profiling (18.6 ms) : 18416, 18783
.   : milestone, 18600,
tracing (17.825 ms) : 17647, 18002
.   : milestone, 17825,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.482 ms [18.288 ms, 18.676 ms]	-
appsec	18.557 ms [18.368 ms, 18.746 ms]	75.051 µs (0.4%)
code_origins	17.651 ms [17.477 ms, 17.824 ms]	-831.284 µs (-4.5%)
iast	17.805 ms [17.631 ms, 17.98 ms]	-676.421 µs (-3.7%)
profiling	19.912 ms [19.706 ms, 20.119 ms]	1.43 ms (7.7%)
tracing	17.735 ms [17.558 ms, 17.912 ms]	-746.846 µs (-4.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.522 ms [19.32 ms, 19.725 ms]	-
appsec	18.932 ms [18.739 ms, 19.125 ms]	-589.937 µs (-3.0%)
code_origins	17.901 ms [17.722 ms, 18.081 ms]	-1.621 ms (-8.3%)
iast	17.687 ms [17.514 ms, 17.86 ms]	-1.835 ms (-9.4%)
profiling	18.6 ms [18.416 ms, 18.783 ms]	-922.612 µs (-4.7%)
tracing	17.825 ms [17.647 ms, 18.002 ms]	-1.698 ms (-8.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/improper-test-isolation
git_commit_date	1773751598	1773779280
git_commit_sha	`99d47ca`	`25009ba`
release_version	1.61.0-SNAPSHOT~99d47ca835	1.61.0-SNAPSHOT~25009ba8e9

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1773781358	1773781358
ci_job_id	1515166805	1515166805
ci_pipeline_id	103135836	103135836
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-wmnhkaj7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-wmnhkaj7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.276 s) : 15276000, 15276000
.   : milestone, 15276000,
appsec (14.81 s) : 14810000, 14810000
.   : milestone, 14810000,
iast (18.289 s) : 18289000, 18289000
.   : milestone, 18289000,
iast_GLOBAL (17.853 s) : 17853000, 17853000
.   : milestone, 17853000,
profiling (14.838 s) : 14838000, 14838000
.   : milestone, 14838000,
tracing (14.908 s) : 14908000, 14908000
.   : milestone, 14908000,
section candidate
no_agent (14.939 s) : 14939000, 14939000
.   : milestone, 14939000,
appsec (14.74 s) : 14740000, 14740000
.   : milestone, 14740000,
iast (18.229 s) : 18229000, 18229000
.   : milestone, 18229000,
iast_GLOBAL (18.394 s) : 18394000, 18394000
.   : milestone, 18394000,
profiling (14.989 s) : 14989000, 14989000
.   : milestone, 14989000,
tracing (14.793 s) : 14793000, 14793000
.   : milestone, 14793000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.276 s [15.276 s, 15.276 s]	-
appsec	14.81 s [14.81 s, 14.81 s]	-466.0 ms (-3.1%)
iast	18.289 s [18.289 s, 18.289 s]	3.013 s (19.7%)
iast_GLOBAL	17.853 s [17.853 s, 17.853 s]	2.577 s (16.9%)
profiling	14.838 s [14.838 s, 14.838 s]	-438.0 ms (-2.9%)
tracing	14.908 s [14.908 s, 14.908 s]	-368.0 ms (-2.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.939 s [14.939 s, 14.939 s]	-
appsec	14.74 s [14.74 s, 14.74 s]	-199.0 ms (-1.3%)
iast	18.229 s [18.229 s, 18.229 s]	3.29 s (22.0%)
iast_GLOBAL	18.394 s [18.394 s, 18.394 s]	3.455 s (23.1%)
profiling	14.989 s [14.989 s, 14.989 s]	50.0 ms (0.3%)
tracing	14.793 s [14.793 s, 14.793 s]	-146.0 ms (-1.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~25009ba8e9, baseline=1.61.0-SNAPSHOT~99d47ca835
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (3.785 ms) : 3564, 4006
.   : milestone, 3785,
iast (2.247 ms) : 2178, 2317
.   : milestone, 2247,
iast_GLOBAL (2.295 ms) : 2225, 2364
.   : milestone, 2295,
profiling (2.075 ms) : 2020, 2130
.   : milestone, 2075,
tracing (2.056 ms) : 2002, 2110
.   : milestone, 2056,
section candidate
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (3.788 ms) : 3566, 4011
.   : milestone, 3788,
iast (2.242 ms) : 2172, 2311
.   : milestone, 2242,
iast_GLOBAL (2.298 ms) : 2228, 2368
.   : milestone, 2298,
profiling (2.51 ms) : 2343, 2676
.   : milestone, 2510,
tracing (2.051 ms) : 1997, 2105
.   : milestone, 2051,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.457 ms, 1.48 ms]	-
appsec	3.785 ms [3.564 ms, 4.006 ms]	2.316 ms (157.7%)
iast	2.247 ms [2.178 ms, 2.317 ms]	778.689 µs (53.0%)
iast_GLOBAL	2.295 ms [2.225 ms, 2.364 ms]	825.891 µs (56.2%)
profiling	2.075 ms [2.02 ms, 2.13 ms]	606.144 µs (41.3%)
tracing	2.056 ms [2.002 ms, 2.11 ms]	587.251 µs (40.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.467 ms [1.455 ms, 1.478 ms]	-
appsec	3.788 ms [3.566 ms, 4.011 ms]	2.322 ms (158.3%)
iast	2.242 ms [2.172 ms, 2.311 ms]	775.111 µs (52.8%)
iast_GLOBAL	2.298 ms [2.228 ms, 2.368 ms]	831.269 µs (56.7%)
profiling	2.51 ms [2.343 ms, 2.676 ms]	1.043 ms (71.1%)
tracing	2.051 ms [1.997 ms, 2.105 ms]	584.643 µs (39.9%)

…tests MuzzleMavenRepoUtils.MUZZLE_REPOS is a lazy singleton initialized once per JVM from MAVEN_REPOSITORY_PROXY. When the Gradle daemon was reused across test cases, the env var value from the first test was cached, causing version range resolution to target a non-existent or unrelated local Maven repo in subsequent tests. Passing --no-daemon ensures each GradleRunner invocation starts a fresh JVM, so MUZZLE_REPOS is re-evaluated with the correct repo URL for each test. As a side effect, this also prevents the daemon from holding file handles on @tempdir directories at JUnit cleanup time.

bric3 added 2 commits March 17, 2026 15:55

fix: buildSrc result files were never collected

6f9da4c

bric3 requested review from a team as code owners March 17, 2026 15:14

bric3 requested review from randomanderson and ygree and removed request for a team March 17, 2026 15:14

bric3 added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Mar 17, 2026

AlexeyKuznetsov-DD approved these changes Mar 17, 2026

View reviewed changes

bric3 added 4 commits March 17, 2026 18:58

test(build): Replace --no-daemon by system property

d6aaa9a

t(build): disable daemin via gradle.properties

4d6fb07

test(build): Use project local test kit dir

25009ba

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improper test isolation for muzzle plugin tests#10875

Improper test isolation for muzzle plugin tests#10875
bric3 wants to merge 6 commits intomasterfrom
bdu/improper-test-isolation

bric3 commented Mar 17, 2026

Uh oh!

pr-commenter bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bric3 commented Mar 17, 2026

What Does This Do

Uh oh!

pr-commenter bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pr-commenter bot commented Mar 17, 2026 •

edited

Loading