Add openai-java v3.0+ instrumentation #9959

ygree · 2025-11-13T20:25:51Z

What Does This Do

Adds openai-java v3.0+ instrumentation for completions, chat completions, embeddings, and responses.

APM shared tests:

test_openai.py::TestOpenAiApm::test_chat_completion[java-test-ml-app-tcp-True] PASSED [ 12%]
test_openai.py::TestOpenAiApm::test_chat_completion[java-test-ml-app-tcp-False] PASSED [ 25%]
test_openai.py::TestOpenAiApm::test_completion[java-test-ml-app-tcp] PASSED [ 37%]
test_openai.py::TestOpenAiApm::test_embedding[java-test-ml-app-tcp] PASSED [ 50%]
test_openai.py::TestOpenAiApm::test_chat_completion_tool_call[java-test-ml-app-tcp-True] PASSED [ 62%]
test_openai.py::TestOpenAiApm::test_chat_completion_tool_call[java-test-ml-app-tcp-False] PASSED [ 75%]
test_openai.py::TestOpenAiApm::test_responses_create[java-test-ml-app-tcp-True] PASSED [ 87%]
test_openai.py::TestOpenAiApm::test_responses_create[java-test-ml-app-tcp-False] PASSED [100%]

LLMObs shared tests 14*/15:

test_openai.py::TestOpenAiLlmObs::test_chat_completion[java-test-ml-app-tcp-True] PASSED [  6%]
test_openai.py::TestOpenAiLlmObs::test_chat_completion[java-test-ml-app-tcp-False] PASSED [ 13%]
test_openai.py::TestOpenAiLlmObs::test_completion[java-test-ml-app-tcp] PASSED [ 20%]
test_openai.py::TestOpenAiLlmObs::test_embedding[java-test-ml-app-tcp] PASSED [ 26%]
test_openai.py::TestOpenAiLlmObs::test_chat_completion_tool_call[java-test-ml-app-tcp-True] PASSED [ 33%]
test_openai.py::TestOpenAiLlmObs::test_chat_completion_tool_call[java-test-ml-app-tcp-False] PASSED [ 40%]

test_openai.py::TestOpenAiLlmObs::test_chat_completion_telemetry[java-test-ml-app-tcp] FAILED* [ 46%]

test_openai.py::TestOpenAiLlmObs::test_responses_create[java-test-ml-app-tcp-True] PASSED [ 53%]
test_openai.py::TestOpenAiLlmObs::test_responses_create[java-test-ml-app-tcp-False] PASSED [ 60%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_call[java-test-ml-app-tcp-True] PASSED [ 66%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_call[java-test-ml-app-tcp-False] PASSED [ 73%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_reasoning[java-test-ml-app-tcp-True] PASSED [ 80%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_reasoning[java-test-ml-app-tcp-False] PASSED [ 86%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_input[java-test-ml-app-tcp-True] PASSED [ 93%]
test_openai.py::TestOpenAiLlmObs::test_responses_create_tool_input[java-test-ml-app-tcp-False] PASSED [100%]

- test_chat_completion_telemetry FAILED because the shared test expects a namespace for the entire batch, but it should allow the namespace to be part of the metric data. Since Java batches telemetry data across subsystems and cannot guarantee that all the data belongs to the same namespace.

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: AIDM-163

…stead of forcing it

…d "streamed async request completion test with withRawResponse"

Call decorateWithResponse from the wrappers

Rename span resources to be aligned with trace-py Add http.client resource assertion

Test case renaming

Reorder tests by synch, async

Fix Embeddings fixture for the latestDepTest when base64

github-actions · 2025-12-23T21:40:00Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

PerfectSlayer

First round of comments for the core part. I will let IDM review the instrumentation part.

dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/InstrumenterModule.java

dd-java-agent/testing/src/main/groovy/datadog/trace/agent/test/asserts/TagsAssert.groovy

dd-java-agent/instrumentation/openai-java/openai-java-3.0/build.gradle

dd-trace-core/src/main/java/datadog/trace/llmobs/writer/ddintake/LLMObsSpanMapper.java

...ava-agent/testing/src/main/groovy/datadog/trace/agent/test/server/http/TestHttpServer.groovy

…ge data

vandonr · 2026-01-22T14:10:27Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37a8fd12f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dd-trace-core/src/main/java/datadog/trace/llmobs/writer/ddintake/LLMObsSpanMapper.java

PerfectSlayer

Looking for for LP. I left aside the instrumentation part.
Thanks for the follow up changes 👍

...ava-agent/testing/src/main/groovy/datadog/trace/agent/test/server/http/TestHttpServer.groovy

vandonr · 2026-01-27T09:19:39Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d9a6fcfbce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-01-27T09:23:35Z

...ai-java-3.0/src/main/java/datadog/trace/instrumentation/openai_java/HttpResponseWrapper.java

+  public static <T> CompletableFuture<HttpResponseFor<T>> wrapFuture(
+      CompletableFuture<HttpResponseFor<T>> future,
+      AgentSpan span,
+      BiConsumer<AgentSpan, T> decorate) {
+    return future
+        .thenApply(response -> wrap(response, span, decorate))
+        .whenComplete((_r, t) -> DECORATE.finishSpan(span, t));


Avoid finishing async spans before response parsing

The wrapFuture path finishes the span as soon as the CompletableFuture completes successfully, but the OpenAI async APIs return a response that is parsed later; the output decorators run in parse(). That means for any async create call (chat/completions/responses) where the future resolves before parse() is invoked—which is the normal usage—the span is already finished and the output tags/metrics set in parse() can be dropped or ignored. This effectively loses LLMObs output data for async calls. Consider only finishing on error in whenComplete and letting parse()/close() finish the span on success.

Useful? React with 👍 / 👎.

🤔 we have tests on async requests though, which seem to be asserting the correct tags are applied. Is it that we do all the operations in a row in the test, which is not representative of a "real world" use case ?
Or maybe chatgpt's concerns are unfounded

ygree added 30 commits October 24, 2025 18:03

Initial APM-only openai-java instrumentation with a unit test.

e7bd8d9

Start llmobs system in tests and create llmobs span.

6feb61e

streamed request completion test

d541adb

README

c209bd0

Instrument sync streamed completion

a82856d

Mock Streamed Completion in Tests

3d3def9

Add failing test for Async Completion

3542b9e

Async Single Completion. Simplify tests. Remove println. Cleanup.

c12cd0a

Extract DDHttpResponseFor to intercept when the response is parsed in…

2af4e5f

…stead of forcing it

Instrument and test Async Stream Completion

cec33af

More tests "streamed request completion test with withRawResponse" an…

b2b4141

…d "streamed async request completion test with withRawResponse"

Fix muzzle check

b1b6db2

Add openai APM tags, assert tags, todo tags

baa687d

Wrap HttpResponseFor instead of forcing parsing. Add TODOs

b9ad963

Set response model tag

5bf16f4

Set "openai.organization.name"

be20fa1

Call decorateWithResponse from the wrappers

Set ratelimit metrics

dd230b7

api_base

c3f94b0

"openai.request.method" & "openai.request.endpoint"

ddf0e09

createChatCompletion instrumentation

fb9fd04

Reorder tests single then stream

a4358a8

Async Single Chat Completion

99dedfe

Async Streamed Chat Completion

d66a5ee

Rename assertChatCompletionTrace

f8af3ae

Instrument Embeddings

846e945

Rename span resources to be aligned with trace-py Add http.client resource assertion

ResponseService WIP

b14bc4b

Test case renaming

ResponseService synch

e8daf08

Reorder tests by synch, async

ResponseServiceAsyncInstrumentation

c8a9e6e

Fix Embeddings fixture for the latestDepTest when base64

Setup httpClient for tests WIP

9754cd8

Intercept Http req/resp with TestOpenAiHttpClient

aeaae2a

ygree requested review from a team as code owners December 23, 2025 21:39

ygree requested review from dougqh and removed request for a team December 23, 2025 21:39

ygree added the type: feature request label Dec 23, 2025

PerfectSlayer reviewed Jan 12, 2026

View reviewed changes

Merge branch 'master' into ygree/openai-java

2313a23

ygree force-pushed the ygree/openai-java branch from da72f90 to 3468137 Compare January 21, 2026 20:12

ygree added 2 commits January 21, 2026 12:24

Fix CompletionDecorator to properly accumulate output and collect usa…

3a93165

…ge data

Code review follow-up fixes

aea89e3

ygree force-pushed the ygree/openai-java branch from 6fa5b22 to aea89e3 Compare January 21, 2026 20:24

remove lockfile

35767b7

ygree requested a review from PerfectSlayer January 21, 2026 22:12

ygree added 2 commits January 21, 2026 17:17

multi-choice response tests

ca7ac93

clean up

37a8fd1

ygree force-pushed the ygree/openai-java branch from c1b0c81 to 37a8fd1 Compare January 22, 2026 02:02

chatgpt-codex-connector bot reviewed Jan 22, 2026

View reviewed changes

dd-trace-core/src/main/java/datadog/trace/llmobs/writer/ddintake/LLMObsSpanMapper.java Show resolved Hide resolved

PerfectSlayer approved these changes Jan 22, 2026

View reviewed changes

...ava-agent/testing/src/main/groovy/datadog/trace/agent/test/server/http/TestHttpServer.groovy Outdated Show resolved Hide resolved

PerfectSlayer changed the title ~~feat: openai-java v3.0+ instrumentation~~ Add openai-java v3.0+ instrumentation Jan 22, 2026

ygree added 2 commits January 22, 2026 11:16

minor code review follow up fixes

84065c9

Add common tags

d9a6fcf

ygree force-pushed the ygree/openai-java branch from 96ac8e8 to d9a6fcf Compare January 23, 2026 00:27

ygree added this to the 1.59.0 milestone Jan 26, 2026

ygree requested a review from vandonr January 26, 2026 19:55

chatgpt-codex-connector bot reviewed Jan 27, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add openai-java v3.0+ instrumentation #9959

Add openai-java v3.0+ instrumentation #9959

ygree commented Nov 13, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 23, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vandonr commented Jan 22, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

vandonr commented Jan 27, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Jan 27, 2026

Uh oh!

vandonr Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add openai-java v3.0+ instrumentation #9959

Are you sure you want to change the base?

Add openai-java v3.0+ instrumentation #9959

Conversation

ygree commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Dec 23, 2025

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vandonr commented Jan 22, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vandonr commented Jan 27, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

vandonr Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ygree commented Nov 13, 2025 •

edited

Loading