[WIP] Refactor agentscope instrument using genai util to replace #82

sipercai · 2025-12-08T07:33:24Z

Description

This PR refactors the AgentScope instrumentation to use ExtendedTelemetryHandler from opentelemetry-util-genai instead of the previous custom implementation. This change provides better consistency with other GenAI instrumentations, improved span lifecycle management.

opentelemetry-util-genai: support for metrics

Fixes # (issue)

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Unit Tests:
- test_instrumentor.py: Tests for covering instrumentor initialization, instrumentation, and uninstrumentation
- test_model.py: Tests for chat model and embedding model instrumentation
- test_agent.py: Tests for agent instrumentation with tools
- test_span_content.py: Tests for span content capture
Integration Tests: Verified with real AgentScope examples:
- structured_output/main.py: Tests structured output with tool calls
- stream_printing_messages/multi_agent.py: Tests multi-agent conversations
- rag/agent_usage.py: Tests RAG functionality with embeddings

Test Configuration

# Install the updated package
pip install -e loongsuite-python-agent/instrumentation-loongsuite/loongsuite-instrumentation-agentscope

# Run unit tests
pytest tests/ -v

# Run integration test
opentelemetry-instrument --traces_exporter console --metrics_exporter console \
  python agentscope/examples/functionality/structured_output/main.py

Does This PR Require a Core Repo Change?

Yes. - Link to PR:
No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated

commit 6b8e48ebd909636ce4c2d8631b2b382fb5bb8e32 Author: 流屿 <caishipeng.csp@alibaba-inc.com> Date: Thu Dec 4 20:44:50 2025 +0800 fix: span exporter error Change-Id: I0e7a55eece19a166236d931bf69e590ea3a76ed1 Co-developed-by: Cursor <noreply@cursor.com> commit c1f6e3cc23b1dab15d5e6582211ac679588ef871 Author: 流屿 <caishipeng.csp@alibaba-inc.com> Date: Thu Dec 4 15:51:27 2025 +0800 chore: use new genai util Change-Id: I681e478ffc86cd774aac5476fddf6ec4f22c68b6 Co-developed-by: Cursor <noreply@cursor.com> commit cd2448c018efe4a2f5823e20d57862503b448692 Author: 流屿 <caishipeng.csp@alibaba-inc.com> Date: Thu Dec 4 15:40:38 2025 +0800 refactor: update genai util Change-Id: I338a57c2a2311551bf293132b6dc54224823c3a1 Co-developed-by: Cursor <noreply@cursor.com> commit 3141f49745ab756a2bbc89781cb75dd223186ce5 Author: 流屿 <caishipeng.csp@alibaba-inc.com> Date: Tue Dec 2 23:21:14 2025 +0800 refactor: agentscope use util genai Change-Id: Ia6dd446c2ff04a9bc62f705b8001d2a978ae983d Co-developed-by: Cursor <noreply@cursor.com> commit c6217390ff5a5f704a2ab87c660c9159de41638e Author: 流屿 <caishipeng.csp@alibaba-inc.com> Date: Mon Nov 24 14:11:14 2025 +0800 Refactor agentscope instrumentation to standard structure and drop v0 support Change-Id: Ie267bd9304e8291895d2b4b80914698173598437 Co-developed-by: Cursor <noreply@cursor.com> Change-Id: I16dfafe9ae766d2a36bbee1b773598e143a2af21 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: Ib647add02a3828287eba027514ffc3961f672446 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I5f7cf397133843f23cbf98e1af4eed24dc616dab Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I82757e8cf43cc9210d9050be5925370593daf97a Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I41b0f92aad53265e1fa1f65e0da55d8269a6ce9e Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: Idb02a70c0aa35eddf323518392544af46c23a1d2 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I5fe34b20168badff7efccb0a6345c566c35f4f34 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I811f5942da5b4c74b7c963a854d187351eb547e4 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: Id689c30a61620e97ec29ce9a6d4ae0ca75eb3a2a Co-developed-by: Cursor <noreply@cursor.com>

Cirilla-zmh

Great job! Some changes are needed.

Cirilla-zmh · 2025-12-12T02:30:24Z

instrumentation-loongsuite/loongsuite-instrumentation-agentscope/CHANGELOG.md

+- Message formatting tracing
+
+### Note
+- Only supports AgentScope 1.0.0 and above


This should be highlighted as a breaking change.

Cirilla-zmh · 2025-12-12T02:55:26Z

util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py

        _apply_llm_finish_attributes(span, invocation)
+
+        # LoongSuite Extension
+        span.set_status(Status(StatusCode.OK))


Shouldn't be OK

Cirilla-zmh · 2025-12-12T02:58:06Z

util/opentelemetry-util-genai/src/opentelemetry/util/genai/extended_handler.py

            return invocation

        _apply_create_agent_finish_attributes(invocation.span, invocation)
+        invocation.span.set_status(Status(StatusCode.OK))


Shouldn't be OK here.

Cirilla-zmh · 2025-12-12T03:10:21Z

.../loongsuite-instrumentation-agentscope/src/opentelemetry/instrumentation/agentscope/patch.py

+                try:
+                    result_content = _get_tool_result(last_chunk)
+                    if result_content:
+                        span.set_attribute("gen_ai.tool.call.result", _serialize_to_str(result_content))


Why not modify the implementation with genai-util?

Cirilla-zmh · 2025-12-12T03:10:32Z

.../loongsuite-instrumentation-agentscope/src/opentelemetry/instrumentation/agentscope/patch.py

+    }
+
+    if tool_id:
+        span_attributes[GenAIAttributes.GEN_AI_TOOL_CALL_ID] = tool_id


Why not modify the implementation with genai-util?

Cirilla-zmh · 2025-12-12T03:13:27Z

instrumentation-loongsuite/loongsuite-instrumentation-agentscope/pytest.ini

@@ -0,0 +1,42 @@
+[pytest]


Please add tests into https://github.com/alibaba/loongsuite-python-agent/blob/main/tox-loongsuite.ini rather than here.

Change-Id: Ia9d28b59748a3f367c35529c5d9ac311eec32954 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I3a46ccb35472c2212f475b2ccefc0df51fa51c2a Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: Ie4f1c804a22639e6621fc2229d6813bd1d5f8026 Co-developed-by: Cursor <noreply@cursor.com>

Change-Id: I760e6ba769fe398121bc45bbebdd128fec149f48 Co-developed-by: Cursor <noreply@cursor.com>

sipercai added 9 commits December 8, 2025 14:48

fix(agentscope): add missing trace span attributes

06ce138

Change-Id: Ib647add02a3828287eba027514ffc3961f672446 Co-developed-by: Cursor <noreply@cursor.com>

chore: add __future__.annotations for forward type hints

a70de3b

Change-Id: I5f7cf397133843f23cbf98e1af4eed24dc616dab Co-developed-by: Cursor <noreply@cursor.com>

fix: tool context error

44b7504

Change-Id: I82757e8cf43cc9210d9050be5925370593daf97a Co-developed-by: Cursor <noreply@cursor.com>

feat: add extended metrics

b776f28

Change-Id: I41b0f92aad53265e1fa1f65e0da55d8269a6ce9e Co-developed-by: Cursor <noreply@cursor.com>

feat: add extended metrics

6d38a33

Change-Id: Idb02a70c0aa35eddf323518392544af46c23a1d2 Co-developed-by: Cursor <noreply@cursor.com>

fix: test error fix

42d284d

Change-Id: I5fe34b20168badff7efccb0a6345c566c35f4f34 Co-developed-by: Cursor <noreply@cursor.com>

fix: fix test failed

e6fcd70

Change-Id: I811f5942da5b4c74b7c963a854d187351eb547e4 Co-developed-by: Cursor <noreply@cursor.com>

refactor: simplify structure and remove unused code

9ac5aa3

Change-Id: Id689c30a61620e97ec29ce9a6d4ae0ca75eb3a2a Co-developed-by: Cursor <noreply@cursor.com>

github-actions bot assigned 123liuziming, Cirilla-zmh and ralf0131 Dec 8, 2025

Cirilla-zmh reviewed Dec 12, 2025

View reviewed changes

sipercai added 4 commits December 12, 2025 14:31

feat: add spankind

19c5ece

Change-Id: Ia9d28b59748a3f367c35529c5d9ac311eec32954 Co-developed-by: Cursor <noreply@cursor.com>

using handler

569b3ca

Change-Id: I3a46ccb35472c2212f475b2ccefc0df51fa51c2a Co-developed-by: Cursor <noreply@cursor.com>

fix(agentscope): tool using handler function

3cd5bae

Change-Id: Ie4f1c804a22639e6621fc2229d6813bd1d5f8026 Co-developed-by: Cursor <noreply@cursor.com>

remove pytest.ini add metrics test

2082cef

Change-Id: I760e6ba769fe398121bc45bbebdd128fec149f48 Co-developed-by: Cursor <noreply@cursor.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] Refactor agentscope instrument using genai util to replace #82

[WIP] Refactor agentscope instrument using genai util to replace #82

Uh oh!

sipercai commented Dec 8, 2025

Uh oh!

Cirilla-zmh left a comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Uh oh!

Cirilla-zmh Dec 12, 2025

Uh oh!

Cirilla-zmh Dec 12, 2025

Uh oh!

Cirilla-zmh Dec 12, 2025

Uh oh!

Cirilla-zmh Dec 12, 2025

Uh oh!

Cirilla-zmh Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[WIP] Refactor agentscope instrument using genai util to replace #82

Are you sure you want to change the base?

[WIP] Refactor agentscope instrument using genai util to replace #82

Uh oh!

Conversation

sipercai commented Dec 8, 2025

Description

Type of change

How Has This Been Tested?

Test Configuration

Does This PR Require a Core Repo Change?

Checklist:

Uh oh!

Cirilla-zmh left a comment

Choose a reason for hiding this comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Cirilla-zmh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants