Skip to content

chore: add a script to generate tool_call_metrics.json for telemetry.#1257

Merged
yulunz merged 1 commit intomainfrom
chore/tool-metrics-json
Apr 1, 2026
Merged

chore: add a script to generate tool_call_metrics.json for telemetry.#1257
yulunz merged 1 commit intomainfrom
chore/tool-metrics-json

Conversation

@yulunz
Copy link
Copy Markdown
Collaborator

@yulunz yulunz commented Mar 28, 2026

This adds a script that generates a json file that summarizes all tool calls and arguments for each.

  • The arguments run through the blocklist filtering so arguments containing high entropy ids are filtered out (e.g. "uid", "msgid" etc).
  • It uses existing functions from clearcut logger module to transform the arg name and value (i.e. take the length of the string, take the size of the array, and rename the string to be "string_length", and array "array_count" etc).
  • These functions from the clearcut logger module will be later used to sanitize the params as we start to log them.

This doesn't include the append only / deprecation logic just yet (i.e. it's doesn't handle the case when new tools are added / removed, or arguments of the existing tools are modified). This will be added in following PRs.

The parent PR is #1250.

@yulunz yulunz changed the title Add a script to generate tool_call_metrics.json for telemetry. chore: add a script to generate tool_call_metrics.json for telemetry. Mar 28, 2026
@yulunz yulunz requested a review from OrKoN March 28, 2026 01:04
Comment thread tests/telemetry/toolMetricsUtils.test.ts Outdated
Comment thread package.json
Base automatically changed from chore/filter-sensitive-args to main March 31, 2026 18:22
@yulunz yulunz force-pushed the chore/tool-metrics-json branch from 0435df1 to 4ce4615 Compare March 31, 2026 18:51
@yulunz
Copy link
Copy Markdown
Collaborator Author

yulunz commented Mar 31, 2026

I am roughly still having this like I had it initially in order not to make it too complicated. Follow-up CLs will make this look like what we have discussed.

@OrKoN OrKoN added this pull request to the merge queue Apr 1, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 1, 2026
@yulunz yulunz added this pull request to the merge queue Apr 1, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 1, 2026
@yulunz yulunz force-pushed the chore/tool-metrics-json branch from 4ce4615 to 20bbd55 Compare April 1, 2026 18:32
@yulunz yulunz added this pull request to the merge queue Apr 1, 2026
Merged via the queue into main with commit bc8ef4b Apr 1, 2026
20 checks passed
@yulunz yulunz deleted the chore/tool-metrics-json branch April 1, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants