Skip to content

feat(gcp_stackdriver_logs sink): Support user-defined insertId#24968

Open
garethpelly wants to merge 6 commits intovectordotdev:masterfrom
garethpelly:add-stackdriver-insertId-field
Open

feat(gcp_stackdriver_logs sink): Support user-defined insertId#24968
garethpelly wants to merge 6 commits intovectordotdev:masterfrom
garethpelly:add-stackdriver-insertId-field

Conversation

@garethpelly
Copy link
Contributor

@garethpelly garethpelly commented Mar 19, 2026

Summary

This adds support for setting a custom insertId at the root level of the LogEntry, allowing users to define their own unique identifier for each log entry.

In a similar way to how the severity_key field works with the sink now, we extract a custom insertId from log events via an insert_id_key configuration option. The named field is removed from the log event if present.

Vector configuration

sources:
  dummy_logs:
    type: "demo_logs"
    format: "syslog"
    interval: 1

transforms:
  parse_logs:
    type: "remap"
    inputs: ["dummy_logs"]
    source: |
      . = parse_syslog!(string!(.message))
      .my_insert_id = uuid_v4()

sinks:
  my_sink_id:
    type: gcp_stackdriver_logs
    project_id: "my-project"
    log_id: "test_log"
    inputs:
      - parse_logs
    insert_id_key: "my_insert_id"
    resource:
      type: "gce_instance"
      project_id: "my-project"
      instanceId: "Twilight"
      zone: "us-central1-a"

How did you test this PR?

  • Ran unit tests
  • Tested using my Google account and inspected logs in Google Cloud Logging

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Mar 19, 2026
@garethpelly garethpelly changed the title enhancement(gcp_stackdriver_logs sink): Support user-defined insertId feat(gcp_stackdriver_logs sink): Support user-defined insertId Mar 19, 2026
@garethpelly garethpelly force-pushed the add-stackdriver-insertId-field branch from ad3acb6 to 947e021 Compare March 19, 2026 22:31
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Mar 20, 2026
@garethpelly garethpelly force-pushed the add-stackdriver-insertId-field branch from a36e9c2 to ae35824 Compare March 20, 2026 10:19
@github-actions github-actions bot added domain: ci Anything related to Vector's CI environment domain: external docs Anything related to Vector's external, public documentation and removed domain: external docs Anything related to Vector's external, public documentation labels Mar 20, 2026
@garethpelly garethpelly force-pushed the add-stackdriver-insertId-field branch from 8dd3d4f to ae35824 Compare March 20, 2026 10:56
@github-actions github-actions bot removed the domain: external docs Anything related to Vector's external, public documentation label Mar 20, 2026
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Mar 20, 2026
@garethpelly garethpelly marked this pull request as ready for review March 20, 2026 12:09
@garethpelly garethpelly requested review from a team as code owners March 20, 2026 12:09
Copy link

@OliviaShoup OliviaShoup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: ci Anything related to Vector's CI environment domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support user-defined insertId in gcp_stackdriver_logs sink

2 participants