Skip to content

Conversation

@davidgoli
Copy link
Contributor

@davidgoli davidgoli commented Dec 19, 2025

Type

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Description

Add support for Protocol Buffers binary encoding as an alternative to JSON for OTLP trace, metrics, and log exports. This enables more efficient wire format when communicating with OpenTelemetry collectors. In particular, some OTLP collectors, such as Coralogix, only support protobuf and not json.

  • Add protocol option ("json" | "protobuf") to Otlp.layer and individual OtlpTracer, OtlpMetrics, OtlpLogger layers
  • Implement protobuf wire format encoding following opentelemetry-proto specs
  • Set appropriate Content-Type header (application/x-protobuf vs application/json)
  • No new dependencies - protobuf encoding implemented from scratch

🤖 Generated with Claude Code

Related

  • Related Issue #
  • Closes #

@davidgoli davidgoli requested a review from tim-smart as a code owner December 19, 2025 20:20
@github-project-automation github-project-automation bot moved this to Discussion Ongoing in PR Backlog Dec 19, 2025
@changeset-bot
Copy link

changeset-bot bot commented Dec 19, 2025

🦋 Changeset detected

Latest commit: bf369fa

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@effect/opentelemetry Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Add support for Protocol Buffers binary encoding as an alternative to JSON
for OTLP trace, metrics, and log exports. This enables more efficient
wire format when communicating with OpenTelemetry collectors.

- Add `protocol` option ("json" | "protobuf") to Otlp.layer and individual
  OtlpTracer, OtlpMetrics, OtlpLogger layers
- Implement protobuf wire format encoding following opentelemetry-proto specs
- Set appropriate Content-Type header (application/x-protobuf vs application/json)
- No new dependencies - protobuf encoding implemented from scratch

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@davidgoli davidgoli force-pushed the feat/otel-protobuf-support branch from 09584a8 to 27cd330 Compare December 19, 2025 20:27
@github-project-automation github-project-automation bot moved this from Discussion Ongoing to Waiting on Author in PR Backlog Jan 19, 2026
…tobuf support

Refactors OTLP exporters to use a serializer service pattern, enabling
tree-shaking of protobuf code when only JSON encoding is used.

- Add OtlpSerializer service with JSON layer (default)
- Add OtlpSerializerProtobuf layer for explicit protobuf opt-in
- Remove direct protobuf imports from exporters (key for tree-shaking)
- Update Otlp.layer to provide JSON serializer by default
- Backwards compatible: existing JSON code works unchanged

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@davidgoli davidgoli force-pushed the feat/otel-protobuf-support branch from f7dc1f0 to 93dc380 Compare January 20, 2026 18:31
@davidgoli davidgoli requested a review from tim-smart January 20, 2026 18:37
@tim-smart
Copy link
Contributor

tim-smart commented Jan 21, 2026

I have added this to effect 4.0 here: Effect-TS/effect-smol#1023

Feel free to copy back the tweaks here :)

Simplifies the protobuf support API based on tim-smart's design from effect-smol.

API changes:
- Added Otlp.layerJson and Otlp.layerProtobuf for explicit serialization choice
- Otlp.layer remains as backwards-compatible alias for layerJson
- Removed layerWithSerializer (use layerJson/layerProtobuf instead)
- Replaced OtlpSerializer/OtlpSerializerProtobuf with OtlpSerialization service
- OtlpSerialization now returns HttpBody directly instead of Uint8Array | string

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@davidgoli
Copy link
Contributor Author

@tim-smart done, please take another look

- Use class syntax with type ID for OtlpSerialization (Context.Tag pattern)
- Remove OtlpKind from otlpExporter - exporter doesn't need to know data type
- Lift encode function selection to individual services (OtlpTracer, OtlpMetrics, OtlpLogger)
- Exporter now receives encode function directly via options

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@davidgoli davidgoli requested a review from tim-smart January 21, 2026 02:07
@tim-smart tim-smart force-pushed the feat/otel-protobuf-support branch from 44854e8 to c973e33 Compare January 21, 2026 02:47
@tim-smart tim-smart merged commit f4972ed into Effect-TS:main Jan 21, 2026
11 checks passed
@github-project-automation github-project-automation bot moved this from Waiting on Author to Done in PR Backlog Jan 21, 2026
@github-actions github-actions bot mentioned this pull request Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants