[APMSVLS-197] Add fallback to extract trace context from event.request.headers #1011

Dogbu-cyber · 2026-01-28T18:47:45Z

APMSVLS-197 feat: Add fallback to extract trace context from event.request.headers

Overview

This PR adds automatic trace context extraction from event.request.headers for AppSync integration scenarios. This eliminates the need for customers to use customized extractors for RUM → AppSync → Lambda resolver flows where RUM-injected trace context is nested under event["request"]["headers"].

Problem

Previously, trace context from RUM was not propagated through AppSync to Lambda resolvers because the extension only checked:

Trigger-specific headers (via span inferrer)
event.headers (top-level event headers)
Request headers (HTTP request headers)

For AppSync, the trace context is nested under event.request.headers, which was not checked, causing traces to break at the AppSync boundary.

Solution

Added a fallback extraction step that checks event.request.headers after all other trace context sources have been attempted.

Trace Examples

Before

Without this change, trace context is lost at the AppSync boundary:

Trace showing broken propagation through AppSync

Log showing the passed header value for x-datadog-trace-id does not align with the given trace_id

Flow: RUM → AppSync → Lambda (trace ID not propagated)

After

With this change, trace context is properly extracted and propagated:

Trace showing successful propagation through AppSync

Flow: RUM → AppSync → Lambda resolver (trace continues using value within x-datadog-trace-id)

Testing

Unit Tests

Added 10 unit tests covering:

Extraction from event.request.headers with Datadog headers
Extraction from event.request.headers with W3C TraceContext headers
Edge cases (missing headers, empty objects, invalid data)

Run tests with:

cargo test test_extract_span_context -- --nocapture

Copilot

Pull request overview

This PR adds a fallback mechanism to extract distributed tracing context from event.request.headers for AppSync integration scenarios. This enhancement eliminates the need for custom extractors when RUM injects trace context into AppSync → Lambda resolver flows.

Changes:

Added nested extraction logic to check event.request.headers after primary headers extraction fails

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…request.headers Add automatic trace context extraction from event.request.headers for AppSync integration scenarios. This eliminates the need for customers to use customized extractors for RUM → AppSync → Lambda resolver flows where RUM-injected trace context is nested under event["request"]["headers"].

duncanista · 2026-01-28T23:05:07Z

bottlecap/src/lifecycle/invocation/processor.rs

Could you add some unit testing and show an example trace in the PR description? Just to see what were we missing

…ntext extraction

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

duncanista · 2026-01-29T19:00:33Z

bottlecap/src/lifecycle/invocation/processor.rs

        }

+        if let Some(request_obj) = payload_value.get("request") {
+            if let Some(request_headers) = request_obj.get("headers") {
+                if let Some(sc) = propagator.extract(request_headers) {
+                    debug!("Extracted trace context from event.request.headers");
+                    return Some(sc);
+                }
+            }
+        }
+


Why did you decided that this should be last priority?

I assumed there was some reason for the original order, so I didn't take a stance on where exactly it should go and added it as the last case..

Based on other projects which support AppSync extraction, it seems that they are considered in the same order as inferred spans would do it – I'd suggest to follow that same order

https://github.com/DataDog/datadog-lambda-js/blob/main/src/trace/context/extractor.ts#L92

Thanks! That's a good point.

…or service-specific traces

Dogbu-cyber closed this Jan 28, 2026

Dogbu-cyber changed the title ~~David.ogbureke/extract tracecontext appsync https://datadoghq.atlassian.net/browse/APMSVLS-197~~ [APMSVLS-197] Add fallback to extract trace context from event.request.headers Jan 28, 2026

Dogbu-cyber reopened this Jan 28, 2026

Dogbu-cyber requested a review from Copilot January 28, 2026 18:51

Copilot AI reviewed Jan 28, 2026

View reviewed changes

Dogbu-cyber marked this pull request as draft January 28, 2026 18:54

Dogbu-cyber force-pushed the david.ogbureke/extract-tracecontext-appsync branch from f267045 to 5e936c4 Compare January 28, 2026 18:56

Dogbu-cyber marked this pull request as ready for review January 28, 2026 20:31

Dogbu-cyber requested a review from duncanista January 28, 2026 20:34

Dogbu-cyber marked this pull request as draft January 28, 2026 20:46

Dogbu-cyber force-pushed the david.ogbureke/extract-tracecontext-appsync branch from 5e936c4 to e4d03a2 Compare January 28, 2026 20:47

Dogbu-cyber added the team/serverless label Jan 28, 2026

Dogbu-cyber force-pushed the david.ogbureke/extract-tracecontext-appsync branch 2 times, most recently from 4043feb to 8a2baca Compare January 28, 2026 20:54

Dogbu-cyber marked this pull request as ready for review January 28, 2026 20:59

duncanista reviewed Jan 28, 2026

View reviewed changes

Dogbu-cyber force-pushed the david.ogbureke/extract-tracecontext-appsync branch from 96d4cb4 to b0619c2 Compare January 29, 2026 17:54

[APMSVLS-197] test: Add unit tests for event.request.headers trace co…

391f38c

…ntext extraction

duncanista requested a review from Copilot January 29, 2026 18:48

Copilot AI reviewed Jan 29, 2026

View reviewed changes

Dogbu-cyber force-pushed the david.ogbureke/extract-tracecontext-appsync branch from b0619c2 to 391f38c Compare January 29, 2026 18:50

duncanista reviewed Jan 29, 2026

View reviewed changes

[APMSVLS-197] refactor: Prioritize event.request.headers extraction f…

6d228b0

…or service-specific traces

Dogbu-cyber force-pushed the david.ogbureke/extract-tracecontext-appsync branch from f76c2a8 to 6d228b0 Compare January 29, 2026 19:41

Dogbu-cyber requested a review from duncanista January 30, 2026 15:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APMSVLS-197] Add fallback to extract trace context from event.request.headers #1011

[APMSVLS-197] Add fallback to extract trace context from event.request.headers #1011

Dogbu-cyber commented Jan 28, 2026 •

edited by atlassian bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

duncanista Jan 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

duncanista Jan 29, 2026

Uh oh!

Dogbu-cyber Jan 29, 2026

Uh oh!

duncanista Jan 29, 2026

Uh oh!

Dogbu-cyber Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[APMSVLS-197] Add fallback to extract trace context from event.request.headers #1011

Are you sure you want to change the base?

[APMSVLS-197] Add fallback to extract trace context from event.request.headers #1011

Conversation

Dogbu-cyber commented Jan 28, 2026 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

APMSVLS-197 feat: Add fallback to extract trace context from event.request.headers

Overview

Problem

Solution

Trace Examples

Before

After

Testing

Unit Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

duncanista Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

duncanista Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Dogbu-cyber Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

duncanista Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Dogbu-cyber Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Dogbu-cyber commented Jan 28, 2026 •

edited by atlassian bot

Loading