From 59cc515bf2ed9562ab4d9b98c6f4700a2d986a81 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 28 Aug 2025 23:49:35 +0000 Subject: [PATCH 01/16] A103: xDS Composite Filter --- A103-xds-composite-filter.md | 127 +++++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) create mode 100644 A103-xds-composite-filter.md diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md new file mode 100644 index 000000000..a5456bb1d --- /dev/null +++ b/A103-xds-composite-filter.md @@ -0,0 +1,127 @@ +A103: xDS Composite Filter +---- +* Author(s): markdroth +* Approver: ejona86, dfawley +* Status: {Draft, In Review, Ready for Implementation, Implemented} +* Implemented in: +* Last updated: 2025-08-28 +* Discussion at: (filled after thread exists) + +## Abstract + +gRPC will support the [xDS Composite filter][composite], which is a +"wrapper" filter that dynamically determines which filter to use based +on request attributes. + +[composite]: https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/composite_filter + +## Background + +xDS support in the gRPC client and server are described in [A27] and +[A36], respectively. xDS HTTP filter support is described in [A39]. + +The composite filter will make use of the Unified Matching API and CEL +support described in [A77]. + +### Related Proposals: +* [A27: xDS-Based Global Load Balancing][A27] +* [A39: xDS HTTP Filter Support][A39] +* [A36: xDS-Enabled Servers][A36] +* [A77: xDS Server-Side Rate Limiting][A77] (pending) + +[A27]: A27-xds-global-load-balancing.md +[A36]: A36-xds-for-servers.md +[A39]: A39-xds-http-filters.md +[A77]: https://github.com/grpc/proposal/pull/414 + +## Proposal + +We will support the composite filter in both the gRPC client and gRPC +server. + +### xDS Resource Validation + +Today, the composite filter supports configuring only one filter as a +result of the matching tree. However, we have use-cases where we need +to select a chain of more than one filter based on the matching tree. +As a result, we are proposing a change to the composite filter's config +to allow selecting a chain of filters +(https://github.com/envoyproxy/envoy/pull/40885). + +The composite filter is configured via the +[`envoy.extensions.common.matching.v3.ExtensionWithMatcher` +proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L25). +Within it, gRPC will look at the following fields: +- [extension_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L34C39-L34C55): + This must contain a + [`envoy.extensions.filters.http.composite.v3.Composite` + proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L33), + which has no fields. +- [xds_matcher](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L31): + Specifies a unified matcher tree indicating the config of the filter + to use. The actions in this tree must be one of two types: + - [`envoy.extensions.filters.common.matcher.action.v3.SkipFilter`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/common/matcher/action/v3/skip_action.proto#L24): + This indicates that the no filter will be executed. + - [`envoy.extensions.filters.http.composite.v3.ExecuteFilterAction`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L49): + This indicates which filter should be executed. Within it: + - [typed_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L54C39-L54C51): + The filter to configure. It will be validated using the xDS HTTP filter + registry, just as the top-level filters in the HTTP connection manager + config are. This field is ignored if `filter_chain` is set. + - [dynamic_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L59): + TODO: Do we need to support ECDS here? Also need to define + precedence rules. + - filter_chain (new field added in + https://github.com/envoyproxy/envoy/pull/40885): If set, the + `typed_config` field is ignored. This specifies a chain of + filters to call, in order. Each filter in the chain will be + validated using the xDS HTTP filterregistry, just as the top-level + filters in the HTTP connection manager config are. + - [sample_percent](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L69C43-L69C57): + - Optional; if unset, the specified filter(s) are always executed. + If set, for each RPC, a random number will be generated between + 0 and 100, and if that number is less than the specified + threshold, the specified filter(s) will be executed. + Within this field: + - [default_value](https://github.com/envoyproxy/envoy/blob/cdd19052348f7f6d85910605d957ba4fe0538aec/api/envoy/config/core/v3/base.proto#L648): + This field must be present. The configured value will be capped at + 100%. + - runtime_key: This field will be ignored, since gRPC does not + have a runtime system. +- matcher: gRPC will not support this deprecated field. + +We will also support per-route overrides via the +[`envoy.extensions.common.matching.v3.ExtensionWithMatcherPerRoute` +proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L39C9-L39C37). +In this proto, the +[xds_matcher](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L41) +must be validated the same way as the corresponding field in the +top-level config. The value of this field will replace the value of the +field in the top-level config. + +### CEL Attributes + +In addition to the CEL request attributes described in [A77], we will +also add support for the following [configuration +attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html#configuration-attributes): +- `xds.route_metadata` +- TODO: others? + +### Temporary environment variable protection + +Support for the composite filter will be guarded by the +`GRPC_EXPERIMENTAL_XDS_COMPOSITE_FILTER` environment variable. This +guard will be removed once the feature passes interop tests. + +## Rationale + +The composite filter API seems a little unusual. One would naively +have expected it to be structured by having the matcher tree live +in the config for the composite filter directly, rather than using +`ExtensionWithMatcher`. Unfortunately, that's not the way the API evolved +in Envoy, so we'll stick with what already exists for compatibility +reasons. + +## Implementation + +Will be implemented in C-core, Java, Go, and Node. From 4412a75621c5448e56802c46292118a3690526cc Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 29 Aug 2025 00:20:38 +0000 Subject: [PATCH 02/16] more CEL fields --- A103-xds-composite-filter.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index a5456bb1d..f7e937230 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -104,8 +104,16 @@ field in the top-level config. In addition to the CEL request attributes described in [A77], we will also add support for the following [configuration attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html#configuration-attributes): -- `xds.route_metadata` -- TODO: others? +(TODO: document these fields!) +- `xds.route_metadata`: TODO: tie to metadata registry from A83 +- `source.ip`: TODO: behavior on client side? +- `source.port`: TODO: behavior on client side? +- `connection.requested_server_name`: TODO: behavior on client side? Do + we have access to this data on server side? +- `connection.tls_version`: TODO: behavior on client side? Do we have + access to this data on server side? +- `connection.sha256_peer_certificate_digest`: TODO: behavior on client + side? Do we have access to this data on server side? ### Temporary environment variable protection From 32c8b04b4eb8dc9173ff81833ac32c64504e8ad1 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Sat, 30 Aug 2025 00:04:57 +0000 Subject: [PATCH 03/16] more details --- A103-xds-composite-filter.md | 62 ++++++++++++++++++++++++------------ 1 file changed, 41 insertions(+), 21 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index f7e937230..3dfbc26e0 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2025-08-28 +* Last updated: 2025-08-29 * Discussion at: (filled after thread exists) ## Abstract @@ -28,11 +28,13 @@ support described in [A77]. * [A39: xDS HTTP Filter Support][A39] * [A36: xDS-Enabled Servers][A36] * [A77: xDS Server-Side Rate Limiting][A77] (pending) +* [A83: xDS GCP Authentication Filter][A83] [A27]: A27-xds-global-load-balancing.md [A36]: A36-xds-for-servers.md [A39]: A39-xds-http-filters.md [A77]: https://github.com/grpc/proposal/pull/414 +[A83]: A83-xds-gcp-authn-filter.md ## Proposal @@ -67,16 +69,18 @@ Within it, gRPC will look at the following fields: - [typed_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L54C39-L54C51): The filter to configure. It will be validated using the xDS HTTP filter registry, just as the top-level filters in the HTTP connection manager - config are. This field is ignored if `filter_chain` is set. + config are. This field is ignored if `filter_chain` is set. It + is an error if neither `typed_config` nor `filter_chain` are set. - [dynamic_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L59): - TODO: Do we need to support ECDS here? Also need to define - precedence rules. + This field will be ignored for now, since gRPC does not currently + support ECDS. Support for ECDS will be added in a subsequent gRFC. - filter_chain (new field added in - https://github.com/envoyproxy/envoy/pull/40885): If set, the - `typed_config` field is ignored. This specifies a chain of - filters to call, in order. Each filter in the chain will be - validated using the xDS HTTP filterregistry, just as the top-level - filters in the HTTP connection manager config are. + https://github.com/envoyproxy/envoy/pull/40885): This specifies a + chain of filters to call, in order. Each filter in the chain will + be validated using the xDS HTTP filter registry, just as the + top-level filters in the HTTP connection manager config are. + If set, the `typed_config` field is ignored. It is an error if + neither `typed_config` nor `filter_chain` are set. - [sample_percent](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L69C43-L69C57): - Optional; if unset, the specified filter(s) are always executed. If set, for each RPC, a random number will be generated between @@ -99,21 +103,37 @@ must be validated the same way as the corresponding field in the top-level config. The value of this field will replace the value of the field in the top-level config. +The parsed representation of the composite filter's config will be a +matcher tree, using the same unified matcher API implemented for [A77]. +The actions in the matcher tree will be parsed filter configs. + ### CEL Attributes In addition to the CEL request attributes described in [A77], we will -also add support for the following [configuration -attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html#configuration-attributes): -(TODO: document these fields!) -- `xds.route_metadata`: TODO: tie to metadata registry from A83 -- `source.ip`: TODO: behavior on client side? -- `source.port`: TODO: behavior on client side? -- `connection.requested_server_name`: TODO: behavior on client side? Do - we have access to this data on server side? -- `connection.tls_version`: TODO: behavior on client side? Do we have - access to this data on server side? -- `connection.sha256_peer_certificate_digest`: TODO: behavior on client - side? Do we have access to this data on server side? +also add support for some additional [CEL +attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html#configuration-attributes) +that we expect to be useful for the composite filter. + +On both the gRPC client and server sides, we will add support for the +`xds.route_metadata` attribute. To support this, we will add support +for parsing the [`Route.metadata` +field](https://github.com/envoyproxy/envoy/blob/f384ab2b3e3aa0564ef25f57dc2ed8ad61eaf0cb/api/envoy/config/route/v3/route_components.proto#L319) +in the xDS RouteConfiguration. This field will be validated the same +way as cluster metadata, as described in [A83]. The parsed metadata map +will be added to the route in the parsed RouteConfiguration resource, +and that map will be accessed by this CEL attribute. + +TODO: from CEL's perspective, the `xds.route_metadata` attribute is +supposed to be an xDS metadata proto. how do we make that work without +depending on protobuf? + +We will also add support for the following attributes on the gRPC server +side only (these attributes are not relevant on the client side): +- `source.ip` +- `source.port` +- `connection.requested_server_name` +- `connection.tls_version` +- `connection.sha256_peer_certificate_digest` ### Temporary environment variable protection From b8987fb29a895619b7a1b14d72105d9743299a06 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Sat, 30 Aug 2025 00:20:15 +0000 Subject: [PATCH 04/16] matcher tree encodes skip actions --- A103-xds-composite-filter.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 3dfbc26e0..d0a868968 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -105,7 +105,9 @@ field in the top-level config. The parsed representation of the composite filter's config will be a matcher tree, using the same unified matcher API implemented for [A77]. -The actions in the matcher tree will be parsed filter configs. +The actions in the matcher tree will be one of two possible values: a +parsed filter config, or an indication that the filter should be skipped +(in the case of a `SkipFilter` proto). ### CEL Attributes From c93221303ae80afc1559e0403b1c4bc8c042810c Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 5 Sep 2025 00:57:54 +0000 Subject: [PATCH 05/16] recursion limit --- A103-xds-composite-filter.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index d0a868968..33ab1466f 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2025-08-29 +* Last updated: 2025-09-04 * Discussion at: (filled after thread exists) ## Abstract @@ -109,6 +109,9 @@ The actions in the matcher tree will be one of two possible values: a parsed filter config, or an indication that the filter should be skipped (in the case of a `SkipFilter` proto). +Note that in order to avoid potential stack overflows, we will impose +a maximum recursion depth of 8 when parsing HTTP filter configs. + ### CEL Attributes In addition to the CEL request attributes described in [A77], we will From 94c4bfbdd170535d2b441452b8e2d72b26213203 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 5 Sep 2025 01:18:48 +0000 Subject: [PATCH 06/16] support only filter_metadata, not typed_filter_metadata --- A103-xds-composite-filter.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 33ab1466f..e941029e5 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -120,17 +120,17 @@ attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/adva that we expect to be useful for the composite filter. On both the gRPC client and server sides, we will add support for the -`xds.route_metadata` attribute. To support this, we will add support -for parsing the [`Route.metadata` +`xds.route_metadata.filter_metadata` attribute. To support this, we will +add support for parsing the [`Route.metadata` field](https://github.com/envoyproxy/envoy/blob/f384ab2b3e3aa0564ef25f57dc2ed8ad61eaf0cb/api/envoy/config/route/v3/route_components.proto#L319) in the xDS RouteConfiguration. This field will be validated the same way as cluster metadata, as described in [A83]. The parsed metadata map will be added to the route in the parsed RouteConfiguration resource, -and that map will be accessed by this CEL attribute. - -TODO: from CEL's perspective, the `xds.route_metadata` attribute is -supposed to be an xDS metadata proto. how do we make that work without -depending on protobuf? +and that map will be accessed by this CEL attribute. Note that we will +support only `filter_metadata`, not `typed_filter_metadata`, so that we +do not have to handle protobuf descriptor functionality; to that end, we +will use only those entries in the parsed metadata map that correspond +to `google.protobuf.Struct` type. We will also add support for the following attributes on the gRPC server side only (these attributes are not relevant on the client side): From 4d4f41a236fb8429b784cc63aa6390d963e50ba6 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 5 Sep 2025 01:23:38 +0000 Subject: [PATCH 07/16] add filter behavior section --- A103-xds-composite-filter.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index e941029e5..3afd296a9 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -140,6 +140,17 @@ side only (these attributes are not relevant on the client side): - `connection.tls_version` - `connection.sha256_peer_certificate_digest` +### Filter Behavior + +Because all of the CEL attributes that we are exposing are available +when we see the client's initial metadata, that is the point at which +the filter will evaluate the matcher tree and decide which filter chain +to use. After that point, all other filter hooks will be delegated to +the chosen filter chain. + +If we ever in the future need to support other attributes that are not +yet available at this point, we will need to make changes here. + ### Temporary environment variable protection Support for the composite filter will be guarded by the From 13868f62d3a2821b1a93f45c7a8fd02a10d7ad95 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Tue, 16 Sep 2025 00:24:09 +0000 Subject: [PATCH 08/16] fix typo --- A103-xds-composite-filter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 3afd296a9..6afe48666 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -63,7 +63,7 @@ Within it, gRPC will look at the following fields: Specifies a unified matcher tree indicating the config of the filter to use. The actions in this tree must be one of two types: - [`envoy.extensions.filters.common.matcher.action.v3.SkipFilter`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/common/matcher/action/v3/skip_action.proto#L24): - This indicates that the no filter will be executed. + This indicates that no filter will be executed. - [`envoy.extensions.filters.http.composite.v3.ExecuteFilterAction`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L49): This indicates which filter should be executed. Within it: - [typed_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L54C39-L54C51): From c6bbc1e6c050936e6bcd0717bfae39d2ccbc1e28 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 18 Sep 2025 22:58:43 +0000 Subject: [PATCH 09/16] add mailing list link --- A103-xds-composite-filter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 6afe48666..9f1f39376 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -5,7 +5,7 @@ A103: xDS Composite Filter * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: * Last updated: 2025-09-04 -* Discussion at: (filled after thread exists) +* Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract From 27fd0ad7ba714d343f3e03113649d815a81fb983 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 1 Dec 2025 23:14:55 +0000 Subject: [PATCH 10/16] fix attribute name --- A103-xds-composite-filter.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 9f1f39376..0d694b8fc 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2025-09-04 +* Last updated: 2025-12-01 * Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract @@ -116,7 +116,7 @@ a maximum recursion depth of 8 when parsing HTTP filter configs. In addition to the CEL request attributes described in [A77], we will also add support for some additional [CEL -attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html#configuration-attributes) +attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html) that we expect to be useful for the composite filter. On both the gRPC client and server sides, we will add support for the @@ -134,7 +134,7 @@ to `google.protobuf.Struct` type. We will also add support for the following attributes on the gRPC server side only (these attributes are not relevant on the client side): -- `source.ip` +- `source.address` - `source.port` - `connection.requested_server_name` - `connection.tls_version` From 551505497bad2574732ab9721bd8dd2fad6f458d Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 30 Jan 2026 00:51:01 +0000 Subject: [PATCH 11/16] reference A106 instead of A77 for unified matcher and CEL --- A103-xds-composite-filter.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 0d694b8fc..5aa29fa24 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2025-12-01 +* Last updated: 2026-01-29 * Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract @@ -21,19 +21,19 @@ xDS support in the gRPC client and server are described in [A27] and [A36], respectively. xDS HTTP filter support is described in [A39]. The composite filter will make use of the Unified Matching API and CEL -support described in [A77]. +support described in [A106]. ### Related Proposals: * [A27: xDS-Based Global Load Balancing][A27] * [A39: xDS HTTP Filter Support][A39] * [A36: xDS-Enabled Servers][A36] -* [A77: xDS Server-Side Rate Limiting][A77] (pending) * [A83: xDS GCP Authentication Filter][A83] +* [A106: xDS Unified Matcher and CEL Integration][A106] (pending) [A27]: A27-xds-global-load-balancing.md [A36]: A36-xds-for-servers.md [A39]: A39-xds-http-filters.md -[A77]: https://github.com/grpc/proposal/pull/414 +[A106]: https://github.com/grpc/proposal/pull/520 [A83]: A83-xds-gcp-authn-filter.md ## Proposal @@ -104,7 +104,7 @@ top-level config. The value of this field will replace the value of the field in the top-level config. The parsed representation of the composite filter's config will be a -matcher tree, using the same unified matcher API implemented for [A77]. +matcher tree, using the unified matcher API described in [A106]. The actions in the matcher tree will be one of two possible values: a parsed filter config, or an indication that the filter should be skipped (in the case of a `SkipFilter` proto). @@ -114,7 +114,7 @@ a maximum recursion depth of 8 when parsing HTTP filter configs. ### CEL Attributes -In addition to the CEL request attributes described in [A77], we will +In addition to the CEL request attributes described in [A106], we will also add support for some additional [CEL attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes.html) that we expect to be useful for the composite filter. From b87b76e6e3eeb97078bf658a0476498bd376e093 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Wed, 11 Feb 2026 20:11:21 +0000 Subject: [PATCH 12/16] disallow keep_matching --- A103-xds-composite-filter.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 5aa29fa24..44776acc8 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2026-01-29 +* Last updated: 2026-02-11 * Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract @@ -61,7 +61,9 @@ Within it, gRPC will look at the following fields: which has no fields. - [xds_matcher](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L31): Specifies a unified matcher tree indicating the config of the filter - to use. The actions in this tree must be one of two types: + to use, as described in [A106]. Validation will fail if `keep_matching` + is enabled anywhere in the matcher tree. The actions in this tree must + be one of two types: - [`envoy.extensions.filters.common.matcher.action.v3.SkipFilter`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/common/matcher/action/v3/skip_action.proto#L24): This indicates that no filter will be executed. - [`envoy.extensions.filters.http.composite.v3.ExecuteFilterAction`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L49): From 39a1fe26143b578e0e578a0acda5c71d144e40cb Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 12 Feb 2026 21:52:30 +0000 Subject: [PATCH 13/16] add nested filter validation, and clarify filter behavior --- A103-xds-composite-filter.md | 103 +++++++++++++++++++++++++---------- 1 file changed, 75 insertions(+), 28 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 44776acc8..caddc110b 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2026-02-11 +* Last updated: 2026-02-12 * Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract @@ -52,13 +52,13 @@ to allow selecting a chain of filters The composite filter is configured via the [`envoy.extensions.common.matching.v3.ExtensionWithMatcher` -proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L25). -Within it, gRPC will look at the following fields: +proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L25) +message. Within it, gRPC will look at the following fields: - [extension_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L34C39-L34C55): This must contain a [`envoy.extensions.filters.http.composite.v3.Composite` - proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L33), - which has no fields. + proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L33) + message, which has no fields. - [xds_matcher](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L31): Specifies a unified matcher tree indicating the config of the filter to use, as described in [A106]. Validation will fail if `keep_matching` @@ -67,20 +67,19 @@ Within it, gRPC will look at the following fields: - [`envoy.extensions.filters.common.matcher.action.v3.SkipFilter`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/common/matcher/action/v3/skip_action.proto#L24): This indicates that no filter will be executed. - [`envoy.extensions.filters.http.composite.v3.ExecuteFilterAction`](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L49): - This indicates which filter should be executed. Within it: + This indicates which filter(s) should be executed. Within it: - [typed_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L54C39-L54C51): - The filter to configure. It will be validated using the xDS HTTP filter - registry, just as the top-level filters in the HTTP connection manager - config are. This field is ignored if `filter_chain` is set. It + The filter to configure. See [Nested Filter + Validation](#nested-filter-validation) below for validation rules. + This field is ignored if `filter_chain` is set. It is an error if neither `typed_config` nor `filter_chain` are set. - [dynamic_config](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L59): This field will be ignored for now, since gRPC does not currently support ECDS. Support for ECDS will be added in a subsequent gRFC. - filter_chain (new field added in https://github.com/envoyproxy/envoy/pull/40885): This specifies a - chain of filters to call, in order. Each filter in the chain will - be validated using the xDS HTTP filter registry, just as the - top-level filters in the HTTP connection manager config are. + chain of filters to call, in order. See [Nested Filter + Validation](#nested-filter-validation) below for validation rules. If set, the `typed_config` field is ignored. It is an error if neither `typed_config` nor `filter_chain` are set. - [sample_percent](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/filters/http/composite/v3/composite.proto#L69C43-L69C57): @@ -98,22 +97,51 @@ Within it, gRPC will look at the following fields: We will also support per-route overrides via the [`envoy.extensions.common.matching.v3.ExtensionWithMatcherPerRoute` -proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L39C9-L39C37). -In this proto, the +proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L39C9-L39C37) +proto. In this proto, the [xds_matcher](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L41) -must be validated the same way as the corresponding field in the +field must be validated the same way as the corresponding field in the top-level config. The value of this field will replace the value of the field in the top-level config. -The parsed representation of the composite filter's config will be a -matcher tree, using the unified matcher API described in [A106]. +The parsed representation of the composite filter's config will be +a matcher tree, using the unified matcher API described in [A106]. The actions in the matcher tree will be one of two possible values: a -parsed filter config, or an indication that the filter should be skipped -(in the case of a `SkipFilter` proto). +list of parsed filter configs, or an indication that the filter should +be skipped (in the case of a `SkipFilter` proto). Note that in order to avoid potential stack overflows, we will impose a maximum recursion depth of 8 when parsing HTTP filter configs. +#### Nested Filter Validation + +Filter configs within the matcher tree will be validated using the xDS +HTTP filter registry, just as the top-level filters in the HTTP connection +manager config are. + +It will be considered a configuration error if a nested filter is a +terminal filter (as described in [A39]). While it is conceivably +possible that there could be a use-case where the composite filter is +*intended* to provide a terminal filter, it will be difficult to +determine at config validation time whether this will actually happen +in all cases, so for now we will simply disallow this. + +Note that, as per [A39], any given xDS HTTP filter may be supported on +only the gRPC client or server side. When processing the filter list +in LDS, we know whether the resource is an API listener (client side) +or socket listener (server side), so we reject filters that are not +supported on the side they are being configured on. However, because +the composite filter's per-route override config may be delivered via +RDS instead of LDS, it will in this case not be possible for gRPC to +detect that a filter is being configured on an unsupported side when +validating the RDS resource. Instead, the composite filter will need +to handle this on a per-RPC basis: if a nested filter chain includes a +filter that is not supported on the side that it is running on, it will +fail the RPC with status UNAVAILABLE. Note that if the problematic +filter is not the first filter in the nested filter chain, implementations +may fail the RPC without ever starting to process the RPC on that filter +chain. + ### CEL Attributes In addition to the CEL request attributes described in [A106], we will @@ -144,14 +172,31 @@ side only (these attributes are not relevant on the client side): ### Filter Behavior -Because all of the CEL attributes that we are exposing are available -when we see the client's initial metadata, that is the point at which -the filter will evaluate the matcher tree and decide which filter chain -to use. After that point, all other filter hooks will be delegated to -the chosen filter chain. - -If we ever in the future need to support other attributes that are not -yet available at this point, we will need to make changes here. +When the filter sees the client's initial metadata, it will evaluate the +matcher tree for the RPC. The result of that evaluation will be one of +the following: + +- If the matcher tree does not find a match, the RPC will be failed with + UNAVAILABLE status. +- If the matcher tree finds a `SkipFilter` match, the filter will simply + pass the RPC through to the next filter (the one after the composite + filter), without delegating to any nested filters. +- If the matcher tree finds an `ExecuteFilterAction` match, then the + filter will generate a random number and check the sample_percent + field to determine if the RPC should be sampled. If the RPC is not + sampled, then the filter will pass the RPC through to the next filter + (the one after the composite filter), without delegating to any nested + filters. Otherwise (if the RPC *is* sampled), the RPC will be passed + to the nested filter chain before being sent to the next filter (the + one after the composite filter). + +Note: Because all of the CEL attributes that we are currently supporting +are available when we see the client's initial metadata, that is the +point at which the filter will evaluate the matcher tree and decide which +filter chain to use. After that point, all other filter hooks will be +delegated to the chosen filter chain. (If we ever in the future need +to support other attributes that are not yet available at this point, +such as response attributes, we might need a more complex structure here.) ### Temporary environment variable protection @@ -166,7 +211,9 @@ have expected it to be structured by having the matcher tree live in the config for the composite filter directly, rather than using `ExtensionWithMatcher`. Unfortunately, that's not the way the API evolved in Envoy, so we'll stick with what already exists for compatibility -reasons. +reasons. (Envoy is currently attempting to add a more sane API in +https://github.com/envoyproxy/envoy/pull/43227. In the future, +we can support that new API as well.) ## Implementation From 8ebc93d10bded640523cb647d584a02ff0630417 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 23 Feb 2026 19:30:27 +0000 Subject: [PATCH 14/16] use cluster metadata CEL attribute instead of route metadata --- A103-xds-composite-filter.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index caddc110b..236937fe4 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2026-02-12 +* Last updated: 2026-02-23 * Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract @@ -150,17 +150,19 @@ attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/adva that we expect to be useful for the composite filter. On both the gRPC client and server sides, we will add support for the -`xds.route_metadata.filter_metadata` attribute. To support this, we will -add support for parsing the [`Route.metadata` -field](https://github.com/envoyproxy/envoy/blob/f384ab2b3e3aa0564ef25f57dc2ed8ad61eaf0cb/api/envoy/config/route/v3/route_components.proto#L319) -in the xDS RouteConfiguration. This field will be validated the same -way as cluster metadata, as described in [A83]. The parsed metadata map -will be added to the route in the parsed RouteConfiguration resource, -and that map will be accessed by this CEL attribute. Note that we will -support only `filter_metadata`, not `typed_filter_metadata`, so that we -do not have to handle protobuf descriptor functionality; to that end, we -will use only those entries in the parsed metadata map that correspond -to `google.protobuf.Struct` type. +`xds.cluster_metadata.filter_metadata` attribute. This will be backed +by the same cluster metadata whose parsing was added in [A83]. Note that +we will support only `xds.cluster_metadata.filter_metadata`, not +`xds.cluster_metadata.typed_filter_metadata`, so that we do not have to +handle protobuf descriptor functionality; to that end, we will use only +those entries in the parsed metadata map that correspond to +`google.protobuf.Struct` type. The actual CEL expression we see will be +something like `xds.cluster_metadata.filter_metadata['%s']['%s']`, where +the first `%s` indicates the key of the entry in the parsed metadata map +and the second `%s` indicates the key in the JSON object. If the JSON +value is not an object, this will not resolve. The type of the CEL +value will be based on the type of the underlying JSON value, either +string or integer. We will also add support for the following attributes on the gRPC server side only (these attributes are not relevant on the client side): From 70611bdb31cfca1c5bac1ff93ff9ae686cba2625 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 27 Feb 2026 21:11:39 +0000 Subject: [PATCH 15/16] fix typo --- A103-xds-composite-filter.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index 236937fe4..e178d0529 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -4,7 +4,7 @@ A103: xDS Composite Filter * Approver: ejona86, dfawley * Status: {Draft, In Review, Ready for Implementation, Implemented} * Implemented in: -* Last updated: 2026-02-23 +* Last updated: 2026-02-27 * Discussion at: https://groups.google.com/g/grpc-io/c/es5taH0OZS8 ## Abstract @@ -97,8 +97,8 @@ message. Within it, gRPC will look at the following fields: We will also support per-route overrides via the [`envoy.extensions.common.matching.v3.ExtensionWithMatcherPerRoute` -proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L39C9-L39C37) -proto. In this proto, the +proto](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L39C9-L39C37). +In this proto, the [xds_matcher](https://github.com/envoyproxy/envoy/blob/0685d7bf568485eb112df2a9c73248cb8bfc1c37/api/envoy/extensions/common/matching/v3/extension_matcher.proto#L41) field must be validated the same way as the corresponding field in the top-level config. The value of this field will replace the value of the From 9480b3b3c77a31b49a537c2268515c86af7819ad Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Fri, 27 Feb 2026 21:13:27 +0000 Subject: [PATCH 16/16] clarify wording about failing RPCs --- A103-xds-composite-filter.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/A103-xds-composite-filter.md b/A103-xds-composite-filter.md index e178d0529..26a3b3df8 100644 --- a/A103-xds-composite-filter.md +++ b/A103-xds-composite-filter.md @@ -136,11 +136,12 @@ RDS instead of LDS, it will in this case not be possible for gRPC to detect that a filter is being configured on an unsupported side when validating the RDS resource. Instead, the composite filter will need to handle this on a per-RPC basis: if a nested filter chain includes a -filter that is not supported on the side that it is running on, it will +filter that is not supported on the side that it is running on, we will fail the RPC with status UNAVAILABLE. Note that if the problematic filter is not the first filter in the nested filter chain, implementations may fail the RPC without ever starting to process the RPC on that filter -chain. +chain, or they may wait to fail the RPC until it gets to the problematic +filter. ### CEL Attributes