CSHARP-5495: Support $concatArrays and $setUnion aggregation accumulators#2014
Open
papafe wants to merge 7 commits into
Open
CSHARP-5495: Support $concatArrays and $setUnion aggregation accumulators#2014papafe wants to merge 7 commits into
papafe wants to merge 7 commits into
Conversation
…tors Map g.SelectMany(x => x.Arr) and g.SelectMany(x => x.Arr).Distinct() to the new $concatArrays and $setUnion accumulators inside $group, $bucket and $bucketAuto via two new pattern matches in AstGroupingPipelineOptimizer. Add ConcatArrays and SetUnion extension methods on ISetWindowFieldsPartitionExtensions for use inside $setWindowFields.
sanych-sun
reviewed
Jun 1, 2026
| case "As": DeduceAsMethodSerializers(); break; | ||
| case "AsQueryable": DeduceAsQueryableMethodSerializers(); break; | ||
| case "Concat": DeduceConcatMethodSerializers(); break; | ||
| case "ConcatArrays": DeduceConcatArraysMethodSerializers(); break; |
Member
There was a problem hiding this comment.
Need unit-tests for these new supported methods.
Contributor
Author
There was a problem hiding this comment.
Added SerializerFinder unit tests for those.
…etunion-accumulators # Conflicts: # src/MongoDB.Driver/Core/Misc/Feature.cs
Contributor
There was a problem hiding this comment.
Pull request overview
Adds LINQ3 translation support for MongoDB’s $concatArrays and $setUnion accumulators/window operators, including new window-partition extension methods, AST operator wiring, serializer inference, and integration tests for both $setWindowFields and grouping/bucket scenarios.
Changes:
- Added
ConcatArrays/SetUnionwindow-partition extension methods and reflection/translation plumbing to render$concatArraysand$setUnion. - Updated grouping pipeline optimizer to recognize
SelectMany-shaped ASTs and rewrite them into$concatArrays/$setUnionaccumulators. - Expanded test coverage for new operators in window and grouping/bucket contexts, plus serializer-finder expectations.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MethodTranslators/WindowMethodToAggregationExpressionTranslatorTests.cs | Adds window translation tests for $concatArrays and $setUnion. |
| tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MethodTranslators/SelectManyMethodToAggregationExpressionTranslatorTests.cs | Adds grouping/bucket tests expecting $concatArrays/$setUnion accumulators; adjusts fixture data and existing tests. |
| tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/SerializerFinders/WindowTests.cs | Adds serializer resolution coverage for new window methods. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MethodTranslators/WindowMethodToAggregationExpressionTranslator.cs | Registers ConcatArrays/SetUnion as unary window methods and maps them to AST operators. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MethodTranslators/ConcatArraysMethodToAggregationExpressionTranslator.cs | New method translator entry point delegating to window translation when applicable. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MethodTranslators/SetUnionMethodToAggregationExpressionTranslator.cs | New method translator entry point delegating to window translation when applicable. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MethodCallExpressionToAggregationExpressionTranslator.cs | Routes ConcatArrays/SetUnion method calls to the new translators. |
| src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs | Adds serializer deduction logic for ConcatArrays/SetUnion window methods. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Reflection/WindowMethod.cs | Adds WindowMethod.ConcatArrays and WindowMethod.SetUnion reflection bindings. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Optimizers/AstGroupingPipelineOptimizer.cs | Adds optimizer rewrites to turn SelectMany-shaped $reduce patterns into $concatArrays/$setUnion accumulators. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Expressions/AstUnaryWindowOperator.cs | Adds renderable window operator enum values for $concatArrays/$setUnion. |
| src/MongoDB.Driver/Linq/Linq3Implementation/Ast/Expressions/AstUnaryAccumulatorOperator.cs | Adds renderable accumulator operator enum values for $concatArrays/$setUnion. |
| src/MongoDB.Driver/Linq/ISetWindowFieldsPartitionExtensions.cs | Adds public LINQ window APIs: ConcatArrays and SetUnion. |
| src/MongoDB.Driver/Core/Misc/Feature.cs | Introduces feature gate ConcatArraysAndSetUnionAccumulators (WireVersion.Server81). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+71
to
+75
| var expectedResult = new[] { 1, 10, 2, 20, 3, 30 }; | ||
| foreach (var result in results) | ||
| { | ||
| result["Result"].AsBsonArray.Select(i => i.AsInt32).Should().BeEquivalentTo(expectedResult); | ||
| } |
Comment on lines
+473
to
+481
| public override AstNode VisitReduceExpression(AstReduceExpression node) | ||
| { | ||
| // { $reduce : { input : { $map : { input : "$_elements", as : "x", in : f(x) } }, initialValue : [], in : { $concatArrays : ["$$value", "$$this"] } } } | ||
| // => { __agg0 : { $concatArrays : f(x => element) } } + "$__agg0" | ||
| if (IsSelectManyShapeOverElements(node, out var rewrittenArg)) | ||
| { | ||
| var accumulatorExpression = AstExpression.UnaryAccumulator(AstUnaryAccumulatorOperator.ConcatArrays, rewrittenArg); | ||
| return CreateGetAccumulatorFieldExpression(accumulatorExpression); | ||
| } |
Comment on lines
+499
to
+507
| // { $setUnion : { $reduce : { input : { $map : { input : "$_elements", as : "x", in : f(x) } }, initialValue : [], in : { $concatArrays : ["$$value", "$$this"] } } } } | ||
| // => { __agg0 : { $setUnion : f(x => element) } } + "$__agg0" | ||
| if (node.Operator == AstUnaryOperator.SetUnion && | ||
| node.Arg is AstReduceExpression setUnionReduceArg && | ||
| IsSelectManyShapeOverElements(setUnionReduceArg, out var setUnionArg)) | ||
| { | ||
| var accumulatorExpression = AstExpression.UnaryAccumulator(AstUnaryAccumulatorOperator.SetUnion, setUnionArg); | ||
| return CreateGetAccumulatorFieldExpression(accumulatorExpression); | ||
| } |
The grouping/bucket SelectMany rewrite into $concatArrays/$setUnion accumulators requires server 8.1. Since that query shape was already translatable (as $push + $reduce, which runs on any server), emit the accumulator form only when the compatibility level supports it; otherwise keep the $reduce fallback. The $setWindowFields window methods are new API and stay unconditional.
- Add missing closing brace in SelectMany_Distinct_in_BucketAuto else-branch expected stage - Assert exact UniqueTags contents instead of only HaveCount - Reorder ConcatArrays/SetUnion alphabetically in WindowMethod and unary overload set
Comment on lines
+567
to
+579
| private bool IsSelectManyShapeOverElements(AstReduceExpression node, out AstExpression rewrittenArg) | ||
| { | ||
| if (IsMappedElementsField(node.Input, out var mappedArg) && | ||
| IsEmptyArrayConstant(node.InitialValue) && | ||
| IsConcatArraysOfValueAndThis(node.In)) | ||
| { | ||
| rewrittenArg = mappedArg; | ||
| return true; | ||
| } | ||
|
|
||
| rewrittenArg = null; | ||
| return false; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.