CSHARP-5572: Implement new KnownSerializerFinder #1700

rstam · 2025-05-27T18:33:09Z

No description provided.

Szmiglo · 2025-07-23T09:02:43Z

hi @rstam, what's the status of this effort?

rstam · 2025-07-23T15:47:16Z

hi @rstam, what's the status of this effort?

It is progressing well, but it is a large effort. I work on this whenever I'm not working on some smaller higher priority task.

adelinowona · 2025-10-16T06:42:22Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerMap.cs

+            }
+        }
+
+        if (_map.TryGetValue(node, out var existingSerializer))


I think we should move the duplicate check to the beginning of the AddSerializer method to implement fail-fast behavior. This would avoid unnecessary serializer transformation work when a serializer already exists for the node.

However, I notice the current implementation places this check at the end, likely to preserve information about the conflicting serializer for the exception. While this debugging information could be helpful, I question just how useful it is. Knowing where a duplicate attempt happened seems more useful knowing what the conflicting serializer would have been.

I lean toward the fail-fast approach since the conflicting serializer details seem like a "nice-to-have" rather than essential debugging information.

If we move the error check further up then we have to remove information about the redundant serializer from the error message.

Also, I don't think we need to worry about failing fast because we actually never expect to hit this error. This check is here to help us find bugs. In the absence of bugs this exception will never be thrown.

adelinowona · 2025-10-16T07:00:35Z

...ngoDB.Driver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitor.cs

+    private bool _isMakingProgress = true;
+    private readonly KnownSerializerMap _knownSerializers;
+    private int _oldKnownSerializersCount = 0;
+    private int _pass = 0;


Seems like this is never used?

adelinowona · 2025-10-16T07:11:12Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitor.cs

+        var newKnownSerializersCount = _knownSerializers.Count;
+        if (newKnownSerializersCount == _oldKnownSerializersCount)
+        {
+            if (_useDefaultSerializerForConstants)


Could we maybe make use of a state enum as such:

private enum DiscoveryPhase { InferringFromContext, // Not using defaults UsingDefaultSerializers, // Last attempt with defaults Complete // No more progress possible }

to replace the use of all these booleans used in keeping track of the state of the visitor?

It's somewhat reasonable, but I think the booleans work better here.

There are only 2 booleans.

One issue is that if we ever had more states the order of the enum values could get very confusing and maybe even no order is possible.

sanych-sun

Partial review.

src/MongoDB.Driver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinder.cs

src/MongoDB.Driver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerMap.cs

...iver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitMethodCall.cs

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderHelperMethods.cs

...ver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitConditional.cs

...DB.Driver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitIndex.cs

sanych-sun · 2025-11-25T22:18:25Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMember.cs

+
+        bool IsTupleOrValueTuple(Type type)
+        {
+            return


For net6 and higher there is a better way to check: both Tuple and ValueTuple implements ITuple interface. I suppose we can have #if def here and use interface approach for modern targets, when have something like the proposed code for net472.

Nice to know!

Is there any danger of false positives if unexpected classes also happen to implement ITuple?

Not doing just yet until we have a discussion about false positives.

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitUnary.cs

Copilot

Pull request overview

This PR implements a new KnownSerializerFinder system to replace the previous approach of dynamically determining serializers during expression translation. The new system pre-analyzes expressions to build a map of known serializers for each expression node, enabling more reliable and consistent serializer resolution throughout the translation process.

Key Changes:

Introduced a comprehensive KnownSerializerFinder infrastructure that pre-computes serializers for expression nodes
Updated TranslationContext to accept and use the pre-computed serializer map
Refactored multiple translators to leverage the new known serializer system
Added several new serializer types to support the enhanced type inference

Reviewed changes

Copilot reviewed 98 out of 99 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`KnownSerializerFinder*.cs`	New visitor-based system for discovering and mapping serializers to expression nodes
`TranslationContext.cs`	Extended to store and provide access to known serializers map
`LinqProviderAdapter.cs`	Updated all translation entry points to use the new KnownSerializerFinder
`*Serializer.cs`	Added new serializer types (Polymorphic, Upcasting, Unknowable, etc.) to support enhanced type inference
Test files	Updated test contexts to use new TranslationContext.Create signatures

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-25T22:23:36Z

...Driver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitListInit.cs

+        DeduceSerialiers();
+        base.VisitListInit(node);
+        DeduceSerialiers();
+
+        return node;
+
+        void DeduceSerialiers()


Corrected spelling of 'DeduceSerialiers' to 'DeduceSerializers'.

Suggested change

DeduceSerialiers();

base.VisitListInit(node);

DeduceSerialiers();

return node;

void DeduceSerialiers()

DeduceSerializers();

base.VisitListInit(node);

DeduceSerializers();

return node;

void DeduceSerializers()

Copilot · 2025-11-25T22:23:36Z

...ver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitConditional.cs

+        DeduceConditionaSerializers();
+        base.VisitConditional(node);
+        DeduceConditionaSerializers();
+
+        return node;
+
+        void DeduceConditionaSerializers()


Corrected spelling of 'DeduceConditionaSerializers' to 'DeduceConditionalSerializers'.

Suggested change

DeduceConditionaSerializers();

base.VisitConditional(node);

DeduceConditionaSerializers();

return node;

void DeduceConditionaSerializers()

DeduceConditionalSerializers();

base.VisitConditional(node);

DeduceConditionalSerializers();

return node;

void DeduceConditionalSerializers()

Copilot · 2025-11-25T22:23:37Z

...DB.Driver/Linq/Linq3Implementation/KnownSerializerFinders/KnownSerializerFinderVisitUnary.cs

+                    return sourceSerializer;
+                }
+
+                // handle conversionsn to BsonValue before any others


Corrected spelling of 'conversionsn' to 'conversions'.

Suggested change

// handle conversionsn to BsonValue before any others

// handle conversion to BsonValue before any others

Copilot · 2025-11-25T22:23:37Z

src/MongoDB.Driver/Linq/Linq3Implementation/KnownSerializerFinders/UnknownSerializerFinder.cs

+        _knownSerializers = knownSerializers;
+    }
+
+    public Expression ExpressionWithUnknownSerialier => _expressionWithUnknownSerializer;


Corrected spelling of 'ExpressionWithUnknownSerialier' to 'ExpressionWithUnknownSerializer'.

Suggested change

public Expression ExpressionWithUnknownSerialier => _expressionWithUnknownSerializer;

public Expression ExpressionWithUnknownSerializer => _expressionWithUnknownSerializer;

…equest

…pport $filter with limit : 1

rstam · 2025-12-02T18:17:54Z

tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharp5532Tests.cs

+                }
+            }
+            """;
+        if (!Feature.FilterLimit.IsSupported(CoreTestConfiguration.MaxWireVersion))


We usually create two strings depending on whether a feature is supported or not but, in this case, the MQL is so long it's easier to just remove limit : 1 if it is not supported.

…elper methods

…ySerializerBase

sanych-sun

Please also review all TODOs (remove, address or create a ticket for future if we can postpone the changes) and failing unit-test.

sanych-sun · 2025-12-04T18:29:55Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitBinary.cs

+        }
+
+        if (IsSymmetricalBinaryOperator(@operator) &&
+            CanDeduceSerializer(leftExpression, rightExpression, out var unknownNode, out var otherNodeSerializer))


Minor: I found only 2 usages of CanDeduceSerializer, both looks similar (TODO in the another usage says we have to check value types). I think we can simplify by refactoring to TryDeduceSerializer and do the type check and AddNodeSerializer inside it.

I've removed the CanDeduceSerializer method entirely becase there was another existing method that could be used.

sanych-sun · 2025-12-04T18:33:51Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitBinary.cs

+                }
+                else
+                {
+                    DeduceUnknowableSerializer(node);


It worth to add a comment why we decided stop looking for the serializer for the node.

sanych-sun · 2025-12-04T18:58:58Z

src/MongoDB.Driver/Linq/Linq3Implementation/Serializers/IgnoreNodeSerializer.cs

+    }
+}
+
+internal class IgnoreNodeSerializer<TValue> : SerializerBase<TValue>


What the difference between IgnoreNodeSerializer and UnknowableSerializer?

IgnoreSerializer is when we know what the node means but there is no serializer needed for the node. But since we throw an exception if a node has no serializer we assign this dummy serializer to such nodes.

UnknowableSerializer is when we know what the node means in general, but not in this particular case. An example would be a method call expression with an unknown method (we know about method call expressions in general but can encounter methods we don't know anything about).

UnknowableSerializer comes into play with client-side projections. If we didn't assign the UnknowableSerializer dummy serializer to method call nodes we didn't know about we would throw a missing serializer exception which would prevent us from getting to the client-side projection.

sanych-sun · 2025-12-04T19:02:34Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitListInit.cs

+    protected override Expression VisitListInit(ListInitExpression node)
+    {
+        var newExpression = node.NewExpression;
+        var initializers = node.Initializers;


initializers variable seems to be unused. Should we handle initializers somehow?

Not sure yet. Added a TODO comment for later investigation.

sanych-sun · 2025-12-04T19:03:34Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMember.cs

+            var containingExpression = node.Expression;
+            if (IsKnown(containingExpression, out containingSerializer))
+            {
+                // TODO: handle special cases like DateTime.Year etc.


Is this TODO still actual?

Yes, but I don't know the full list.

I've edited the comment to not mention DateTime.Year (since that one is already done)

sanych-sun · 2025-12-04T19:32:14Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+
+        bool IsDictionaryContainsKeyMethod(out Expression keyExpression)
+        {
+            if (method.DeclaringType.Name.Contains("Dictionary") &&


Let's move this method to DictionaryMethod helper class and avoid Contains("Dictionary") by checking for interface or base class.

sanych-sun · 2025-12-04T19:39:25Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+
+        void DeduceAggregateMethodSerializers()
+        {
+            if (method.IsOneOf(__aggregateMethods))


I cannot find any usage of results of IsItemSerializerKnown method call results, so it probably could be removed. And if it's removed then outer if is redundant.

I've removed this one line.

I don't understand why you think the outer if is redundant.

sanych-sun · 2025-12-04T19:47:29Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+            }
+        }
+
+        void DeduceAsinhMethodSerializers()


We have number of exactly the same methods in this class to process all math trigonometry methods, should we group then into a single collection __trigonometricMathMethods and process in the single method like:

void DeduceTrigonometryMethodSerializers() { if (method.Is(__trigonometricMathMethods)) { DeduceReturnsDoubleSerializer(); } else { DeduceUnknownMethodSerializer(); } }

Good idea. Done.

sanych-sun · 2025-12-04T19:54:42Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+                var isNullable = toType.IsNullable();
+                var valueType = isNullable ? Nullable.GetUnderlyingType(toType) : toType;
+
+                var valueSerializer = (IBsonSerializer)(Type.GetTypeCode(valueType) switch


This logic looks very similar to StandardSerializers.GetSerializer. Can we leverage it here?

Added a TODO for future investigation.

There are 11 types handled here that StandardSerializers doesn't know about and a few of the ones it does know about are configured differentlly here, so it's tricky.

sanych-sun · 2025-12-04T20:04:47Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitNewArray.cs

+
+                case ExpressionType.NewArrayInit:
+                    DeduceNewArrayInitSerializers();
+                    break;


Should we throw in default case?

We could, but I don't think it's necessary.

First of all, there are no other cases here AFAIK.

And if there are, it will result in a missing serializer exception a bit later on.

I don't think we normally throw exceptions when deducing things in the SerializerFinder. We either deduce something or we do nothing.

rstam

Addressed comments and requested changes.

rstam · 2025-12-04T23:14:15Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitBinary.cs

+        }
+
+        if (IsSymmetricalBinaryOperator(@operator) &&
+            CanDeduceSerializer(leftExpression, rightExpression, out var unknownNode, out var otherNodeSerializer))


I've removed the CanDeduceSerializer method entirely becase there was another existing method that could be used.

rstam · 2025-12-04T23:28:40Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMember.cs

+                {
+                    _ when declaringType == typeof(BsonValue) => GetBsonValuePropertySerializer(),
+                    _ when IsCollectionCountOrLengthProperty() => GetCollectionCountOrLengthPropertySerializer(),
+                    _ when declaringType == typeof(DateTime) => GetDateTimePropertySerializer(),


Probably not until we add support for DateTimeOffset in general.

No tests are failing because we didn't do this.

rstam · 2025-12-04T23:29:21Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMember.cs

+            var containingExpression = node.Expression;
+            if (IsKnown(containingExpression, out containingSerializer))
+            {
+                // TODO: handle special cases like DateTime.Year etc.


Yes, but I don't know the full list.

I've edited the comment to not mention DateTime.Year (since that one is already done)

rstam · 2025-12-04T23:30:43Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMember.cs

+
+            return memberName switch
+            {
+                "Keys" => ICollectionSerializer.Create(keySerializer),


Yes this is surprising.

It's because Dictionary<TKey, TValue>.Keys and IDictionary<TKey, TValue>.Keys have different types. Same for Values.

An unexpected glitch in Microsoft's design of these types. (Constrained by having to maintain backward compatibility).

rstam · 2025-12-04T23:35:17Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+
+internal partial class SerializerFinderVisitor
+{
+    private static readonly HashSet<MethodInfo> __absMethods =


Moved to MathMethod.cs from where it can be shared.

rstam · 2025-12-10T16:24:47Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+            }
+        }
+
+        void DeduceAsinhMethodSerializers()


Good idea. Done.

rstam · 2025-12-10T19:58:57Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+                var isNullable = toType.IsNullable();
+                var valueType = isNullable ? Nullable.GetUnderlyingType(toType) : toType;
+
+                var valueSerializer = (IBsonSerializer)(Type.GetTypeCode(valueType) switch


Added a TODO for future investigation.

There are 11 types handled here that StandardSerializers doesn't know about and a few of the ones it does know about are configured differentlly here, so it's tricky.

rstam · 2025-12-10T20:08:17Z

...MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs

+
+        bool IsDictionaryContainsKeyMethod(out Expression keyExpression)
+        {
+            if (method.DeclaringType.Name.Contains("Dictionary") &&


rstam · 2025-12-10T20:09:54Z

src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitNewArray.cs

+
+                case ExpressionType.NewArrayInit:
+                    DeduceNewArrayInitSerializers();
+                    break;


We could, but I don't think it's necessary.

First of all, there are no other cases here AFAIK.

And if there are, it will result in a missing serializer exception a bit later on.

I don't think we normally throw exceptions when deducing things in the SerializerFinder. We either deduce something or we do nothing.

rstam · 2025-12-10T20:14:52Z

src/MongoDB.Driver/Linq/Linq3Implementation/Serializers/IgnoreNodeSerializer.cs

+    }
+}
+
+internal class IgnoreNodeSerializer<TValue> : SerializerBase<TValue>


IgnoreSerializer is when we know what the node means but there is no serializer needed for the node. But since we throw an exception if a node has no serializer we assign this dummy serializer to such nodes.

UnknowableSerializer is when we know what the node means in general, but not in this particular case. An example would be a method call expression with an unknown method (we know about method call expressions in general but can encounter methods we don't know anything about).

UnknowableSerializer comes into play with client-side projections. If we didn't assign the UnknowableSerializer dummy serializer to method call nodes we didn't know about we would throw a missing serializer exception which would prevent us from getting to the client-side projection.

rstam requested review from adelinowona and sanych-sun May 27, 2025 18:33

rstam added bug Fixes issues or unintended behavior. improvement Optimizations or refactoring (no new features or fixes). labels May 27, 2025

rstam changed the title ~~CSHARP-5572: Implement new KnownSerializerFinder.~~ CSHARP-5572: Implement new KnownSerializerFinder May 27, 2025

rstam force-pushed the csharp5572 branch from 89c78f2 to bd9bd3e Compare August 21, 2025 15:24

rstam requested review from a team and removed request for a team August 21, 2025 15:25

rstam force-pushed the csharp5572 branch from bd9bd3e to 8228c21 Compare September 30, 2025 01:13

rstam force-pushed the csharp5572 branch 2 times, most recently from 86fbb24 to 56a464c Compare October 13, 2025 15:15

adelinowona reviewed Oct 16, 2025

View reviewed changes

rstam force-pushed the csharp5572 branch 2 times, most recently from 32aae48 to 7f2a64c Compare October 31, 2025 15:58

sanych-sun marked this pull request as ready for review November 25, 2025 22:22

sanych-sun requested a review from a team as a code owner November 25, 2025 22:22

Copilot AI review requested due to automatic review settings November 25, 2025 22:22

sanych-sun requested changes Nov 25, 2025

View reviewed changes

Copilot AI reviewed Nov 25, 2025

View reviewed changes

rstam added 4 commits December 1, 2025 22:42

CSHARP-5572: Implement new KnownSerializerFinder

adf84bd

CSHARP-5572: Work in progress

92d5316

CSHARP-5572: Requested changes

70fde9b

CSHARP-5572: Rename KnownSerializer to NodeSerializer at reviewer's r…

a0595b9

…equest

rstam force-pushed the csharp5572 branch from 7f2a64c to a0595b9 Compare December 2, 2025 06:42

CSHARP-5572: Fix test that was failing on older servers that don't su…

8298824

…pport $filter with limit : 1

rstam commented Dec 2, 2025

View reviewed changes

rstam added 3 commits December 2, 2025 13:51

CSHARP-5572: Standardize on using existing IsTuple and IsValueTuple h…

75f8411

…elper methods

CSHARP-5572: Refactor to use a single IsSetEqualsMethod implementation

684be5c

CSHARP-5572: Remove unnecessary commented out code

cc0e784

rstam requested review from adelinowona and sanych-sun December 2, 2025 23:12

rstam added 2 commits December 3, 2025 13:05

CSHARP-5572: Revert removal of IBsonDocumentSerializer from Dictionar…

e54c7b7

…ySerializerBase

CSHARP-5572: Introduce IReadOnlySerializerMap interface

4989112

sanych-sun requested changes Dec 4, 2025

View reviewed changes

CSHARP-5572: Requested changes

5cf8dbc

rstam commented Dec 13, 2025

View reviewed changes

rstam requested a review from sanych-sun December 13, 2025 15:38

	// handle conversionsn to BsonValue before any others
	// handle conversion to BsonValue before any others

	public Expression ExpressionWithUnknownSerialier => _expressionWithUnknownSerializer;
	public Expression ExpressionWithUnknownSerializer => _expressionWithUnknownSerializer;

CSHARP-5572: Implement new KnownSerializerFinder #1700

Are you sure you want to change the base?

CSHARP-5572: Implement new KnownSerializerFinder #1700

Uh oh!

Conversation

rstam commented May 27, 2025

Uh oh!

Szmiglo commented Jul 23, 2025

Uh oh!

rstam commented Jul 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanych-sun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanych-sun left a comment

Choose a reason for hiding this comment

Uh oh!

sanych-sun Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

sanych-sun Dec 4, 2025 •

edited

Loading

sanych-sun Dec 4, 2025 •

edited

Loading

sanych-sun Dec 4, 2025 •

edited

Loading

sanych-sun Dec 4, 2025 •

edited

Loading