Fix #37605: Handle null values in IN expressions when using OPENJSON …

[BurakBebek1]

…on SQL Server

• I've read the guidelines for contributing and seen the walkthrough
• I've posted a comment on an issue with a detailed description of how I am planning to contribute and got approval from a member of the team
• The code builds and tests pass locally (also verified by our automated build checks)
• Commit messages follow this format:

        Summary of the changes
        - Detail 1
        - Detail 2

        Fixes #bugnumber

• Tests for the changes have been added (for bug fixes / features)
• Code follows the same patterns and style as existing code in this repo

Summary of the changes

• Added logic to handle null values when translating collection parameters using OPENJSON on SQL Server.
• When a collection contains a null element, the generated SQL now includes an OR [column] IS NULL clause to ensure NULL rows are correctly matched, consistent with LINQ Contains behavior.
• This fix specifically targets scenarios where the 2100 parameter limit is exceeded and the query switches to JSON-based parameterization.

Fixes #37605

[BurakBebek1]

@dotnet-policy-service agree

[BurakBebek1]

Here is the generated SQL for the fix. As shown below, it correctly handles null values by adding the OR ... IS NULL clause when using OPENJSON.

[roji]

@BurakBebek1 above you referenced issue #37494, which seems unrelated. If there's a bug here and there isn't yet an issue tracking it, can you please open a new issue with a clear repro (currently behavior/SQL and how it's wrong)?

[BurakBebek1]

@roji Hi,
I have created a dedicated issue for this bug: #37605. It includes a minimal reproduction test case and the full stack traces for both the InvalidCastException (at execution) and UnreachableException (during translation). This PR addresses the missing nullability handling when OPENJSON is used for large collections.
Thank you.

[roji]

@BurakBebek1 thanks, I finally got around to looking at this.

Your proposed fix has some issues:

• The UnwrapNullableType() call for the ColumnExpression's Type is indeed correct.
• The constants case isn't handled (for old SQL Server versions which have no OPENJSON)
• Note that after adding the item IS NULL clause, the JSON array still contains a null value.

More generally, though, the null handling logic that your PR reimplements already exists in SqlNullabilityProcessor; it's not good to reimplement/duplicate that specifically for the case of SQL Server with lots of parameters. That incidentally had a separate, completely unrelated bug (the InvalidCastOperation) that I looked into as well, and which wasn't directly related to the SQL Server parameters-over-limit problem (I reproduced it in a separate test). After fixing that as well as the UnwrapNullableType() issue, the original parameters-over-limit code works as-is, and properly calls into base.VisitIn() as it should. I've also fixed your testing in various ways.

I've opened #37674 with the fixes - please take a look; thanks for submitting your PR - I'll close it in favor of #37674.