Skip to content

Python: Fix Postgres filter SQL composition#14018

Open
pragnyanramtha wants to merge 2 commits into
microsoft:mainfrom
pragnyanramtha:pragnyan/python-postgres-filter-sql-13595
Open

Python: Fix Postgres filter SQL composition#14018
pragnyanramtha wants to merge 2 commits into
microsoft:mainfrom
pragnyanramtha:pragnyan/python-postgres-filter-sql-13595

Conversation

@pragnyanramtha
Copy link
Copy Markdown

Motivation and Context

Fixes #13595.

Postgres vector-search lambda filters were parsed into plain strings and then passed through psycopg.sql.SQL.format(). psycopg treats non-composable values as SQL literals, so filter predicates could be embedded as quoted strings instead of SQL expressions. The filtered query also missed a leading space before WHERE.

Description

  • Build Postgres filter clauses with psycopg.sql composables instead of plain strings
  • Use sql.Identifier for field names and sql.Literal for constants
  • Preserve existing boolean/comparison/list filter behavior while composing AND, OR, NOT, IN, and chained comparisons as SQL fragments
  • Add regression tests for single and multiple filtered vector-search queries

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the SK Contribution Guidelines
  • The code follows the Python coding conventions
  • All unit tests pass, and I have added new tests where possible
  • I didn't break anyone 😄

Validation

From python/:

  • .venv/bin/python -m pytest tests/unit/connectors/memory/test_postgres_store.py -q -> 22 passed
  • .venv/bin/ruff check semantic_kernel/connectors/postgres.py tests/unit/connectors/memory/test_postgres_store.py -> passed
  • .venv/bin/ruff format --check semantic_kernel/connectors/postgres.py tests/unit/connectors/memory/test_postgres_store.py -> passed
  • git diff --check -> passed

@moonbox3 moonbox3 added the python Pull requests for the Python Semantic Kernel label May 16, 2026
@pragnyanramtha pragnyanramtha marked this pull request as ready for review May 16, 2026 21:03
@pragnyanramtha pragnyanramtha requested a review from a team as a code owner May 16, 2026 21:03
Copilot AI review requested due to automatic review settings May 16, 2026 21:03
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 91%

✓ Correctness

The PR correctly migrates Postgres filter SQL construction from plain f-strings to psycopg's sql.Composable types (sql.SQL, sql.Identifier, sql.Literal), fixing both the missing space before WHERE and the risk of filter predicates being embedded as quoted string literals instead of SQL expressions. The logic is preserved faithfully across all comparison, boolean, unary, attribute, constant, and list/tuple AST node cases. The _build_filter return contract (single composable vs list of composables) is properly handled by the existing isinstance(where_clauses, list) check. No correctness issues found.

✓ Security Reliability

This PR correctly migrates Postgres filter construction from manual string interpolation to psycopg's sql.Composable system (sql.Identifier for field names, sql.Literal for constants, sql.SQL for operators). The old code's manual escaping (e.g., replace("'", "'")) was fragile and led to the bug described in the PR rationale where filter predicates were treated as string literals rather than SQL expressions. The fix also addresses a missing space before WHERE. No new security or reliability issues are introduced.

✓ Test Coverage

The PR adds two well-structured regression tests that verify the core bug fix (filter predicates embedded as proper SQL composables with correct WHERE spacing). However, several changed code paths remain untested: the newly added ast.Tuple() handler, None/boolean literal rendering via sql.Literal (behavioral change from explicit string formatting), OR boolean operations, and chain comparisons. These represent moderate coverage gaps for a security-sensitive change (SQL composition).

✗ Design Approach

The composable SQL rewrite fixes the quoted-predicate bug, but it still leaves one supported filter shape semantically incorrect: null equality/inequality. Because ast.Constant() now accepts None as a literal while ast.Eq and ast.NotEq still always emit = and <>, a filter like lambda record: record.content_type == None compiles to "content_type" = NULL, which silently filters out matching rows in Postgres.

Flagged Issues

  • python/semantic_kernel/connectors/postgres.py:874-876 and 916 still compose null comparisons as = NULL / <> NULL, so supported filters such as lambda record: record.content_type == None or != None will never behave correctly in Postgres. Null equality should be special-cased to emit IS NULL / IS NOT NULL.

Automated review by pragnyanramtha's agents

Comment thread python/semantic_kernel/connectors/postgres.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python postgres connector

3 participants