Skip to content

feat: add async PostgreSQL fixture support and configurable fixture scope#1295

Open
tboy1337 wants to merge 10 commits intodbfixtures:mainfrom
tboy1337:async
Open

feat: add async PostgreSQL fixture support and configurable fixture scope#1295
tboy1337 wants to merge 10 commits intodbfixtures:mainfrom
tboy1337:async

Conversation

@tboy1337
Copy link

@tboy1337 tboy1337 commented Mar 13, 2026

  • Adds postgresql_async fixture factory and AsyncDatabaseJanitor class for testing async code against PostgreSQL using psycopg.AsyncConnection.
  • Adds an optional async extra (pip install pytest-postgresql[async]) that pulls in pytest-asyncio and aiofiles as dependencies.
  • Adds a configurable scope parameter to the postgresql, postgresql_async, postgresql_proc, and postgresql_noproc factories, allowing consumers to override the fixture scope. Defaults are preserved ("function" for client fixtures, "session" for process fixtures).

Changes

  • pytest_postgresql/factories/client.py: new postgresql_async factory using pytest_asyncio.fixture; scope parameter added to postgresql
  • pytest_postgresql/factories/process.py / noprocess.py: scope parameter added to postgresql_proc and postgresql_noproc
  • pytest_postgresql/janitor.py: new AsyncDatabaseJanitor with init, drop, load, cursor, and async context manager support
  • pytest_postgresql/loader.py: new build_loader_async and sql_async for async SQL file loading via aiofiles
  • pytest_postgresql/retry.py: new retry_async coroutine mirroring the existing retry helper
  • pytest_postgresql/types.py: new FixtureScopeT literal type shared across factories
  • pyproject.toml: new [async] optional dependency group
  • tests/: comprehensive unit and integration tests covering all new async paths, including mock-based janitor tests and docker-based fixture tests

Summary by CodeRabbit

  • New Features

    • Async PostgreSQL fixtures and an AsyncDatabaseJanitor with configurable fixture scopes ("session", "package", "module", "class", "function").
    • Optional "async" extra dependency group (pytest-asyncio, aiofiles).
    • Async SQL loader (sql_async, build_loader_async) and async retry utility (retry_async).
  • Tests

    • Extensive async test coverage for fixtures, loader, janitor and retry.
  • Chores

    • Public API extended to expose async variants alongside existing sync APIs.

tboy1337 and others added 6 commits March 13, 2026 20:17
- Introduced AsyncDatabaseJanitor for managing database state asynchronously.
- Added async loading capabilities with build_loader_async and sql_async functions.
- Updated factories to include async versions of PostgreSQL fixtures.
- Enhanced tests to cover async functionality for janitor and loader.
- Updated pyproject.toml to include optional dependencies for async support.
- Added tests for custom database name handling in AsyncDatabaseJanitor.
- Implemented tests for database initialization and dropping with various configurations.
- Included tests for callable resolution in build_loader and build_loader_async with dot-separated paths.
- Improved error handling tests for sql_async when aiofiles is not installed.
…c functions

- Updated the loader and client to check for aiofiles and pytest_asyncio imports at runtime, raising ImportError with clear messages if they are not installed.
- Refactored tests to mock the absence of these dependencies more effectively, ensuring proper error handling is validated.
… default postgresql_async fixture"

This reverts commit 5c92310.
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 13, 2026

📝 Walkthrough

Walkthrough

Adds first-class async PostgreSQL support: new postgresql_async fixture factory, AsyncDatabaseJanitor, async SQL loader and retry_async; factories gain a configurable scope; optional [async] extras (pytest-asyncio, aiofiles); plus corresponding async tests and documentation fragment.

Changes

Cohort / File(s) Summary
Async janitor & types
pytest_postgresql/janitor.py, pytest_postgresql/types.py
Adds AsyncDatabaseJanitor and AsyncDatabaseJanitorType; introduces FixtureScopeT Literal for fixture scope values.
Client factories (async + scope export)
pytest_postgresql/factories/client.py, pytest_postgresql/factories/__init__.py
Adds postgresql_async(...) async fixture factory; updates postgresql(...) to accept scope; exports postgresql_async in __all__.
Process / no-process factories (scope)
pytest_postgresql/factories/process.py, pytest_postgresql/factories/noprocess.py
Adds FixtureScopeT import and scope parameter; fixture decorators now use @pytest.fixture(scope=scope) instead of a fixed "session".
Async loader & retry
pytest_postgresql/loader.py, pytest_postgresql/retry.py
Adds build_loader_async and sql_async (uses optional aiofiles) and retry_async coroutine; loader uses importlib for module resolution and guards optional imports.
Packaging & newsfragment
pyproject.toml, newsfragments/1235.feature.rst
Adds [project.optional-dependencies] extras group async with pytest-asyncio >= 0.21 and aiofiles >= 23.0; documents feature in newsfragment.
Tests (async coverage)
tests/conftest.py, tests/test_postgresql.py, tests/test_template_database.py, tests/docker/test_noproc_docker.py, tests/test_janitor.py, tests/test_loader.py, tests/test_retry.py, tests/test_factory_errors.py
Adds async fixtures and many async test variants exercising postgresql_async, AsyncDatabaseJanitor, async loader and retry_async; includes tests for missing optional dependency errors.

Sequence Diagram

sequenceDiagram
    participant Test as Test Function
    participant Factory as postgresql_async Factory
    participant Janitor as AsyncDatabaseJanitor
    participant AsyncConn as AsyncConnection
    participant DB as PostgreSQL

    Test->>Factory: request async fixture
    Factory->>Janitor: create & initialise test DB
    Janitor->>DB: CREATE / prepare DB
    Factory-->>Test: yield AsyncConnection (via AsyncConnection.connect)
    Test->>AsyncConn: execute async queries
    AsyncConn->>DB: run SQL, return results
    Test->>Factory: end of test (fixture finaliser)
    Factory->>Janitor: trigger async teardown
    Janitor->>DB: DROP test DB
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Suggested labels

Feature

Poem

🐰 I hop through awaits, tidy schemas with care,

Async janitors hum and breeze through database air,
Fixtures stretch their scopes, yielding connections anew,
aiofiles reads the scripts while retries see tests through,
A happy rabbit cheers — async tests now hop too!

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Newsfragment Check ⚠️ Warning Newsfragment file exists with valid content and correct format, but the filename (1235.feature.rst) does not match the PR number (1295). Rename newsfragments/1235.feature.rst to newsfragments/1295.feature.rst to align with the actual PR number and ensure proper changelog generation.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarises the main changes: adding async PostgreSQL fixture support and configurable fixture scopes, which are the primary features introduced across the entire changeset.
Docstring Coverage ✅ Passed Docstring coverage is 84.52% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tboy1337
Copy link
Author

pre-commit broke coderabbit @fizyk

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pytest_postgresql/factories/process.py (1)

85-110: ⚠️ Potential issue | 🟠 Major

Configurable process scopes currently leak the port lock file.

Line 110 now allows repeated setup/teardown cycles, but the postgresql-<port>.port sentinel created on Line 140 is never removed if the fixture later tears down or fails. With scope="function" and a fixed port or narrow range, the second invocation can fail with PortForException even though the previous server has already stopped.

Possible fix
-        postgresql_executor = PostgreSQLExecutor(
-            executable=postgresql_ctl,
-            host=host or config.host,
-            port=pg_port,
-            user=user or config.user,
-            password=password or config.password,
-            dbname=pg_dbname,
-            options=options or config.options,
-            datadir=str(datadir),
-            unixsocketdir=unixsocketdir or config.unixsocketdir,
-            logfile=str(logfile_path),
-            startparams=startparams or config.startparams,
-            postgres_options=postgres_options or config.postgres_options,
-        )
-        # start server
-        with postgresql_executor:
-            postgresql_executor.wait_for_postgres()
-            janitor = DatabaseJanitor(
-                user=postgresql_executor.user,
-                host=postgresql_executor.host,
-                port=postgresql_executor.port,
-                dbname=postgresql_executor.template_dbname,
-                as_template=True,
-                version=postgresql_executor.version,
-                password=postgresql_executor.password,
-            )
-            if config.drop_test_database:
-                janitor.drop()
-            with janitor:
-                for load_element in pg_load:
-                    janitor.load(load_element)
-                yield postgresql_executor
+        try:
+            postgresql_executor = PostgreSQLExecutor(
+                executable=postgresql_ctl,
+                host=host or config.host,
+                port=pg_port,
+                user=user or config.user,
+                password=password or config.password,
+                dbname=pg_dbname,
+                options=options or config.options,
+                datadir=str(datadir),
+                unixsocketdir=unixsocketdir or config.unixsocketdir,
+                logfile=str(logfile_path),
+                startparams=startparams or config.startparams,
+                postgres_options=postgres_options or config.postgres_options,
+            )
+            # start server
+            with postgresql_executor:
+                postgresql_executor.wait_for_postgres()
+                janitor = DatabaseJanitor(
+                    user=postgresql_executor.user,
+                    host=postgresql_executor.host,
+                    port=postgresql_executor.port,
+                    dbname=postgresql_executor.template_dbname,
+                    as_template=True,
+                    version=postgresql_executor.version,
+                    password=postgresql_executor.password,
+                )
+                if config.drop_test_database:
+                    janitor.drop()
+                with janitor:
+                    for load_element in pg_load:
+                        janitor.load(load_element)
+                    yield postgresql_executor
+        finally:
+            port_filename_path.unlink(missing_ok=True)

Also applies to: 133-141, 155-186

🧹 Nitpick comments (1)
tests/test_retry.py (1)

18-32: Avoid the real backoff in the failure-path tests.

These two tests add roughly three seconds of wall-clock delay and can become timing-sensitive on busy runners. Patching pytest_postgresql.retry.asyncio.sleep (and the clock helper for the timeout case) would keep the async retry coverage fast and deterministic.

Also applies to: 35-43

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_retry.py` around lines 18 - 32, The tests
test_retry_async_succeeds_after_failures (and the related timeout test) are
incurring real backoff delays; patch the sleep and clock used by retry_async to
make them deterministic and fast: in the tests monkeypatch
pytest_postgresql.retry.asyncio.sleep to an async no-op (or a function that
records calls) and also patch the retry module's clock/timer helper used for
timeout handling so the timeout case can advance without waiting; update the
tests to use these monkeypatched replacements while calling retry_async to
eliminate wall-clock backoff and keep behavior identical.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pytest_postgresql/factories/client.py`:
- Around line 102-119: The async fixture decorator call for
postgresql_async_factory must set loop_scope to match the fixture cache scope to
avoid event-loop lifetime misconfiguration; update the `@pytest_asyncio.fixture`
invocation (the decorator on postgresql_async_factory) to pass loop_scope=scope
so the event loop scope is at least as broad as the fixture scope (keeping the
existing scope=scope argument).

In `@tests/test_postgresql.py`:
- Around line 109-127: The async test test_postgres_terminate_connection_async
is missing the pytest-xdist isolation marker; add the same
`@pytest.mark.xdist_group`(name="terminate_connection") decorator to the async
test (alongside the existing `@pytest.mark.asyncio` and `@pytest.mark.parametrize`)
so it gets run in the same xdist group as the synchronous counterpart and avoids
cross-test interference.

---

Nitpick comments:
In `@tests/test_retry.py`:
- Around line 18-32: The tests test_retry_async_succeeds_after_failures (and the
related timeout test) are incurring real backoff delays; patch the sleep and
clock used by retry_async to make them deterministic and fast: in the tests
monkeypatch pytest_postgresql.retry.asyncio.sleep to an async no-op (or a
function that records calls) and also patch the retry module's clock/timer
helper used for timeout handling so the timeout case can advance without
waiting; update the tests to use these monkeypatched replacements while calling
retry_async to eliminate wall-clock backoff and keep behavior identical.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 821e8573-8a73-4dae-a6fc-dde60894e840

📥 Commits

Reviewing files that changed from the base of the PR and between 524e05a and 990496a.

📒 Files selected for processing (18)
  • newsfragments/1235.feature.rst
  • pyproject.toml
  • pytest_postgresql/factories/__init__.py
  • pytest_postgresql/factories/client.py
  • pytest_postgresql/factories/noprocess.py
  • pytest_postgresql/factories/process.py
  • pytest_postgresql/janitor.py
  • pytest_postgresql/loader.py
  • pytest_postgresql/retry.py
  • pytest_postgresql/types.py
  • tests/conftest.py
  • tests/docker/test_noproc_docker.py
  • tests/test_factory_errors.py
  • tests/test_janitor.py
  • tests/test_loader.py
  • tests/test_postgresql.py
  • tests/test_retry.py
  • tests/test_template_database.py

- Updated the postgresql_async fixture to include loop_scope for better async handling.
- Added xdist_group marker to the test_postgres_terminate_connection_async for improved test organization.
- Refactored retry tests to mock sleep and current time, ensuring accurate timeout handling and retry logic validation.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/test_retry.py (1)

49-64: Optionally harden timeout assertions.

You may want to assert no sleep occurred and only one failing call happened in this scenario, to lock in current timeout behaviour more explicitly.

Optional tightening
-    async def always_fail() -> None:
-        raise ValueError("boom")
+    always_fail = AsyncMock(side_effect=ValueError("boom"))
@@
         with pytest.raises(TimeoutError, match="Failed after"):
             await retry_async(always_fail, timeout=1, possible_exception=ValueError)
+    sleep_mock.assert_not_awaited()
+    assert always_fail.await_count == 1
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_retry.py` around lines 49 - 64, Add explicit assertions that no
sleep was awaited and that only the expected number of time-checks/calls
happened: after the with-block raising TimeoutError, call
sleep_mock.assert_not_awaited() to ensure no backoff was used and assert
call_count == 2 (verifying get_current_datetime was called exactly once to
capture the start and once to detect the timeout) to lock the current timeout
behavior for retry_async when always_fail raises immediately.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/test_retry.py`:
- Around line 49-64: Add explicit assertions that no sleep was awaited and that
only the expected number of time-checks/calls happened: after the with-block
raising TimeoutError, call sleep_mock.assert_not_awaited() to ensure no backoff
was used and assert call_count == 2 (verifying get_current_datetime was called
exactly once to capture the start and once to detect the timeout) to lock the
current timeout behavior for retry_async when always_fail raises immediately.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5bf34b69-b624-4e9d-8cb3-1fdaeffb1f94

📥 Commits

Reviewing files that changed from the base of the PR and between 990496a and 1384100.

📒 Files selected for processing (3)
  • pytest_postgresql/factories/client.py
  • tests/test_postgresql.py
  • tests/test_retry.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/test_postgresql.py

- Changed isolation level and autocommit settings to direct attribute assignment instead of using methods.
- Removed unnecessary mock methods in test for improved clarity and performance.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/test_janitor.py (1)

124-143: Consider adding test coverage for async loaders.

This test validates that AsyncDatabaseJanitor.load() works with synchronous loaders via the inspect.isawaitable check. Consider adding a companion test that exercises the async loader path (e.g., using a Path argument to trigger sql_async), ensuring the await result branch is also covered.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/test_janitor.py` around lines 124 - 143, Add a companion async test
that exercises the AsyncDatabaseJanitor.load() path where the loader is
asynchronous (so the inspect.isawaitable branch and the await result path are
executed): create an async loader (or point to an async loader via a Path to
trigger sql_async), patch tests.loader.psycopg.connect like in
test_janitor_populate_async, instantiate AsyncDatabaseJanitor(version=10,
**call_kwargs), await janitor.load(async_loader) and assert connect_mock.called
and connect_mock.call_args.kwargs == call_kwargs; ensure the test uses
`@pytest.mark.asyncio` and param or fixture to provide the async loader so
sql_async/inspect.isawaitable logic in AsyncDatabaseJanitor.load is covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pytest_postgresql/janitor.py`:
- Around line 252-259: The SQL string concatenation in the async helper
_terminate_connection (async def _terminate_connection(cur: AsyncCursor, dbname:
str)) currently joins two literals without a space, producing "...pid)FROM..."
and causing a syntax error; fix it by ensuring there's whitespace between ")”
and "FROM" (e.g., include a trailing space on the first literal or a leading
space on the second) so the executed query becomes "...pid) FROM
pg_stat_activity ...", and apply the same whitespace fix to the synchronous
DatabaseJanitor._terminate_connection query to avoid the identical runtime
error.

---

Nitpick comments:
In `@tests/test_janitor.py`:
- Around line 124-143: Add a companion async test that exercises the
AsyncDatabaseJanitor.load() path where the loader is asynchronous (so the
inspect.isawaitable branch and the await result path are executed): create an
async loader (or point to an async loader via a Path to trigger sql_async),
patch tests.loader.psycopg.connect like in test_janitor_populate_async,
instantiate AsyncDatabaseJanitor(version=10, **call_kwargs), await
janitor.load(async_loader) and assert connect_mock.called and
connect_mock.call_args.kwargs == call_kwargs; ensure the test uses
`@pytest.mark.asyncio` and param or fixture to provide the async loader so
sql_async/inspect.isawaitable logic in AsyncDatabaseJanitor.load is covered.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 951653a7-eb09-47eb-a5cc-adc4bbea59c1

📥 Commits

Reviewing files that changed from the base of the PR and between 1384100 and f16583a.

📒 Files selected for processing (2)
  • pytest_postgresql/janitor.py
  • tests/test_janitor.py

- Added a space at the end of the SQL query string in both DatabaseJanitor and AsyncDatabaseJanitor to ensure proper formatting.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pytest_postgresql/janitor.py`:
- Around line 224-231: The CREATE DATABASE DDL currently builds SQL with
f-strings using self.dbname and self.template_dbname (in the method that calls
cur.execute with query), which can break on embedded quotes; replace this string
interpolation with psycopg.sql.Identifier composition: import psycopg.sql as
sql, build the base sql.SQL("CREATE DATABASE
{}").format(sql.Identifier(self.dbname)), conditionally append sql.SQL("
TEMPLATE {}").format(sql.Identifier(self.template_dbname)) when present and
sql.SQL(" IS_TEMPLATE = true") when self.as_template is true, then pass the
composed sql.SQL object to await cur.execute(...) instead of the f-string.
Ensure all occurrences that construct database identifiers (the blocks using
self.dbname, self.template_dbname, and as_template) use sql.Identifier.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3dceab04-c112-4e1c-8a96-71c78d839bfd

📥 Commits

Reviewing files that changed from the base of the PR and between f16583a and 38a5ae4.

📒 Files selected for processing (1)
  • pytest_postgresql/janitor.py

Comment on lines +224 to +231
query = f'CREATE DATABASE "{self.dbname}" TEMPLATE "{self.template_dbname}"'
else:
query = f'CREATE DATABASE "{self.dbname}"'

if self.as_template:
query += " IS_TEMPLATE = true"

await cur.execute(f"{query};")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

find . -name "janitor.py" -type f

Repository: dbfixtures/pytest-postgresql

Length of output: 102


🏁 Script executed:

head -n 50 pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 1729


🏁 Script executed:

sed -n '200,280p' pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 3251


🏁 Script executed:

rg "CREATE DATABASE" pytest_postgresql/janitor.py -A 3 -B 3

Repository: dbfixtures/pytest-postgresql

Length of output: 1030


🏁 Script executed:

rg "from psycopg import" pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 123


🏁 Script executed:

rg "sql\." pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 372


🏁 Script executed:

sed -n '220,260p' pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 1930


🏁 Script executed:

grep -n "class AsyncDatabaseJanitor" pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 103


🏁 Script executed:

sed -n '1,20p' pytest_postgresql/janitor.py | cat -n

Repository: dbfixtures/pytest-postgresql

Length of output: 866


🏁 Script executed:

python3 << 'EOF'
import psycopg
# Check if sql module is available
print("psycopg version:", psycopg.__version__ if hasattr(psycopg, '__version__') else "unknown")
print("Has sql module:", hasattr(psycopg, 'sql'))
if hasattr(psycopg, 'sql'):
    print("Has Identifier:", hasattr(psycopg.sql, 'Identifier'))
EOF

Repository: dbfixtures/pytest-postgresql

Length of output: 191


🏁 Script executed:

sed -n '80,180p' pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 3373


🏁 Script executed:

rg "def init|def drop|def _dont_datallowconn" pytest_postgresql/janitor.py -A 1 -B 1

Repository: dbfixtures/pytest-postgresql

Length of output: 753


🏁 Script executed:

wc -l pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 104


🏁 Script executed:

sed -n '222,232p' pytest_postgresql/janitor.py | cat -n

Repository: dbfixtures/pytest-postgresql

Length of output: 591


🏁 Script executed:

sed -n '244,252p' pytest_postgresql/janitor.py | cat -n

Repository: dbfixtures/pytest-postgresql

Length of output: 558


🏁 Script executed:

grep -n "await cur.execute" pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 415


🏁 Script executed:

rg "sql.Identifier|psycopg.sql" pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 54


🏁 Script executed:

web_search
psycopg 3 sql.Identifier documentation API

Repository: dbfixtures/pytest-postgresql

Length of output: 168


🌐 Web query:

psycopg 3 sql.Identifier documentation

💡 Result:

psycopg.sql.Identifier (Psycopg 3) is documented in the Psycopg 3 API → sql – SQL string composition page, which explains how to safely compose dynamic SQL (e.g., table/column names) using sql.SQL(...).format(sql.Identifier(...)), including qualified names via multiple strings (e.g. sql.Identifier("schema", "table")). [1]

https://www.psycopg.org/psycopg3/docs/api/sql.html

Example (from the docs):

from psycopg import sql

query = sql.SQL("SELECT {field} FROM {table} WHERE {pkey} = %s").format(
    field=sql.Identifier("my_name"),
    table=sql.Identifier("some_table"),
    pkey=sql.Identifier("id"),
)

[1]


🏁 Script executed:

sed -n '90,130p' pytest_postgresql/janitor.py | cat -n

Repository: dbfixtures/pytest-postgresql

Length of output: 1878


🏁 Script executed:

sed -n '55,95p' pytest_postgresql/janitor.py | cat -n

Repository: dbfixtures/pytest-postgresql

Length of output: 2009


🏁 Script executed:

rg "cur.execute\(" pytest_postgresql/janitor.py -B 1 -A 1 | head -50

Repository: dbfixtures/pytest-postgresql

Length of output: 1242


🏁 Script executed:

sed -n '254,259p' pytest_postgresql/janitor.py

Repository: dbfixtures/pytest-postgresql

Length of output: 285


Use psycopg.sql.Identifier for safe database identifier composition in async DDL statements.

Database identifiers (dbname, template_dbname) are interpolated directly into DDL statements via f-strings at lines 224, 226, 245–246, and 250. This lacks proper escaping and breaks on embedded quotes. Use psycopg.sql.Identifier composition to safely handle identifiers consistently with Psycopg 3 best practices (the codebase already uses parameterised queries for the same values elsewhere, in _terminate_connection).

Suggested fix
-from psycopg import AsyncCursor, Connection, Cursor
+from psycopg import AsyncCursor, Connection, Cursor, sql
...
-                query = f'CREATE DATABASE "{self.dbname}" TEMPLATE "{self.template_dbname}"'
+                query = sql.SQL("CREATE DATABASE {} TEMPLATE {}").format(
+                    sql.Identifier(self.dbname),
+                    sql.Identifier(self.template_dbname),
+                )
             else:
-                query = f'CREATE DATABASE "{self.dbname}"'
+                query = sql.SQL("CREATE DATABASE {}").format(sql.Identifier(self.dbname))
 
             if self.as_template:
-                query += " IS_TEMPLATE = true"
+                query += sql.SQL(" IS_TEMPLATE = true")
 
-            await cur.execute(f"{query};")
+            await cur.execute(query + sql.SQL(";"))
...
-                await cur.execute(f'ALTER DATABASE "{self.dbname}" with is_template false;')
-            await cur.execute(f'DROP DATABASE IF EXISTS "{self.dbname}";')
+                await cur.execute(
+                    sql.SQL("ALTER DATABASE {} WITH is_template false;").format(sql.Identifier(self.dbname))
+                )
+            await cur.execute(sql.SQL("DROP DATABASE IF EXISTS {};").format(sql.Identifier(self.dbname)))
...
-        await cur.execute(f'ALTER DATABASE "{dbname}" with allow_connections false;')
+        await cur.execute(sql.SQL("ALTER DATABASE {} WITH allow_connections false;").format(sql.Identifier(dbname)))
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
query = f'CREATE DATABASE "{self.dbname}" TEMPLATE "{self.template_dbname}"'
else:
query = f'CREATE DATABASE "{self.dbname}"'
if self.as_template:
query += " IS_TEMPLATE = true"
await cur.execute(f"{query};")
query = sql.SQL("CREATE DATABASE {} TEMPLATE {}").format(
sql.Identifier(self.dbname),
sql.Identifier(self.template_dbname),
)
else:
query = sql.SQL("CREATE DATABASE {}").format(sql.Identifier(self.dbname))
if self.as_template:
query += sql.SQL(" IS_TEMPLATE = true")
await cur.execute(query + sql.SQL(";"))
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pytest_postgresql/janitor.py` around lines 224 - 231, The CREATE DATABASE DDL
currently builds SQL with f-strings using self.dbname and self.template_dbname
(in the method that calls cur.execute with query), which can break on embedded
quotes; replace this string interpolation with psycopg.sql.Identifier
composition: import psycopg.sql as sql, build the base sql.SQL("CREATE DATABASE
{}").format(sql.Identifier(self.dbname)), conditionally append sql.SQL("
TEMPLATE {}").format(sql.Identifier(self.template_dbname)) when present and
sql.SQL(" IS_TEMPLATE = true") when self.as_template is true, then pass the
composed sql.SQL object to await cur.execute(...) instead of the f-string.
Ensure all occurrences that construct database identifiers (the blocks using
self.dbname, self.template_dbname, and as_template) use sql.Identifier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant