Skip to content

feat: add CursorTracker for at-least-once delivery via drain_changes() polling#1110

Merged
pyramation merged 2 commits intomainfrom
feat/realtime-cursor-tracking
May 10, 2026
Merged

feat: add CursorTracker for at-least-once delivery via drain_changes() polling#1110
pyramation merged 2 commits intomainfrom
feat/realtime-cursor-tracking

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

@pyramation pyramation commented May 10, 2026

Summary

Adds a CursorTracker class to graphile-realtime-subscriptions that manages the listener_node lifecycle and periodic drain_changes() polling, enabling at-least-once delivery semantics using the existing database-side cursor tracking infrastructure (change_log, listener_node, drain_changes, touch_listener, cleanup_ephemeral).

New files:

  • src/cursor-tracker.tsCursorTracker class with start(), stop(), drain(), touchListener(), cleanupEphemeral() methods
  • __tests__/cursor-tracker.test.ts — 25 tests covering lifecycle, polling, error handling, concurrency guards, schema quoting

Modified files:

  • src/types.ts — New types: PgClient, WithPgClient, ChangeLogEntry, CursorTrackerOptions
  • src/index.ts — Exports CursorTracker and new types
  • src/plugin.ts — Re-exports CursorTracker, updated docstring documenting cursor tracking
  • package.json — Added @pgsql/quotes dependency for identifier quoting

The CursorTracker is a standalone component — it is exported for external use but is not wired into the plugin's Grafast subscription plan execution. Callers instantiate it with a withPgClient callback and an onChanges handler to receive ChangeLogEntry[] results from drain_changes(). The wiring of cursor-tracked changes into the GraphQL subscription delivery path is expected to happen in a subsequent integration step (e.g., server preset or middleware).

Updates since last revision

  • Replaced hand-rolled quoteIdent with QuoteUtils.quoteIdentifier() from @pgsql/quotes — uses the same identifier quoting library already used by query-builder, graphile-settings, and other packages in the monorepo. QuoteUtils.quoteIdentifier() only adds double-quotes when the identifier requires them (e.g., contains spaces or special characters); simple identifiers like realtime_public are returned unquoted.
  • Updated tests to match QuoteUtils.quoteIdentifier() behavior — schema quoting tests no longer assert double-quotes around simple identifiers. Added a new test verifying that identifiers with spaces are properly quoted.

Review & Testing Checklist for Human

  • Verify ChangeLogEntry type matches drain_changes() return shape — The type assumes drain_changes returns JSONB rows with {id, occurred_at, source_schema, source_table, operation, payload_after, payload_before, payload_diff, subscriber_ids}. Cross-reference with proc_drain_changes_body in constructive-db.
  • Verify the standalone integration pattern is acceptableCursorTracker is not wired into subscribePlan/buildPlans; it's a building block. There is no code path that connects onChanges entries to GraphQL subscriber delivery yet. Confirm this is the intended phasing.
  • Review SQL query construction in drain() — Uses SELECT * FROM ${schema}.drain_changes($1, $2) where schema is quoted via QuoteUtils.quoteIdentifier(). Verify this produces valid SQL for both simple schemas (realtime_public) and schemas requiring quoting.
  • Test plan: Instantiate CursorTracker against a real database with realtime_module deployed, call start(), insert rows into a realtime-enabled table, verify onChanges receives the expected entries, then call stop() and verify listener_node row is removed.

Notes

  • QuoteUtils.quoteIdentifier() from @pgsql/quotes only adds double-quotes when necessary — simple identifiers like realtime_public are emitted unquoted, while identifiers with spaces or special characters get properly quoted. This is safe since the quoting follows PostgreSQL's identifier rules.
  • The concurrent drain guard (draining boolean) silently skips poll cycles if a previous drain is still in-flight. This is intentional to prevent query pileup.
  • CursorTracker and its types are exported from both src/plugin.ts and src/index.ts (dual export paths).

Link to Devin session: https://app.devin.ai/sessions/19485cf5cc58416a9f86068563d512f5
Requested by: @pyramation

…) polling

Implements cursor tracking integration for the realtime subscriptions plugin:

- CursorTracker class manages listener_node lifecycle (register, heartbeat, cleanup)
- Periodic drain_changes() polling fetches change_log entries matched against subscribers
- Periodic touch_listener() heartbeat keeps the node alive
- cleanup_ephemeral() on stop removes ephemeral subscriptions and deletes the node
- Configurable poll interval, heartbeat interval, batch limit, and schema
- WithPgClient callback pattern keeps connection management external
- onChanges callback delivers ChangeLogEntry[] with subscriber_ids for fan-out
- Error handling with onError callback, no unhandled rejections
- Guard against concurrent drains

New types: PgClient, WithPgClient, ChangeLogEntry, CursorTrackerOptions
24 new tests covering lifecycle, polling, error handling, schema quoting
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@socket-security
Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​types/​geojson@​7946.0.161001007280100
Addedjs-yaml@​4.1.19710010081100
Addedrimraf@​6.1.39910010083100
Addedcors@​2.8.610010010084100
Addedpg@​8.20.0991009985100
Addedform-data@​4.0.59910010088100
Addedjiti@​2.6.19710010088100
Addedajv@​8.18.09910010090100
Addedtypescript@​5.9.3100100909890
Addedprettier@​3.8.1901009795100
Addedsemver@​7.7.410010010091100

View full report

@socket-security
Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
Obfuscated code: npm entities is 91.0% likely obfuscated

Confidence: 0.91

Location: Package overview

From: pnpm-lock.yamlnpm/entities@4.5.0

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/entities@4.5.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: npm entities is 91.0% likely obfuscated

Confidence: 0.91

Location: Package overview

From: pnpm-lock.yamlnpm/entities@6.0.1

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/entities@6.0.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: npm markdown-it is 91.0% likely obfuscated

Confidence: 0.91

Location: Package overview

From: pnpm-lock.yamlnpm/markdown-it@14.1.1

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/markdown-it@14.1.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

@pyramation pyramation merged commit 4276758 into main May 10, 2026
54 checks passed
@pyramation pyramation deleted the feat/realtime-cursor-tracking branch May 10, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant