Skip to content

refactor(appkit): split SQLWarehouseConnector into submit/get/poll/transform#372

Closed
ditadi wants to merge 1 commit into
taskflow/service-registryfrom
taskflow/sql-warehouse-split
Closed

refactor(appkit): split SQLWarehouseConnector into submit/get/poll/transform#372
ditadi wants to merge 1 commit into
taskflow/service-registryfrom
taskflow/sql-warehouse-split

Conversation

@ditadi
Copy link
Copy Markdown
Contributor

@ditadi ditadi commented May 11, 2026

Stack

Based on #371. Review independently — SQLClient public contract is unchanged.


SQLClient.executeStatement was a single block: submit the SQL, poll
until terminal, transform the Arrow payload to JSON. Splitting it into
four narrower public APIs lets durable executors compose them without
holding the orchestrator open across the wait:

  • submitStatement(sql, params, opts) — POST /sql/statements,
    returns the raw initial response. Adds a dedicated sql.submit span.
  • getStatement(id) — GET /sql/statements/{id}, single status read.
  • pollStatement(id, opts) — block until the statement reaches a
    terminal state (SUCCEEDED / FAILED / CANCELED / CLOSED), respecting
    the same timeout, signal, and error semantics the old monolithic
    method had.
  • transformResult(response) — Arrow → JSON row transform, no I/O.

executeStatement(...) is preserved and now composes the four publics
(submitpolltransform). No private wrapper-only helpers
remain. Every error path, abort branch, and status state machine of
the old method is exercised by the new per-API test suites (21 new
tests against submit / get / poll / transform).

Motivation (documented inline in JSDoc): durable callers — e.g. a
future TaskFlow-based analytics handler — emit a statement_submitted
event with the warehouse-side statement ID right after submitStatement
returns, so on crash recovery they can re-attach via pollStatement
without re-running the SQL. The TaskFlow integration itself is not in
this PR.

executeStatement's contract is unchanged; analytics (the only external
caller) keeps working without modification. The added sql.submit span
is purely additive for OTLP collectors.

Verified: pnpm -r typecheck, pnpm build, full pnpm test
(122 files, 2276 tests) all green.

Signed-off-by: ditadi victordperd@gmail.com


Stack created with GitHub Stacks CLIGive Feedback 💬

…ansform

`SQLClient.executeStatement` was a single block: submit the SQL, poll
until terminal, transform the Arrow payload to JSON. Splitting it into
four narrower public APIs lets durable executors compose them without
holding the orchestrator open across the wait:

- `submitStatement(sql, params, opts)` — POST `/sql/statements`,
  returns the raw initial response. Adds a dedicated `sql.submit` span.
- `getStatement(id)` — GET `/sql/statements/{id}`, single status read.
- `pollStatement(id, opts)` — block until the statement reaches a
  terminal state (SUCCEEDED / FAILED / CANCELED / CLOSED), respecting
  the same timeout, signal, and error semantics the old monolithic
  method had.
- `transformResult(response)` — Arrow → JSON row transform, no I/O.

`executeStatement(...)` is preserved and now composes the four publics
(`submit` → `poll` → `transform`). No private wrapper-only helpers
remain. Every error path, abort branch, and status state machine of
the old method is exercised by the new per-API test suites (21 new
tests against `submit` / `get` / `poll` / `transform`).

Motivation (documented inline in JSDoc): durable callers — e.g. a
future TaskFlow-based analytics handler — emit a `statement_submitted`
event with the warehouse-side statement ID right after `submitStatement`
returns, so on crash recovery they can re-attach via `pollStatement`
without re-running the SQL. The TaskFlow integration itself is not in
this PR.

`executeStatement`'s contract is unchanged; analytics (the only external
caller) keeps working without modification. The added `sql.submit` span
is purely additive for OTLP collectors.

Verified: pnpm -r typecheck, pnpm build, full pnpm test
(122 files, 2276 tests) all green.

Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi
Copy link
Copy Markdown
Contributor Author

ditadi commented May 11, 2026

Superseded by #375 — replatformed onto git stack with branch name stack/taskflow/sql-warehouse-split.

@ditadi ditadi closed this May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant