feat: cross-run lineage via reserved run attributes#2153
Conversation
🦋 Changeset detectedLatest commit: e0b62e9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 21 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@rchasman is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
|
since now that #2088 has been shipped in v5 too, you can also use attributes to have parent and child runs link to each other by setting the run IDs as you want. We could have a reserved wdyt @rchasman ? |
Relate runs without a new primitive: `start()` records lineage as reserved
run attributes (`$rootId`, `$parentRunId`) rather than a dedicated column.
A run started with no parent is its own root (`$rootId === runId`); a run
started from inside another run inherits the parent's `$rootId`, so a chain
of any depth stays flat under one root. `start()` reads the parent from the
ambient step context.
- world: `attributes` on `CreateWorkflowRunRequest`, the run_created and
resilient run_started event data, and queue `RunInput`; an `attributes`
filter on `ListWorkflowRunsParams` so a lineage is queryable.
- core: `start()` resolves `$rootId`/`$parentRunId` and merges any
caller-provided attributes on top.
A lineage is a query (`list({ attributes: { $rootId } })`), not an entity,
so the model stays run-centric: a cron is just a workflow that starts its
own successor, and `cron_abc123` is the root run's id.
Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
…ibutes
Apply attributes provided at run creation (run_created and the resilient
run_started path), and support `list({ attributes })`, matching runs whose
attributes contain every requested key/value. Because lineage rides on
attributes — which are already preserved across lifecycle updates — it
survives started/completed/failed/cancelled with no extra handling.
Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
Add a daisy-chain fixture (each tick starts its successor), a `/runs` endpoint that filters by the `$rootId` attribute plus a `listRuns` fetcher, and an e2e test asserting the chain runs through the real runtime and queue and groups under one lineage, with an unrelated run excluded. Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
world-vercel and world-postgres are intentionally out of scope here. Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
rchasman
force-pushed
the
feat/cross-run-lineage-rootid
branch
from
bb30012 to
e0b62e9
Compare
rchasman
changed the title
|
Re-spun onto attributes per your suggestion. Good call.. Going through attributes turned out cleaner than the column: lineage is preserved across lifecycle updates for free (attributes already carry forward), and the same filter generalizes to any attribute, not just lineage. I record both Also fixed the resilient-start path the review bot flagged: attributes now flow through Proven end-to-end with a real daisy-chain; unit + e2e green, existing suites unaffected. Still world-local only and one |
2 participants
Problem
Workflow has no way to relate one run to another.
ListWorkflowRunsParamsfilters byworkflowNameandstatus; a run carries no lineage. That's fine for one-shot runs but breaks every multi-run pattern the SDK already encourages:cron_abc123" — can't be built on a library alone, because the runs aren't grouped.deploymentId: 'latest'hops — they all orphan.Solution
Record lineage as reserved run attributes (
$rootId,$parentRunId) and make attributes filterable inlist(). This follows @pranaygp's suggestion on #1649 to reuse the attributes mechanism (#2088) rather than add a dedicated field — the only missing piece was queryability.$rootId === runId).$rootId, so a chain of any depth stays flat under one root.start()reads the parent from the ambient step context (it's a'use step').list({ attributes: { $rootId } })— not a new entity.Design: stays run-centric
Workflow is run-centric — event log, observability, cancellation, version pinning, and IDs are all per-run; the run is the atom (the run-listed UI is a symptom of this). Lineage is deliberately not a new primitive above the run:
cron_abc123is just the root run's id.Presentation follows and stays open-ended. A run with no lineage is unaffected, so the existing runs view is unchanged; if a consumer ignores the attributes, nothing breaks. The view can optionally collapse a lineage and filter by
$rootId. A tree view can come later for free and also stays run-centric:$parentRunIdis already recorded for the edge, so a tree just renders the same runs hierarchically — no containing "parent run" entity, just runs pointing at runs. Flat ships today with zero migration; nested timelines layer on whenever the UI wants them. Flat also keeps grouping O(1) by one id rather than an N-hop parent walk.What's included
@workflow/world:attributesonCreateWorkflowRunRequest, the run_created and resilient run_started event data, and queueRunInput; anattributesfilter onListWorkflowRunsParams.@workflow/core:start()resolves$rootId/$parentRunIdfrom the ambient step context and merges any caller-provided attributes.@workflow/world-local: apply attributes at creation;list({ attributes }).@workflow/world-testing: an end-to-end daisy-chain test.Testing
$rootId, listable via the attribute filter, with an unrelated run excluded.Why attributes, on parent vs root
$parentRunIdalone (the immediate edge) needs an N-hop walk to group a lineage. A reserved$rootIdgives O(1) grouping in onelist()filter — a run inherits its parent's$rootId, a root's is itself. Both are recorded:$parentRunIdfor the tree edge,$rootIdfor the group key.Scope and caveats
A working prototype opened for design direction, not a finished feature:
start()are done, but persistence/filter is wired only in@workflow/world-local.world-vercelandworld-postgreshave their own run-creation, lifecycle, andlistpaths that each need the same wiring.start()reads the parent's$rootIdwith oneworld.runs.getper nested start. The zero-I/O shape is to thread the root through the step context (WorkflowMetadata+ step invoke payload). Left as a documented follow-up.deploymentId: 'latest'per hop) is a real RFC question, not decided here.A changeset is included (
minorfor the three touched packages, world-vercel/postgres gap noted).