[DRAFT][STACKED on #206] Add Dep marker for explicit container dependencies in @Flow.model#213
Draft
NeejWeej wants to merge 16 commits into
Draft
[DRAFT][STACKED on #206] Add Dep marker for explicit container dependencies in @Flow.model#213NeejWeej wants to merge 16 commits into
NeejWeej wants to merge 16 commits into
Conversation
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
… @Flow.model Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
…-containers Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #213 +/- ##
==========================================
- Coverage 95.37% 93.81% -1.56%
==========================================
Files 142 151 +9
Lines 11404 17614 +6210
Branches 620 1156 +536
==========================================
+ Hits 10876 16525 +5649
- Misses 399 865 +466
- Partials 129 224 +95 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
85c61fa to
7b76d13
Compare
Signed-off-by: Nijat K <nijat.khanbabayev@gmail.com>
f864ab1 to
6938b95
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Summary:
nk/explicit-deps-containersStacked PR Note
This builds on the
@Flow.modelPR: #206.That PR should be reviewed/merged first. This PR has to target
mainbecause GitHub cannot target the fork-only parent branch, please review this as
the follow-on
Dep[...]container-dependency layer rather than re-reviewingthe base
@Flow.modelwork.TL;DR
This lets a flow consume a dynamic collection of upstream model outputs without
hard-coding every branch into the model signature. A fan-out step can produce
many sibling models, and a fan-in step can accept them as
list[Dep[T]],tuple[...], ordict[..., Dep[T]]while still receiving plain resolvedvalues inside the function. The marker keeps that power explicit: normal
containers stay literal unless the exact nested dependency slot is marked.
Tiny example:
Without
Dep,combine(rows=branch_models)would be rejected because regularcontainers stay literal by default. With
list[Dep[int]], each list item may beeither a literal
intor a model producing anint, andcombine()still seesplain
list[int]at execution time.What Changed
This PR adds
Dep[T], a narrow@Flow.modelannotation marker for explicitdependency leaves inside regular container inputs.
Regular parameters already accepted either a literal value or a direct
CallableModelsupplying the whole parameter. That behavior is unchanged.Dep[T]only adds marked nested slots inside literal containers:The function body receives
list[int]; generated@Flow.modelcode resolvesthe marked model leaves before calling the function.
Semantics
list[int]still accepts a literallist[int]or a direct model returninglist[int].list[Dep[int]]additionally accepts model leaves inside the literal list.Dep[T]means "literalTorCallableModelwhose unwrapped resultvalidates as
T" at that exact annotation position.Dep[...]is interpreted only by generated@Flow.modelregular parameters.Dep[...]is supported insidelist,tuple, anddictvalues.Dep[...]is rejected because direct whole-parameter dependenciesalready cover that case.
list[Dep[int | None]].Depmarker, including optional containerforms like
list[Dep[int]] | None.Dep[...].Dep[...]markers are rejected.Dep[...]does not add automatic behavior to handwrittenCallableModelfields; there it behaves like ordinary
Annotatedmetadata.Why This Shape
This keeps automatic dependency discovery simple and explicit. We do not scan
every container for models. Instead, users mark the exact nested positions where
model leaves are allowed. That preserves existing literal-container behavior
while supporting generic fan-in patterns such as lists of branch results.
Implementation Notes
Depas anAnnotated[T, _DepMarker()]marker.__deps__, and effectiveidentity to walk only marked container positions.
marked dependency leaves.
Annotatedmetadata when removing theDepmarker, soconstraints such as
Field(gt=0)still apply to literals and dependencyresults.
Tests
Covered by
ccflow/tests/test_flow_model.py:list[Dep[int]]andlist[Dep[list[int]]];dependency leaves;
DepAnnotatedmetadata;Dep, nestedDep, dict-keyDep, and set usage;CallableModelfields ignoringDepas normal annotationmetadata.