feat: add SCOPED_TASKFILES experiment #2659
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vmaerten
changed the title
Add new experiment flag TASK_X_SCOPED_INCLUDES for scoped variable resolution in included Taskfiles. When enabled, variables from included Taskfiles will be isolated rather than merged globally. This is the first step towards implementing lazy DAG-based variable resolution with strict isolation between includes.
Store the TaskfileGraph in the Executor so it can be used for lazy variable resolution when SCOPED_INCLUDES experiment is enabled. The graph is now preserved after reading, before merging into the final Taskfile. This allows traversing the DAG at runtime to resolve variables from the correct scope.
Add Root() method to TaskfileGraph to get the root vertex (entrypoint Taskfile). This will be used for lazy variable resolution. Note: Tasks already have Location.Taskfile which can be used to find their source Taskfile in the graph, so GetVertexByNamespace is not needed.
When the SCOPED_INCLUDES experiment is enabled, variables from included Taskfiles are no longer merged globally. They remain in their original Taskfile within the DAG. Exception: flatten includes still merge variables globally to allow sharing common variables across multiple Taskfiles.
When SCOPED_INCLUDES experiment is enabled: - Resolve vars from DAG instead of merged vars - Apply root Taskfile vars first (inheritance) - Then apply task's source Taskfile vars from DAG - Apply IncludeVars passed via includes: section - Skip IncludedTaskfileVars (contains parent's vars, not source's) This ensures tasks in included Taskfiles see: 1. Root vars (inheritance from parent) 2. Their own Taskfile's vars 3. Vars passed through includes: section 4. Call vars and task-level vars
Tests verify: - Legacy mode: vars merged globally (A sees B's VAR, can access UNIQUE_B) - Scoped mode: vars isolated (A sees own VAR, cannot access UNIQUE_B) - Inheritance: includes can still access root vars (ROOT_VAR) Test structure: - testdata/scoped_includes/ with main Taskfile and two includes - inc_a and inc_b both define VAR with different values - Cross-include test shows A trying to access B's UNIQUE_B
… env namespace
Rename the experiment from SCOPED_INCLUDES to SCOPED_TASKFILES to better
reflect its expanded scope. This experiment now provides:
1. Variable scoping (existing): includes see only their own vars + parent vars
2. Environment namespace (new): env vars accessible via {{.env.XXX}}
With TASK_X_SCOPED_TASKFILES=1:
- {{.VAR}} accesses vars only (scoped per include)
- {{.env.VAR}} accesses env (OS + Taskfile env:, inherited)
- {{.TASK}} and other special vars remain at root level
This is a breaking change for the experimental feature:
- {{.PATH}} no longer works, use {{.env.PATH}} instead
- Env vars are no longer at root level in templates
In scoped mode, CLI vars (e.g., `task foo VAR=value`) now correctly
override task-level vars. This is achieved by:
1. Adding a `CLIVars` field to the Compiler struct
2. Storing CLI globals in this field after parsing
3. Applying CLI vars last in scoped mode to ensure they override everything
The order of variable resolution in scoped mode is now:
1. OS env → {{.env.XXX}}
2. Root taskfile env → {{.env.XXX}}
3. Root taskfile vars → {{.VAR}}
4. Include taskfile env/vars (if applicable)
5. IncludeVars (vars passed via includes: section)
6. Task-level vars
7. CLI vars (highest priority)
Legacy mode behavior is unchanged.
Document the new experiment with:
- Environment namespace ({{.env.XXX}}) explanation
- Variable scoping between includes
- CLI variables priority
- Migration guide from legacy mode
- Comparison table between legacy and scoped modes
When calling a task with vars (e.g., `task: name` with `vars:`), those vars were not being applied in scoped mode. This fix adds call.Vars to the variable resolution chain. Variable priority (lowest to highest): 1. Root Taskfile vars 2. Include Taskfile vars 3. Include passthrough vars 4. Task vars 5. Call vars (NEW) 6. CLI vars
Refactor compiler.go for better maintainability: - Extract isScopedMode() helper function - Split getVariables() into getScopedVariables() and getLegacyVariables() - Fix directory resolution: parent chain env/vars now resolve from their own directory instead of the current task's directory Add nested includes support and tests: - Add testdata/scoped_taskfiles/inc_a/nested/Taskfile.yml (3 levels deep) - Add test case for nested include inheritance (root → a → nested) - Verify nested includes inherit vars from full parent chain Fix flaky tests: - Remove VAR from print tasks (defined in both inc_a and inc_b) - Test only unique variables (UNIQUE_A, UNIQUE_B, ROOT_VAR) Document flatten: true escape hatch: - Add migration guide step for using flatten: true - Add new section explaining flatten bypasses scoping - Include example and usage recommendations
Collaborator
📦 Build artifacts ready!Download binaries from this workflow run. Available platforms: Linux, macOS, Windows (amd64, arm64) |
- Fix import order in setup.go (gci) - Fix variable alignment in experiments.go (gofmt) - Add nolint:paralleltest directive for TestScopedTaskfiles
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces the
TASK_X_SCOPED_TASKFILESexperiment which fundamentally changes how variables work in included Taskfiles. The goal is to eliminate variable conflicts and unexpected leakage between includes.related to #2035
What this experiment does
1. Environment namespace
Environment variables (OS +
env:section) move to a dedicated namespace:Special variables like
{{.TASK}},{{.ROOT_DIR}}stay at root level.2. Variable isolation
Included Taskfiles can only see:
includes: name: vars:They can NOT see variables from sibling includes anymore.
3. Clear priority order
From lowest to highest:
includes: foo: vars:)vars:)task foo VAR=value)4. Escape hatch:
flatten: trueFor gradual migration, you can use
flatten: trueon specific includes to restore legacy behavior:Breaking changes
{{.HOME}}→{{.env.HOME}}for all env vars