feat: Add CEL-based conditional function execution (#4388)#4391
feat: Add CEL-based conditional function execution (#4388)#4391SurbhiAgarwal1 wants to merge 1 commit intokptdev:mainfrom
Conversation
✅ Deploy Preview for kptdocs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
32fe3c0 to
3d2bde5
Compare
.agent/github_comment_4382.md
Outdated
There was a problem hiding this comment.
Did you mean to post these files on the issue rather than on the PR? It looks like AI generated content.
nagygergo
left a comment
There was a problem hiding this comment.
Hey, good to see a new contributor.
Some general comments:
- Clean up AI instructions.
- Add e2e tests.
- Add documentation.
Some specific comments are inline.
|
|
||
| // NewCELEvaluator creates a new CEL evaluator with the standard environment | ||
| func NewCELEvaluator() (*CELEvaluator, error) { | ||
| env, err := cel.NewEnv( |
There was a problem hiding this comment.
This is a totally unprotected cel executor. There should be limitations on the number of CPU cycles it can consume, the amount of characters it can output, the max complexity of the ast.
There was a problem hiding this comment.
This is still unresolved. Please read up on how cost estimation works in cel environments. The goal is to protect the kpt and porch runtime from exploits using arbitrary code execution allowed here
There was a problem hiding this comment.
Hi @nagygergo! I've researched the CEL cost estimation API and found that env.EstimateCost() requires runtime size information about variables (like the resources list) which isn't available at compile time. When I tried using it with a nil estimator, it panics during size computation.
I see two approaches:
Option 1: Runtime Cost Tracking - Use cel.CostTracking() in the Program to enforce limits during evaluation (similar to Kubernetes ValidatingAdmissionPolicy)
Option 2: Custom Estimator - Implement checker.CostEstimator interface with estimated sizes for the resources variable
Which approach would you prefer? I'm happy to implement either one properly. All tests are currently passing with cel-go v0.26.0.
| return NewFunctionRunner(ctx, fltr, pkgPath, fnResult, fnResults, opts) | ||
|
|
||
| // Initialize CEL evaluator if a condition is specified | ||
| var evaluator *CELEvaluator |
There was a problem hiding this comment.
Why do you need to create a new CEL env for each function evaluation? The env can be the same.
internal/fnruntime/runner.go
Outdated
| pr := printer.FromContextOrDie(fr.ctx) | ||
|
|
||
| // Check condition before executing function | ||
| if fr.condition != "" && fr.evaluator != nil { |
There was a problem hiding this comment.
Why check if fr.evaluator exists or not. If the function runner was created with a condition appearing for it's Runner, then must have an evaluator. It's ok to run to a panic if it doesn't exist at this point.
| "github.com/stretchr/testify/require" | ||
| "sigs.k8s.io/kustomize/kyaml/yaml" | ||
| ) | ||
|
|
There was a problem hiding this comment.
Add a testcase that makes sure that the cel functions can't mutate the resourcelist that is the input. The function signature can allow for it, as it hands over the *yaml.RNode list.
There was a problem hiding this comment.
Hi @nagygergo! Done! I've added TestEvaluateCondition_Immutability that verifies CEL functions can't mutate the input resource list.
The test creates a ConfigMap, stores the original YAML, evaluates a CEL condition, then compares before/after to ensure no mutation. Even though the function signature accepts *yaml.RNode list (which could be mutated), CEL only receives converted map[string]interface{} copies, so the original RNodes remain unchanged.
|
|
||
| // Create function runner with condition | ||
| fnResult := &fnresult.Result{} | ||
| fnResults := &fnresult.ResultList{} |
There was a problem hiding this comment.
Are these needed for testware when initialising it?
| // NewCELEvaluator creates a new CEL evaluator with the standard environment | ||
| func NewCELEvaluator() (*CELEvaluator, error) { | ||
| env, err := cel.NewEnv( | ||
| cel.Variable("resources", cel.ListType(cel.DynType)), |
There was a problem hiding this comment.
Probably advanced strings libraries would be good to include. https://pkg.go.dev/github.com/google/cel-go/ext#Strings
internal/fnruntime/celeval.go
Outdated
| } | ||
|
|
||
| // Evaluate the expression | ||
| out, _, err := prg.Eval(map[string]interface{}{ |
There was a problem hiding this comment.
There should be a context passed to this to protect against long-hanging operations.
| } | ||
|
|
||
| // Convert resources to a format suitable for CEL | ||
| resourceList, err := e.resourcesToList(resources) |
There was a problem hiding this comment.
Is serialising all the yaml.RNode actually needed? As it's a map[string]any type anyways (with no strange subtypes), probably the CEL interpreter can deal with it directly. Serialising the whole package for the cel execution, then not reusing it can cause a significant memory footprint bloat.
There was a problem hiding this comment.
Hi @nagygergo! I've fixed the RNode serialization issue you mentioned.
Changed from resource.Map() (which serializes to YAML and back) to using resource.YNode().Decode() directly. This avoids the intermediate serialization step and reduces memory overhead.
The new implementation:
- Gets the underlying yaml.Node directly via
YNode() - Decodes it to
map[string]interface{}without serialization - Maintains the same CEL evaluation behavior
All tests are passing with this change.
5f1dce7 to
98ac05d
Compare
|
Thanks @nagygergo for the detailed review! I've addressed all the feedback: Changes Made:1. Cleaned up AI-generated files
2. Added CEL protection
3. Reuse CEL environment
4. Added context parameter
5. Removed unnecessary nil check
6. Added immutability test
7. Updated all tests
8. Added documentation
9. Resource serialization
10. fnResult/fnResults in tests
All review comments have been addressed. Let me know if you need any other changes! |
cce25d3 to
e0c5faa
Compare
|
@SurbhiAgarwal1 please fix the build issues. I'll get back to reviewing it after the build and tests are passing. |
155590a to
fc9326d
Compare
cf1b03a to
4ccfe93
Compare
04e12f6 to
588479d
Compare
1af8cd4 to
d2288d6
Compare
d2288d6 to
588479d
Compare
Signed-off-by: Surbhi <agarwalsurbhi1807@gmail.com>
ef74d9f to
fa984be
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Create the program with cost tracking to prevent resource exhaustion | ||
| // This enforces a runtime cost limit similar to Kubernetes ValidatingAdmissionPolicy | ||
| prg, err := env.Program(ast, cel.CostTracking(nil)) |
There was a problem hiding this comment.
The CEL cost tracking is enabled but with a nil limit, which means there's no actual runtime cost limit enforced. According to the comment, this should prevent resource exhaustion similar to Kubernetes ValidatingAdmissionPolicy, but a nil cost tracker doesn't enforce any limit. Consider using cel.CostLimit to set an actual cost limit, or if unlimited evaluation is intentional, the comment should be updated to reflect that cost tracking is enabled for monitoring purposes only without enforcement.
6b864d3 to
fa984be
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // NewCELEvaluator creates a new CEL evaluator with the standard environment | ||
| // The environment is created once and reused for all evaluations | ||
| func NewCELEvaluator(condition string) (*CELEvaluator, error) { | ||
| env, err := cel.NewEnv( | ||
| cel.Variable("resources", cel.ListType(cel.DynType)), | ||
| ) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to create CEL environment: %w", err) | ||
| } |
There was a problem hiding this comment.
The doc comment claims “The environment is created once and reused for all evaluations”, but NewCELEvaluator currently creates a new cel.Env on every call. Either adjust the comment to reflect per-evaluator creation, or make the env a shared singleton if reuse is intended.
|
|
||
| // Check condition before executing function | ||
| if fr.evaluator != nil { | ||
| shouldExecute, err := fr.evaluator.EvaluateCondition(fr.ctx, input) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to evaluate condition for function %q: %w", fr.name, err) | ||
| } | ||
|
|
||
| if !shouldExecute { | ||
| if !fr.disableCLIOutput { | ||
| pr.Printf("[SKIPPED] %q (condition not met)\n", fr.name) | ||
| } | ||
| // Return input unchanged - function is skipped | ||
| return input, nil | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
When a function is skipped due to a false condition, Filter returns early without updating fr.fnResult / appending to fr.fnResults.Items. This makes the structured FunctionResultList omit the function entirely (unlike the normal do() path, which always appends a Result), which can break consumers that expect one result per pipeline step. Consider appending a Result with ExitCode=0 (and optionally a structured message indicating it was skipped) before returning.
| // Check condition before executing function | |
| if fr.evaluator != nil { | |
| shouldExecute, err := fr.evaluator.EvaluateCondition(fr.ctx, input) | |
| if err != nil { | |
| return nil, fmt.Errorf("failed to evaluate condition for function %q: %w", fr.name, err) | |
| } | |
| if !shouldExecute { | |
| if !fr.disableCLIOutput { | |
| pr.Printf("[SKIPPED] %q (condition not met)\n", fr.name) | |
| } | |
| // Return input unchanged - function is skipped | |
| return input, nil | |
| } | |
| } | |
| // Check condition before executing function | |
| if fr.evaluator != nil { | |
| shouldExecute, err := fr.evaluator.EvaluateCondition(fr.ctx, input) | |
| if err != nil { | |
| return nil, fmt.Errorf("failed to evaluate condition for function %q: %w", fr.name, err) | |
| } | |
| if !shouldExecute { | |
| if !fr.disableCLIOutput { | |
| pr.Printf("[SKIPPED] %q (condition not met)\n", fr.name) | |
| } | |
| // Record a synthetic successful result for the skipped function so that | |
| // consumers that expect one result per pipeline step still see an entry. | |
| skippedResult := &fnresult.Result{ | |
| ExitCode: 0, | |
| } | |
| fr.fnResult = skippedResult | |
| if fr.fnResults != nil { | |
| fr.fnResults.Items = append(fr.fnResults.Items, skippedResult) | |
| } | |
| // Return input unchanged - function is skipped | |
| return input, nil | |
| } | |
| } |
| // Check condition before executing function | ||
| if fr.evaluator != nil { | ||
| shouldExecute, err := fr.evaluator.EvaluateCondition(fr.ctx, input) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to evaluate condition for function %q: %w", fr.name, err) |
There was a problem hiding this comment.
The condition is evaluated against the input passed into FunctionRunner.Filter. In the render/hydration flow, inputs are often pre-filtered via SelectInput (selectors/exclusions) before calling Filter, so this ends up evaluating the condition against only the selected subset rather than the full package resources (contradicting the schema comment and issue intent). Consider evaluating conditions at a higher level where the full resource set is available, or changing the call pattern so Filter receives the full resource list for condition evaluation.
Implements #4388
Changes
conditionfield to Function schema for CEL expressionsExample Usage