[Enhancement](udf) support volatility for udaf && udtf#63611
Open
linrrzqqq wants to merge 2 commits into
Open
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Collaborator
Author
|
/review |
Collaborator
Author
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Automated review completed. I did not find blocking correctness issues in the changed UDAF/UDTF volatility paths.
Critical checkpoint conclusions:
- Goal/test coverage: The PR extends volatility parsing, SHOW/SHOW CREATE emission, and Nereids catalog-function conversion for Java/Python UDAF and UDTF. Added unit tests cover default UDAF volatility, explicit UDTF volatility, SHOW properties, SHOW CREATE SQL, and Nereids conversion preservation.
- Scope/focus: The implementation is small and localized to function creation, display, replay SQL, and UDF expression wrappers.
- Concurrency/lifecycle: No new shared mutable state, locks, threads, static initialization, or lifecycle-sensitive ownership were introduced.
- Config/compatibility: No new config items. Existing functions without persisted volatility still use Function.getVolatility() fallback to IMMUTABLE, and replay SQL now includes a property that CreateFunctionCommand accepts for Java/Python UDAF/UDTF.
- Parallel paths: Java/Python and UDAF/UDTF conversion paths appear updated consistently; RPC/non-Java/Python paths still reject volatility.
- Error handling: Invalid volatility continues through FunctionVolatility.fromString and is converted to AnalysisException.
- Transaction/persistence/data correctness: No transaction or storage visibility paths are touched.
- Performance/observability: No hot-path or observability concerns found for these metadata-only changes.
User focus: No additional user-provided review focus was supplied.
Contributor
TPC-H: Total hot run time: 31897 ms |
Contributor
TPC-DS: Total hot run time: 172942 ms |
Contributor
FE Regression Coverage ReportIncrement line coverage |
yujun777
reviewed
May 26, 2026
|
|
||
| @Override | ||
| public boolean isDeterministic() { | ||
| return volatility == FunctionVolatility.IMMUTABLE; |
Collaborator
Author
|
/review |
Collaborator
Author
|
run buildall |
linrrzqqq
force-pushed
the
volatile-udaf-udtf
branch
from
d1c79a3 to
6114612
Compare
Collaborator
Author
|
run buildall |
Collaborator
Author
|
/review |
Contributor
There was a problem hiding this comment.
Review result: no blocking findings found in the PR diff.
Critical checkpoint conclusions:
- Goal and coverage: The PR extends volatility metadata handling from scalar UDFs to UDAF/UDTF creation, Nereids function wrappers, SHOW FUNCTIONS, and SHOW CREATE FUNCTION. The changed paths appear to accomplish that goal, with unit coverage added for parser/default behavior, SQL rendering, SHOW FUNCTIONS properties, and Nereids wrapper propagation.
- Scope/focus: The implementation is small and focused. The existing review thread about moving deterministic logic to the common UDF layer is already addressed by the final patch, so I did not duplicate it.
- Concurrency/lifecycle: No new shared mutable state, threads, locks, static initialization dependencies, or non-obvious lifecycle management were introduced.
- Configuration/compatibility: No new config items or storage-format changes. Function volatility is already represented in function metadata; this PR primarily allows and propagates it for aggregate/table UDF forms.
- Parallel code paths: Java/Python and UDAF/UDTF paths were updated consistently for create, catalog conversion, and display.
- Error handling: Existing analysis exception patterns are preserved; invalid volatility still fails through FunctionVolatility parsing.
- Tests/results: Added FE unit coverage is relevant. I attempted to run mvn -pl fe-core -am test -Dtest=CreateFunctionTest,FunctionToSqlConverterTest,UdfVolatilityTest,ShowFunctionsCommandTest -DfailIfNoTests=false -Dskip.doc=true, but this runner is missing thirdparty/installed/bin/thrift, so Maven failed in fe-thrift before reaching fe-core tests.
- Observability/performance/data correctness: No runtime data read/write, transaction, memory tracking, or observability-sensitive path is changed; no performance concern found.
User focus: No additional user-provided review focus was specified.
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
TPC-H: Total hot run time: 31327 ms |
Contributor
TPC-DS: Total hot run time: 171602 ms |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
|
PR approved by anyone and no changes requested. |
yujun777
reviewed
May 27, 2026
Contributor
There was a problem hiding this comment.
Udf also extends VolatileExpression ?
yujun777
reviewed
May 27, 2026
| @@ -50,6 +51,7 @@ public class JavaUdaf extends AggregateFunction implements ExplicitlyCastableSig | |||
| private final FunctionSignature signature; | |||
| private final DataType intermediateType; | |||
| private final NullableMode nullableMode; | |||
| private final FunctionVolatility volatility; | |||
Contributor
There was a problem hiding this comment.
need add VolatileIdentity field for case FunctionValatility.VOLATILE
yujun777
reviewed
May 27, 2026
| @@ -49,6 +50,7 @@ public class JavaUdtf extends TableGeneratingFunction implements ExplicitlyCasta | |||
| private final Function.BinaryType binaryType; | |||
| private final FunctionSignature signature; | |||
| private final NullableMode nullableMode; | |||
| private final FunctionVolatility volatility; | |||
Contributor
There was a problem hiding this comment.
need add VolatileIdentity field for case FunctionValatility.VOLATILE
yujun777
reviewed
May 27, 2026
| @@ -50,6 +51,7 @@ public class PythonUdaf extends AggregateFunction implements ExplicitlyCastableS | |||
| private final FunctionSignature signature; | |||
| private final DataType intermediateType; | |||
| private final NullableMode nullableMode; | |||
| private final FunctionVolatility volatility; | |||
Contributor
There was a problem hiding this comment.
need add VolatileIdentity field for case FunctionValatility.VOLATILE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related PR: #62698
Problem Summary:
Add volatility metadata support for UDAF and UDTF definitions, so user-defined aggregate and table functions can preserve and expose their volatility semantics consistently with UDFs.