forked from zstackio/zstack
-
Notifications
You must be signed in to change notification settings - Fork 0
<feature>[ai] ZSTAC-84025: schema + SDK actions for inference template auto-match #3891
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
zstack-robot-2
wants to merge
7
commits into
5.5.22
Choose a base branch
from
sync/ye.zou/feat/ZSTAC-84025-auto-match@@2
base: 5.5.22
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
fe02151
<feature>[ai]: schema add ModelVO.pipelineTag/manifestJson and ModelS…
AlanJager 0311282
<feature>[ai]: sdk AutoMatchModelServiceByModel action and UpdateMode…
AlanJager 4b352ab
<chore>[ai]: regenerate ApiHelper.groovy with autoMatchModelServiceBy…
AlanJager 23cb764
<chore>[sdk]: Update all sdks
AlanJager 165548d
<fix>[ai]: make schema strict-mode safe
AlanJager 1b617a7
<fix>[ai]: avoid second timestamp default
AlanJager 621c295
<fix>[ai]: drop redundant isDefault backfill
AlanJager File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| -- ZSTAC-84025: Add pipelineTag to ModelVO for inference template auto-matching | ||
| CALL ADD_COLUMN('ModelVO', 'pipelineTag', 'VARCHAR(64)', 1, NULL); | ||
|
|
||
| -- ZSTAC-84025: Add isDefault to ModelServiceRefVO to mark the default inference template per model | ||
| ALTER TABLE `zstack`.`ModelServiceRefVO` ADD COLUMN `isDefault` TINYINT(1) NOT NULL DEFAULT 0; | ||
|
|
||
| -- ZSTAC-84025-F2: Add manifestJson to ModelVO so Step 1 (file format) of the auto-match Matcher can | ||
| -- parse file_types/file_extensions from the manifest returned by the aios agent. | ||
| CALL ADD_COLUMN('ModelVO', 'manifestJson', 'TEXT', 1, NULL); | ||
|
|
||
| -- ZSTAC-84025: Add createDate/lastOpDate to ModelServiceRefVO so the auto-match Matcher can | ||
| -- pick the earliest isDefault=true row when DB has the rare 2+ defaults anomaly (Q5). | ||
| ALTER TABLE `zstack`.`ModelServiceRefVO` ADD COLUMN `lastOpDate` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP; | ||
| ALTER TABLE `zstack`.`ModelServiceRefVO` ADD COLUMN `createDate` TIMESTAMP NULL DEFAULT NULL; | ||
|
|
||
| DROP PROCEDURE IF EXISTS backfill_model_service_ref_create_date; | ||
| DELIMITER $$ | ||
| CREATE PROCEDURE backfill_model_service_ref_create_date() | ||
| BEGIN | ||
| UPDATE `zstack`.`ModelServiceRefVO` | ||
| SET `createDate` = CURRENT_TIMESTAMP | ||
| WHERE `createDate` IS NULL OR `createDate` = '0000-00-00 00:00:00'; | ||
| END $$ | ||
| DELIMITER ; | ||
| CALL backfill_model_service_ref_create_date(); | ||
| DROP PROCEDURE IF EXISTS backfill_model_service_ref_create_date; | ||
|
|
||
| -- Older MySQL/MariaDB versions allow only one TIMESTAMP column with CURRENT_TIMESTAMP | ||
| -- in DEFAULT or ON UPDATE. lastOpDate already uses it, so keep createDate non-zero | ||
| -- and let ModelServiceRefVO.@PrePersist populate the real creation time for new rows. | ||
| ALTER TABLE `zstack`.`ModelServiceRefVO` MODIFY COLUMN `createDate` TIMESTAMP NOT NULL DEFAULT '2000-01-01 00:00:00'; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
95 changes: 95 additions & 0 deletions
95
sdk/src/main/java/org/zstack/sdk/AutoMatchModelServiceByModelAction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| package org.zstack.sdk; | ||
|
|
||
| import java.util.HashMap; | ||
| import java.util.Map; | ||
| import org.zstack.sdk.*; | ||
|
|
||
| public class AutoMatchModelServiceByModelAction extends AbstractAction { | ||
|
|
||
| private static final HashMap<String, Parameter> parameterMap = new HashMap<>(); | ||
|
|
||
| private static final HashMap<String, Parameter> nonAPIParameterMap = new HashMap<>(); | ||
|
|
||
| public static class Result { | ||
| public ErrorCode error; | ||
| public org.zstack.sdk.AutoMatchModelServiceByModelResult value; | ||
|
|
||
| public Result throwExceptionIfError() { | ||
| if (error != null) { | ||
| throw new ApiException( | ||
| String.format("error[code: %s, description: %s, details: %s, globalErrorCode: %s]", error.code, error.description, error.details, error.globalErrorCode) | ||
| ); | ||
| } | ||
|
|
||
| return this; | ||
| } | ||
| } | ||
|
|
||
| @Param(required = true, nonempty = false, nullElements = false, emptyString = true, noTrim = false) | ||
| public java.lang.String modelUuid; | ||
|
|
||
| @Param(required = false) | ||
| public java.util.List systemTags; | ||
|
|
||
| @Param(required = false) | ||
| public java.util.List userTags; | ||
|
|
||
| @Param(required = false) | ||
| public String sessionId; | ||
|
|
||
| @Param(required = false) | ||
| public String accessKeyId; | ||
|
|
||
| @Param(required = false) | ||
| public String accessKeySecret; | ||
|
|
||
| @Param(required = false) | ||
| public String requestIp; | ||
|
|
||
|
|
||
| private Result makeResult(ApiResult res) { | ||
| Result ret = new Result(); | ||
| if (res.error != null) { | ||
| ret.error = res.error; | ||
| return ret; | ||
| } | ||
|
|
||
| org.zstack.sdk.AutoMatchModelServiceByModelResult value = res.getResult(org.zstack.sdk.AutoMatchModelServiceByModelResult.class); | ||
| ret.value = value == null ? new org.zstack.sdk.AutoMatchModelServiceByModelResult() : value; | ||
|
|
||
| return ret; | ||
| } | ||
|
|
||
| public Result call() { | ||
| ApiResult res = ZSClient.call(this); | ||
| return makeResult(res); | ||
| } | ||
|
|
||
| public void call(final Completion<Result> completion) { | ||
| ZSClient.call(this, new InternalCompletion() { | ||
| @Override | ||
| public void complete(ApiResult res) { | ||
| completion.complete(makeResult(res)); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| protected Map<String, Parameter> getParameterMap() { | ||
| return parameterMap; | ||
| } | ||
|
|
||
| protected Map<String, Parameter> getNonAPIParameterMap() { | ||
| return nonAPIParameterMap; | ||
| } | ||
|
|
||
| protected RestInfo getRestInfo() { | ||
| RestInfo info = new RestInfo(); | ||
| info.httpMethod = "GET"; | ||
| info.path = "/ai/models/{modelUuid}/auto-match-service"; | ||
| info.needSession = true; | ||
| info.needPoll = false; | ||
| info.parameterName = ""; | ||
| return info; | ||
| } | ||
|
|
||
| } |
31 changes: 31 additions & 0 deletions
31
sdk/src/main/java/org/zstack/sdk/AutoMatchModelServiceByModelResult.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| package org.zstack.sdk; | ||
|
|
||
| import org.zstack.sdk.MatchedStep; | ||
| import org.zstack.sdk.MatchEvidence; | ||
|
|
||
| public class AutoMatchModelServiceByModelResult { | ||
| public java.lang.String recommendedServiceUuid; | ||
| public void setRecommendedServiceUuid(java.lang.String recommendedServiceUuid) { | ||
| this.recommendedServiceUuid = recommendedServiceUuid; | ||
| } | ||
| public java.lang.String getRecommendedServiceUuid() { | ||
| return this.recommendedServiceUuid; | ||
| } | ||
|
|
||
| public MatchedStep matchedByStep; | ||
| public void setMatchedByStep(MatchedStep matchedByStep) { | ||
| this.matchedByStep = matchedByStep; | ||
| } | ||
| public MatchedStep getMatchedByStep() { | ||
| return this.matchedByStep; | ||
| } | ||
|
|
||
| public MatchEvidence evidence; | ||
| public void setEvidence(MatchEvidence evidence) { | ||
| this.evidence = evidence; | ||
| } | ||
| public MatchEvidence getEvidence() { | ||
| return this.evidence; | ||
| } | ||
|
|
||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| package org.zstack.sdk; | ||
|
|
||
|
|
||
|
|
||
| public class MatchEvidence { | ||
|
|
||
| public java.lang.String matchedRefUuid; | ||
| public void setMatchedRefUuid(java.lang.String matchedRefUuid) { | ||
| this.matchedRefUuid = matchedRefUuid; | ||
| } | ||
| public java.lang.String getMatchedRefUuid() { | ||
| return this.matchedRefUuid; | ||
| } | ||
|
|
||
| public java.lang.String matchedFileExtension; | ||
| public void setMatchedFileExtension(java.lang.String matchedFileExtension) { | ||
| this.matchedFileExtension = matchedFileExtension; | ||
| } | ||
| public java.lang.String getMatchedFileExtension() { | ||
| return this.matchedFileExtension; | ||
| } | ||
|
|
||
| public java.lang.String matchedPipelineTag; | ||
| public void setMatchedPipelineTag(java.lang.String matchedPipelineTag) { | ||
| this.matchedPipelineTag = matchedPipelineTag; | ||
| } | ||
| public java.lang.String getMatchedPipelineTag() { | ||
| return this.matchedPipelineTag; | ||
| } | ||
|
|
||
| public java.util.List detectedPipelineTags; | ||
| public void setDetectedPipelineTags(java.util.List detectedPipelineTags) { | ||
| this.detectedPipelineTags = detectedPipelineTags; | ||
| } | ||
| public java.util.List getDetectedPipelineTags() { | ||
| return this.detectedPipelineTags; | ||
| } | ||
|
|
||
| public java.lang.String warning; | ||
| public void setWarning(java.lang.String warning) { | ||
| this.warning = warning; | ||
| } | ||
| public java.lang.String getWarning() { | ||
| return this.warning; | ||
| } | ||
|
|
||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| package org.zstack.sdk; | ||
|
|
||
| public enum MatchedStep { | ||
| USER_PRESET, | ||
| FILE_FORMAT, | ||
| PIPELINE_TAG, | ||
| FALLBACK, | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: MatheMatrix/zstack
Length of output: 222
🏁 Script executed:
Repository: MatheMatrix/zstack
Length of output: 1304
🌐 Web query:
MySQL 5.6.5 TIMESTAMP CURRENT_TIMESTAMP multiple columns restriction history💡 Result:
In MySQL 5.6.5 and later, you can have multiple TIMESTAMP columns in the same table that use the special automatic clauses DEFAULT CURRENT_TIMESTAMP and/or ON UPDATE CURRENT_TIMESTAMP; this restriction (only one TIMESTAMP column per table could have those automatic properties) was lifted in MySQL 5.6.5 [1][2]. What the “restriction history” looks like - Before 5.6.5, at most one TIMESTAMP column per table could be automatically initialized or updated to the current date and time; attempting to use CURRENT_TIMESTAMP defaults/updates on more than one TIMESTAMP column would fail (or be disallowed by the server/UI), depending on context [1][3]. - In MySQL 5.6.5 (released 2012-04-10 per 5.6 release notes), “This restriction has been lifted. Any TIMESTAMP column definition can have any combination of DEFAULT CURRENT_TIMESTAMP and ON UPDATE CURRENT_TIMESTAMP clauses.” The same changelog text also notes that these clauses can now be used with DATETIME definitions [1]. - MySQL’s current documentation (the referenced “Automatic Initialization and Updating for TIMESTAMP and DATETIME” section) describes the modern rules: TIMESTAMP/DATETIME columns can be specified to have current-timestamp behavior for default and/or auto-update, and different columns can have different combinations [2]. Practical notes for your specific query - If you’re on MySQL 5.6.5+: you can define multiple columns like: - col1 TIMESTAMP DEFAULT CURRENT_TIMESTAMP - col2 TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP and MySQL will allow it [2][1]. - The server behavior is also affected by explicit_defaults_for_timestamp (documented): when it’s disabled, MySQL may implicitly apply DEFAULT CURRENT_TIMESTAMP and ON UPDATE CURRENT_TIMESTAMP to the first TIMESTAMP column if neither is specified explicitly [2]. That’s not a “5.6.5 multiple columns restriction” anymore, but it can affect which columns end up with automatic defaults.
Citations:
🏁 Script executed:
Repository: MatheMatrix/zstack
Length of output: 2085
createDate的哨兵默认值'2000-01-01 00:00:00'基于已过时的 MySQL 限制,应改为CURRENT_TIMESTAMP第 28–30 行的注释解释了不能使用
DEFAULT CURRENT_TIMESTAMP的原因——"旧版 MySQL/MariaDB 每张表只允许一列 TIMESTAMP 使用 CURRENT_TIMESTAMP"。然而,MySQL 5.6.5(2012 年 4 月发布)已经取消了该限制,允许任意数量的 TIMESTAMP 列使用DEFAULT CURRENT_TIMESTAMP与ON UPDATE CURRENT_TIMESTAMP的任意组合。ZStack 的 pom.xml 依赖 mysql-connector-j 8.2.0,明确表示项目针对 MySQL 8.x 开发和测试。这说明没有对 MySQL 5.6.5 之前版本的支持需求。因此,当前的
DEFAULT '2000-01-01 00:00:00'绕过方案依据的前提已不成立。该哨兵值带来的风险是:如果
ModelServiceRefVO.@PrePersist因故障、异常或被绕过而无法执行,任何未显式设置createDate的新 INSERT 都会记录一个在 2000 年的错误时间戳,导致该行数据的时间信息彻底失真,难以在后续业务逻辑(如"挑选最早的 isDefault=true 行")中正确处理。🛠️ 建议修复
📝 Committable suggestion
🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comment from ye.zou:
这条建议我不采纳,当前改成非 CURRENT_TIMESTAMP 是为了兼容本分支实际验证环境里的 MySQL 5.7 行为:
explicit_defaults_for_timestamp=OFF时,lastOpDate已带ON UPDATE CURRENT_TIMESTAMP,再把createDate改为DEFAULT CURRENT_TIMESTAMP会直接触发本次失败日志里的错误:Error Code : 1293 Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause另外 Java 侧
ModelServiceRefVO.@PrePersist已在新行写入真实创建时间,2000-01-01 00:00:00只是为了让历史/绕过 JPA 的插入不违反 NOT NULL,并不作为业务排序依赖。这个 schema 改动已经在本地 PR docker 跑过AutoMatchModelServiceCase,可以通过该迁移阶段。There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zstack-robot-2,感谢 ye.zou 的详细解释,完全理解了。在 MySQL 5.7 且explicit_defaults_for_timestamp=OFF的环境下,Error 1293 确实会重新生效——即便 MySQL 5.6.5 在标准模式下解除了该限制,该模式下的行为仍然是每表只允许一列 TIMESTAMP 使用 CURRENT_TIMESTAMP。lastOpDate已占用该位,createDate使用'2000-01-01 00:00:00'作为 NOT NULL 占位是正确的处理方式,撤回该建议。✏️ Learnings added
🧠 Learnings used