Resolving database upgrade issues with multiple database drivers#965
Resolving database upgrade issues with multiple database drivers#965
Conversation
…MySQL database auto-creation The core issue was a naming mismatch between: - FQN in SchemaRegistry: sys__user (double underscore) - Physical table name from ObjectSchema.create(): sys_user (single underscore) - Table name expected by auth adapter (SystemObjectName): sys_user Changes: 1. SchemaRegistry.getObject() - Added fallback to match by tableName property 2. ObjectQL.resolveObjectName() - Returns tableName instead of FQN 3. ObjectQLPlugin.syncRegisteredSchemas() - Uses tableName for DDL operations 4. SqlDriver.ensureDatabaseExists() - Extended for MySQL support 5. SqlDriver.createDatabase() - Added MySQL-specific database creation Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> Agent-Logs-Url: https://github.com/objectstack-ai/spec/sessions/79aaa91e-29db-42ee-8a83-2041d708a8f7
…MySQL config handling - Removed unnecessary `(... as any)` type casts since ServiceObject already includes tableName - Fixed syncRegisteredSchemas to pass original obj instead of spreading with name override - Fixed MySQL createDatabase to use destructuring instead of setting database: undefined Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> Agent-Logs-Url: https://github.com/objectstack-ai/spec/sessions/79aaa91e-29db-42ee-8a83-2041d708a8f7
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR addresses database upgrade/sync problems across multiple SQL drivers by aligning schema/table naming (FQN vs physical table name) and expanding automatic database creation beyond PostgreSQL.
Changes:
- Extend
SqlDriver.ensureDatabaseExists()/createDatabase()to handle MySQL and explicitly no-op for SQLite. - Add
tableName-based resolution inSchemaRegistry.getObject()and prefertableNamein ObjectQL driver operations / schema sync. - Add tests for tableName resolution + syncSchema argument behavior; update
CHANGELOG.md.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/plugins/driver-sql/src/sql-driver.ts | Add MySQL support to DB existence checks/DB creation; skip SQLite. |
| packages/plugins/driver-sql/src/sql-driver-schema.test.ts | Add a SQLite “ensureDatabaseExists is a no-op” test. |
| packages/objectql/src/registry.ts | Add tableName fallback in SchemaRegistry.getObject(). |
| packages/objectql/src/registry.test.ts | Add tests for resolving objects by tableName. |
| packages/objectql/src/plugin.ts | Use obj.tableName (when present) for syncSchema() DDL target. |
| packages/objectql/src/plugin.integration.test.ts | Assert syncSchema() is called with tableName rather than FQN. |
| packages/objectql/src/engine.ts | Prefer schema.tableName when resolving object names for driver operations. |
| CHANGELOG.md | Document fixes around naming mismatch and MySQL DB auto-creation. |
| // Use the physical table name (e.g., 'sys_user') for DDL operations | ||
| // instead of the FQN (e.g., 'sys__user'). ObjectSchema.create() | ||
| // auto-derives tableName as {namespace}_{name}. | ||
| const tableName = obj.tableName || obj.name; | ||
|
|
||
| try { | ||
| await driver.syncSchema(obj.name, obj); | ||
| await driver.syncSchema(tableName, obj); | ||
| synced++; |
There was a problem hiding this comment.
syncRegisteredSchemas() now passes tableName as the first argument to driver.syncSchema(), but SqlDriver.syncSchema() currently ignores that argument and uses schema.name when calling initObjects(). Since SchemaRegistry.registerObject() overwrites schema.name with the FQN (e.g. sys__user), SqlDriver will still create/alter the FQN table instead of the physical tableName (e.g. sys_user). To make this change effective, either update SqlDriver.syncSchema to use the object parameter as the physical table name, or pass a schema copy with name rewritten to tableName for drivers that derive DDL names from schema.name.
| const adminKnex = knex(adminConfig); | ||
| try { | ||
| await adminKnex.raw(`CREATE DATABASE "${dbName}"`); | ||
| if (this.isPostgres) { | ||
| await adminKnex.raw(`CREATE DATABASE "${dbName}"`); | ||
| } else if (this.isMysql) { | ||
| await adminKnex.raw(`CREATE DATABASE IF NOT EXISTS \`${dbName}\``); | ||
| } |
There was a problem hiding this comment.
createDatabase() interpolates dbName directly into raw SQL (CREATE DATABASE "${dbName}" / ``CREATE DATABASE IF NOT EXISTS `${dbName}```), but dbName comes from the connection string/object and is not validated/escaped. This can fail for empty/undefined names and also risks SQL injection if the connection config can be influenced. Prefer Knex identifier bindings (e.g., `??`) or a strict identifier allowlist validation before executing the raw statement, and explicitly error if `dbName` is missing.
| protected async ensureDatabaseExists() { | ||
| if (!this.isPostgres) return; | ||
| // SQLite auto-creates database files — no need to check | ||
| if (this.isSqlite) return; | ||
|
|
||
| // Only PostgreSQL and MySQL support programmatic database creation | ||
| if (!this.isPostgres && !this.isMysql) return; | ||
|
|
||
| try { | ||
| await this.knex.raw('SELECT 1'); | ||
| } catch (e: any) { | ||
| if (e.code === '3D000') { | ||
| // PostgreSQL: '3D000' = database does not exist | ||
| // MySQL: 'ER_BAD_DB_ERROR' (errno 1049) = unknown database | ||
| if ( | ||
| e.code === '3D000' || | ||
| e.code === 'ER_BAD_DB_ERROR' || | ||
| e.errno === 1049 | ||
| ) { | ||
| await this.createDatabase(); | ||
| } else { |
There was a problem hiding this comment.
The new MySQL handling in ensureDatabaseExists() (ER_BAD_DB_ERROR / errno 1049) and the MySQL branch in createDatabase() are not covered by tests. The added SQLite test still passes even with the old behavior (SQLite already skipped the PostgreSQL-only check), so it doesn’t validate the new MySQL upgrade fix. Consider adding unit-level tests that stub knex.raw() to throw MySQL/PG "unknown database" errors and assert createDatabase() is invoked (and not invoked for other error codes).
| for (const fqn of this.objectContributors.keys()) { | ||
| const resolved = this.resolveObject(fqn); | ||
| if (resolved?.tableName === name) { | ||
| return resolved; |
There was a problem hiding this comment.
SchemaRegistry.getObject() now adds a second full scan over objectContributors and calls resolveObject() for each entry to match by tableName. Since getObject() is used on every engine operation (resolve + driver selection), resolving by physical table name can become O(n) per call and scale poorly as the number of objects grows. Consider maintaining an index/map from tableName -> fqn (updated on register/unregister/reset) so tableName lookups are O(1) and don’t require merging every object during resolution.
| for (const fqn of this.objectContributors.keys()) { | |
| const resolved = this.resolveObject(fqn); | |
| if (resolved?.tableName === name) { | |
| return resolved; | |
| // | |
| // To avoid repeatedly merging objects on every iteration, we derive the | |
| // physical table name directly from the FQN using the convention: | |
| // tableName = `${namespace}_${name}` | |
| // and only resolve the object once we find a match. | |
| for (const fqn of this.objectContributors.keys()) { | |
| const { namespace, shortName } = parseFQN(fqn); | |
| const tableName = `${namespace}_${shortName}`; | |
| if (tableName === name) { | |
| return this.resolveObject(fqn); |
| private resolveObjectName(name: string): string { | ||
| const schema = SchemaRegistry.getObject(name); | ||
| if (schema) { | ||
| return schema.name; // FQN from registry (e.g., 'todo__task') | ||
| // Prefer the physical table name (e.g., 'sys_user') over the FQN | ||
| // (e.g., 'sys__user'). ObjectSchema.create() auto-derives tableName | ||
| // as {namespace}_{name} which matches the storage convention. | ||
| return schema.tableName || schema.name; | ||
| } |
There was a problem hiding this comment.
Changing resolveObjectName() to return schema.tableName makes the engine’s canonical object identifier become the physical table name for objects that define tableName. This affects more than driver I/O: hook/middleware matching uses context.object string equality, and action keys are built from the object name, so existing code registering hooks/actions against the FQN (e.g. sys__user) will no longer match when operations run as sys_user. Consider keeping FQN as the canonical engine object identifier (for hooks/actions/registry) and passing tableName only at the driver boundary (or carrying both values in OperationContext) to avoid a breaking semantic change.
Pull request created by AI Agent