feat(codegen): parse DDL with sqlparser, drop hand-rolled scanner (#38)#111
Merged
Conversation
Replace src/codegen/parser.rs's uppercase-and-split scanner with the sqlparser crate (0.50). parse_sql_schema now walks Statement::CreateTable, trying PostgreSQL → SQLite → generic dialects. Public IR (ParsedSchema/TableDef/ColumnDef) and the parse_sql_schema / parse_schema_file signatures are unchanged, so overlay/query consumers are unaffected. split_respecting_parens, parse_create_table and parse_column_def are deleted. Fixes the documented misclassifications: schema-qualified names (bare table identifier extracted), quoted identifiers with whitespace, column- and table-level CHECK clauses (ignored, not split into bogus columns), GENERATED columns, and semicolons inside comments. Invalid SQL is now a hard error instead of a silent empty schema. 6 original parser tests retained + 6 new acceptance tests (schema-qualified, quoted-whitespace, CHECK, GENERATED, semicolon-in-comment, invalid-SQL). Suite: 113 lib + 9 integration green. Closes #38. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
🔍 Hypatia Security ScanFindings: 20 issues detected
View findings[
{
"reason": "Required file missing",
"type": "missing",
"file": "SECURITY.md",
"action": "create",
"rule_module": "root_hygiene",
"severity": "high"
},
{
"reason": "Issue in quality.yml",
"type": "missing_workflow",
"file": "quality.yml",
"action": "create",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Issue in security-policy.yml",
"type": "missing_workflow",
"file": "security-policy.yml",
"action": "create",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Action actions/checkout@v4 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action Swatinem/rust-cache@v2 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action actions/checkout@v4 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action dtolnay/rust-toolchain@master needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Action Swatinem/rust-cache@v2 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Required file missing (condition: public_repo)",
"type": "missing_requirement",
"file": "SECURITY.md",
"action": "create",
"rule_module": "cicd_rules",
"severity": "high"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
🔍 Hypatia Security ScanFindings: 20 issues detected
View findings[
{
"reason": "Required file missing",
"type": "missing",
"file": "SECURITY.md",
"action": "create",
"rule_module": "root_hygiene",
"severity": "high"
},
{
"reason": "Issue in quality.yml",
"type": "missing_workflow",
"file": "quality.yml",
"action": "create",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Issue in security-policy.yml",
"type": "missing_workflow",
"file": "security-policy.yml",
"action": "create",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Action actions/checkout@v4 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action Swatinem/rust-cache@v2 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action actions/checkout@v4 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Action dtolnay/rust-toolchain@master needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Action Swatinem/rust-cache@v2 needs attention",
"type": "unpinned_action",
"file": "rust-ci.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Required file missing (condition: public_repo)",
"type": "missing_requirement",
"file": "SECURITY.md",
"action": "create",
"rule_module": "cicd_rules",
"severity": "high"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #38 (V-L2-A1) — replace the hand-rolled SQL DDL scanner with
sqlparser.What changed
sqlparser = "0.50"dependency added.src/codegen/parser.rs::parse_sql_schemanow parses withsqlparser, trying PostgreSQL → SQLite → generic dialects (coversSERIAL,AUTOINCREMENT, generated columns), and walksStatement::CreateTableinto the IR.ParsedSchema/TableDef/ColumnDef) and theparse_sql_schema/parse_schema_filesignatures are unchanged — overlay and query codegen consume the same shape, no ripple.split_respecting_parens,parse_create_table,parse_column_def.Defects fixed (all were silent corruption before)
analytics.events) → bare table identifier."audit log","user name").CHECK (...)— ignored cleanly instead of comma-split into bogus columns.GENERATED ALWAYS AS (...) STOREDcolumns.--comments no longer break statement boundaries.Acceptance
sqlparserdependency addedSuite: 113 lib + 9 integration green. The single red is the pre-existing failing-by-design
provenance_fork_test(#104, fixed by open PR #109) — onmain, unrelated to this branch.🤖 Generated with Claude Code