Skip to content

Data declaration extensions#216

Open
comnik wants to merge 9 commits intomainfrom
ncg-load-cdc
Open

Data declaration extensions#216
comnik wants to merge 9 commits intomainfrom
ncg-load-cdc

Conversation

@comnik
Copy link
Collaborator

@comnik comnik commented Feb 25, 2026

RelEDB → EDB, dropping the "Rel" prefix which we've been using to indicate constructs that will go away once we drop Rel support. This is not the case for EDB references.

CSVColumn → GNFColumn, generalizing to make column declarations reusable between CSV and Iceberg data. GNFColumn can represent nested paths like /:append/:METADATA$KEY.

Generalize Snapshot to support multiple mappings. The snapshot implementation on the engine is more awkward than I had hoped for, and this allows us to at least do it more efficiently.

@comnik comnik self-assigned this Feb 25, 2026
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude says: the grammar's gnf_column_path rule indexes into a Sequence[String] with $$[0], which requires the sequence/list indexing support added in that file.

@comnik comnik marked this pull request as ready for review February 25, 2026 21:41
@@ -6,9 +6,9 @@
;; Define the EDBs we're going to be referring to.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't comment on it, but I'm assuming tests/lqp/snapshot.bin should be removed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nice catch, yes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

return builtin.list_sort(result)



Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

| "[" STRING* "]"
construct: $$ = $2
deconstruct if builtin.length($$) != 1:
$2: Sequence[String] = $$
Copy link
Contributor

@minsungc minsungc Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there semantics for when gnf_column_path is a sequence of length 1? Is it something we can explicitly disallow?

Copy link
Contributor

@minsungc minsungc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick clarifying question otherwise LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants