Skip to content

sql: Convert transforms to use repr types#35073

Draft
mgree wants to merge 2 commits intoMaterializeInc:mainfrom
mgree:repr-types-migrate-xforms
Draft

sql: Convert transforms to use repr types#35073
mgree wants to merge 2 commits intoMaterializeInc:mainfrom
mgree:repr-types-migrate-xforms

Conversation

@mgree
Copy link
Contributor

@mgree mgree commented Feb 18, 2026

WIP.

Motivation

#27239

Description

#35067 ensures that everyone working at the MIR level will be working with repr types either as canonical SQL types or proper repr types. This PR makes it so that (all!) transforms explicitly use repr types internally throughout, rather than moving in and out of (canonical) SQL types.

NB that some external structures---MRE, MSE, etc., AvailableCollections---will still use SQL types.

Verification

CI + nightly, with special attention during the PR process to determine when we can turn on soft panics for when we "fall back" to repr type comparisons.

@github-actions
Copy link

github-actions bot commented Feb 18, 2026

Pre-merge checklist

  • The PR title is descriptive and will make sense in the git log.
  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).

mgree added 2 commits February 18, 2026 17:36
@mgree
convert columnknowledge to use repr types
db8fe93
@mgree
convert eqprop, canonicalization, filter fusion, join impl, literal c…
02c81b9 
…onstraints, predicate pushdown, redundant join, MSE::reduce, and a number of helper functions
@mgree mgree force-pushed the repr-types-migrate-xforms branch from 84ba9fb to 02c81b9 Compare February 18, 2026 22:37
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 24, 2026
@ggevay
Switch optimizer transforms to use ReprRelationType instead of SqlRel…
a8dd6b2 
…ationType analysis

Partially inspired by MaterializeInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 24, 2026
@ggevay
Simplify DatumKnowledge to use ReprScalarType, inspired by Materializ…
d6be949 
…eInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 24, 2026
@ggevay
Switch transform files to use repr types (inspired by MaterializeInc#…
be43624 
…35073)

Add infrastructure:
- SqlRelationType::from_repr(&ReprRelationType) in src/repr/src/relation.rs
- MirRelationExpr::repr_typ() -> ReprRelationType in src/expr/src/relation.rs
- MirRelationExpr::repr_typ_with_input_types(&[ReprRelationType]) in src/expr/src/relation.rs

Switch the following transform files to work with ReprRelationType/ReprColumnType:
- column_knowledge.rs: optimize() now takes &[ReprColumnType], all .typ() sites
  produce Vec<ReprColumnType>, scalar.repr_typ() for output types.
- predicate_pushdown.rs: repr_typ() for nullable checks, extract_equal_or_both_null
  and helpers take &[ReprColumnType], uses repr_typ()/repr_typ_with_input_types
  throughout.
- fusion/filter.rs: uses input.repr_typ().column_types directly.
- join_implementation.rs: input_types is Vec<ReprRelationType> via repr_typ(),
  uses new_from_input_repr_types.
- redundant_join.rs: same pattern as join_implementation.rs.
- literal_constraints.rs: inp_typ switched to ReprRelationType, converts back to
  SqlRelationType only for MIR node construction (Constant, take_safely).

Convert back to SqlColumnType/SqlRelationType only where structurally required
(reduce() calls, MIR node construction).
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 24, 2026
@ggevay
Simplify DatumKnowledge to use ReprScalarType, inspired by Materializ…
eb35701 
…eInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 24, 2026
@ggevay
Switch transforms to use repr types (inspired by MaterializeInc#35073)
6c39115 
Add infrastructure:
- SqlRelationType::from_repr(&ReprRelationType) in src/repr/src/relation.rs
- MirRelationExpr::repr_typ() -> ReprRelationType in src/expr/src/relation.rs
- MirRelationExpr::repr_typ_with_input_types(&[ReprRelationType]) in src/expr/src/relation.rs

Switch the following transform files to work with ReprRelationType/ReprColumnType:
- column_knowledge.rs: optimize() now takes &[ReprColumnType], all .typ() sites
  produce Vec<ReprColumnType>, scalar.repr_typ() for output types.
- predicate_pushdown.rs: repr_typ() for nullable checks, extract_equal_or_both_null
  and helpers take &[ReprColumnType], uses repr_typ()/repr_typ_with_input_types
  throughout.
- fusion/filter.rs: uses input.repr_typ().column_types directly.
- join_implementation.rs: input_types is Vec<ReprRelationType> via repr_typ(),
  uses new_from_input_repr_types.
- redundant_join.rs: same pattern as join_implementation.rs.
- literal_constraints.rs: inp_typ switched to ReprRelationType, converts back to
  SqlRelationType only for MIR node construction (Constant, take_safely).

Convert back to SqlColumnType/SqlRelationType only where structurally required
(reduce() calls, MIR node construction).
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 24, 2026
@ggevay
Switch transforms to use repr types (inspired by MaterializeInc#35073)
3b77832 
Add infrastructure:
- SqlRelationType::from_repr(&ReprRelationType) in src/repr/src/relation.rs
- MirRelationExpr::repr_typ() -> ReprRelationType in src/expr/src/relation.rs
- MirRelationExpr::repr_typ_with_input_types(&[ReprRelationType]) in src/expr/src/relation.rs

Switch the following transform files to work with ReprRelationType/ReprColumnType:
- column_knowledge.rs: optimize() now takes &[ReprColumnType], all .typ() sites
  produce Vec<ReprColumnType>, scalar.repr_typ() for output types.
- predicate_pushdown.rs: repr_typ() for nullable checks, extract_equal_or_both_null
  and helpers take &[ReprColumnType], uses repr_typ()/repr_typ_with_input_types
  throughout.
- fusion/filter.rs: uses input.repr_typ().column_types directly.
- join_implementation.rs: input_types is Vec<ReprRelationType> via repr_typ(),
  uses new_from_input_repr_types.
- redundant_join.rs: same pattern as join_implementation.rs.
- literal_constraints.rs: inp_typ switched to ReprRelationType, converts back to
  SqlRelationType only for MIR node construction (Constant, take_safely).

Convert back to SqlColumnType/SqlRelationType only where structurally required
(reduce() calls, MIR node construction).
@ggevay ggevay mentioned this pull request Feb 25, 2026
Merged
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 25, 2026
@ggevay
Switch optimizer transforms to use ReprRelationType instead of SqlRel…
b631f66 
…ationType analysis

Partially inspired by MaterializeInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 25, 2026
@ggevay
Simplify DatumKnowledge to use ReprScalarType, inspired by Materializ…
5ab379b 
…eInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 25, 2026
@ggevay
Switch transforms to use repr types (inspired by MaterializeInc#35073)
e1b0606 
Add infrastructure:
- SqlRelationType::from_repr(&ReprRelationType) in src/repr/src/relation.rs
- MirRelationExpr::repr_typ() -> ReprRelationType in src/expr/src/relation.rs
- MirRelationExpr::repr_typ_with_input_types(&[ReprRelationType]) in src/expr/src/relation.rs

Switch the following transform files to work with ReprRelationType/ReprColumnType:
- column_knowledge.rs: optimize() now takes &[ReprColumnType], all .typ() sites
  produce Vec<ReprColumnType>, scalar.repr_typ() for output types.
- predicate_pushdown.rs: repr_typ() for nullable checks, extract_equal_or_both_null
  and helpers take &[ReprColumnType], uses repr_typ()/repr_typ_with_input_types
  throughout.
- fusion/filter.rs: uses input.repr_typ().column_types directly.
- join_implementation.rs: input_types is Vec<ReprRelationType> via repr_typ(),
  uses new_from_input_repr_types.
- redundant_join.rs: same pattern as join_implementation.rs.
- literal_constraints.rs: inp_typ switched to ReprRelationType, converts back to
  SqlRelationType only for MIR node construction (Constant, take_safely).

Convert back to SqlColumnType/SqlRelationType only where structurally required
(reduce() calls, MIR node construction).
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 26, 2026
@ggevay
Switch optimizer transforms to use ReprRelationType instead of SqlRel…
e2fae57 
…ationType analysis

Partially inspired by MaterializeInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 26, 2026
@ggevay
Simplify DatumKnowledge to use ReprScalarType, inspired by Materializ…
9261fc7 
…eInc#35073
ggevay added a commit to ggevay/materialize that referenced this pull request Feb 26, 2026
@ggevay
Switch transforms to use repr types (inspired by MaterializeInc#35073)
486ea0d 
Add infrastructure:
- SqlRelationType::from_repr(&ReprRelationType) in src/repr/src/relation.rs
- MirRelationExpr::repr_typ() -> ReprRelationType in src/expr/src/relation.rs
- MirRelationExpr::repr_typ_with_input_types(&[ReprRelationType]) in src/expr/src/relation.rs

Switch the following transform files to work with ReprRelationType/ReprColumnType:
- column_knowledge.rs: optimize() now takes &[ReprColumnType], all .typ() sites
  produce Vec<ReprColumnType>, scalar.repr_typ() for output types.
- predicate_pushdown.rs: repr_typ() for nullable checks, extract_equal_or_both_null
  and helpers take &[ReprColumnType], uses repr_typ()/repr_typ_with_input_types
  throughout.
- fusion/filter.rs: uses input.repr_typ().column_types directly.
- join_implementation.rs: input_types is Vec<ReprRelationType> via repr_typ(),
  uses new_from_input_repr_types.
- redundant_join.rs: same pattern as join_implementation.rs.
- literal_constraints.rs: inp_typ switched to ReprRelationType, converts back to
  SqlRelationType only for MIR node construction (Constant, take_safely).

Convert back to SqlColumnType/SqlRelationType only where structurally required
(reduce() calls, MIR node construction).
ggevay added a commit that referenced this pull request Feb 26, 2026
@ggevay @mgree
Repr types: Introduce repr types in all MIR code, and then change the…
52af3ba 
… MIR structs (#35084)

This is pushing on the repr types work stream. It's on top of
#35067, and stealing
stuff from #35073.

The "Add soft_panic_or_log! canaries for repr type mismatches" commit
turns some error logs into soft_panics, and the other commits attempt to
make CI not run into any of these panics.

Nightlies:
- https://buildkite.com/materialize/nightly/builds/15255 (stale)
- https://buildkite.com/materialize/nightly/builds/15259 (stale)
- https://buildkite.com/materialize/nightly/builds/15260 (stale)
- https://buildkite.com/materialize/nightly/builds/15296 (stale)
- https://buildkite.com/materialize/nightly/builds/15337 (stale)
- https://buildkite.com/materialize/nightly/builds/15346 (stale)
- https://buildkite.com/materialize/nightly/builds/15350 (stale)
- https://buildkite.com/materialize/nightly/builds/15351 (stale)
- https://buildkite.com/materialize/nightly/builds/15355 (limited to
RQGs + SLTs)
- https://buildkite.com/materialize/nightly/builds/15358 (stale)
- https://buildkite.com/materialize/nightly/builds/15359 (slightly
stale)
- https://buildkite.com/materialize/nightly/builds/15366

Release qualifications:
- https://buildkite.com/materialize/release-qualification/builds/1115
(slightly stale)

#35219 does some
naming/cleanup that we'll keep separate (since it involves touching the
`*Func` macros).

---------

Co-authored-by: Michael Greenberg <michael.greenberg@materialize.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mgree