type_variation_reference not implemented for User Defined Types in substrait-java

[asolimando]

Context and Background

In a downstream project using Substrait to send query plans between a query planner and the execution engine, the latter has multiple variants for some user defined data types (UDT), specialized for some specific use-cases.

The planner has enough context to suggest the right variant, and we would like to be able to set type_variation_reference when building the Substrait POJO.

At the moment we have to set this information at the proto level, since the io.substrait.type/Type::UserDefined class doesn't expose this information, which is a bit cumbersome.

Current Situation

• The Substrait proto clearly defines type_variation_reference as field #2 in the UserDefined message
• The proto comments even mention that UserDefined was created to "allow nullability and variations to be specified"
• But substrait-java completely ignores this field in both directions: proto -> POJO and POJO -> proto only read/write type_reference and nullability, respectively
• There are no references to "variation" or "typeVariationReference" anywhere in the codebase (at least for substrait-java, haven't checked for other languages)

What Changes are Proposed

• Adding a typeVariationReference (type int?) to https://github.com/substrait-io/substrait-java/blob/main/core/src/main/java/io/substrait/type/Type.java#L390
• Updating proto -> POJO and POJO -> proto accordingly