proto: add proto converter reference to PhysicalExtensionCodec trait

[jayshrivastava]

Which issue does this PR close?

• Closes add PhysicalProtoConverterExtension to PhysicalExtensionCodec #21056

Rationale for this change

Custom ExecutionPlan nodes that store PhysicalExpr fields cannot participate in expression deduplication during serialization because PhysicalExtensionCodec::try_encode/try_decode lack access to the PhysicalProtoConverterExtension.

What changes are included in this PR?

• Added proto_converter: &dyn PhysicalProtoConverterExtension parameter
  to PhysicalExtensionCodec::try_decode and try_encode
• Updated all implementations: DefaultPhysicalExtensionCodec,
  ComposedPhysicalExtensionCodec, ForeignPhysicalExtensionCodec (FFI),
  and all examples
• Updated call sites in PhysicalPlanNode serialization/deserialization
• FFI bridge passes DefaultPhysicalProtoConverter as a fallback

Are these changes tested?

Yes — added test_custom_node_with_dynamic_filter_dedup_roundtrip which
verifies that a DynamicFilterPhysicalExpr shared between a FilterExec
and a custom ExecutionPlan node preserves its shared inner state after a
roundtrip through DeduplicatingProtoConverter.

Are there any user-facing changes?

Breaking API change: PhysicalExtensionCodec::try_decode and try_encode
now take an additional &dyn PhysicalProtoConverterExtension parameter.
All custom codec implementations must be updated.

[github-actions]

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion-ffi v53.1.0 (current)
       Built [  57.638s] (current)
     Parsing datafusion-ffi v53.1.0 (current)
      Parsed [   0.056s] (current)
    Building datafusion-ffi v53.1.0 (baseline)
       Built [  56.177s] (baseline)
     Parsing datafusion-ffi v53.1.0 (baseline)
      Parsed [   0.056s] (baseline)
    Checking datafusion-ffi v53.1.0 -> v53.1.0 (no change; assume patch)
     Checked [   0.223s] 222 checks: 222 pass, 30 skip
     Summary no semver update required
    Finished [ 116.201s] datafusion-ffi
    Building datafusion-proto v53.1.0 (current)
       Built [  53.301s] (current)
     Parsing datafusion-proto v53.1.0 (current)
      Parsed [   0.133s] (current)
    Building datafusion-proto v53.1.0 (baseline)
       Built [  52.749s] (baseline)
     Parsing datafusion-proto v53.1.0 (baseline)
      Parsed [   0.133s] (baseline)
    Checking datafusion-proto v53.1.0 -> v53.1.0 (no change; assume patch)
     Checked [   1.572s] 222 checks: 221 pass, 1 fail, 0 warn, 30 skip

--- failure trait_method_parameter_count_changed: pub trait method parameter count changed ---

Description:
A trait method now takes a different number of parameters.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-item-signature
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.47.0/src/lints/trait_method_parameter_count_changed.ron

Failed in:
  PhysicalExtensionCodec::try_decode now takes 4 instead of 3 parameters, in file /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:3705
  PhysicalExtensionCodec::try_encode now takes 3 instead of 2 parameters, in file /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:3713

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [ 110.948s] datafusion-proto

[adriangb]

Currently this is a breaking change. Would you be okay holding it until we release DF 54? Either way if we keep a breaking change (or add new recommended methods) we should add a note the upgrade guide.

[adriangb]

We could add new methods with the param that default to the old methods to avoid a breaking change. Did you consider that as an option?

[adriangb]

Naming these converter would match try_from_physical_plan_with_converter and make the arg name shorter.

[adriangb]

@timsaucer not sure if you have any suggestions but I think this is fine. Generally we have not made any effort to make dynamic filters / expr deduplication work with plans that go through FFI.