Skip to content

[Bug] [spark] Align spark.paimon.branch option with explicit branch syntax for chain tables #8015

@juntaozhang

Description

@juntaozhang

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

master

Compute Engine

spark

Minimal reproduce step

CREATE DATABASE IF NOT EXISTS my_db1;
USE spark_catalog.my_db1;
DROP TABLE IF EXISTS `my_db1`.`chain_test`;
CREATE TABLE IF NOT EXISTS `chain_test` (
  `t1` BIGINT,
  `t2` BIGINT,
  `t3` STRING
) PARTITIONED BY (`dt` STRING)
TBLPROPERTIES (
  'bucket-key' = 't1',
  'primary-key' = 'dt,t1',
  'partition.timestamp-pattern' = '$dt',
  'partition.timestamp-formatter' = 'yyyyMMdd',
  'chain-table.enabled' = 'true',
  'bucket' = '1',
  'merge-engine' = 'deduplicate',
  'sequence.field' = 't2'
);

SET spark.paimon.branch = delta;
INSERT OVERWRITE TABLE `my_db1`.`chain_test` PARTITION (dt = '20250810') VALUES (1, 2, '1');
INSERT OVERWRITE TABLE `my_db1`.`chain_test$branch_delta` PARTITION (dt = '20250811') VALUES (2, 2, '1');
SELECT * FROM `my_db1`.`chain_test$snapshots`;

RESET spark.paimon.branch;
//SELECT * FROM `my_db1`.`chain_test$branch_snapshot` WHERE dt = '20250811';

//SET spark.paimon.branch = snapshot;
SELECT * FROM `my_db1`.`chain_test` where dt = '20250811';

DROP TABLE IF EXISTS `my_db1`.`chain_test`;

What doesn't meet your expectations?

① ② should be empty.
but currently ② is not empty.

Anything else?

#5460 deprecated the branch option.
#7084 discussed the priority between Paimon dynamic options and table identifiers.

Before the branch option is fully removed, we should unify their behaviors first.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions