Skip to content

[Bug] Doris 配置 nessie rest catalog 连接 iceberg ,可能查看库和表,但是无法查询表数据 #61191

@easyinplay

Description

@easyinplay

Search before asking

  • I had searched in the issues and found no similar issues.

Version

Doris 4.0.3

What's Wrong?

CREATE CATALOG iceberg PROPERTIES (
  'type'='iceberg',
  'iceberg.catalog.type'='rest',
  'iceberg.rest.uri'='http://192.168.10.61:19120/iceberg/main',
  'warehouse'='nessie-warehouse',
  'iceberg.rest.security.type'='none',
  -- 'iceberg.rest.vended-credentials-enabled'='true',
  'hdfs.authentication.type'='simple',
  -- 'hdfs.impersonation.enabled'='true',
  'hadoop.username'='hadoop',
  'hadoop.config.resources'='hadoop61/core-site.xml,hadoop61/hdfs-site.xml',
  -- 'hadoop.conf.dir' = '/opt/module/doris-4.0.0-bin-x64/fe/plugins/hadoop_conf',
  'fs.defaultFS'='hdfs://venndata61:9000'
);

--正常
show databases from jdclean;
--正常
show tables from iceberg.jdclean;
--报错
[2026-03-10 21:46:14] jdclean> select *
from iceberg.jdclean.tb_clean_jd_image_m

[2026-03-10 21:46:14] [HY000][1105] errCode = 2, detailMessage = IllegalArgumentException: java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to read table metadata from hdfs://192.168.10.61:9000/iceberg/warehouse/jdclean/tb_clean_jd_image_m-6489e2f7e41e4d03976ebc56fb593c4f/metadata/00014-58d752fc-c028-4060-acf7-187b5d3740de.metadata.json

Image Image Image

nessie的服务看起来是正常的:

hadoop@venndata61:~$ curl "http://192.168.10.61:19120/iceberg/main/v1/config"
{
  "defaults" : {
    "rest-metrics-reporting-enabled" : "false",
    "warehouse" : "hdfs://venndata61:9000/user/iceberg/warehouse",
    "rest-page-size" : "200",
    "fs.defaultFS" : "hdfs://venndata61:9000",
    "fs.impl" : "org.apache.hadoop.hdfs.DistributedFileSystem",
    "fs.file.impl" : "org.apache.hadoop.fs.LocalFileSystem",
    "io-impl" : "org.apache.iceberg.hadoop.HadoopFileIO",
    "hadoop.security.authentication" : "simple",
    "fs.hdfs.impl" : "org.apache.hadoop.hdfs.DistributedFileSystem",
    "prefix" : "main"
  },
  "overrides" : {
    "nessie.core-base-uri" : "http://192.168.10.61:19120/api/",
    "nessie.catalog-base-uri" : "http://192.168.10.61:19120/catalog/v1/",
    "nessie.iceberg-base-uri" : "http://192.168.10.61:19120/iceberg/",
    "uri" : "http://192.168.10.61:19120/iceberg/",
    "commit.retry.num-retries" : "10",
    "logging.level" : "DEBUG",
    "trace.enabled" : "true",
    "commit.retry.min-wait-ms" : "1000",
    "fs.defaultFS" : "hdfs://venndata61:9000",
    "fs.impl" : "org.apache.hadoop.hdfs.DistributedFileSystem",
    "hadoop.security.authentication" : "simple",
    "io-impl" : "org.apache.iceberg.hadoop.HadoopFileIO",
    "write.format.default" : "parquet",
    "table.metadata.cache-enabled" : "false",
    "parquet.row-group-size-bytes" : "134217728",
    "fs.hdfs.impl" : "org.apache.hadoop.hdfs.DistributedFileSystem",
    "parquet.page-size-bytes" : "1048576",
    "nessie.is-nessie-catalog" : "true",
    "nessie.prefix-pattern" : "{ref}|{warehouse}",
    "nessie.default-branch.name" : "main"
  },
  "endpoints" : [ "GET /v1/{prefix}/namespaces", "GET /v1/{prefix}/namespaces/{namespace}", "HEAD /v1/{prefix}/namespaces/{namespace}", "POST /v1/{prefix}/namespaces", "POST /v1/{prefix}/namespaces/{namespace}/properties", "DELETE /v1/{prefix}/namespaces/{namespace}", "POST /v1/{prefix}/transactions/commit", "GET /v1/{prefix}/namespaces/{namespace}/tables", "GET /v1/{prefix}/namespaces/{namespace}/tables/{table}", "HEAD /v1/{prefix}/namespaces/{namespace}/tables/{table}", "POST /v1/{prefix}/namespaces/{namespace}/tables", "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}", "DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table}", "POST /v1/{prefix}/tables/rename", "POST /v1/{prefix}/namespaces/{namespace}/register", "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics", "GET /v1/{prefix}/namespaces/{namespace}/views", "GET /v1/{prefix}/namespaces/{namespace}/views/{view}", "HEAD /v1/{prefix}/namespaces/{namespace}/views/{view}", "POST /v1/{prefix}/namespaces/{namespace}/views", "POST /v1/{prefix}/namespaces/{namespace}/views/{view}", "DELETE /v1/{prefix}/namespaces/{namespace}/views/{view}", "POST /v1/{prefix}/views/rename" ]

What You Expected?

可以正常查看表中数据

How to Reproduce?

目前doris访问其他,比如hive sqlserver之类是正常的;其他引擎如trino spark 访问 nessie也是正常的(它们使用nesssie api/v2),但是doris只支持 nessie rest api。根据上面描述,curl "http://192.168.10.61:19120/iceberg/main/v1/config" 我的nessie的rest服务应该是正常的,目前doris 可以读取到库和表,但是无法读取表中数据。

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions