[PECOBLR-1746] Implementing support for listing procedures by msrathore-db · Pull Request #1238 · databricks/databricks-jdbc

msrathore-db · 2026-02-27T16:44:11Z

Description

Implements getProcedures and getProcedureColumns in DatabricksDatabaseMetaData by querying information_schema.routines and information_schema.parameters via SQL.

Unlike other metadata operations that use SHOW commands or Thrift RPCs, these use direct SQL SELECT queries against information_schema views. This works for both Thrift and SEA transports.

getProcedures:

Queries information_schema.routines filtered by routine_type = 'PROCEDURE'
Returns 9-column JDBC-spec result set (PROCEDURE_CAT, PROCEDURE_SCHEM, PROCEDURE_NAME, reserved x3, REMARKS, PROCEDURE_TYPE, SPECIFIC_NAME)
PROCEDURE_TYPE is always procedureNoResult (1)

getProcedureColumns:

Queries information_schema.parameters JOINed with information_schema.routines to filter for procedures
Returns 20-column JDBC-spec result set with parameter metadata
Maps parameter_mode (IN/OUT/INOUT) to JDBC COLUMN_TYPE constants (1/4/2)
Maps Databricks type names to java.sql.Types codes via existing getCode()

Catalog resolution:

NULL catalog → queries system.information_schema.routines (cross-catalog)
Specific catalog → queries <catalog>.information_schema.routines
Empty string → returns empty result set

Shared SQL builders in CommandConstants eliminate duplication between SDK and Thrift clients.

Testing

Unit tests (DatabricksMetadataSdkClientTest):

4 parameterized tests for listProcedures SQL generation (catalog+schema+name, null schema, null name, null catalog)
3 parameterized tests for listProcedureColumns SQL generation (all filters, partial filters, all nulls)

Integration test (MetadataIntegrationTests#testGetProceduresAndProcedureColumns):

Creates a procedure jdbc_test_compute_area(x DOUBLE, y DOUBLE, OUT area DOUBLE) with COMMENT
Verifies getProcedures returns correct name, schema, catalog, remarks, type
Verifies getProcedureColumns returns 3 params with correct COLUMN_TYPE (IN=1, OUT=4), DATA_TYPE (DOUBLE=8), ordinal positions
Tests column name filtering
Cleans up procedure after test
WireMock stubs recorded for REPLAY mode

Existing tests: All 248 DatabricksDatabaseMetaDataTest + 51 DatabricksMetadataSdkClientTest pass.

Additional Notes to the Reviewer

information_schema.parameters is used (not routine_columns) because routine_columns contains table-valued function output columns, not procedure parameters.
NULLABLE is always procedureNullableUnknown (2) since the server does not track parameter nullability.
When catalog is NULL, system.information_schema is queried which requires system table access permissions. If the user lacks this permission, the driver returns an empty result set. This will be addressed server-side in a future release.

gopalldb · 2026-03-11T06:35:25Z

+      sql.append(" AND routine_schema LIKE '").append(schemaPattern).append("'");
+    }
+    if (procedureNamePattern != null) {
+      sql.append(" AND routine_name LIKE '").append(procedureNamePattern).append("'");


Is it possible to use parameterized statements here and provide user provided values as parameters? This will prevent SQL injection issues

Done. Now using ? placeholders with ImmutableSqlParameter for server-side binding.

gopalldb · 2026-03-11T07:18:13Z

+  }
+
+  private static String getCatalogPrefix(String catalog) {
+    return (catalog == null) ? "system" : "`" + catalog + "`";


what if catalogName already contains a backtick character? Should we escape that?

Catalog names cannot contain backticks, so the user input would be invalid in that case. No escaping needed.

gopalldb · 2026-03-11T07:38:05Z

+      return (short) procedureColumnUnknown;
+    }
+    switch (parameterMode.toUpperCase()) {
+      case "IN":


declare as constants

Done. Extracted PARAM_MODE_IN, PARAM_MODE_OUT, PARAM_MODE_INOUT, and IS_RESULT_YES as constants.

gopalldb · 2026-03-11T07:38:40Z

+    try {
+      Object val = resultSet.getObject(columnName);
+      return val != null ? val.toString() : null;
+    } catch (SQLException e) {


add logging

Done. Added debug logging to getRowsForProcedures, getRowsForProcedureColumns, and mapParameterModeToColumnType.

gopalldb · 2026-03-11T07:39:00Z

+      if (val == null) return null;
+      if (val instanceof Number) return ((Number) val).intValue();
+      return Integer.parseInt(val.toString());
+    } catch (SQLException | NumberFormatException e) {


add logging

Done — see above.

gopalldb · 2026-03-11T07:39:07Z

+      if (val == null) return null;
+      if (val instanceof Number) return ((Number) val).shortValue();
+      return Short.parseShort(val.toString());
+    } catch (SQLException | NumberFormatException e) {


add logging

Done — see above.

gopalldb · 2026-03-11T08:16:02Z

+      Integer charMaxLength = getIntOrNull(resultSet, "character_maximum_length");
+      Integer charOctetLength = getIntOrNull(resultSet, "character_octet_length");
+      row.add(numericPrecision != null ? numericPrecision : charMaxLength); // PRECISION
+      row.add(charOctetLength != null ? charOctetLength : numericPrecision); // LENGTH


but numericPrecision can be null also

Correct — if both numericPrecision and charMaxLength are null, COLUMN_SIZE returns null, which is the expected behavior per JDBC spec for types where precision is not applicable.

gopalldb · 2026-03-11T08:16:20Z

+      Integer numericPrecision = getIntOrNull(resultSet, "numeric_precision");
+      Integer charMaxLength = getIntOrNull(resultSet, "character_maximum_length");
+      Integer charOctetLength = getIntOrNull(resultSet, "character_octet_length");
+      row.add(numericPrecision != null ? numericPrecision : charMaxLength); // PRECISION


charMaxLength itself can be null

Same as above — null fallback is intentional. Returns null when neither numeric precision nor character length applies to the type.

gopalldb · 2026-03-11T08:17:10Z

+      Integer charMaxLength = getIntOrNull(resultSet, "character_maximum_length");
+      Integer charOctetLength = getIntOrNull(resultSet, "character_octet_length");
+      row.add(numericPrecision != null ? numericPrecision : charMaxLength); // PRECISION
+      row.add(charOctetLength != null ? charOctetLength : numericPrecision); // LENGTH


why is length being fallback to precision?

Fixed. BUFFER_LENGTH now uses charOctetLength directly (null for non-character types) instead of falling back to numericPrecision.

gopalldb · 2026-03-11T08:18:16Z

+        Arguments.of(
+            "SELECT p.specific_catalog, p.specific_schema, p.specific_name,"
+                + " p.parameter_name, p.parameter_mode, p.is_result,"
+                + " p.data_type, p.full_data_type,"


where is full_data_type used?

Removed. It was unused — numeric_precision and numeric_scale are available as separate columns so full_data_type is not needed.

…SQL queries Implement JDBC-compliant getProcedures and getProcedureColumns by querying information_schema.ROUTINES and information_schema.parameters via SQL. This approach works for both thrift and SEA transports without using thrift RPC. - getProcedures: queries ROUTINES filtered by routine_type='PROCEDURE' - getProcedureColumns: queries parameters joined with ROUTINES for procedures - Null catalog uses system.information_schema; specific catalog uses <catalog>.information_schema - Shared SQL builders in CommandConstants to avoid duplication across SDK/Thrift clients - Maps parameter_mode (IN/OUT/INOUT) to JDBC COLUMN_TYPE constants - Maps Databricks type names to java.sql.Types via existing getCode() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…line helpers - Extract table names, column lists, and filter into named constants in CommandConstants - Remove getProcedureColumnPrecision/getProcedureColumnLength helpers with unused dataType param - Inline precision/length logic directly in getRowsForProcedureColumns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ocedureColumns Unit tests (DatabricksMetadataSdkClientTest): - testListProcedures: parameterized tests for SQL generation with various catalog/schema/name combinations including null catalog (system prefix) - testListProcedureColumns: parameterized tests for SQL generation with JOIN to routines table, all filter combinations Integration test (MetadataIntegrationTests): - testGetProceduresAndProcedureColumns: creates a test procedure with IN and OUT params, verifies getProcedures returns correct metadata (name, remarks, type), verifies getProcedureColumns returns correct parameter info (column types IN=1/OUT=4, data types, ordinal positions), tests column name filtering, cleans up procedure after test Also renames ROUTINES -> routines in CommandConstants for consistency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…I changes Add GET_PROCEDURES and GET_PROCEDURE_COLUMNS to MetadataOperationType enum. Update getResultSet/executeStatement calls to pass the new required MetadataOperationType parameter after upstream API signature change. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

String.format treats the single quote before %s as a flag character, causing UnknownFormatConversion when patterns like '%' are passed. Replace String.format with StringBuilder.append for SQL LIKE clauses. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

… '%' The LOGGER.error(e, "message {}", e) call uses String.format internally. When the exception message contains SQL with LIKE '%', the '%' is interpreted as a format specifier causing UnknownFormatConversionException. Use plain log message without exception interpolation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…m values Add GET_PROCEDURES and GET_PROCEDURE_COLUMNS to the enum count assertion and parameterized header value tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Replace string concatenation with ? placeholders and ImmutableSqlParameter for all LIKE clause values in buildProceduresSQL and buildProcedureColumnsSQL. This prevents SQL injection via user-provided schema/procedure/column patterns. The build methods now accept a Map<Integer, ImmutableSqlParameter> that they populate with the parameter bindings. Callers pass this map through to executeStatement instead of an empty HashMap. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…o prevent injection Use ? placeholders with ImmutableSqlParameter for LIKE clause values in buildProceduresSQL and buildProcedureColumnsSQL. Parameters are passed to the server via executeStatement's parameter map for server-side binding, the same mechanism used by PreparedStatement. Also resolves merge conflict with upstream rename of DatabricksMetadataSdkClient to DatabricksMetadataQueryClient. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Extract string constants for parameter mode values (IN/OUT/INOUT/YES) - Add debug logging to getRowsForProcedures, getRowsForProcedureColumns, and mapParameterModeToColumnType - Fix BUFFER_LENGTH: use charOctetLength directly instead of falling back to numericPrecision which is semantically wrong - Remove unused full_data_type from PARAMETERS_SELECT_COLUMNS Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ries The SQL now uses ? placeholders with server-side parameter binding, so the WireMock stubs need to match the new request format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

… and fix column count The PARAMETERS_SELECT_COLUMNS no longer includes full_data_type (removed per PR review), so unit test expected SQL strings and column count mock need to match. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

gopalldb · 2026-03-16T05:41:52Z

+    int paramIndex = 1;
+
+    StringBuilder sql = new StringBuilder();
+    sql.append("SELECT ").append(PARAMETERS_SELECT_COLUMNS);


can we use utility like mybatis-3 for creating such SQL strings?

gopalldb · 2026-03-16T05:43:53Z

+    List<List<Object>> rows = new ArrayList<>();
+    while (resultSet.next()) {
+      List<Object> row = new ArrayList<>();
+      row.add(getStringOrNull(resultSet, "routine_catalog")); // PROCEDURE_CAT


declare as constants, all of hard coded values

Done. Extracted all column name strings to private static final constants (COL_ROUTINE_CATALOG, COL_SPECIFIC_NAME, COL_DATA_TYPE, etc.).

gopalldb · 2026-03-16T05:44:10Z

+    LOGGER.debug("Building rows for getProcedureColumns result set");
+    List<List<Object>> rows = new ArrayList<>();
+    while (resultSet.next()) {
+      String dataType = getStringOrNull(resultSet, "data_type");


declare as constants

Done — see above.

gopalldb · 2026-03-16T05:44:46Z

+      String isResult = getStringOrNull(resultSet, "is_result");
+
+      List<Object> row = new ArrayList<>();
+      row.add(getStringOrNull(resultSet, "specific_catalog")); // PROCEDURE_CAT


is null value allowed for catalog and schema?

Yes, per the JDBC spec both PROCEDURE_CAT and PROCEDURE_SCHEM are nullable. From the spec: "PROCEDURE_CAT String => procedure catalog (may be null)" and "PROCEDURE_SCHEM String => procedure schema (may be null)".

gopalldb · 2026-03-16T05:45:48Z

+      row.add(getStringOrNull(resultSet, "parameter_default")); // COLUMN_DEF
+      row.add(null); // SQL_DATA_TYPE (reserved)
+      row.add(null); // SQL_DATETIME_SUB (reserved)
+      row.add(getIntOrNull(resultSet, "character_octet_length")); // CHAR_OCTET_LENGTH


is this same as buffer_length? and you are reading it again

Good catch. Fixed — now reusing the charOctetLength variable for both BUFFER_LENGTH (col 9) and CHAR_OCTET_LENGTH (col 17) instead of calling getIntOrNull twice.

…e charOctetLength - Extract all hardcoded column name strings to private static final constants (COL_ROUTINE_CATALOG, COL_SPECIFIC_NAME, COL_DATA_TYPE, etc.) - Reuse charOctetLength variable for CHAR_OCTET_LENGTH instead of calling getIntOrNull a second time - Add nullable annotations in comments per JDBC spec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…eColumns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

## Summary - Implement `getProcedures`, `getProcedureColumns` metadata operations - Follow the existing `getTables`/`getColumns` pattern with trait methods, SQL builders, SeaClient impl, FFI functions, and C header declarations - Add comprehensive E2E metadata tests covering all metadata operations ### getProcedures - Queries `information_schema.routines` filtered by `routine_type = 'PROCEDURE'` - Catalog resolution: `NULL` → `system` (cross-catalog), specific value → that catalog, empty string → empty result - Pattern filtering via SQL `LIKE` on `routine_schema` and `routine_name` ### getProcedureColumns - Queries `information_schema.parameters` joined with `information_schema.routines` - Same catalog resolution and pattern filtering as getProcedures - Selects all columns needed for ODBC `SQLProcedureColumns`: parameter_name, parameter_mode, is_result, data_type, full_data_type, numeric_precision/scale, character_maximum/octet_length, ordinal_position, parameter_default, comment ### getCrossReferences - Uses `SHOW FOREIGN KEYS` (same query as `getForeignKeys`) - Takes all 6 ODBC `SQLCrossReferences` parameters (pk_catalog/schema/table + fk_catalog/schema/table) - Parent table filtering is done client-side by the ODBC C++ layer ### Design references - JDBC implementation: databricks/databricks-jdbc#1238 - Design doc: [SQLProcedures and SQLProcedureColumns Support](https://docs.google.com/document/d/1WV1hOiJA8Obs9q3o47u7cvZCwnRvKg8kWI8gZhuFFaY) ## Test plan - [x] 12 unit tests for SQL builders (catalog resolution, pattern filters, escaping, column selection) - [x] 12 E2E metadata tests (`#[ignore]`, require real Databricks connection) covering all operations: catalogs, schemas, tables, columns, primary keys, foreign keys, cross-references, procedures, procedure columns - [x] `cargo test` — 237 tests pass - [x] `cargo clippy --all-targets -- -D warnings` — clean - [x] `cargo +stable fmt --all` — clean - [ ] E2E validation against live workspace (blocked on token — tests compile and are properly `#[ignore]`d) This pull request was AI-assisted by Isaac.

msrathore-db requested a review from gopalldb February 27, 2026 17:17

msrathore-db force-pushed the PECOBLR-1746 branch from 33cc3e9 to 32e1fcd Compare February 27, 2026 17:31

gopalldb reviewed Mar 11, 2026

View reviewed changes

msrathore-db and others added 13 commits March 16, 2026 02:10

retrigger CI

6e44923

[PECOBLR-1746] Update MetadataOperationTypeTest for new procedure enu…

0c483d2

…m values Add GET_PROCEDURES and GET_PROCEDURE_COLUMNS to the enum count assertion and parameterized header value tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

updated next_changelog

8a81acd

[PECOBLR-1746] Re-record integration test stubs for parameterized que…

02b19f1

…ries The SQL now uses ? placeholders with server-side parameter binding, so the WireMock stubs need to match the new request format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

msrathore-db force-pushed the PECOBLR-1746 branch from fbbf4db to 02b19f1 Compare March 15, 2026 20:42

gopalldb reviewed Mar 16, 2026

View reviewed changes

gopalldb approved these changes Mar 16, 2026

View reviewed changes

msrathore-db and others added 2 commits March 17, 2026 00:29

Merge branch 'main' into PECOBLR-1746

4dd561e

[PECOBLR-1746] Add NEXT_CHANGELOG entry for getProcedures/getProcedur…

cdbdb5b

…eColumns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

msrathore-db merged commit 28d6a13 into databricks:main Mar 16, 2026
14 of 15 checks passed

gopalldb mentioned this pull request Mar 22, 2026

feat(rust): implement getProcedures, getProcedureColumns adbc-drivers/databricks#366

Merged

6 tasks

Conversation

msrathore-db commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Additional Notes to the Reviewer

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

msrathore-db commented Feb 27, 2026 •

edited

Loading