feat(DirectoryNamespace): add alter table column operations (add/alter/drop)

[XuQianJin-Stars]

close: #6274

Implement the three DDL operations for altering table columns in both ManifestNamespace (manifest.rs) and DirectoryNamespace (dir.rs):

• alter_table_add_columns: Add new columns via SQL expressions
• alter_table_alter_columns: Rename, change type, or change nullability
• alter_table_drop_columns: Remove columns from a table

These methods were previously returning 'not implemented' errors. The implementation delegates to Lance Dataset's add_columns, alter_columns, and drop_columns APIs respectively.

Add comprehensive unit tests :

manifest.rs tests (12 tests, with/without optimization):

• test_alter_table_add_columns: add column via SQL expression, verify schema
• test_alter_table_add_columns_missing_id: error on missing table ID
• test_alter_table_add_columns_nonexistent_table: error on non-existent table
• test_alter_table_alter_columns_rename: rename column, verify schema change
• test_alter_table_alter_columns_missing_id: error on missing table ID
• test_alter_table_drop_columns: drop column, verify schema change
• test_alter_table_drop_columns_missing_id: error on missing table ID
• test_alter_table_drop_columns_nonexistent_table: error on non-existent table

dir.rs tests (8 tests):

• test_alter_table_add_columns: add column, verify schema
• test_alter_table_add_columns_missing_id: error on missing ID
• test_alter_table_alter_columns_rename: rename column, verify schema
• test_alter_table_alter_columns_missing_id: error on missing ID
• test_alter_table_drop_columns: drop column, verify schema
• test_alter_table_drop_columns_missing_id: error on missing ID
• test_alter_table_drop_columns_nonexistent_table: error on non-existent table

[github-actions]

PR Review

P0: Silent failure on data type conversion in alter_table_alter_columns

In both dir.rs and manifest.rs, when data_type is provided but convert_json_arrow_type fails, the error is silently swallowed:

if let Ok(dt) = lance_namespace::schema::convert_json_arrow_type(&json_type) {
    alteration = alteration.cast_to(dt);
}

If a user requests a type cast and the type string is invalid or unsupported, the operation silently succeeds without performing the cast. This should return an error instead — silently ignoring user intent is a bug. Per CLAUDE.md: "Validate inputs and reject invalid values with descriptive errors at API boundaries — never silently clamp or adjust."

Similarly, data_type.as_str() silently skips non-string JSON values (e.g. objects). If the type is provided but not a string, that should also be an error.

P1: unwrap_or_default() on missing SQL expression

let expression = col.expression.clone().unwrap_or_default();

If expression is None, this passes an empty string "" as a SQL expression to Dataset::add_columns. This will likely produce a confusing downstream error from DataFusion rather than a clear validation error at the API boundary. Consider validating that expression is Some and returning a descriptive error if not.

P1: Massive code duplication between dir.rs and manifest.rs

The non-manifest path in dir.rs duplicates essentially the same logic as manifest.rs (open dataset, convert request, call Dataset API, map errors). The dir.rs non-manifest path also lacks the input validation (table ID required/empty checks) that manifest.rs has. Consider extracting the shared logic into a helper, or having the dir.rs non-manifest path validate inputs consistently.

Minor

• The version field is cast as i64 from u64 — this is fine for practical version numbers but worth noting if versions could theoretically exceed i64::MAX.

🤖 Generated with Claude Code

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

[codecov]

Codecov Report

❌ Patch coverage is 65.64885% with 90 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-namespace-impls/src/dir.rs	74.64%	50 Missing and 4 partials ⚠️
rust/lance-namespace-impls/src/dir/manifest.rs	26.53%	36 Missing ⚠️

📢 Thoughts on this report? Let us know!