diff --git a/TOC-tidb-cloud-lake.md b/TOC-tidb-cloud-lake.md index 7c10d48d6ee12..2c7157f3df310 100644 --- a/TOC-tidb-cloud-lake.md +++ b/TOC-tidb-cloud-lake.md @@ -85,6 +85,7 @@ - [Query Avro Files](/tidb-cloud-lake/guides/query-avro-files-in-stage.md) - [Query Staged ORC Files](/tidb-cloud-lake/guides/query-staged-orc-files-in-stage.md) - [Transform Data on Load](/tidb-cloud-lake/guides/transform-data-on-load.md) + - [Schema Evolution](/tidb-cloud-lake/guides/schema-evolution.md) - Continuous Data Pipelines - [Overview](/tidb-cloud-lake/guides/continuous-data-pipelines.md) - [Track and Transform Data via Streams](/tidb-cloud-lake/guides/track-and-transform-data-via-streams.md) @@ -128,6 +129,7 @@ - [Masking Policy](/tidb-cloud-lake/guides/masking-policy.md) - [Network Policy](/tidb-cloud-lake/guides/network-policy.md) - [Password Policy](/tidb-cloud-lake/guides/password-policy.md) + - [Row Access Policy](/tidb-cloud-lake/guides/row-access-policy.md) - [Recovery from Operational Errors](/tidb-cloud-lake/guides/recovery-from-operational-errors.md) - Data Management - [Overview](/tidb-cloud-lake/guides/data-management.md) @@ -417,6 +419,11 @@ - [DROP PASSWORD POLICY](/tidb-cloud-lake/sql/drop-password-policy.md) - [Password Policy](/tidb-cloud-lake/sql/password-policy-sql.md) - [SHOW PASSWORD POLICIES](/tidb-cloud-lake/sql/show-password-policies.md) + - Row Access Policy + - [Overview](/tidb-cloud-lake/sql/row-access-policy-overview.md) + - [CREATE ROW ACCESS POLICY](/tidb-cloud-lake/sql/create-row-access-policy.md) + - [DESC ROW ACCESS POLICY](/tidb-cloud-lake/sql/desc-row-access-policy.md) + - [DROP ROW ACCESS POLICY](/tidb-cloud-lake/sql/drop-row-access-policy.md) - Dictionary - [Overview](/tidb-cloud-lake/sql/dictionary.md) - [CREATE DICTIONARY](/tidb-cloud-lake/sql/create-dictionary.md) @@ -939,9 +946,13 @@ - [ST_ASTEXT](/tidb-cloud-lake/sql/st-astext.md) - [ST_ASWKB](/tidb-cloud-lake/sql/st-aswkb.md) - [ST_ASWKT](/tidb-cloud-lake/sql/st-aswkt.md) + - [ST_AZIMUTH](/tidb-cloud-lake/sql/st-azimuth.md) + - [ST_BUFFER](/tidb-cloud-lake/sql/st-buffer.md) - [ST_CENTROID](/tidb-cloud-lake/sql/st-centroid.md) - [ST_CONTAINS](/tidb-cloud-lake/sql/st-contains.md) - [ST_CONVEXHULL](/tidb-cloud-lake/sql/st-convexhull.md) + - [ST_COVEREDBY](/tidb-cloud-lake/sql/st-coveredby.md) + - [ST_COVERS](/tidb-cloud-lake/sql/st-covers.md) - [ST_DIFFERENCE](/tidb-cloud-lake/sql/st-difference.md) - [ST_DIMENSION](/tidb-cloud-lake/sql/st-dimension.md) - [ST_DISJOINT](/tidb-cloud-lake/sql/st-disjoint.md) @@ -975,21 +986,26 @@ - [ST_GEOMFROMWKB](/tidb-cloud-lake/sql/st-geomfromwkb.md) - [ST_GEOMFROMWKT](/tidb-cloud-lake/sql/st-geomfromwkt.md) - [ST_GEOMPOINTFROMGEOHASH](/tidb-cloud-lake/sql/st-geompointfromgeohash.md) + - [ST_HAUSDORFFDISTANCE](/tidb-cloud-lake/sql/st-hausdorffdistance.md) - [ST_HILBERT](/tidb-cloud-lake/sql/st-hilbert.md) - [ST_INTERSECTION](/tidb-cloud-lake/sql/st-intersection.md) - [ST_INTERSECTS](/tidb-cloud-lake/sql/st-intersects.md) + - [ST_ISVALID](/tidb-cloud-lake/sql/st-isvalid.md) - [ST_LENGTH](/tidb-cloud-lake/sql/st-length.md) - [ST_MAKE_LINE](/tidb-cloud-lake/sql/st-make-line.md) - [ST_MAKEGEOMPOINT](/tidb-cloud-lake/sql/st-makegeompoint.md) - [ST_MAKELINE](/tidb-cloud-lake/sql/st-makeline.md) - [ST_MAKEPOINT](/tidb-cloud-lake/sql/st-makepoint.md) - [ST_MAKEPOLYGON](/tidb-cloud-lake/sql/st-makepolygon.md) + - [ST_MAKEPOLYGONORIENTED](/tidb-cloud-lake/sql/st-makepolygonoriented.md) - [ST_NPOINTS](/tidb-cloud-lake/sql/st-npoints.md) - [ST_NUMPOINTS](/tidb-cloud-lake/sql/st-numpoints.md) + - [ST_PERIMETER](/tidb-cloud-lake/sql/st-perimeter.md) - [ST_POINT](/tidb-cloud-lake/sql/st-point.md) - [ST_POINTN](/tidb-cloud-lake/sql/st-pointn.md) - [ST_POLYGON](/tidb-cloud-lake/sql/st-polygon.md) - [ST_SETSRID](/tidb-cloud-lake/sql/st-setsrid.md) + - [ST_SIMPLIFY](/tidb-cloud-lake/sql/st-simplify.md) - [ST_SRID](/tidb-cloud-lake/sql/st-srid.md) - [ST_STARTPOINT](/tidb-cloud-lake/sql/st-startpoint.md) - [ST_SYMDIFFERENCE](/tidb-cloud-lake/sql/st-symdifference.md) diff --git a/tidb-cloud-lake/guides/query-stage.md b/tidb-cloud-lake/guides/query-stage.md index b805dd5486ea8..75a723e81232c 100644 --- a/tidb-cloud-lake/guides/query-stage.md +++ b/tidb-cloud-lake/guides/query-stage.md @@ -155,3 +155,7 @@ The following file-level metadata fields are available for the supported file fo - [Querying Avro Files](/tidb-cloud-lake/guides/query-avro-files-in-stage.md) - [Querying CSV Files](/tidb-cloud-lake/guides/query-csv-files-in-stage.md) - [Querying TSV Files](/tidb-cloud-lake/guides/query-tsv-files-in-stage.md) + +## Schema Evolution + +- [Schema Evolution](/tidb-cloud-lake/guides/schema-evolution.md): Automatically add new columns to a table when loading Parquet files with evolving schemas. diff --git a/tidb-cloud-lake/guides/row-access-policy.md b/tidb-cloud-lake/guides/row-access-policy.md new file mode 100644 index 0000000000000..be1a59703e1b0 --- /dev/null +++ b/tidb-cloud-lake/guides/row-access-policy.md @@ -0,0 +1,278 @@ +--- +title: Row Access Policy +summary: Row access policies protect data by filtering table rows at query time. They let you define centralized row-level predicates once, attach them to tables, and ensure users only see rows that satisfy the policy. +--- + +# Row Access Policy + +Row access policies protect data by filtering table rows at query time. They let you define centralized row-level predicates once, attach them to tables, and ensure users only see rows that satisfy the policy. + +> **Note:** +> +> Row access policy is currently experimental. Enable it with `SET enable_experimental_row_access_policy = 1` for the current session or `SET GLOBAL enable_experimental_row_access_policy = 1` for the account. + +## How Row Access Works + +Policies evaluate a Boolean expression for each row. Only rows where the expression returns `TRUE` are visible. + +**Admin role sees all rows** + +```sql +id | name | department +---|---------|------------- + 1 | Alice | Engineering + 2 | Bob | Sales + 3 | Charlie | Engineering +``` + +**Other roles see policy-visible rows** + +```sql +id | name | department +---|---------|------------- + 1 | Alice | Engineering + 3 | Charlie | Engineering +``` + +### Key Traits + +- **Query-time** - rows are filtered during SELECT and DML planning; stored data never changes. +- **Role-aware** - expressions can reference `current_role()` or other context functions. +- **Table-scoped** - attach one row access policy to a table and map one or more table columns to policy arguments. +- **Reusable** - use the same policy across tables when the mapped columns have compatible types. + +## End-to-End Workflow + +### 1. Enable the feature + +```sql +SET enable_experimental_row_access_policy = 1; +``` + +### 2. Create the target table + +```sql +CREATE TABLE employees ( + id INT, + name STRING, + department STRING +); + +INSERT INTO employees VALUES + (1, 'Alice', 'Engineering'), + (2, 'Bob', 'Sales'), + (3, 'Charlie', 'Engineering'); +``` + +### 3. Define the row access policy + +```sql +CREATE ROW ACCESS POLICY rap_engineering +AS (dept STRING) +RETURNS BOOLEAN -> +CASE + WHEN current_role() = 'admin' THEN true + WHEN dept = 'Engineering' THEN true + ELSE false +END; +``` + +### 4. Attach the policy + +```sql +ALTER TABLE employees ADD ROW ACCESS POLICY rap_engineering ON (department); +``` + +The `ON (department)` clause maps the table column `department` to the policy argument `dept`. + +### 5. Query the table + +```sql +SELECT id, name, department FROM employees ORDER BY id; +``` + +**Result** + +```sql +id | name | department +---|---------|------------- + 1 | Alice | Engineering + 3 | Charlie | Engineering +``` + +## Read and Write Behavior + +Row access policies are enforced on reads and on DML target-row matching. `UPDATE`, `DELETE`, and `MERGE` only affect rows visible through the policy; invisible rows are not matched or modified. + +`INSERT` is different: it writes rows normally. If an inserted row does not satisfy the policy, it is stored but hidden from policy-protected reads and DML. + +### Complete DML Example + +The example below uses a policy that exposes only `sales` rows. It temporarily detaches the policy after each operation only to inspect the stored data. + +```sql +SET enable_experimental_row_access_policy = 1; + +DROP TABLE IF EXISTS rap_dml_orders; +DROP TABLE IF EXISTS rap_dml_src; +DROP ROW ACCESS POLICY IF EXISTS rap_sales_only; + +CREATE ROW ACCESS POLICY rap_sales_only +AS (dept STRING) +RETURNS BOOLEAN -> dept = 'sales'; + +CREATE TABLE rap_dml_orders(id INT, dept STRING, amount INT); +ALTER TABLE rap_dml_orders ADD ROW ACCESS POLICY rap_sales_only ON (dept); + +-- INSERT is not affected: both visible and invisible rows are stored. +INSERT INTO rap_dml_orders VALUES + (1, 'sales', 100), + (2, 'eng', 200), + (3, 'sales', 300); + +-- SELECT is affected: only policy-visible rows are returned. +SELECT id, dept, amount FROM rap_dml_orders ORDER BY id; + +id | dept | amount +---|-------|------- + 1 | sales | 100 + 3 | sales | 300 + +-- Detach only for inspection: the inserted 'eng' row exists. +ALTER TABLE rap_dml_orders DROP ROW ACCESS POLICY rap_sales_only; +SELECT id, dept, amount FROM rap_dml_orders ORDER BY id; + +id | dept | amount +---|-------|------- + 1 | sales | 100 + 2 | eng | 200 + 3 | sales | 300 + +ALTER TABLE rap_dml_orders ADD ROW ACCESS POLICY rap_sales_only ON (dept); + +-- UPDATE is affected: only visible target rows are updated. +UPDATE rap_dml_orders SET amount = amount + 10; + +ALTER TABLE rap_dml_orders DROP ROW ACCESS POLICY rap_sales_only; +SELECT id, dept, amount FROM rap_dml_orders ORDER BY id; + +id | dept | amount +---|-------|------- + 1 | sales | 110 + 2 | eng | 200 + 3 | sales | 310 + +ALTER TABLE rap_dml_orders ADD ROW ACCESS POLICY rap_sales_only ON (dept); + +-- DELETE is affected: invisible target rows are not deleted. +DELETE FROM rap_dml_orders WHERE dept = 'eng'; + +ALTER TABLE rap_dml_orders DROP ROW ACCESS POLICY rap_sales_only; +SELECT id, dept, amount FROM rap_dml_orders ORDER BY id; + +id | dept | amount +---|-------|------- + 1 | sales | 110 + 2 | eng | 200 + 3 | sales | 310 + +ALTER TABLE rap_dml_orders ADD ROW ACCESS POLICY rap_sales_only ON (dept); + +-- DELETE still affects visible target rows. +DELETE FROM rap_dml_orders WHERE id = 1; + +ALTER TABLE rap_dml_orders DROP ROW ACCESS POLICY rap_sales_only; +SELECT id, dept, amount FROM rap_dml_orders ORDER BY id; + +id | dept | amount +---|-------|------- + 2 | eng | 200 + 3 | sales | 310 + +CREATE TABLE rap_dml_src(id INT, new_amount INT); +INSERT INTO rap_dml_src VALUES (2, 777), (3, 888); + +ALTER TABLE rap_dml_orders ADD ROW ACCESS POLICY rap_sales_only ON (dept); + +-- MERGE is affected: invisible target rows are not matched or updated. +MERGE INTO rap_dml_orders AS t +USING rap_dml_src AS s +ON t.id = s.id +WHEN MATCHED THEN UPDATE SET t.amount = s.new_amount; + +ALTER TABLE rap_dml_orders DROP ROW ACCESS POLICY rap_sales_only; +SELECT id, dept, amount FROM rap_dml_orders ORDER BY id; + +id | dept | amount +---|-------|------- + 2 | eng | 200 + 3 | sales | 888 + +DROP TABLE rap_dml_src; +DROP TABLE rap_dml_orders; +DROP ROW ACCESS POLICY rap_sales_only; +``` + +To inspect or modify all rows, use a role that satisfies the policy or detach the policy. + +## Multiple Arguments + +A policy can depend on multiple columns. The columns in `ON (...)` bind to policy arguments by position, not by name. + +```sql +CREATE ROW ACCESS POLICY rap_region_dept +AS (region STRING, dept STRING) +RETURNS BOOLEAN -> + region = 'APAC' AND dept = 'Engineering'; + +ALTER TABLE employees ADD ROW ACCESS POLICY rap_region_dept ON (office_region, department); +``` + +## Managing Policies + +### DESCRIBE ROW ACCESS POLICY + +View metadata, including creation time, signature, return type, body, and comment. + +```sql +DESC ROW ACCESS POLICY rap_engineering; +``` + +### DROP ROW ACCESS POLICY + +Remove a policy definition you no longer need. Detach it from every table first. + +```sql +ALTER TABLE employees DROP ROW ACCESS POLICY rap_engineering; +DROP ROW ACCESS POLICY rap_engineering; +``` + +### Drop All Row Access Policies From a Table + +```sql +ALTER TABLE employees DROP ALL ROW ACCESS POLICIES; +``` + +## Limits and Requirements + +- A table can have at most one row access policy at a time. +- Row access policies can only be attached to regular tables. Views, streams, and temporary tables do not allow `ADD ROW ACCESS POLICY`. +- Each policy argument must map to one table column with a compatible data type. +- A column can belong to at most one security policy, either masking or row access. +- `INSERT` is not filtered by row access policies. Inserted rows are stored even when they do not satisfy the policy predicate. +- `SELECT` is filtered by row access policies and only returns policy-visible rows. +- `UPDATE`, `DELETE`, and `MERGE` are filtered by row access policies when matching target rows. Invisible target rows are not updated, deleted, or merged. +- Drop or detach the policy before altering or dropping protected columns. +- `CREATE OR REPLACE ROW ACCESS POLICY` and `ALTER ROW ACCESS POLICY` are not supported. + +## Privileges & References + +- Grant `CREATE ROW ACCESS POLICY` on `*.*` to roles responsible for creating policies; the creator automatically owns the policy. +- Grant `ALTER` on the target table plus the global `APPLY ROW ACCESS POLICY` privilege or `APPLY ON ROW ACCESS POLICY ` to roles that attach or detach policies via `ALTER TABLE`. +- Audit access with `SHOW GRANTS ON ROW ACCESS POLICY `. +- Find policy usage with [`POLICY_REFERENCES`](/tidb-cloud-lake/sql/policy-references.md): `POLICY_REFERENCES(POLICY_NAME => '')`. +- Additional references: + - [User & Role](/tidb-cloud-lake/sql/user-role.md) + - [CREATE ROW ACCESS POLICY](/tidb-cloud-lake/sql/create-row-access-policy.md) + - [ALTER TABLE](/tidb-cloud-lake/sql/alter-table.md#row-access-policy-operations) + - [Row Access Policy Commands](/tidb-cloud-lake/sql/row-access-policy-overview.md) diff --git a/tidb-cloud-lake/guides/schema-evolution.md b/tidb-cloud-lake/guides/schema-evolution.md new file mode 100644 index 0000000000000..96c82693fa6f2 --- /dev/null +++ b/tidb-cloud-lake/guides/schema-evolution.md @@ -0,0 +1,124 @@ +--- +title: Schema Evolution +summary: Automatically evolve table schemas when loading data with COPY INTO. +--- + +# Schema Evolution + +Schema evolution allows {{{ .lake }}} to automatically add new columns to a table during `COPY INTO` when the source Parquet files contain columns not yet present in the table. + +## How It Works + +When enabled, `COPY INTO`: + +1. Infers the schema from source Parquet files. +2. Adds any new columns (not in the table) as nullable columns. +3. Loads the data, filling missing values with `NULL`. + +## Enabling Schema Evolution + +Set the table option `ENABLE_SCHEMA_EVOLUTION` to `true`: + +```sql +-- On an existing table +ALTER TABLE my_table SET OPTIONS(ENABLE_SCHEMA_EVOLUTION = true); + +-- Or when creating a new table +CREATE TABLE my_table(id INT) ENABLE_SCHEMA_EVOLUTION = true; +``` + +To disable, set it back to `false`: + +```sql +ALTER TABLE my_table SET OPTIONS(ENABLE_SCHEMA_EVOLUTION = false); +``` + +## Tutorial + +This tutorial uses a fully runnable example to demonstrate schema evolution. + +### Step 1: Create a Table and Stage + +```sql +CREATE OR REPLACE TABLE invoices(order_id INT); +CREATE OR REPLACE STAGE my_stage; +``` + +### Step 2: Generate Parquet Files with Different Schemas + +```sql +-- File with columns: order_id, amount, currency +COPY INTO @my_stage FROM ( + SELECT 1 AS order_id, 100.50::DOUBLE AS amount, 'USD' AS currency + UNION ALL + SELECT 2, 250.50::DOUBLE, 'EUR' +) FILE_FORMAT = (TYPE = parquet); + +-- File with columns: order_id, amount (no currency) +COPY INTO @my_stage FROM ( + SELECT 3 AS order_id, 75.50::DOUBLE AS amount +) FILE_FORMAT = (TYPE = parquet); +``` + +### Step 3: Enable Schema Evolution and Load + +```sql +ALTER TABLE invoices SET OPTIONS(ENABLE_SCHEMA_EVOLUTION = true); + +COPY INTO invoices +FROM @my_stage/ +FILE_FORMAT = (TYPE = parquet MISSING_FIELD_AS = FIELD_DEFAULT); +``` + +### Step 4: Verify Results + +The table now has three columns — `amount` and `currency` were added automatically: + +```sql +DESC invoices; +``` + +```text +┌─────────────────────────────────────────────────────────────┐ +│ Field │ Type │ Null │ Default │ Extra │ +├──────────┼────────────────┼────────┼─────────┼──────────────┤ +│ order_id │ INT │ YES │ NULL │ │ +│ amount │ DOUBLE │ YES │ NULL │ │ +│ currency │ VARCHAR │ YES │ NULL │ │ +└─────────────────────────────────────────────────────────────┘ +``` + +```sql +SELECT * FROM invoices ORDER BY order_id; +``` + +```text +┌──────────────────────────────────────────────────┐ +│ order_id │ amount │ currency │ +├──────────┼──────────┼─────────────────────────────┤ +│ 1 │ 100.50 │ USD │ +│ 2 │ 250.50 │ EUR │ +│ 3 │ 75.50 │ NULL │ +└──────────────────────────────────────────────────┘ +``` + +Row 3 has `currency = NULL` because its source file did not contain that column. + +## Column Match Mode + +By default, column names are matched case-insensitively. Use `COLUMN_MATCH_MODE` for case-sensitive matching: + +```sql +COPY INTO invoices +FROM @my_stage/ +FILE_FORMAT = (TYPE = parquet MISSING_FIELD_AS = FIELD_DEFAULT) +COLUMN_MATCH_MODE = CASE_SENSITIVE; +``` + +## Limitations + +- Supported for **Parquet** files only. +- New columns are appended to the end of the table and are always nullable. +- If the same column name appears in multiple files with **different data types**, the load fails. +- No automatic type promotion (e.g., `INT` → `BIGINT`). +- Column drops and renames are not supported through schema evolution. diff --git a/tidb-cloud-lake/guides/security-reliability.md b/tidb-cloud-lake/guides/security-reliability.md index 3aa3e267a7c7e..2235ed67c54fa 100644 --- a/tidb-cloud-lake/guides/security-reliability.md +++ b/tidb-cloud-lake/guides/security-reliability.md @@ -14,5 +14,6 @@ summary: "{{{ .lake }}} offers enterprise-grade security and reliability feature | [**Network Policy**](/tidb-cloud-lake/guides/network-policy.md) | Restrict network access | When you want to limit connections to specific IP ranges even with valid credentials | | [**Password Policy**](/tidb-cloud-lake/guides/password-policy.md) | Set password requirements | When you need to enforce password complexity, rotation, and account lockout rules | | [**Masking Policy**](/tidb-cloud-lake/guides/masking-policy.md) | Hide sensitive data | When you need to protect confidential data while still allowing authorized access | +| [**Row Access Policy**](/tidb-cloud-lake/guides/row-access-policy.md) | Filter rows dynamically | When users should only see rows that match role-aware access rules | | [**Fail-Safe**](/tidb-cloud-lake/guides/fail-safe.md) | Prevent data loss | When you need to recover accidentally deleted data from S3-compatible storage | | [**Recovery from Errors**](/tidb-cloud-lake/guides/recovery-from-operational-errors.md) | Fix operational mistakes | When you need to recover from dropped databases/tables or incorrect data modifications | diff --git a/tidb-cloud-lake/sql/alter-table.md b/tidb-cloud-lake/sql/alter-table.md index 60060a3c53f89..6e80e9729ca16 100644 --- a/tidb-cloud-lake/sql/alter-table.md +++ b/tidb-cloud-lake/sql/alter-table.md @@ -74,7 +74,7 @@ DROP [ COLUMN ] - Masking policies can only be attached to regular tables. Views, streams, and temporary tables do not allow `SET MASKING POLICY`. - A column can belong to at most one security policy (masking or row-level). Remove the existing policy before attaching a new one. - Attaching, detaching, describing, or dropping a masking policy requires the global `APPLY MASKING POLICY` privilege or APPLY/OWNERSHIP on the specific masking policy. -- Adding, removing, describing, or dropping a row access policy requires the global `APPLY ROW ACCESS POLICY` privilege or APPLY/OWNERSHIP on that policy. +- Adding or removing a row access policy requires `ALTER` on the target table plus the global `APPLY ROW ACCESS POLICY` privilege or APPLY/OWNERSHIP on that policy. Describing or dropping a policy requires the same policy privilege. > **Note:** > @@ -182,7 +182,58 @@ ALTER TABLE users MODIFY COLUMN email UNSET MASKING POLICY; ``` -## Table Comment {#table-comment} +## Row Access Policy Operations + +Attach or detach a row access policy for a table. A row access policy filters rows at query time and during DML target-row matching. + +### Syntax + +```sql +-- Add a row access policy to a table +ALTER TABLE [ IF EXISTS ] [ . ] +ADD ROW ACCESS POLICY ON ( [ , ... ] ) + +-- Drop a specific row access policy from a table +ALTER TABLE [ IF EXISTS ] [ . ] +DROP ROW ACCESS POLICY + +-- Drop all row access policies from a table +ALTER TABLE [ IF EXISTS ] [ . ] +DROP ALL ROW ACCESS POLICIES +``` + +> **Note:** +> +> - Row access policy is currently experimental. Enable it with `SET enable_experimental_row_access_policy = 1` or `SET GLOBAL enable_experimental_row_access_policy = 1`. +> - A table can have at most one row access policy at a time. +> - The columns in `ON (...)` bind to the policy arguments by position. The number of columns and their data types must match the policy signature. +> - Row access policies can only be attached to regular tables. Views, streams, and temporary tables do not allow `ADD ROW ACCESS POLICY`. +> - A column can belong to at most one security policy, either masking or row access. +> - Adding or removing a row access policy requires `ALTER` on the target table plus the global `APPLY ROW ACCESS POLICY` privilege or APPLY/OWNERSHIP on that policy. Describing or dropping a policy requires the same policy privilege. + +> **Warning:** +> +> You must detach the row access policy before changing or dropping a protected column. Otherwise the statement fails because the column is still referenced by a security policy. + +### Example + +```sql +SET enable_experimental_row_access_policy = 1; + +CREATE TABLE employees(id INT, name STRING, department STRING); + +CREATE ROW ACCESS POLICY rap_engineering +AS (dept STRING) +RETURNS BOOLEAN -> dept = 'Engineering'; + +ALTER TABLE employees +ADD ROW ACCESS POLICY rap_engineering ON (department); + +ALTER TABLE employees +DROP ROW ACCESS POLICY rap_engineering; +``` + +## Table Comment Modifies the comment of a table. If the table does not have a comment yet, this command adds the specified comment to the table. @@ -247,6 +298,7 @@ Only the following Fuse Engine options can be unset: - `block_size_threshold` - `data_retention_period_in_hours` - `data_retention_num_snapshots_to_keep` +- `enable_schema_evolution` - `row_avg_depth_threshold` - `row_per_block` - `row_per_page` diff --git a/tidb-cloud-lake/sql/copy-into-table.md b/tidb-cloud-lake/sql/copy-into-table.md index 6cb39a962d986..b815d8a240f70 100644 --- a/tidb-cloud-lake/sql/copy-into-table.md +++ b/tidb-cloud-lake/sql/copy-into-table.md @@ -710,3 +710,25 @@ SELECT * FROM t2; │ 6 │ null │ └──────────────────────────────────────┘ ``` + +### Example 8: Loading with Schema Evolution + +When loading Parquet files whose schemas contain columns not present in the target table, you can use schema evolution to automatically add the missing columns. First, enable schema evolution on the table: + +```sql +CREATE OR REPLACE TABLE invoices(order_id INT); + +-- Enable schema evolution +ALTER TABLE invoices SET OPTIONS(ENABLE_SCHEMA_EVOLUTION = true); +``` + +Then load Parquet files with different schemas. {{{ .lake }}} automatically adds new columns and fills missing values with `NULL`: + +```sql +-- Assume @my_stage contains Parquet files with extra columns (e.g., amount, currency) +COPY INTO invoices + FROM @my_stage/ + FILE_FORMAT = (TYPE = PARQUET MISSING_FIELD_AS = FIELD_DEFAULT); +``` + +For more details, see [Schema Evolution](/tidb-cloud-lake/guides/schema-evolution.md). diff --git a/tidb-cloud-lake/sql/create-row-access-policy.md b/tidb-cloud-lake/sql/create-row-access-policy.md new file mode 100644 index 0000000000000..fe9f7130d81aa --- /dev/null +++ b/tidb-cloud-lake/sql/create-row-access-policy.md @@ -0,0 +1,87 @@ +--- +title: CREATE ROW ACCESS POLICY +summary: "Creates a new row access policy in {{{ .lake }}}. A row access policy defines a Boolean predicate that {{{ .lake }}} applies to rows when the policy is attached to a table." +--- + +# CREATE ROW ACCESS POLICY + +> **Note:** +> +> Introduced or updated in v1.2.845. + +Creates a new row access policy in {{{ .lake }}}. A row access policy defines a Boolean predicate that {{{ .lake }}} applies to rows when the policy is attached to a table. + +## Syntax + +```sql +CREATE ROW ACCESS POLICY [ IF NOT EXISTS ] AS + ( [ , ... ] ) + RETURNS BOOLEAN -> + [ COMMENT = '' ] +``` + +| Parameter | Description | +|-----------|-------------| +| `policy_name` | Name of the row access policy to create. Policy names share the same namespace as masking policies. | +| `arg_name` | Policy argument name used inside the predicate expression. Argument names do not need to match table column names. | +| `arg_type` | Data type for the argument. When the policy is attached, each listed table column must match the corresponding argument type. | +| `predicate_expression` | Boolean expression that decides whether a row is visible. Rows are returned only when this expression evaluates to `TRUE`. | +| `comment` | Optional comment that stores notes about the policy. | + +> **Note:** +> +> - Row access policy is currently experimental. Enable it with `SET enable_experimental_row_access_policy = 1` or `SET GLOBAL enable_experimental_row_access_policy = 1`. +> - The policy must return `BOOLEAN`. +> - The columns listed in `ALTER TABLE ... ADD ROW ACCESS POLICY ... ON (...)` bind to policy arguments by position. +> - Subquery predicates are not supported in row access policy definitions. + +## Access Control Requirements + +| Privilege | Description | +|:----------|:------------| +| CREATE ROW ACCESS POLICY | Required to create a row access policy. Typically granted on `*.*`. | + +{{{ .lake }}} automatically grants OWNERSHIP on the new row access policy to the current role so that it can manage the policy with others. + +## Examples + +This example creates a policy that only exposes rows from the `Engineering` department, unless the current role is `admin`. + +```sql +SET enable_experimental_row_access_policy = 1; + +CREATE TABLE employees ( + id INT, + name STRING, + department STRING +); + +INSERT INTO employees VALUES + (1, 'Alice', 'Engineering'), + (2, 'Bob', 'Sales'), + (3, 'Charlie', 'Engineering'); + +CREATE ROW ACCESS POLICY rap_engineering +AS (dept STRING) +RETURNS BOOLEAN -> + CASE + WHEN current_role() = 'admin' THEN true + WHEN dept = 'Engineering' THEN true + ELSE false + END + COMMENT = 'show engineering rows'; + +ALTER TABLE employees +ADD ROW ACCESS POLICY rap_engineering ON (department); + +SELECT id, name, department FROM employees ORDER BY id; + +┌────┬─────────┬─────────────┐ +│ id │ name │ department │ +├────┼─────────┼─────────────┤ +│ 1 │ Alice │ Engineering │ +│ 3 │ Charlie │ Engineering │ +└────┴─────────┴─────────────┘ +``` + +The `ON (department)` clause maps the table column `department` to the policy argument `dept`. diff --git a/tidb-cloud-lake/sql/ddl.md b/tidb-cloud-lake/sql/ddl.md index 8d460e616f1c9..5f4613f7f3d52 100644 --- a/tidb-cloud-lake/sql/ddl.md +++ b/tidb-cloud-lake/sql/ddl.md @@ -38,6 +38,7 @@ These topics provide reference information for the DDL (Data Definition Language | **[Network Policy](/tidb-cloud-lake/sql/network-policy-sql.md)** | Control network access to databases | | **[Mask Policy](/tidb-cloud-lake/sql/masking-policy-sql.md)** | Apply data masking for sensitive information | | **[Password Policy](/tidb-cloud-lake/sql/password-policy-sql.md)** | Enforce password requirements and rotation | +| **[Row Access Policy](/tidb-cloud-lake/sql/row-access-policy-overview.md)** | Filter table rows with centralized row-level predicates | ## Data Integration & Processing diff --git a/tidb-cloud-lake/sql/desc-row-access-policy.md b/tidb-cloud-lake/sql/desc-row-access-policy.md new file mode 100644 index 0000000000000..9e20a839efb13 --- /dev/null +++ b/tidb-cloud-lake/sql/desc-row-access-policy.md @@ -0,0 +1,50 @@ +--- +title: DESC ROW ACCESS POLICY +summary: "Displays detailed information about a specific row access policy in {{{ .lake }}}." +--- + +# DESC ROW ACCESS POLICY + +> **Note:** +> +> Introduced or updated in v1.2.845. + +Displays detailed information about a specific row access policy in {{{ .lake }}}. + +## Syntax + +```sql +DESC ROW ACCESS POLICY +``` + +`DESCRIBE ROW ACCESS POLICY` is also supported. + +## Access Control Requirements + +| Privilege | Description | +|:----------|:------------| +| APPLY ROW ACCESS POLICY | Required to describe a row access policy unless you own that policy. | + +Either the global `APPLY ROW ACCESS POLICY` privilege or APPLY/OWNERSHIP on the specific row access policy satisfies this requirement. + +## Examples + +```sql +SET enable_experimental_row_access_policy = 1; + +CREATE ROW ACCESS POLICY rap_engineering +AS (dept STRING) +RETURNS BOOLEAN -> + CASE + WHEN current_role() = 'admin' THEN true + WHEN dept = 'Engineering' THEN true + ELSE false + END + COMMENT = 'show engineering rows'; + +DESC ROW ACCESS POLICY rap_engineering; + +Name | Created On | Signature | Return Type | Body | Comment +----------------+-----------------------------+---------------+-------------+------------------------------------------------------------+---------------------- +rap_engineering | 2026-05-15 08:42:10.949 UTC | (dept STRING) | BOOLEAN | CASE WHEN current_role() = 'admin' THEN true WHEN... | show engineering rows +``` diff --git a/tidb-cloud-lake/sql/drop-row-access-policy.md b/tidb-cloud-lake/sql/drop-row-access-policy.md new file mode 100644 index 0000000000000..a9b81e5c5a58a --- /dev/null +++ b/tidb-cloud-lake/sql/drop-row-access-policy.md @@ -0,0 +1,44 @@ +--- +title: DROP ROW ACCESS POLICY +summary: "Deletes an existing row access policy from {{{ .lake }}}. Before dropping a policy, detach it from all tables that reference it." +--- + +# DROP ROW ACCESS POLICY + +> **Note:** +> +> Introduced or updated in v1.2.845. + +Deletes an existing row access policy from {{{ .lake }}}. Before dropping a policy, detach it from all tables that reference it. + +## Syntax + +```sql +DROP ROW ACCESS POLICY [ IF EXISTS ] +``` + +## Access Control Requirements + +| Privilege | Description | +|:----------|:------------| +| APPLY ROW ACCESS POLICY | Required to drop a row access policy unless you own that policy. | + +You must have the global `APPLY ROW ACCESS POLICY` privilege or APPLY/OWNERSHIP on the target policy. {{{ .lake }}} automatically revokes OWNERSHIP from the creator role after the policy is dropped. + +## Examples + +```sql +SET enable_experimental_row_access_policy = 1; + +CREATE ROW ACCESS POLICY rap_engineering +AS (dept STRING) +RETURNS BOOLEAN -> dept = 'Engineering'; + +CREATE TABLE employees(id INT, department STRING); +ALTER TABLE employees ADD ROW ACCESS POLICY rap_engineering ON (department); + +-- Detach the policy before dropping it. +ALTER TABLE employees DROP ROW ACCESS POLICY rap_engineering; + +DROP ROW ACCESS POLICY rap_engineering; +``` diff --git a/tidb-cloud-lake/sql/fuse-engine-tables.md b/tidb-cloud-lake/sql/fuse-engine-tables.md index c236480b0449c..0614e40d7317d 100644 --- a/tidb-cloud-lake/sql/fuse-engine-tables.md +++ b/tidb-cloud-lake/sql/fuse-engine-tables.md @@ -189,3 +189,23 @@ SET GLOBAL data_retention_num_snapshots_to_keep = 20; ALTER TABLE t1 SET OPTIONS(data_retention_num_snapshots_to_keep = 3); -- Now t1 will keep 3 snapshots when vacuum is triggered ``` + +### `enable_schema_evolution` + +- **Syntax:** + + `enable_schema_evolution = True / False` + +- **Description:** + + Controls whether the table schema can be automatically evolved during `COPY INTO` operations. When enabled (set to `True`), {{{ .lake }}} automatically adds missing columns to the table when loading Parquet files whose schemas contain columns not present in the destination table. Missing values for existing rows are filled with `NULL`. For more information, see [Schema Evolution](/tidb-cloud-lake/guides/schema-evolution.md). + +**Examples:** + +```sql +-- Enable schema evolution for an existing table +ALTER TABLE invoices SET OPTIONS(ENABLE_SCHEMA_EVOLUTION = true); + +-- Create a new table with schema evolution enabled +CREATE OR REPLACE TABLE invoices (order_id INT) ENABLE_SCHEMA_EVOLUTION = true; +``` diff --git a/tidb-cloud-lake/sql/grant.md b/tidb-cloud-lake/sql/grant.md index 3e720d527361e..4ec6d3f136df2 100644 --- a/tidb-cloud-lake/sql/grant.md +++ b/tidb-cloud-lake/sql/grant.md @@ -137,6 +137,10 @@ GRANT ROLE TO GRANT ROLE TO ROLE ``` +> **Note:** +> +> `default_role` is a user property — it's set when you `CREATE USER` or `ALTER USER`, and `GRANT`/`REVOKE` won't touch it. So if you later revoke a role that happens to be someone's `default_role`, the setting stays but the role membership is gone. Use `ALTER USER ... WITH DEFAULT_ROLE` to update it explicitly. + ### Granting Ownership To understand what ownership is and how it works, see [Ownership](/tidb-cloud-lake/guides/ownership.md). diff --git a/tidb-cloud-lake/sql/policy-references.md b/tidb-cloud-lake/sql/policy-references.md index a01997f42444a..8f360812e60b3 100644 --- a/tidb-cloud-lake/sql/policy-references.md +++ b/tidb-cloud-lake/sql/policy-references.md @@ -10,6 +10,7 @@ Returns the associations between security policies (Masking Policy or Row Access See also: - [MASKING POLICY](/tidb-cloud-lake/guides/masking-policy.md) +- [ROW ACCESS POLICY](/tidb-cloud-lake/guides/row-access-policy.md) ## Syntax diff --git a/tidb-cloud-lake/sql/revoke.md b/tidb-cloud-lake/sql/revoke.md index b2cf4adb696e5..ed81d222b2171 100644 --- a/tidb-cloud-lake/sql/revoke.md +++ b/tidb-cloud-lake/sql/revoke.md @@ -19,6 +19,10 @@ See also: - [GRANT](/tidb-cloud-lake/sql/grant.md) - [SHOW GRANTS](/tidb-cloud-lake/sql/show-grants.md) +> **Note:** +> +> After changing privileges or roles with `REVOKE`, run [SYSTEM FLUSH PRIVILEGES](/tidb-cloud-lake/sql/system-flush-privileges.md) to broadcast the updates to every query node immediately. + ## Syntax ### Revoking Privileges @@ -192,6 +196,36 @@ REVOKE ROLE role1 FROM USER user1; SHOW GRANTS FOR user1; ``` +> **Tip:** +> +> Why does `default_role` still show after revoke? +> +> `default_role` is a user property, not a grant. `REVOKE` removes role membership but won't reset `default_role`. Here's what that looks like: +> +> ```sql +> CREATE ROLE analyst; +> CREATE USER bob IDENTIFIED BY 'password123' WITH DEFAULT_ROLE = 'analyst'; +> GRANT ROLE analyst TO bob; +> +> DESC USER bob; +> +------+----------+----------------------+--------------+---------+ +> | name | hostname | auth_type | default_role | roles | +> +------+----------+----------------------+--------------+---------+ +> | bob | % | double_sha1_password | analyst | analyst | +> +------+----------+----------------------+--------------+---------+ +> +> REVOKE ROLE analyst FROM bob; +> +> DESC USER bob; +> +------+----------+----------------------+--------------+-------+ +> | name | hostname | auth_type | default_role | roles | +> +------+----------+----------------------+--------------+-------+ +> | bob | % | double_sha1_password | analyst | | +> +------+----------+----------------------+--------------+-------+ +> ``` +> +> Notice `roles` is now empty but `default_role` stays. The user no longer has `analyst`'s privileges. To clean up, run `ALTER USER bob WITH DEFAULT_ROLE = 'public'`. + ### Example 4: Revoking Masking Policy Privileges ```sql diff --git a/tidb-cloud-lake/sql/row-access-policy-overview.md b/tidb-cloud-lake/sql/row-access-policy-overview.md new file mode 100644 index 0000000000000..2d2b2b2a1eb88 --- /dev/null +++ b/tidb-cloud-lake/sql/row-access-policy-overview.md @@ -0,0 +1,26 @@ +--- +title: Row Access Policy Overview +summary: "A comprehensive overview of Row Access Policy operations in {{{ .lake }}}, organized by functionality for easy reference." +--- + +# Row Access Policy Overview + +This page provides a comprehensive overview of Row Access Policy operations in {{{ .lake }}}, organized by functionality for easy reference. + +## Row Access Policy Management + +| Command | Description | +|---------|-------------| +| [CREATE ROW ACCESS POLICY](/tidb-cloud-lake/sql/create-row-access-policy.md) | Creates a row-level filter policy | +| [DESCRIBE ROW ACCESS POLICY](/tidb-cloud-lake/sql/desc-row-access-policy.md) | Shows details of a row access policy | +| [DROP ROW ACCESS POLICY](/tidb-cloud-lake/sql/drop-row-access-policy.md) | Removes a row access policy | + +## Related Topics + +- [Row Access Policy](/tidb-cloud-lake/guides/row-access-policy.md) +- [ALTER TABLE](/tidb-cloud-lake/sql/alter-table.md#row-access-policy-operations) +- [POLICY_REFERENCES](/tidb-cloud-lake/sql/policy-references.md) + +> **Note:** +> +> Row access policies filter rows at query time. A protected table returns only the rows for which the policy expression evaluates to `TRUE`. diff --git a/tidb-cloud-lake/sql/st-azimuth.md b/tidb-cloud-lake/sql/st-azimuth.md new file mode 100644 index 0000000000000..31225720a08fa --- /dev/null +++ b/tidb-cloud-lake/sql/st-azimuth.md @@ -0,0 +1,73 @@ +--- +title: ST_AZIMUTH +summary: Returns the azimuth in radians of the line segment from one Point to another, measured clockwise from the positive Y-axis (north). Returns NULL if the two points are identical. +--- + +# ST_AZIMUTH + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns the azimuth in radians of the line segment from one Point to another, measured clockwise from the positive Y-axis (north). Returns NULL if the two points are identical. + +## Syntax + +```sql +ST_AZIMUTH(, ) +``` + +## Arguments + +| Arguments | Description | +|------------|------------------------------------------------------| +| `` | A GEOMETRY expression of type Point (origin). | +| `` | A GEOMETRY expression of type Point (target). | + +> **Note:** +> +> Both arguments must be Point geometries. Other types produce an error. + +## Return Type + +Double (nullable). + +## Examples + +```sql +-- Due north (along positive Y-axis): 0 radians +SELECT ST_AZIMUTH(TO_GEOMETRY('POINT(0 0)'), TO_GEOMETRY('POINT(0 1)')); + +┌────────┐ +│ result │ +├────────┤ +│ 0.0 │ +└────────┘ + +-- Due east: π/2 radians +SELECT ST_AZIMUTH(TO_GEOMETRY('POINT(0 0)'), TO_GEOMETRY('POINT(1 0)')); + +┌─────────────┐ +│ result │ +├─────────────┤ +│ 1.570796327 │ +└─────────────┘ + +-- Due south: π radians +SELECT ST_AZIMUTH(TO_GEOMETRY('POINT(0 1)'), TO_GEOMETRY('POINT(0 0)')); + +┌─────────────┐ +│ result │ +├─────────────┤ +│ 3.141592654 │ +└─────────────┘ + +-- Identical points: NULL +SELECT ST_AZIMUTH(TO_GEOMETRY('POINT(0 0)'), TO_GEOMETRY('POINT(0 0)')); + +┌────────┐ +│ result │ +├────────┤ +│ NULL │ +└────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-buffer.md b/tidb-cloud-lake/sql/st-buffer.md new file mode 100644 index 0000000000000..c2843c7ba696b --- /dev/null +++ b/tidb-cloud-lake/sql/st-buffer.md @@ -0,0 +1,79 @@ +--- +title: ST_BUFFER +summary: Returns a GEOMETRY representing all points whose distance from the input geometry is less than or equal to the specified distance. The result is a MultiPolygon or NULL. +--- + +# ST_BUFFER + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns a GEOMETRY representing all points whose distance from the input geometry is less than or equal to the specified distance. The result is a MultiPolygon or NULL. + +## Syntax + +```sql +ST_BUFFER(, ) +``` + +## Arguments + +| Arguments | Description | +|--------------|-----------------------------------------------------------------------------| +| `` | A GEOMETRY expression. GeometryCollection is not supported. | +| `` | Buffer distance. Units match the coordinate system of the input geometry. | + +> **Note:** +> +> - For Point, MultiPoint, LineString, and MultiLineString: the absolute value of distance is used (negative behaves the same as positive). +> - For Polygon and MultiPolygon: positive distance inflates, negative distance deflates. +> - Returns NULL when the result is empty (e.g., zero distance on a Point, or deflating a polygon past zero area). +> - For Polygon with distance 0: returns the polygon wrapped as a MultiPolygon. +> - SRID is preserved in the output. + +## Return Type + +Geometry (nullable). + +## Examples + +```sql +-- Buffer a point (produces a polygon approximating a circle) +SELECT ST_BUFFER(TO_GEOMETRY('POINT(0 0)'), 1) IS NOT NULL; + +┌────────┐ +│ result │ +├────────┤ +│ true │ +└────────┘ + +-- Zero distance on a polygon returns itself as MultiPolygon +SELECT ST_ASWKT( + ST_BUFFER(TO_GEOMETRY('POLYGON((0 0, 4 0, 4 4, 0 4, 0 0))'), 0) +); + +┌─────────────────────────────────────────────────┐ +│ result │ +├─────────────────────────────────────────────────┤ +│ MULTIPOLYGON(((0 0,4 0,4 4,0 4,0 0))) │ +└─────────────────────────────────────────────────┘ + +-- Zero distance on a point returns NULL +SELECT ST_ASWKT(ST_BUFFER(TO_GEOMETRY('POINT(0 0)'), 0)); + +┌────────┐ +│ result │ +├────────┤ +│ NULL │ +└────────┘ + +-- SRID is preserved +SELECT ST_SRID(ST_BUFFER(ST_GEOMETRYFROMWKT('POINT(0 0)', 4326), 1)); + +┌────────┐ +│ result │ +├────────┤ +│ 4326 │ +└────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-coveredby.md b/tidb-cloud-lake/sql/st-coveredby.md new file mode 100644 index 0000000000000..60ae6d523195f --- /dev/null +++ b/tidb-cloud-lake/sql/st-coveredby.md @@ -0,0 +1,68 @@ +--- +title: ST_COVEREDBY +summary: Returns TRUE if no point in the first GEOMETRY object lies outside the second GEOMETRY object. +--- + +# ST_COVEREDBY + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns TRUE if no point in the first GEOMETRY object lies outside the second GEOMETRY object. + +See also: [ST_COVERS](/tidb-cloud-lake/sql/st-covers.md) + +## Syntax + +```sql +ST_COVEREDBY(, ) +``` + +## Arguments + +| Arguments | Description | +|---------------|------------------------------------------------------| +| `` | A GEOMETRY expression (the object being tested). | +| `` | A GEOMETRY expression (the covering object). | + +## Return Type + +Boolean. + +## Examples + +```sql +SELECT ST_COVEREDBY( + TO_GEOMETRY('POINT(1 1)'), + TO_GEOMETRY('POLYGON((0 0, 3 0, 3 3, 0 3, 0 0))') +); + +┌────────┐ +│ result │ +├────────┤ +│ true │ +└────────┘ + +SELECT ST_COVEREDBY( + TO_GEOMETRY('POLYGON((1 1, 2 1, 2 2, 1 2, 1 1))'), + TO_GEOMETRY('POLYGON((0 0, 3 0, 3 3, 0 3, 0 0))') +); + +┌────────┐ +│ result │ +├────────┤ +│ true │ +└────────┘ + +SELECT ST_COVEREDBY( + TO_GEOMETRY('POINT(5 5)'), + TO_GEOMETRY('POLYGON((0 0, 3 0, 3 3, 0 3, 0 0))') +); + +┌────────┐ +│ result │ +├────────┤ +│ false │ +└────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-covers.md b/tidb-cloud-lake/sql/st-covers.md new file mode 100644 index 0000000000000..4f151fdacdb4b --- /dev/null +++ b/tidb-cloud-lake/sql/st-covers.md @@ -0,0 +1,71 @@ +--- +title: ST_COVERS +summary: Returns TRUE if no point in the second GEOMETRY object lies outside the first GEOMETRY object. +--- + +# ST_COVERS + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns TRUE if no point in the second GEOMETRY object lies outside the first GEOMETRY object. + +See also: [ST_COVEREDBY](/tidb-cloud-lake/sql/st-coveredby.md) + +## Syntax + +```sql +ST_COVERS(, ) +``` + +## Arguments + +| Arguments | Description | +|---------------|------------------------------------------------------| +| `` | A GEOMETRY expression (the covering object). | +| `` | A GEOMETRY expression (the object being tested). | + +## Return Type + +Boolean. + +## Examples + +```sql +-- A polygon covers a smaller polygon inside it +SELECT ST_COVERS( + TO_GEOMETRY('POLYGON((-2 0, 0 2, 2 0, -2 0))'), + TO_GEOMETRY('POLYGON((-1 0, 0 1, 1 0, -1 0))') +); + +┌────────┐ +│ result │ +├────────┤ +│ true │ +└────────┘ + +-- A polygon covers a linestring on its boundary +SELECT ST_COVERS( + TO_GEOMETRY('POLYGON((-2 0, 0 2, 2 0, -2 0))'), + TO_GEOMETRY('LINESTRING(-1 1, 0 2, 1 1)') +); + +┌────────┐ +│ result │ +├────────┤ +│ true │ +└────────┘ + +-- A point outside the polygon is not covered +SELECT ST_COVERS( + TO_GEOMETRY('POLYGON((0 0, 3 0, 3 3, 0 3, 0 0))'), + TO_GEOMETRY('POINT(5 5)') +); + +┌────────┐ +│ result │ +├────────┤ +│ false │ +└────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-hausdorffdistance.md b/tidb-cloud-lake/sql/st-hausdorffdistance.md new file mode 100644 index 0000000000000..49dec968f4c98 --- /dev/null +++ b/tidb-cloud-lake/sql/st-hausdorffdistance.md @@ -0,0 +1,66 @@ +--- +title: ST_HAUSDORFFDISTANCE +summary: Returns the discrete Hausdorff distance between two GEOMETRY objects. This measures how far apart two geometries are by finding the greatest distance from any vertex in one object to the nearest vertex in the other. +--- + +# ST_HAUSDORFFDISTANCE + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns the discrete Hausdorff distance between two GEOMETRY objects. This measures how far apart two geometries are by finding the greatest distance from any vertex in one object to the nearest vertex in the other. + +## Syntax + +```sql +ST_HAUSDORFFDISTANCE(, ) +``` + +## Arguments + +| Arguments | Description | +|---------------|------------------------------------------------------| +| `` | A GEOMETRY expression. | +| `` | A GEOMETRY expression. | + +## Return Type + +Double. + +## Examples + +```sql +SELECT ST_HAUSDORFFDISTANCE( + TO_GEOMETRY('POINT(0 0)'), + TO_GEOMETRY('POINT(0 1)') +); + +┌────────┐ +│ result │ +├────────┤ +│ 1.0 │ +└────────┘ + +SELECT ST_HAUSDORFFDISTANCE( + TO_GEOMETRY('LINESTRING(0 0, 1 0)'), + TO_GEOMETRY('LINESTRING(0 1, 1 1)') +); + +┌────────┐ +│ result │ +├────────┤ +│ 1.0 │ +└────────┘ + +SELECT ST_HAUSDORFFDISTANCE( + TO_GEOMETRY('POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))'), + TO_GEOMETRY('POLYGON((2 0, 3 0, 3 1, 2 1, 2 0))') +); + +┌────────┐ +│ result │ +├────────┤ +│ 2.0 │ +└────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-isvalid.md b/tidb-cloud-lake/sql/st-isvalid.md new file mode 100644 index 0000000000000..a9b4586a5cdb6 --- /dev/null +++ b/tidb-cloud-lake/sql/st-isvalid.md @@ -0,0 +1,49 @@ +--- +title: ST_ISVALID +summary: Returns TRUE if the GEOMETRY object is geometrically valid as defined by the OGC specification. +--- + +# ST_ISVALID + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns TRUE if the GEOMETRY object is geometrically valid as defined by the OGC specification. + +## Syntax + +```sql +ST_ISVALID() +``` + +## Arguments + +| Arguments | Description | +|--------------|------------------------------------------------------| +| `` | A GEOMETRY expression. | + +## Return Type + +Boolean. + +## Examples + +```sql +SELECT ST_ISVALID(TO_GEOMETRY('POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))')); + +┌──────────────────────────────────────────────────────────┐ +│ st_isvalid(to_geometry('polygon((0 0, 1 0, 1 1, 0 1, 0 0))')) │ +├──────────────────────────────────────────────────────────┤ +│ true │ +└──────────────────────────────────────────────────────────┘ + +-- Self-intersecting polygon (bowtie shape) is invalid +SELECT ST_ISVALID(TO_GEOMETRY('POLYGON((0 0, 2 2, 2 0, 0 2, 0 0))')); + +┌──────────────────────────────────────────────────────────────┐ +│ st_isvalid(to_geometry('polygon((0 0, 2 2, 2 0, 0 2, 0 0))')) │ +├──────────────────────────────────────────────────────────────┤ +│ false │ +└──────────────────────────────────────────────────────────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-makepolygonoriented.md b/tidb-cloud-lake/sql/st-makepolygonoriented.md new file mode 100644 index 0000000000000..115270443406a --- /dev/null +++ b/tidb-cloud-lake/sql/st-makepolygonoriented.md @@ -0,0 +1,58 @@ +--- +title: ST_MAKEPOLYGONORIENTED +summary: Creates a Polygon from a LineString input, preserving the vertex order as given. Unlike ST_MAKEPOLYGON, this function does not reorder vertices to enforce a specific winding direction. +--- + +# ST_MAKEPOLYGONORIENTED + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Creates a Polygon from a LineString input, preserving the vertex order as given. Unlike [ST_MAKEPOLYGON](/tidb-cloud-lake/sql/st-makepolygon.md), this function does not reorder vertices to enforce a specific winding direction. + +## Syntax + +```sql +ST_MAKEPOLYGONORIENTED() +``` + +## Arguments + +| Arguments | Description | +|--------------|-----------------------------------------------------------------------------| +| `` | A GEOMETRY expression of type LineString. Must have at least 4 points with the first and last point being identical. | + +> **Note:** +> +> - Only LineString input is accepted. Other types produce an error. +> - The LineString must form a valid polygon (no self-intersections). + +## Return Type + +Geometry. + +## Examples + +```sql +SELECT ST_ASWKT( + ST_MAKEPOLYGONORIENTED(TO_GEOMETRY('LINESTRING(0 0, 1 0, 1 2, 0 2, 0 0)')) +); + +┌──────────────────────────────────┐ +│ result │ +├──────────────────────────────────┤ +│ POLYGON((0 0,1 0,1 2,0 2,0 0)) │ +└──────────────────────────────────┘ + +-- Reversed winding order is preserved +SELECT ST_ASWKT( + ST_MAKEPOLYGONORIENTED(TO_GEOMETRY('LINESTRING(0 0, 0 2, 1 2, 1 0, 0 0)')) +); + +┌──────────────────────────────────┐ +│ result │ +├──────────────────────────────────┤ +│ POLYGON((0 0,0 2,1 2,1 0,0 0)) │ +└──────────────────────────────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-perimeter.md b/tidb-cloud-lake/sql/st-perimeter.md new file mode 100644 index 0000000000000..2b67320c05935 --- /dev/null +++ b/tidb-cloud-lake/sql/st-perimeter.md @@ -0,0 +1,61 @@ +--- +title: ST_PERIMETER +summary: Returns the perimeter of polygon(s) in a GEOMETRY object, measured in the units of the coordinate system. +--- + +# ST_PERIMETER + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns the perimeter of polygon(s) in a GEOMETRY object, measured in the units of the coordinate system. + +## Syntax + +```sql +ST_PERIMETER() +``` + +## Arguments + +| Arguments | Description | +|--------------|------------------------------------------------------| +| `` | A GEOMETRY expression. | + +> **Note:** +> +> Returns 0 if the input is not a Polygon or MultiPolygon. + +## Return Type + +Double. + +## Examples + +```sql +SELECT ST_PERIMETER(TO_GEOMETRY('POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))')); + +┌─────────────────────────────────────────────────────────────────┐ +│ st_perimeter(to_geometry('polygon((0 0, 0 1, 1 1, 1 0, 0 0))')) │ +├─────────────────────────────────────────────────────────────────┤ +│ 4.0 │ +└─────────────────────────────────────────────────────────────────┘ + +SELECT ST_PERIMETER(TO_GEOMETRY('POLYGON((0 0, 0 3, 4 3, 4 0, 0 0))')); + +┌─────────────────────────────────────────────────────────────────┐ +│ st_perimeter(to_geometry('polygon((0 0, 0 3, 4 3, 4 0, 0 0))')) │ +├─────────────────────────────────────────────────────────────────┤ +│ 14.0 │ +└─────────────────────────────────────────────────────────────────┘ + +-- Non-polygon types return 0 +SELECT ST_PERIMETER(TO_GEOMETRY('POINT(1 1)')); + +┌──────────────────────────────────────┐ +│ st_perimeter(to_geometry('point(1 1)')) │ +├──────────────────────────────────────┤ +│ 0.0 │ +└──────────────────────────────────────┘ +``` diff --git a/tidb-cloud-lake/sql/st-simplify.md b/tidb-cloud-lake/sql/st-simplify.md new file mode 100644 index 0000000000000..f773e27d0274c --- /dev/null +++ b/tidb-cloud-lake/sql/st-simplify.md @@ -0,0 +1,73 @@ +--- +title: ST_SIMPLIFY +summary: Returns a simplified version of a GEOMETRY object by removing vertices where the distance to the resulting edge is within the specified tolerance. Uses the Ramer-Douglas-Peucker algorithm. +--- + +# ST_SIMPLIFY + +> **Note:** +> +> Introduced or updated in v1.2.911. + +Returns a simplified version of a GEOMETRY object by removing vertices where the distance to the resulting edge is within the specified tolerance. Uses the Ramer-Douglas-Peucker algorithm. + +## Syntax + +```sql +ST_SIMPLIFY(, ) +``` + +## Arguments + +| Arguments | Description | +|---------------|-----------------------------------------------------------------------------| +| `` | A GEOMETRY expression. Works on LineString, MultiLineString, Polygon, and MultiPolygon. No effect on Point or MultiPoint. | +| `` | The maximum distance tolerance for vertex removal. | + +> **Note:** +> +> GeometryCollection is not supported. + +## Return Type + +Geometry. + +## Examples + +```sql +SELECT ST_ASWKT( + ST_SIMPLIFY( + TO_GEOMETRY('LINESTRING(0 0, 1 0, 1 1, 2 1)'), 0.5 + ) +) AS simplified; + +┌──────────────────────┐ +│ simplified │ +├──────────────────────┤ +│ LINESTRING(0 0,2 1) │ +└──────────────────────┘ + +SELECT ST_ASWKT( + ST_SIMPLIFY( + TO_GEOMETRY('LINESTRING(1100 1100, 2500 2100, 3100 3100, 4900 1100, 3100 1900)'), 500 + ) +) AS simplified; + +┌──────────────────────────────────────────────────────┐ +│ simplified │ +├──────────────────────────────────────────────────────┤ +│ LINESTRING(1100 1100,3100 3100,4900 1100,3100 1900) │ +└──────────────────────────────────────────────────────┘ + +SELECT ST_ASWKT( + ST_SIMPLIFY( + TO_GEOMETRY('POLYGON((0 0, 1 0, 1 1, 0.5 0.5, 0 1, 0 0))'), 0.6 + ) +) AS simplified; + +┌──────────────────────────────────┐ +│ simplified │ +├──────────────────────────────────┤ +│ POLYGON((0 0,1 0,1 1,0 1,0 0)) │ +└──────────────────────────────────┘ +``` diff --git a/tidb-cloud-lake/sql/system-settings.md b/tidb-cloud-lake/sql/system-settings.md index c00a81b6a9e76..14d3747f7b453 100644 --- a/tidb-cloud-lake/sql/system-settings.md +++ b/tidb-cloud-lake/sql/system-settings.md @@ -41,7 +41,7 @@ enable_distributed_recluster |0 |0 |SESSION|E enable_distributed_replace_into |0 |0 |SESSION|Enable distributed execution of replace into. |UInt64| enable_dphyp |1 |1 |SESSION|Enables dphyp join order algorithm. |UInt64| enable_experimental_merge_into |0 |0 |SESSION|Enable experimental merge into. |UInt64| -enable_experimental_row_access_policy |0 |0 |SESSION|Enables the experimental row access policy feature. Required to use CREATE/DROP ROW ACCESS POLICY. |UInt64| +enable_experimental_row_access_policy |0 |0 |SESSION|Enables the experimental row access policy feature. Required to use row access policy statements. |UInt64| enable_experimental_virtual_column |0 |0 |SESSION|Enables the experimental virtual column feature. |UInt64| enable_geo_create_table |0 |0 |SESSION|Enables the creation of geographic tables. |UInt64| enable_hive_parquet_predict_pushdown |1 |1 |SESSION|Enable hive parquet predict pushdown by setting this variable to 1, default value: 1 |UInt64|