You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(appkit): support Lakebase Autoscaling x Apps integration natively (#132)
* feat(appkit): support zero-config Lakebase Autoscaling x Apps integration
* chore: update template to avoid perms issues
* docs: simplify Lakebase database permissions section
Restructure to prioritize the UI-based databricks_superuser flow as the
primary path, clarify DML vs DDL access in the local dev steps, and wrap
the fine-grained SQL script in a collapsible details block.
Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
* fix: align ResourceFieldEntry types with JSON schema and update Lakebase docs
Add missing localOnly, value, resolve, and examples fields to all
ResourceFieldEntry TypeScript types. Fix bundleIgnore description to
reference databricks.yml instead of app.yaml. Simplify Lakebase local
dev setup docs since CLI now auto-generates .env.
Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
---------
Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>
Currently, the Lakebase plugin currently requires a one-time manual setup to connect your Databricks App with your Lakebase database. An automated setup process is planned for an upcoming future release.
9
-
:::
10
-
11
7
Provides a PostgreSQL connection pool for Databricks Lakebase Autoscaling with automatic OAuth token refresh.
12
8
13
9
**Key features:**
14
10
- Standard `pg.Pool` compatible with any PostgreSQL library or ORM
- Built-in OpenTelemetry instrumentation (query duration, pool connections, token refresh)
14
+
- AppKit logger configured by default for query and connection events
18
15
19
-
## Setting up Lakebase
20
-
21
-
Before using the plugin, you need to connect your Databricks App's service principal to your Lakebase database.
22
-
23
-
### 1. Find your app's service principal
24
-
25
-
Create a Databricks App from the UI (`Compute > Apps > Create App > Create a custom app`). Navigate to the **Environment** tab and note the `DATABRICKS_CLIENT_ID` value — this is the service principal that will connect to your Lakebase database.
Create a new Lakebase Postgres Autoscaling project. Navigate to your Lakebase project's branch details and switch to the **Compute** tab. Note the **Project ID**and **Branch ID** from the URL.
18
+
The easiest way to get started with the Lakebase plugin is to use the Databricks CLI to create a new Databricks app with AppKit installed and the Lakebase plugin.
-[Node.js](https://nodejs.org) v22+ environment with `npm`
23
+
- Databricks CLI (v0.287.0 or higher): install and configure it according to the [official tutorial](https://docs.databricks.com/aws/en/dev-tools/cli/tutorial).
24
+
- A new Databricks app with AppKit installed. See [Bootstrap a new Databricks app](../index.md#quick-start-options) for more details.
53
25
54
-
### 4. Get connection parameters
26
+
### Steps
55
27
56
-
Click the **Connect** button on your Lakebase branch and copy the `PGHOST` and `PGDATABASE` values for later.
Navigate to the **SQL Editor** tab on your Lakebase branch. Run the following SQL against the `databricks_postgres` database, replacing the service principal ID in the `DECLARE` block with the `DATABRICKS_CLIENT_ID` value from step 1:
63
-
64
-
```sql
65
-
CREATE EXTENSION IF NOT EXISTS databricks_auth;
66
-
67
-
DO $$
68
-
DECLARE
69
-
sp TEXT :='your-service-principal-id'; -- Replace with DATABRICKS_CLIENT_ID from Step 1
1. Firstly, create a new Lakebase Postgres Autoscaling project according to the [Get started documentation](https://docs.databricks.com/aws/en/oltp/projects/get-started).
29
+
1. To add the Lakebase plugin to your project, run the `databricks apps init` command and interactively select the **Lakebase** plugin. The CLI will guide you through picking a Lakebase project, branch, and database.
30
+
- When asked, select **Yes** to deploy the app to Databricks Apps right after its creation.
For the full configuration reference (SSL, pool size, timeouts, logging, ORM examples), see the [`@databricks/lakebase` README](https://github.com/databricks/appkit/blob/main/packages/lakebase/README.md).
136
-
137
42
## Accessing the pool
138
43
139
44
After initialization, access Lakebase through the `AppKit.lakebase` object:
|`PGHOST`| Lakebase host (auto-injected in production by the `postgres` Databricks Apps resource) |
83
+
|`PGDATABASE`| Database name (auto-injected in production by the `postgres` Databricks Apps resource) |
84
+
|`PGSSLMODE`| TLS mode - set to `require` (auto-injected in production by the `postgres` Databricks Apps resource) |
85
+
86
+
When deployed to Databricks Apps with a `postgres` database resource configured, `PGHOST`, `PGDATABASE`, `PGSSLMODE`, `PGUSER`, `PGPORT`, and `PGAPPNAME` are automatically injected by the platform. Only `LAKEBASE_ENDPOINT` must be set explicitly:
87
+
88
+
```yaml
89
+
env:
90
+
- name: LAKEBASE_ENDPOINT
91
+
valueFrom: postgres
92
+
```
93
+
94
+
For local development, the `.env` file is automatically generated by `databricks apps init` with the correct values for your Lakebase project.
95
+
96
+
For the full configuration reference (SSL, pool size, timeouts, logging, ORM examples), see the [`@databricks/lakebase` README](https://github.com/databricks/appkit/blob/main/packages/lakebase/README.md).
97
+
98
+
### Pool configuration
161
99
162
100
Pass a `pool` object to override any defaults:
163
101
@@ -174,3 +112,75 @@ await createApp({
174
112
],
175
113
});
176
114
```
115
+
116
+
## Database Permissions
117
+
118
+
When you create the app with the Lakebase resource using the [Getting started](#getting-started-with-the-lakebase) guide, the Service Principal is automatically granted `CONNECT_AND_CREATE` permission on the `postgres` resource. This lets the Service Principal connect to the database and create new objects, but **not access any existing schemas or tables.**
119
+
120
+
### Local development
121
+
122
+
To develop locally against a deployed Lakebase database:
123
+
124
+
1. **Deploy the app first.** The Service Principal creates the database schema and tables on first deploy. Apps generated from `databricks apps init` handle this automatically - they check if tables exist on startup and skip creation if they do.
125
+
126
+
2. **Grant `databricks_superuser` via the Lakebase UI:**
127
+
1. Open the Lakebase Autoscaling UI and navigate to your project's **Branch Overview** page.
128
+
2. Click **Add role** (or **Edit role** if your OAuth role already exists).
129
+
3. Select your Databricks identity as the principal and check the **`databricks_superuser`** system role.
130
+
131
+
3. **Run locally** - your Databricks user identity (email) is used for OAuth authentication. The `databricks_superuser` role gives full **DML access** (read/write data) but **not DDL** (creating schemas or tables) - that's why deploying first matters (see note below).
132
+
133
+
For other users, use the same **Add role** flow in the Lakebase UI to create an OAuth role with `databricks_superuser` for each user.
134
+
135
+
:::tip
136
+
[Postgres password authentication](https://docs.databricks.com/aws/en/oltp/projects/authentication#overview) is a simpler alternative that avoids OAuth role permission complexity. However, it requires you to set up a password for the user in the **Branch Overview** page in the Lakebase Autoscaling UI.
137
+
:::
138
+
139
+
:::info[Why deploy first?]
140
+
When the app is deployed, the Service Principal creates schemas and tables and becomes their owner. A `databricks_superuser` has full **DML access** (SELECT, INSERT, UPDATE, DELETE) to these objects, but **cannot run DDL** (CREATE SCHEMA, CREATE TABLE) on schemas owned by the Service Principal. Deploying first ensures all objects exist before local development begins.
141
+
:::
142
+
143
+
### Fine-grained permissions
144
+
145
+
For most use cases, `databricks_superuser` is sufficient. If you need schema-level grants instead, refer to the official documentation:
0 commit comments