Skip to content

Always write out.test.toml; check drift on CI with git diff#4737

Merged
denik merged 6 commits intomainfrom
denik/modify-out-test
Mar 13, 2026
Merged

Always write out.test.toml; check drift on CI with git diff#4737
denik merged 6 commits intomainfrom
denik/modify-out-test

Conversation

@denik
Copy link
Contributor

@denik denik commented Mar 13, 2026 •
edited
Loading

Changes

Instead of failing tests when out.test.toml disagrees, always write it to the source directory. CI now runs git diff --exit-code after tests to catch stale files.

Why

We avoid failing test just because out.test.toml is different. Otherwise you (or agent) needs multiple steps: "make test" to see the difference, run test-update or generate-out-test-toml to fix it then run test again.

We still enforce that out.test.toml is up to date by checking "git diff" afterwards.

denik and others added 2 commits March 13, 2026 16:43
@denik @claude
Always write out.test.toml; check drift on CI with git diff
63358ba 
Instead of failing tests when out.test.toml disagrees, always write it
to the source directory. CI now runs git diff --exit-code after tests
to catch stale files, consistent with how schema drift is detected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@denik @claude
Remove redundant second write of out.test.toml
077639a 
The first write in testAccept covers all tests including skipped ones,
making the write in runTest and the outputs tracking unnecessary.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@denik denik temporarily deployed to test-trigger-is March 13, 2026 15:56 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 13, 2026 •
edited
Loading

Commit: 5fbe2e3

Run: 23073852938

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 8 7 268 787 5:29
💚​ aws windows 8 7 270 785 4:40
💚​ aws-ucws linux 8 7 365 702 6:01
🔄​ aws-ucws windows 2 7 7 366 700 5:59
💚​ azure linux 2 9 271 785 5:51
💚​ azure windows 2 9 273 783 4:13
🔄​ azure-ucws linux 2 1 9 369 698 6:45
🔄​ azure-ucws windows 2 1 9 371 696 5:47
💚​ gcp linux 2 9 267 788 5:57
💚​ gcp windows 2 9 269 786 5:24
16 interesting tests: 7 SKIP, 6 RECOVERED, 3 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🔄​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/ssh/connect-serverless-gpu 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R
Top 20 slowest tests (at least 2 minutes):
duration env testname
4:15 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:35 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:34 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:07 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:05 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:04 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:53 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:52 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:46 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:43 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:39 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:39 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:37 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:36 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:33 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:32 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:11 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:08 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:06 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@denik denik temporarily deployed to test-trigger-is March 13, 2026 16:21 — with GitHub Actions Inactive
@denik @claude
Remove unused skipRepls parameter from doComparison
3397e4e 
The only caller that passed skipRepls=true was for out.test.toml, which
is no longer compared via doComparison.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@denik denik enabled auto-merge March 13, 2026 16:23
@denik denik temporarily deployed to test-trigger-is March 13, 2026 16:24 — with GitHub Actions Inactive
@denik @claude
Regenerate test outputs after out.test.toml handling change
562966f 
out.test.toml is no longer copied to the test tmpdir, so it no longer
appears in sync/upload output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@denik denik temporarily deployed to test-trigger-is March 13, 2026 17:03 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is March 13, 2026 22:57 — with GitHub Actions Inactive
@denik denik added this pull request to the merge queue Mar 13, 2026
Merged via the queue into main with commit 3e38fe2 Mar 13, 2026
18 checks passed
@denik denik deleted the denik/modify-out-test branch March 13, 2026 23:39
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 14, 2026

Commit: 3e38fe2

Run: 23074949027

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 1 1 2 7 7 494 740 52:35
❌​ aws windows 1 1 2 7 7 466 748 53:00
❌​ aws-ucws linux 12 1 2 23 1 755 588 115:13
❌​ aws-ucws windows 12 1 3 23 1 717 599 95:57
❌​ azure linux 1 1 1 9 499 738 51:17
❌​ azure windows 1 1 1 9 471 746 46:03
❌​ azure-ucws linux 50 1 4 8 697 593 89:56
🔄​ azure-ucws windows 2 4 8 709 604 75:39
❌​ gcp linux 1 1 2 1 9 480 746 56:23
❌​ gcp windows 1 1 1 9 454 754 50:26
93 interesting tests: 63 FAIL, 23 RECOVERED, 5 flaky, 1 KNOWN, 1 SKIP
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🔄​f 🟨​K 🟨​K
🔄​ TestAccept/bundle/apps/compute_size ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/apps/compute_size/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/apps/job_permissions ❌​F ❌​F ✅​p ✅​p ❌​F ❌​F ✅​p ✅​p ❌​F ❌​F
❌​ TestAccept/bundle/deployment/bind/model-serving-endpoint ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/deployment/bind/model-serving-endpoint/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/deployment/bind/model-serving-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/integration_whl/serverless 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ❌​F ✅​p 🙈​s 🙈​s
❌​ TestAccept/bundle/integration_whl/serverless/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ❌​F ✅​p
❌​ TestAccept/bundle/invariant/migrate 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ❌​F ✅​p 🙈​s 🙈​s
❌​ TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl ✅​p ✅​p ❌​F ✅​p
❌​ TestAccept/bundle/invariant/no_drift 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ❌​F ✅​p 🙈​s 🙈​s
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl ✅​p ✅​p ❌​F ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl ✅​p ✅​p ❌​F ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume.yml.tmpl ✅​p ✅​p ❌​F ✅​p
❌​ TestAccept/bundle/local_state_staleness ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/local_state_staleness/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/local_state_staleness/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/alerts/basic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/alerts/basic/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/alerts/basic/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/alerts/with_file ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/apps/inline_config ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p 🔄​f ✅​p
❌​ TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ❌​F ✅​p 🔄​f ✅​p
❌​ TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/data_security_mode ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/simple ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/run/spark_python_task ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/change-parent-path ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/change-parent-path/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/change-parent-path/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/change-serialized-dashboard ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/dataset-catalog-schema ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/dataset-catalog-schema/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/dataset-catalog-schema/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/delete-trashed-out-of-band ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/delete-trashed-out-of-band/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/delete-trashed-out-of-band/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/destroy ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/destroy/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/resources/dashboards/destroy/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/postgres_branches/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/postgres_branches/recreate/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/recreate/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/postgres_branches/update_protected/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/update_protected/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/postgres_branches/without_branch_id/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/without_branch_id/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R
❌​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
💚​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
❌​ TestAccept/bundle/resources/postgres_projects/basic 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_projects/basic/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_projects/basic/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_projects/recreate 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_projects/recreate/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_projects/recreate/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_projects/update_display_name/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_projects/update_display_name/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
💚​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/run/app-with-job 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/ssh/connect-serverless-gpu 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s
💚​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
16:11 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
14:54 aws-ucws windows TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
14:44 aws-ucws windows TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct
14:39 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct
14:33 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
14:27 gcp linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
13:42 gcp windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
13:06 aws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
12:00 gcp windows TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=terraform
11:43 gcp linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
10:56 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
10:39 azure linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION
9:58 aws linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
9:20 gcp windows TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct
9:15 aws-ucws windows TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_15.4_LTS
9:15 aws-ucws windows TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_13.3_LTS
9:13 aws-ucws windows TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_14.3_LTS
8:53 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
8:52 azure-ucws linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
8:47 azure windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
8:44 azure linux TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform
8:43 aws-ucws windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
8:40 aws-ucws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:37 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=USER_ISOLATION
8:31 gcp windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:30 aws-ucws linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION
8:27 aws-ucws linux TestAccept/bundle/resources/database_instances/single-instance/DATABRICKS_BUNDLE_ENGINE=direct
8:26 azure-ucws windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
8:18 gcp linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:16 aws-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:16 gcp linux TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS
8:14 gcp windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
8:08 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=SINGLE_USER
8:07 aws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
8:02 aws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:55 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=SINGLE_USER
7:52 aws-ucws windows TestAccept/bundle/resources/permissions/factcheck/DATABRICKS_BUNDLE_ENGINE=terraform
7:49 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION
7:49 aws-ucws linux TestAccept/bundle/resources/database_catalogs/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:48 aws-ucws windows TestAccept/bundle/resources/database_catalogs/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:48 gcp windows TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct
7:47 aws windows TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS
7:47 aws-ucws linux TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_15.4_LTS
7:47 aws-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:46 gcp windows TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform
7:46 aws linux TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS
7:44 gcp linux TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform
7:38 azure linux TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS
7:31 azure-ucws linux TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_15.4_LTS
7:29 gcp linux TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewnester andrewnester andrewnester approved these changes

@anton-107 anton-107 Awaiting requested review from anton-107 anton-107 is a code owner

@pietern pietern Awaiting requested review from pietern pietern is a code owner

@shreyas-goenka shreyas-goenka Awaiting requested review from shreyas-goenka shreyas-goenka is a code owner

@simonfaltum simonfaltum Awaiting requested review from simonfaltum simonfaltum is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@denik @eng-dev-ecosystem-bot @andrewnester