Persist state to .wal immediately after resource update#5149

Open

denik wants to merge 86 commits into

mainfrom

denik/wal-review1

Commits on May 18, 2026

Add WAL for direct deployment state recovery

Varun Deep Saini
authored and
Tester
committed
Updated tests and enhanced kill caller with an offset

Varun Deep Saini
authored and
Tester
committed
Updated existing tests

Varun Deep Saini
authored and
Tester
committed
test fixes

Varun Deep Saini
authored and
Tester
committed
Fixes

Varun Deep Saini
authored and
Tester
committed
fixed tests

Varun Deep Saini
authored and
Tester
committed
updated tests

varundeepsaini
authored and
Tester
committed
dedup

varundeepsaini
authored and
Tester
committed
Update WAL corrupted entry outputs

varundeepsaini
authored and
Tester
committed
WIP

denik
authored and
Tester
committed
Updated tests and enhanced kill caller with an offset

Varun Deep Saini
authored and
Tester
committed
Updated existing tests

Varun Deep Saini
authored and
Tester
committed
Merge simplified WAL handling into state.go

denik
authored and
Tester
committed
fixes

denik
authored and
Tester
committed
fixes

denik
authored and
Tester
committed
rm unnecessary assert

denik
authored and
Tester
committed
Centralize state open/close lifecycle for direct engine

denik
authored and
Tester
committed
lint

denik
authored and
Tester
committed
fixes

denik
authored and
Tester
committed
lint

denik
authored and
Tester
committed
restore test

denik
authored and
Tester
committed
Skip state file write when WAL has no resource entries

denik
authored and
Tester
committed
Revert per-engine test splits for no-resource deploys

denik
authored and
Tester
committed
fmt

denik
authored and
Tester
committed
Maintain stateIDs as single source of truth for resource IDs

denik
authored and
Tester
committed
Remove defer Close from processBundleRetInternal; align with main approach

denik
authored and
Tester
committed
Rename Close to Finalize; make plan a local var in processBundleRetInternal

denik
authored and
Tester
committed
Restore process.go structure to match main more closely

denik
authored and
Tester
committed
Fix migration count, remove unnecessary defer Finalize, fix errcheck

denik
authored and
Tester
committed
Fix WAL validation: lowercase suffix, partial recovery, directory creation

denik
authored and
Tester
committed
restore non-material changes: assertions and comment

denik
authored and
Tester
committed
deduplicate UpgradeToWrite+defer Finalize in Deploy

denik
authored and
Tester
committed
update out.test.toml

denik
authored and
Tester
committed
fix compilation in configsync/variables.go

denik
authored and
Tester
committed
use OpenWithData+UpgradeToWrite in migrate to avoid disk roundtrip

denik
authored and
Tester
committed
use OpenWithData+UpgradeToWrite in uploadStateForYamlSync

denik
authored and
Tester
committed
remove redundant defer Finalize in Deploy

denik
authored and
Tester
committed
move Finalize into destroyCore before files.Delete

denik
authored and
Tester
committed
remove noise comment from bundle_apply.go

denik
authored and
Tester
committed
fix gofumpt and test output

denik
authored and
Tester
committed
shrink chain-10-jobs to chain-3-jobs

denik
authored and
Tester
committed
fix test names in state_test.go: Close -> Finalize, restore SaveFinalize

denik
authored and
Tester
committed
clean up WAL acceptance tests

denik
authored and
Tester
committed
fix crash-after-create: handle Linux exit code 1 after KillCaller

denik
authored and
Tester
committed
update selftest

denik
authored and
Tester
committed
fix WAL acceptance test hygiene

denik
authored and
Tester
committed
destroyCore: warn on Finalize failure instead of aborting

denik
authored and
Tester
committed
deployCore: use Finalize return value instead of re-opening state

denik
authored and
Tester
committed
statemgmt.Load: accept state directly instead of engine

denik
authored and
Tester
committed
fmt

denik
authored and
Tester
committed
deployCore: move ParseResourcesState before PushResourcesState

denik
authored and
Tester
committed
simplify test

denik
authored and
Tester
committed
update outputs

denik
authored and
Tester
committed
fix Windows replacement for process kill during deployment

denik
authored and
Tester
committed
formatting

denik
authored and
Tester
committed
clean up

denik
authored and
Tester
committed
rm unnecessarial SERIAL replacement

denik
authored and
Tester
committed
rm noop replacement for lineage

denik
authored and
Tester
committed
clean up

denik
authored and
Tester
committed
testserver: replace KillCaller config with HTTP kill API

denik
authored and
Tester
committed
remove blank line

denik
authored and
Tester
committed
wal tests: remove redundant server stubs covered by default handlers

denik
authored and
Tester
committed
wal tests: move test.toml comments to script, remove empty test.toml files

denik
authored and
Tester
committed
Add databricks.yml
Tester
committed
clean up
Tester
committed
add replace_ids.py
Tester
committed
clean up
Tester
committed
test more commands for validation
Tester
committed
remove normal-deploy test
Tester
committed
clean up
Tester
committed
test recover in plan/deploy/summary
Tester
committed
clean up
Tester
committed
add assert_*.py
Tester
committed
corrupted-wal-entry: use envsubst + template file for WAL generation
Tester
committed
kill_caller selftests: move test.toml comments to script, remove empty test.toml files
Tester
committed
formatting
Tester
committed
fix CI: commit missing test.tomls and fix assert_*.py permissions
Tester
committed
fix: use TOML basic strings with \n escapes in Repls to avoid CRLF on Windows

Tester
and
denik
committed
refactor: merge duplicate IsDirect() blocks in dashboard.go

Tester
and
denik
committed
refactor: merge duplicate IsDirect() blocks in deploy.go

Tester
and
denik
committed
restore comment: Finalize is called even on Apply failure to save partial progress

Tester
and
denik
committed
update NEXT_CHANGELOG.md
Tester
committed
rm databricksyyml
Tester
committed
add a warning on Close error
Tester
committed
comment fix
Tester
committed
update max entry to 10MB
Tester
committed