New deploy command

[josephjclark]

This PR basically makes deploy --beta functional (or it will do, when I finish)

Re-write the openfn project deploy command:

• Deploys from the working dir only (whatever you have checked out)
• After deploy, updates the local project file
• Throws on various error cases: mismatching project UUIDs, diverged remote,
• Confirm by default, override with --no-confirm or -y
• Very simple diff
• --dry-run works

This PR also cleans up some general issues in sync and v1 compatibility:

• Fix the v1 state schemas - turns out I've been using the wrong structure this whole time. Sorted now.
• better support for sandboxing metadata
• Add a mode to merge which allows all project properties to merge (needed for deploy)

Future work (real soon):

• Deploy any project file by uuid, alias, id or path. It'll just take what's there locally and push it
• Allow a way to "eject" a project into a v1 state file, for gh sync and as an escape hatch. Probably project export will take the working or OR any project by identifier, and will write it to the file you give it in the format you want
• Better diffing

Fixes #1165

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

• Code generation (copilot but not intellisense)
• Learning or fact checking
• Strategy / design
• Optimisation / refactoring
• Translation / spellchecking / doc gen
• Other
• I have not used AI

You can read more details in our Responsible AI Policy

[josephjclark]

TODO: add an env var PREFER_NEW_SYNC

This can be set in .env for projects which use the new sync stuff, and mean you don't need to pass the --beta flag

This should not be set on projects which need github sync

[josephjclark]

At this point deploy is almost totally written. But after a lot of struggle it's working!

I hit a major problem in projects: turns out we were serializing the v1 state file in the wrong format! Which meant that what I was deploying was getting rejected.

Also hit some problems in merge code where not all properties of project and workflow get merged. I knew that was coming, but I'd totally forgotten it. So far I've dropped in a quick fix

Still to do:

• handle divergence
• testing of deploy
• sneak in sandbox object
• testing of new project merge stuff (and changeset)
• deploy from any arbitrary state or project file
• probably go ahead and add --format to fetch so I can pull a state file raw. Really useful for debugging

[josephjclark]

Almost done, and I've tested that it basically works. There are no tests on deploy - I think I'll manually test a bit more and then come back and do tests later

I want to check:

• No POST if there's no diff
• Error if the remote history has diverged
• Project credentials are stable
• step credentials are stable
• sandbox metadata can be changed (not really expected but it should work)
• collections are stable
• adaptors are stable