osbuild: use bootc install to deploy the container#4224
osbuild: use bootc install to deploy the container#4224dustymabe merged 9 commits intocoreos:mainfrom
Conversation
|
Skipping CI for Draft Pull Request. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request introduces changes to use bootc install to deploy the container, which simplifies the image build process. There are a few critical issues in the YAML manifest related to copy-paste errors that lead to incorrect configurations for the 4k image builds and missing options for loopback devices. These issues need to be addressed.
|
I switched the CI on this to run against |
|
A few diffs picked up by We should probably profile each diff (maybe in coreos/fedora-coreos-tracker#1827) and evaluate whether it's a change we want to make or not. |
|
I can't get a built qemu image to boot. I suspect probably the root= and boot= UUIDs added on the kernel command line? |
do you mind sharing more logs ? What I am getting locally is ignition failing on coreos/fedora-coreos-tracker#1250 |
|
Ahh. I see that too now: |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
looks like removing those make the boot process go further (ignition completes), and out of the initramfs but fail to mount the boot partition. |
|
Blocked on bootc-dev/bootc#1441 |
|
ok this works with the following PRs :
for the bootc PR, it can be built then added into the image through |
jbtrystram
force-pushed
the
osbuild-bootc-install-fs
branch
from
59f1061 to
254f877
Compare
|
follow-up : either find a way to get the boot components inside cosa, or change the bootc code to call bootupd from the deployed root . I think the latter is preferable. |
Made bootc-dev/bootc#1460 |
jbtrystram
force-pushed
the
osbuild-bootc-install-fs
branch
4 times, most recently
from
bb4270f to
310bd60
Compare
|
Alright, marking this as ready for review as all the bits are in place. This will need a release of bootc. |
dustymabe
left a comment
dustymabe
left a comment
There was a problem hiding this comment.
Some comments.
I think there are a few things we need to iron out before we can really move forward with this:
- supporting both old and new paths at the same time
Do we need to? Usually when we make a change this large we roll it out slowly, which means we have to support both ways for some time.
This PR is ignoring that fact, but TBH looking at OSBuild configs that support both would be pretty intimidating, so I'm not excited about trying to do that either. I'd be interested in @jlebon or @travier's thoughts.
- We need to make sure any/every diff that exists between images generated this way and the old way are considered and acknowleged as acceptable before we'd make this change.
|
Ok I just built the failure is expected here, that's because I just built the image. |
|
Filed bootc-dev/bootc#2130 for the |
jbtrystram
force-pushed
the
osbuild-bootc-install-fs
branch
from
e39271a to
c880bc3
Compare
|
This looks reasonable to me. I found #4526 while reviewing this one and it will likely conflict. |
travier
left a comment
travier
left a comment
There was a problem hiding this comment.
Approved but let's wait for the sync with Dusty before merging.
jbtrystram
force-pushed
the
osbuild-bootc-install-fs
branch
from
c880bc3 to
8a83d5c
Compare
|
Test run with this PR and this config : https://jenkins-fedora-coreos-pipeline.apps.ocp.stg.fedoraproject.org/job/build/140/ |
Thanks. Can you add a commit here with the following patch (and also link to bootc-dev/bootc#2130 in a comment). diff --git a/src/cmd-diff b/src/cmd-diff
index 5424fa497..78051e544 100755
--- a/src/cmd-diff
+++ b/src/cmd-diff
@@ -568,6 +568,7 @@ def diff_metal_helper(diff_from, diff_to):
# in filenames with XXXXXXXXXXXXXXXX so that we can get a real diff between
# two of the same files in different builds.
runcmd(['tar', '-xf', tmp_tar.name, '-C', diff_dir,
+ '--exclude', '*backingFsBlockDev',
'--transform', 's|[[:xdigit:]]{64}|XXXXXXXXXXXXXXXX|gx'])
except Exception as e: |
Added a commit. |
jbtrystram
force-pushed
the
osbuild-bootc-install-fs
branch
from
1fb0d0b to
9acecf5
Compare
|
The last push here is purely cosmetic, I addressed all the review comments, a build in jenkins passed all the tests : https://jenkins-fedora-coreos-pipeline.apps.ocp.stg.fedoraproject.org/job/build/140/console Let's merge this ! |
Introduce a new overlay to ship configuration files for bootc and image-builder. These file are sourced from the container during `bootc install to-filesystem`. We can also use this later to ship other bits as we make the container more and more the source of truth, e.g. the partition table definition. This is prep work for [1] [1] coreos/fedora-coreos-tracker#1827 See also coreos/coreos-assembler#4224
The log disk usage message comming every 10 seconds is quite noisy, hide it when we are in a shell in osbuild. I aslo added a couple of helpful tips in comments given by @dustymabe to work with osbuild.
This adds raw-{,4k}-image-bootc manifests that are alternative versions
of the raw-{,4k}-image manifests. This will allow to keep the legacy build
path with a new path that leverages bootc install to filesystem.
In this mode instead of deploying the container to the tree then copy all
the contents to the disk image, use bootc to directly manage the installation
to the target filesystems.
We can conditionalize this until we are confident to roll this to all streams
or move to image-builder.
Requires:
bootc-dev/bootc#1460
bootc-dev/bootc#1451
osbuild/osbuild#2149
osbuild/osbuild#2152
bootc-dev/bootc#1978
bootc-dev/bootc#1909
Create symlinks to the aleph file created by bootc so our tests and tooling find the aleph at the expected path. Note that when moving to image-build we will likely move this to an overlay in the config, that's way easier than having to wire up a blueprint option to allow creating arbitrary symlinks.
By default bootc calls bootups with the `--write-uuid` option that write a stamp file with the boot partition UUID in the UEFI parition. We want to restamp those UUID at first boot, so adding this option make sure bootc does not pass that flag to bootupd. See bootc-dev/bootc#1978
Bootc is looking for the prepare-root config file in the buildroot environnement because the main assumption is that it's run from the target container. However, in osbuild, it's run from te buildroot, because podman inside bwrap (inside supermin in our case) causes issues. It's fine for RHCOS and SCOS where we use the target container as the buildroot but we cannot do that for FCOS because we require python in the buildroot. For now, insert a prepare-root file in the supermin VM (use as the buildroot for osbuild) until either : - bootc learn to look into the container for it [1] - we ship python in our images and can use them as buildroot. Another approach would be to layer python and the osbuild dependencies on top of our image and use that as the buildroot, but that would create room for packages drift (what was in the repos at build time?). At least using COSA it's easier to keep track of versions. [1] bootc-dev/bootc#1410
Tell bootc to enforce that `/etc/containers/policy.json` include a default policy that verify our images signature. When moving to image-builder, this config can be moved into the container itself but as long as we are using osbuild manually we have to carry this in the buildroot. TODO: uncomment this when bootc-dev/bootc#2116 is merged and released See coreos/fedora-coreos-config#4093 (comment)
Until we are able to tune this through and install config file, let's always pass the flag at the osbuild label. This can be dropped once we have bootc-dev/bootc#2116 in bootc.
Add a bootc install config file[1] to set ostree repo options so we inject the `grub_users` config on non-default entries. [1] https://bootc-dev.github.io/bootc/man/bootc-install-config.5.html#ostree See bootc-dev/bootc#1909
bootc install to-filesystem currently leave a leftover blockdev file in the resulting image. This is causing tar to fail on a permission denied error, so let's filter it out for now. This is tracked in bootc-dev/bootc#2130
|
Since there is a merge conflict now (with the merge of #4526) I'll rebase this. |
dustymabe
force-pushed
the
osbuild-bootc-install-fs
branch
from
9acecf5 to
b29f433
Compare
7 participants
Instead of deploying the container to the tree then copy all the contents to the disk image, use bootc to directly manage the installation to the target filesystems.
Right now this requires to use the image as the buildroot so this requires python (for osbuild). This is tracked in [1].
[1] bootc-dev/bootc#1410 Requires osbuild/osbuild#2149