Skip to content

build-sys: Rework sealing to be one build step #1898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
cgwalters wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

build-sys: Rework sealing to be one build step #1898

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
264cbd0
ci: Add gating property to allow non-blocking test failures
cgwalters Jan 14, 2026
be32547
build-sys: Rework sealing to be one build step
cgwalters Jan 8, 2026
e034260
build-sys: Add BOOTC_extra_src for local dependency development
cgwalters Jan 14, 2026
f6b6da8
build-sys: Add bcvk-run target for quick VM testing
cgwalters Jan 14, 2026
52a83ed
Update composefs-rs to cgwalters pending PR
cgwalters Jan 14, 2026
4468c3f
xtask: Add validate-composefs-digest for debugging mtime issues
cgwalters Jan 14, 2026
3e9520d
build-sys: Rework sealing to be one build step
cgwalters Jan 8, 2026
35cbf82
ci: Add composefs digest validation for sealed UKI builds
cgwalters Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,11 +154,24 @@ jobs:
matrix:
test_os: [fedora-42, fedora-43, fedora-44, centos-9, centos-10]
variant: [ostree, composefs-sealeduki-sdboot]
gating: [true]
exclude:
# centos-9 UKI is experimental/broken (https://github.com/bootc-dev/bootc/issues/1812)
- test_os: centos-9
variant: composefs-sealeduki-sdboot

- test_os: fedora-44
gating: true
include:
# fedora-44 non-gating due to grub2 regression
# https://bugzilla.redhat.com/show_bug.cgi?id=2429501
- test_os: fedora-44
gating: false
variant: ostree
- test_os: fedora-44
gating: false
variant: composefs-sealeduki-sdboot
# Non-gating jobs are allowed to fail without blocking the PR
continue-on-error: ${{ !matrix.gating }}
runs-on: ubuntu-24.04

steps:
Expand Down Expand Up @@ -197,6 +210,10 @@ jobs:
- name: Unit and container integration tests
run: just test-container

- name: Validate composefs digest (sealed UKI only)
if: matrix.variant == 'composefs-sealeduki-sdboot'
run: just validate-composefs-digest

- name: Run TMT integration tests
run: |
if [ "${{ matrix.variant }}" = "composefs-sealeduki-sdboot" ]; then
Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 11 additions & 3 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,17 @@ clap = "4.5.4"
clap_mangen = { version = "0.2.20" }
# Reviewers (including AI tools): The composefs-rs git revision is duplicated for each crate.
# If adding/removing crates here, also update docs/Dockerfile.mdbook and docs/src/internals.md.
composefs = { git = "https://github.com/containers/composefs-rs", rev = "e9008489375044022e90d26656960725a76f4620", package = "composefs", features = ["rhel9"] }
composefs-boot = { git = "https://github.com/containers/composefs-rs", rev = "e9008489375044022e90d26656960725a76f4620", package = "composefs-boot" }
composefs-oci = { git = "https://github.com/containers/composefs-rs", rev = "e9008489375044022e90d26656960725a76f4620", package = "composefs-oci" }
#
# To develop against a local composefs-rs checkout:
# 1. Set BOOTC_extra_src to your composefs-rs path when building:
# BOOTC_extra_src=$HOME/src/composefs-rs just build
# 2. Comment out the git refs below and uncomment the path refs:
composefs = { git = "https://github.com/cgwalters/composefs-rs", rev = "c7257e95b80704adc7e2a4a37e5a10b27cd5237d", package = "composefs", features = ["rhel9"] }
composefs-boot = { git = "https://github.com/cgwalters/composefs-rs", rev = "c7257e95b80704adc7e2a4a37e5a10b27cd5237d", package = "composefs-boot" }
composefs-oci = { git = "https://github.com/cgwalters/composefs-rs", rev = "c7257e95b80704adc7e2a4a37e5a10b27cd5237d", package = "composefs-oci" }
# composefs = { path = "/run/extra-src/crates/composefs", package = "composefs", features = ["rhel9"] }
# composefs-boot = { path = "/run/extra-src/crates/composefs-boot", package = "composefs-boot" }
# composefs-oci = { path = "/run/extra-src/crates/composefs-oci", package = "composefs-oci" }
fn-error-context = "0.2.1"
hex = "0.4.3"
indicatif = "0.18.0"
Expand Down
102 changes: 83 additions & 19 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,9 @@ WORKDIR /src
# See https://www.reddit.com/r/rust/comments/126xeyx/exploring_the_problem_of_faster_cargo_docker/
# We aren't using the full recommendations there, just the simple bits.
# First we download all of our Rust dependencies
# Note: /run/extra-src is optionally bind-mounted via BOOTC_extra_src for local composefs-rs development
RUN --mount=type=cache,target=/src/target --mount=type=cache,target=/var/roothome cargo fetch

FROM buildroot as sdboot-content
# Writes to /out
RUN /src/contrib/packaging/configure-systemdboot download

# We always do a "from scratch" build
# https://docs.fedoraproject.org/en-US/bootc/building-from-scratch/
# because this fixes https://github.com/containers/composefs-rs/issues/132
Expand Down Expand Up @@ -65,6 +62,11 @@ ENV container=oci
STOPSIGNAL SIGRTMIN+3
CMD ["/sbin/init"]

# This layer contains things which aren't in the default image and may
# be used for sealing images in particular.
FROM base as tools
RUN --mount=type=bind,from=packaging,target=/run/packaging /run/packaging/initialize-sealing-tools

# -------------
# external dependency cutoff point:
# NOTE: Every RUN instruction past this point should use `--network=none`; we want to ensure
Expand All @@ -81,14 +83,35 @@ ENV SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH}
# Build RPM directly from source, using cached target directory
RUN --network=none --mount=type=cache,target=/src/target --mount=type=cache,target=/var/roothome RPM_VERSION="${pkgversion}" /src/contrib/packaging/build-rpm

FROM buildroot as sdboot-signed
# This image signs systemd-boot using our key, and writes the resulting binary into /out
FROM tools as sdboot-signed
# The secureboot key and cert are passed via Justfile
# We write the signed binary into /out
# Note: /out already contains systemd-boot-unsigned RPM from initialize-sealing-tools
RUN --network=none \
--mount=type=bind,from=sdboot-content,target=/run/sdboot-package \
--mount=type=secret,id=secureboot_key \
--mount=type=secret,id=secureboot_cert \
/src/contrib/packaging/configure-systemdboot sign
--mount=type=secret,id=secureboot_cert <<EORUN
set -xeuo pipefail

# Extract the unsigned systemd-boot binary from the downloaded RPM
cd /tmp
rpm2cpio /out/*.rpm | cpio -idmv
# Find the extracted unsigned binary
sdboot_unsigned=$(ls ./usr/lib/systemd/boot/efi/systemd-boot*.efi)
sdboot_bn=$(basename ${sdboot_unsigned})
Comment on lines +100 to +101
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The use of ls with a glob to find the systemd-boot EFI binary can be fragile. If the glob matches more than one file, the sdboot_unsigned variable will contain a multi-line string, which could cause sbsign to behave unexpectedly. It's safer to ensure exactly one file is found.

sdboot_unsigned_files=(./usr/lib/systemd/boot/efi/systemd-boot*.efi)
if [ ${#sdboot_unsigned_files[@]} -ne 1 ]; then
    echo "Error: Expected 1 systemd-boot EFI file, but found ${#sdboot_unsigned_files[@]}" >&2
    ls -l ./usr/lib/systemd/boot/efi/
    exit 1
fi
sdboot_unsigned="${sdboot_unsigned_files[0]}"
sdboot_bn=$(basename "${sdboot_unsigned}")

# Sign with sbsign using db certificate and key
sbsign --key /run/secrets/secureboot_key \
--cert /run/secrets/secureboot_cert \
--output /out/${sdboot_bn} \
${sdboot_unsigned}
ls -al /out/${sdboot_bn}
EORUN

# ----
# Unit and integration tests
# The section here (up until the last `FROM` line which acts as the default target)
# is non-default images for unit and source code validation.
# ----

# This "build" includes our unit tests
FROM build as units
Expand All @@ -101,20 +124,61 @@ RUN --network=none --mount=type=cache,target=/src/target --mount=type=cache,targ
FROM buildroot as validate
RUN --network=none --mount=type=cache,target=/src/target --mount=type=cache,target=/var/roothome make validate

# Common base for final images: configures variant, rootfs, and injects extra content
FROM base as final-common
# ----
# Stages for the final image
# ----

# Perform all filesystem transformations except generating the sealed UKI (if configured)
FROM base as base-penultimate
ARG variant
# Switch to a signed systemd-boot, if configured
RUN --network=none --mount=type=bind,from=packaging,target=/run/packaging \
--mount=type=bind,from=sdboot-content,target=/run/sdboot-content \
--mount=type=bind,from=sdboot-signed,target=/run/sdboot-signed \
/run/packaging/configure-variant "${variant}"
--mount=type=bind,from=sdboot-signed,target=/run/sdboot-signed <<EORUN
set -xeuo pipefail
if test "${variant}" = "composefs-sealeduki-sdboot"; then
/run/packaging/switch-to-sdboot /run/sdboot-signed
fi
EORUN
# Configure the rootfs
ARG rootfs=""
RUN --network=none --mount=type=bind,from=packaging,target=/run/packaging /run/packaging/configure-rootfs "${variant}" "${rootfs}"
COPY --from=packaging /usr-extras/ /usr/

# Final target: installs pre-built packages from /run/packages volume mount.
# Use with: podman build --target=final -v path/to/packages:/run/packages:ro
FROM final-common as final
RUN --network=none --mount=type=bind,from=packaging,target=/run/packaging \
/run/packaging/configure-rootfs "${variant}" "${rootfs}"
# Override with our built package
RUN --network=none \
--mount=type=bind,from=packaging,target=/run/packaging \
/run/packaging/install-rpm-and-setup /run/packages
# Inject some other configuration
COPY --from=packaging /usr-extras/ /usr/

# Generate the sealed UKI in a separate stage
# This computes the composefs digest from base-penultimate and creates a signed UKI
# We need our newly-built bootc for the compute-composefs-digest command
FROM tools as sealed-uki
ARG variant
# Install our bootc package (only needed for the compute-composefs-digest command)
RUN --network=none rpm -Uvh --oldpackage /run/packages/bootc-*.rpm
RUN --network=none \
--mount=type=secret,id=secureboot_key \
--mount=type=secret,id=secureboot_cert \
--mount=type=bind,from=packaging,target=/run/packaging \
--mount=type=bind,from=base-penultimate,target=/run/target <<EORUN
set -xeuo pipefail
if test "${variant}" = "composefs-sealeduki-sdboot"; then
/run/packaging/seal-uki /run/target /out /run/secrets
fi
EORUN

# And now the final image
FROM base-penultimate
ARG variant
# Copy the sealed UKI and finalize the image (remove raw kernel, create symlinks)
RUN --network=none \
--mount=type=bind,from=packaging,target=/run/packaging \
--mount=type=bind,from=sealed-uki,target=/run/sealed-uki <<EORUN
set -xeuo pipefail
if test "${variant}" = "composefs-sealeduki-sdboot"; then
/run/packaging/finalize-uki /run/sealed-uki/out
fi
EORUN
# And finally, test our linting
RUN --network=none bootc container lint --fatal-warnings
67 changes: 0 additions & 67 deletions Dockerfile.cfsuki
View file
Open in desktop

This file was deleted.

34 changes: 23 additions & 11 deletions Justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,16 +39,22 @@ lbi_images := "quay.io/curl/curl:latest quay.io/curl/curl-base:latest registry.a
# ```
# TODO: Gather more info and file a buildah bug
generic_buildargs := ""
# Optional: path to extra source directory (e.g. composefs-rs) to bind mount into builds.
# Usage: BOOTC_extra_src=$HOME/src/github/containers/composefs-rs just build
# The directory will be mounted at /run/extra-src inside the container.
# When using this, you must also patch Cargo.toml to use path dependencies:
# composefs = { path = "/run/extra-src/crates/composefs", ... }
# Note: This disables SELinux labeling for the mount.
extra_src := env("BOOTC_extra_src", "")
# Generate podman args for extra source mount if configured
_extra_src_args := if extra_src != "" { "-v " + extra_src + ":/run/extra-src:ro --security-opt=label=disable" } else { "" }
# Args for package building (no secrets needed, just builds RPMs)
base_buildargs := generic_buildargs + " --build-arg=base=" + base + " --build-arg=variant=" + variant
base_buildargs := generic_buildargs + " " + _extra_src_args + " --build-arg=base=" + base + " --build-arg=variant=" + variant
# - scratch builds need extra perms per https://docs.fedoraproject.org/en-US/bootc/building-from-scratch/
# - we do secure boot signing here, so provide the keys
buildargs := base_buildargs \
+ " --cap-add=all --security-opt=label=type:container_runtime_t --device /dev/fuse" \
+ " --secret=id=secureboot_key,src=target/test-secureboot/db.key --secret=id=secureboot_cert,src=target/test-secureboot/db.crt"
# Args for build-sealed (no base arg, it sets that itself)
sealed_buildargs := "--build-arg=variant=" + variant + " --secret=id=secureboot_key,src=target/test-secureboot/db.key --secret=id=secureboot_cert,src=target/test-secureboot/db.crt"

# The default target: build the container image from current sources.
# Note commonly you might want to override the base image via e.g.
# `just build --build-arg=base=quay.io/fedora/fedora-bootc:42`
Expand All @@ -62,8 +68,7 @@ build: package _keygen && _pull-lbi-images
# Resolve to absolute path for podman volume mount
# Use :z for SELinux relabeling
pkg_path=$(realpath target/packages)
podman build --target=final -v "${pkg_path}":/run/packages:ro,z -t {{base_img}}-bin {{buildargs}} .
./hack/build-sealed {{variant}} {{base_img}}-bin {{base_img}} {{sealed_buildargs}}
podman build -v "${pkg_path}":/run/packages:ro,z -t {{base_img}} {{buildargs}} .

# Pull images used by hack/lbi
_pull-lbi-images:
Expand Down Expand Up @@ -126,6 +131,11 @@ package:
test-composefs:
just variant=composefs-sealeduki-sdboot test-tmt readonly local-upgrade-reboot

# Validate composefs digests match between build-time and install-time views.
# This catches mtime/metadata issues that cause sealed boot failures.
validate-composefs-digest:
cargo xtask validate-composefs-digest --base {{base}}

# Only used by ci.yml right now
build-install-test-image: build
cd hack && podman build {{base_buildargs}} -t {{base_img}}-install -f Containerfile.drop-lbis
Expand Down Expand Up @@ -156,8 +166,7 @@ test-tmt *ARGS: build

# Generate a local synthetic upgrade
_build-upgrade-image:
cat tmt/tests/Dockerfile.upgrade | podman build -t {{upgrade_img}}-bin --from={{base_img}}-bin -
./hack/build-sealed {{variant}} {{upgrade_img}}-bin {{upgrade_img}} {{sealed_buildargs}}
cat tmt/tests/Dockerfile.upgrade | podman build -t {{upgrade_img}} --from={{base_img}} -

# Assume the localhost/bootc image is up to date, and just run tests.
# Useful for iterating on tests quickly.
Expand All @@ -170,10 +179,9 @@ build-testimage-coreos PATH: _keygen
#!/bin/bash
set -xeuo pipefail
pkg_path=$(realpath "{{PATH}}")
podman build --target=final -v "${pkg_path}":/run/packages:ro,z \
podman build -v "${pkg_path}":/run/packages:ro,z \
--build-arg SKIP_CONFIGS=1 \
-t {{base_img}}-coreos-bin {{buildargs}} .
./hack/build-sealed {{variant}} {{base_img}}-coreos-bin {{base_img}}-coreos {{sealed_buildargs}}
-t {{base_img}}-coreos {{buildargs}} .

# Run test bootc install on FCOS
# BOOTC_target is `bootc-coreos`, it will be used for bootc install.
Expand All @@ -186,6 +194,10 @@ test-tmt-on-coreos *ARGS:
tmt-vm-cleanup:
bcvk libvirt rm --stop --force --label bootc.test=1

# Run the built image in a transient VM for manual testing/debugging
bcvk-run *ARGS:
bcvk libvirt run --replace --transient --name bootc-dev --secure-boot-keys=$(pwd)/target/test-secureboot --label=bootc.test=1 {{base_img}} {{ARGS}}

# Run tests (unit and integration) that are containerized
test-container: build build-units
podman run --rm --read-only localhost/bootc-units /usr/bin/bootc-units
Expand Down
Loading
Loading