Allow installation preset from file for void-installer

[melezhik]

Hi! I have not found discussion feature in this repo so I decided to share my thoughts in this ticket

I am trying to help void Linux devs to automate testing of void-Linux installer and thus the process of bootstrapping of new versions of void Linux from the scratch.

( already informally discussed with @classabbyamp via irc in voidlinux channel )

I see the process like that:

• get a bootable iso ( which is tested )
• kick off qemu vm bootable from the iso
• automate installation via void-installer ( this is where we have a problem )
• shut vm
• start qemu vm without bootable iso
• check that configuration is ok ( like partitioning is ok, locale is set, user account is set, etc )
• optionally runs additional tests like installing packages, configure services, etc ( more like end to end testing )
• last two steps are carried out over ssh

So the only hurdle I see here is void-installer ncurse app is hard to automate , there is way to use tmux , send-keys but it feels clunky and hard to maintain

What if we can allow to have a void-installer configuration preset stored in plain file, so void-installer could run unattended way ?

It would allow us automate many testing scenarios which could be beneficial to void Linux testing …

[hippi777]

a config and/or arguments for everything can solve the problem, the hardest part is to add flexibility to various fs setups, as that can be layered and combined (luks, lvm, mdadm, btrfs.......) with tons of options for tuning and environmental stuffs to align, but a default or some flavors can solve anything, as well as a skip option and doing it before anyhow... (thats where i started years ago (flavors) and then i felt like it wont be cool but messy, and then i reached to native pseudo trees in bash, but this shouldnt block anything here, and jq is still an option...) the other thing is storing secrets inside a config, but defaults can be sufficient here too with a big red warning to change them... my take on that is gocryptfs (after a pretty well-developed solution based on a luks loopback file...)

[Vaelatern]

There is also an autoinstaller present though, which is what I use to build my void boxen. That one is configured by overriding shell functions to achieve different results, including installing additional packages, adding services. The difficulty there is a need to override a kernel command line argument to trigger the installer.

Here is a working tree for a VM I use to host my bambu-farm instance. It's a glibc install despite looking like musl (previous I'd copied it from was musl so I still had that around).

This does not solve testing void-installer. It solves autoinstalling void for a VM. It also uses qemu for VMs with packer as an orchestration layer.

$ cat void-autoinstall.pkr.hcl
source "qemu" "void-musl" {
  iso_url          = "https://repo-fastly.voidlinux.org/live/current/void-live-x86_64-musl-20250202-base.iso"
  iso_checksum     = "sha256:0c46bf13a7a019c1fec0b2969201aecf09ff9a7646ef749e04ea63ead0c86404"
  output_directory = "output-void-musl"
  shutdown_command = "echo 'voidlinux' | sudo -S shutdown -P now"
  disk_size        = "30G"
  format           = "qcow2"
  accelerator      = "kvm"
  headless         = true
  http_directory   = "http"

  memory = 768

  ssh_username = "admin"
  ssh_password = "voidlinux"
  ssh_timeout  = "90m" # How long we give the system to get installed

  boot_wait = "2s"
  boot_command = [
    "<tab><wait>",
    "auto autourl=http://{{.HTTPIP}}:{{.HTTPPort}}/basic-void.cfg",
    "<enter>"
  ]
}

build {
  source "source.qemu.void-musl" {
    vm_name = "void-nova"
  }
}
$ cat http/basic-void.cfg
#!/bin/sh

export username=admin
export password=voidlinux

export XBPS_ARCH=x86_64-musl
export xbpsrepository="https://repo-fastly.voidlinux.org/current/musl"
export XBPS_ARCH=x86_64
export xbpsrepository="https://repo-fastly.voidlinux.org/current"

export timezone="America/New_York"

export pkgs="docker gearlever openjdk21 flatpak mesa-dri xfce4 avahi xorg dbus gnulib xtools libsoup webkit2gtk libOSMesa"

export end_action=func
end_function() {
        printf "Prepare to be useful\n"
        mkdir "${target}/shared"
        mkdir "${target}/etc/sv/toyam-bambujar"
        cat >> "${target}/etc/sv/toyam-bambujar/run" <<EOF
#!/bin/sh
[ ! -d /shared/apps ] && mount /shared || sleep 60
exec chpst -u admin -C /shared/apps/ java -jar bambu-web-*-runner.jar
EOF
        chmod +x "${target}/etc/sv/toyam-bambujar/run"

        printf "Linking default services\n"
        ln -s /etc/sv/docker "${target}/etc/runit/runsvdir/default/docker"
        ln -s /etc/sv/dhcpcd "${target}/etc/runit/runsvdir/default/dhcpcd"
        ln -s /etc/sv/dbus "${target}/etc/runit/runsvdir/default/dbus"
        ln -s /etc/sv/avahi-daemon "${target}/etc/runit/runsvdir/default/avahi-daemon"
        ln -s /etc/sv/sshd "${target}/etc/runit/runsvdir/default/sshd"
        ln -s /etc/sv/agetty-ttyS0 "${target}/etc/runit/runsvdir/default/agetty-ttyS0"
        ln -s /etc/sv/toyam-bambujar "${target}/etc/runit/runsvdir/default/toyam-bambujar"
        echo "***"

        echo "***"
        printf "Let's continue with our life.\n"
        sync
        reboot -f
}

# Skip boot partition
VAI_partition_disk() {
        sfdisk "${disk}" <<EOF
,${swapsize},S
;
EOF
}
VAI_format_disk() {
        mkfs.ext4 -F "${disk}2"
        mkswap -f "${disk}1"
}
export swapsize="1G"

VAI_mount_target() {
        mkdir -p "${target}"
        mount "${disk}2" "${target}"
}

VAI_configure_fstab() {
        uuid_root="$(blkid -s UUID -o value "${disk}2")"
        uuid_swap="$(blkid -s UUID -o value "${disk}1")"
        echo "UUID=$uuid_root / ext4 defaults,errors=remount-ro,noatime 0 1" >> "${target}/etc/fstab"
        echo "UUID=$uuid_swap swap swap defaults 0 0" >> "${target}/etc/fstab"
        echo "host0     /shared 9p      noauto,trans=virtio     0       0" >> "${target}/etc/fstab"
}

# Removable when this is closed https://github.com/void-linux/void-mklive/pull/241
VAI_install_base_system() {
    # Install a base system
    XBPS_ARCH="${XBPS_ARCH}" xbps-install -Sy -R "${xbpsrepository}" -r /mnt base-system grub

    # Install additional packages
    if [  -n "${pkgs}" ] ; then
        # shellcheck disable=SC2086
        XBPS_ARCH="${XBPS_ARCH}" xbps-install -Sy -R "${xbpsrepository}" -r /mnt ${pkgs}
    fi
}

I then run the above VM, just as an FYI, with:

$ cat play.qemuconf
binary  /usr/bin/qemu-system-x86_64
smp     2
m       2048
enable-kvm
drive:
        if      virtio
        file    output-void-musl/void-nova.overlay
boot    c

netdev:
        bridge
        br      $REDACTED
        id      br4
device:
        virtio-net
        netdev  br4
        mac     $REDACTED


#curses
#display none
vnc:
        :1
vga:
        std

virtfs:
        local
        path    ./sync
        id      host0
        mount_tag=host0
        security_model=passthrough

This results in a fully running Void Linux system reachable by SSH and VNC. Fully automated setup.

[Vaelatern]

Biggest problem with my approach is secrets which I am improving, and the fact that any filesystem layout is possible. That means any filesystem layout, so long as I can describe creating it as a shell function. It also doesn't involve the void-installer at all so proves useless to test that piece of software.

[hippi777]

flavors as functions/scripts (preferably with a descriptive name based on its structure, i dunno whether i have an old version around for an example, but likely) are still good for testing, this is a kinda simple approach, while it allows some tuning without weird data structures... in the meantime i realized that jq in itself is horrible when base paths, arbitrary names or whatever are beyond strict regulation (like having spaces and what not in a command line), but proper escaping of commands with eval (innocent in this case) is doable with or without jq, i have a freestanding posix implementation for this that i can share without dumping all of my halfbaked bash codebase for the trees... (im not ready to collab on that, but tell me if u want to see the escaping stuff, all i want is to keep it public domain, but i expect no further development on it, like doing so inside a licensed code base by anyone else, that i couldnt take back after that without any worry...)

the pseudo trees are made of arrays managed in the global scope, they are backed by a lot of codes, they provide structural flexibility (beyond flavors)... arrays play kind with assembling commands+arguments, and the escaping is mainly interesting when there are no arrays (posix shells), it allows to store complex data without messing it up, but it can allow to go beyond shell internals, so combining this with jq can give all the flexibility without either strict forms or degradation... however, at system level, strict forms can be expected, so storing anything in a string or iterating it with for and feeling no pain from the narrow possibilities of a posix shell seems to be fine... an another approach for structuring data is to use things like square brackets, colons and commas as arguments that can be processed as structural elements, but it cant go endlessly, and they should be escaped for the actual commands to prevent misinterpretation, which is most likely with [ x ], but that can be test x... (i developed miracles upon parentheses, commas and empty values in c preprocessor (well, i didnt invent those wheels, just made something way prettier/purer for all that matters, than all the mess i saw in the wild), which seems to be harder, but i still suggest escaping and jq...)

the problem of secrets can be managed by various means... mine based on gocryptfs is almost ready, i can show it... (i had an initial version for all the basic convenience for my needs, a 2nd which blocks writing unencrypted data when it isnt mounted, and a 3rd ongoing that gives a better organization for individual vaults (beyond 2 dirs) and a collection of many, with more possible tuning beyond editing...) but secrets can be asked interactively, they can be stored on tmpfs (unencrypted swap can leak them), individual lines of /etc/shadow can be managed with sed, so they are hashed, and password protected ssh keys can be considered fine too, while testing can simply go with some dummy passwords too...