Add scripts for building, publishing images and deploying azure VM#15953
Add scripts for building, publishing images and deploying azure VM#15953christopherco merged 8 commits intotomls/base/mainfrom
Conversation
scripts/common.sh
Outdated
| # Set gallery configuration | ||
| export GALLERY_NAME="basevmimagegallery" | ||
| export GALLERY_IMAGE_DEFINITION="basevmimage" | ||
| export VM_NAME="azl4-vm-1" No newline at end of file |
There was a problem hiding this comment.
comment(non-blocking): It would be good to tag the default name with $USER. Even better would be to add user and a date string or some such
7b45ca2 to
15d1d30
Compare
| } | ||
| } | ||
| tags: {} | ||
| } No newline at end of file |
There was a problem hiding this comment.
issue: please update your text editor configurations to end files with newlines. It's not an issue with this file in particular, but can be an issue if newlines are missed at the end of unix config files
scripts/build-vm-images.sh
Outdated
| @@ -0,0 +1,29 @@ | |||
| #!/bin/bash | |||
| set -euxo pipefail | |||
| TARGET_DIR="/var/tmp/azl-vm-images" | |||
There was a problem hiding this comment.
issue: This creates a target directory in the user's /var/tmp namespace, which is not guaranteed to be safe to manipulate.
Instead, consider reworking to utilize mktemp -d invocation so you get a unique temp directory. This command also has the possible benefit, depending on how the system is configured, to create the directory in a tmpfs, which is backed by RAM for faster I/O transactions than a directory backed by a storage disk media
There was a problem hiding this comment.
I removed the TARGET_DIR which was used for hosting the vhd outside my WSL for experiments.
scripts/build-vm-images.sh
Outdated
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd | ||
|
|
||
| # Copy the resulting VHD to Windows Downloads folder | ||
| cp ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd "$HOME/Downloads/azl4-vm-base.x86_64-0.1.vhd" |
There was a problem hiding this comment.
issue: We don't own the user's $HOME/Downloads directory so we shouldn't copy files into this path
I suggest to remove this line
scripts/deploy-azure-vm.sh
Outdated
| --resource-group "$RESOURCE_GROUP_NAME" \ | ||
| --name "$VM_NAME" \ | ||
| --size "$TEST_VM_SIZE" \ | ||
| --os-disk-size-gb 60 \ |
There was a problem hiding this comment.
question: do we need to set this os disk size to 60GB? Can we successfully deploy with the default size?
scripts/publish-sig-image.sh
Outdated
| storage_account_resource_id="/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME" | ||
|
|
||
| replicationMode="Shallow" | ||
| image_version=$VERSION |
There was a problem hiding this comment.
suggestion (non-blocking): Consider a future change to automatically update the image_version value so users can keep uploading new images to the same SIG, but with different increasing version numbers.
scripts/publish-sig-image.sh
Outdated
| --auth-mode login | ||
| fi | ||
|
|
||
| # Upload the image artifact to Steamboat Storage Account |
There was a problem hiding this comment.
chore: remove references to steamboat
595edac to
ce5cb9d
Compare
There was a problem hiding this comment.
Pull request overview
Adds a small set of developer-facing scripts to build Azure Linux VM images locally, publish them to an Azure Shared Image Gallery (SIG), and deploy a test Azure VM from the published image.
Changes:
- Add
build-vm-images.shto build thevm-baseimage and convert it to a fixed-size VHD. - Add
publish-sig-image.shplus a Bicep template to upload the VHD to Storage and create a SIG image version. - Add
deploy-azure-vm.shand sharedcommon.shconfiguration/helpers to deploy a VM from the SIG image version.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/build-vm-images.sh | Builds the VM image via azldev and converts output VHDX → VHD. |
| scripts/common.sh | Centralizes Azure/SIG variables and helper functions for versioning. |
| scripts/publish-sig-image.sh | Uploads VHD to Storage and creates SIG image definition/version via Bicep deployment. |
| scripts/deploy-azure-vm.sh | Creates an Azure VM using the SIG image version. |
| scripts/azure-gallery-image-base.bicep | Defines the SIG image version resource created from a VHD in Storage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
scripts/build-vm-images.sh
Outdated
| sudo rm -rf ./base/out/images/* | ||
| sudo rm -rf ./base/build/work/vm-base/* | ||
| # Find the absolute path of the directory containing this script | ||
| SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )" | ||
| . "$SCRIPTS_DIR/common.sh" | ||
|
|
||
| # Build vm-base image using azldev | ||
| azldev image build vm-base --local-repo ./base/out --remote-repo "$REMOTE_KOJI_REPO_URL" | ||
|
|
||
| # Convert VHDX to VPC fixed size | ||
| qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \ | ||
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx \ | ||
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd |
There was a problem hiding this comment.
This script deletes/builds using repo-relative paths (./base/...) before ensuring the current working directory is the repo root. If invoked from another CWD it may operate on the wrong paths. Consider deriving a REPO_ROOT from SCRIPTS_DIR (or asserting repo root like scripts/demo-build.sh) and using absolute paths.
| sudo rm -rf ./base/out/images/* | |
| sudo rm -rf ./base/build/work/vm-base/* | |
| # Find the absolute path of the directory containing this script | |
| SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )" | |
| . "$SCRIPTS_DIR/common.sh" | |
| # Build vm-base image using azldev | |
| azldev image build vm-base --local-repo ./base/out --remote-repo "$REMOTE_KOJI_REPO_URL" | |
| # Convert VHDX to VPC fixed size | |
| qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \ | |
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx \ | |
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd | |
| # Find the absolute path of the directory containing this script | |
| SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )" | |
| REPO_ROOT="$( cd "$SCRIPTS_DIR/.." &> /dev/null && pwd )" | |
| . "$SCRIPTS_DIR/common.sh" | |
| sudo rm -rf "$REPO_ROOT/base/out/images/"* | |
| sudo rm -rf "$REPO_ROOT/base/build/work/vm-base/"* | |
| # Build vm-base image using azldev | |
| azldev image build vm-base --local-repo "$REPO_ROOT/base/out" --remote-repo "$REMOTE_KOJI_REPO_URL" | |
| # Convert VHDX to VPC fixed size | |
| qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \ | |
| "$REPO_ROOT/base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx" \ | |
| "$REPO_ROOT/base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd" |
scripts/common.sh
Outdated
| export STORAGE_CONTAINER_NAME="<your-storage-container-name>" | ||
| export PUBLISHER="<your-publisher-name>" | ||
| export OFFER="<your-offer-name>" | ||
| export storage_blob_name="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd" |
There was a problem hiding this comment.
The variable storage_blob_name is exported in lowercase, while the rest of the configuration exports are uppercase. Consider renaming this to STORAGE_BLOB_NAME (and updating references) for consistency with the other environment-style variables.
| export storage_blob_name="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd" | |
| export STORAGE_BLOB_NAME="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd" | |
| export storage_blob_name="$STORAGE_BLOB_NAME" |
scripts/publish-sig-image.sh
Outdated
| fi | ||
|
|
||
| # Ensure STORAGE_ACCOUNT_NAME exists and the managed identity has access | ||
| storage_account_resource_id="/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME" |
There was a problem hiding this comment.
storage_account_resource_id is redefined here even though it was already set earlier in the script. Consider defining it once and reusing it to avoid drift and reduce duplication.
| storage_account_resource_id="/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME" |
scripts/build-vm-images.sh
Outdated
| # Convert VHDX to VPC fixed size | ||
| qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \ | ||
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx \ | ||
| ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd |
There was a problem hiding this comment.
Suggestion: I have the same observation as copilot. An alternative is to find git root git rev-parse --show-toplevel and base paths from that. It would also help to build out common prefix paths, that is a coding pattern.
scripts/common.sh
Outdated
| export STORAGE_CONTAINER_NAME="<your-storage-container-name>" | ||
| export PUBLISHER="<your-publisher-name>" | ||
| export OFFER="<your-offer-name>" | ||
| export storage_blob_name="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd" |
There was a problem hiding this comment.
suggestion (refactor): The time string tag is used elsewhere too. You could make this available via a variable and use it for all artifacts that need a unique-id. VM name, or dir-name and such.
scripts/common.sh
Outdated
| } | ||
|
|
||
| function get-image-version() { | ||
| local OP="${1:-}" |
There was a problem hiding this comment.
nit (non-blocking): A getter should not have side effects even at the cost of more code. I would strongly recommend splitting this into get-image-version() and a separate increment-version() function. You already have all of the required code in hand.
There was a problem hiding this comment.
nit (design, non-blocking, opinion): This script can be stand alone and maybe is meant to be used that way. It does not take any command line arguments but expects environment variables to be used. While that pattern works well there is a locality issue for arguments and context. i.e. any of the environment variables can be set at any moment prior to the invocation. That applies to interactive use and use in a script.
This kind of usage is very common in pipeline scripts I have seen. Unless there is a precedent or coding guideline which recommends this pattern, I would rather use explicit command line arguments for all passed-in parameters.
| @@ -0,0 +1,99 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
Here and other scripts, you may want '-e' even though most commands seems to be in a conditional. Safer to do that.
binujp
left a comment
There was a problem hiding this comment.
Looks good from a functionality standpoint. I would like Chris to chime in from a sig, image, vm etc. location perspective. Holding the approve for that.
b5276c9 to
d6151a1
Compare
… bugs - Rename bicep params sourceDiskId/sourceDiskUrl to sourceStorageAccountId/sourceVhdUri to reflect actual usage - Merge get-latest-version() and get-image-version() into a single get-image-version(); add separate increment-version() - Move REPO_ROOT into common.sh so all scripts share it - Fix variable casing (storage_blob_name → STORAGE_BLOB_NAME) - Remove duplicate storage_account_resource_id assignment
| printf "]"; | ||
| }') | ||
| # Create Image Version from storage account blob | ||
| az deployment group create \ |
There was a problem hiding this comment.
issue: this deployment is failing due to "The source blob is not a page blob" error
There was a problem hiding this comment.
I was able to fix it by updating the azcopy command to add the --blob-type=PageBlob option
azcopy copy "$IMAGE_PATH" "$storage_blob_endpoint" --blob-type=PageBlob
scripts/common.sh
Outdated
| export VM_NAME="${USER}-azl-vm-${TIME_TAG}" | ||
| export SSH_USER="<your-ssh-username>" | ||
| export SSH_PUBLIC_KEY_PATH="<path-to-your-ssh-public-key>" | ||
| export TEST_VM_SIZE="<your-test-vm-size>" |
There was a problem hiding this comment.
suggestion(non-blocking): can this variable take some default value, personally I often have a hard time remembering available SKUs for different image types, and will have to search up docs/notes to figure it out
There was a problem hiding this comment.
+1 on this suggestion. Maybe we can use "Standard_D4s_v7" as a default x86_64 VM Size and "Standard_D4ps_v6" for ARM64 VM size
- Replace hardcoded TEST_VM_SIZE placeholder with arch-based selection: Standard_D4s_v7 for x86_64, Standard_D4ps_v6 for aarch64. - Add --blob-type=PageBlob to azcopy for VHD upload.
v6 and above SKUs need NVMe support advertised in the image definition. While we can likely enable NVMe support advertisement in the image defintion, let's first get a VM working. v5 SKU supports SCSI which is what our image definition is defaulted to advertising. Signed-off-by: Chris Co <chrco@microsoft.com>
… VM (#15953) This PR includes a couple of dev scripts. One can change the variable names in common.sh and use build-vm-images.sh to build VM images locally and publish the image to SIG via publish-sig-image.sh and deploy an Azure VM with deploy-azure-vm.sh for testing purposes.
… VM (#15953) This PR includes a couple of dev scripts. One can change the variable names in common.sh and use build-vm-images.sh to build VM images locally and publish the image to SIG via publish-sig-image.sh and deploy an Azure VM with deploy-azure-vm.sh for testing purposes.
This PR includes a couple of dev scripts. One can change the variable names in
common.shand usebuild-vm-images.shto build VM images locally and publish the image to SIG viapublish-sig-image.shand deploy an Azure VM withdeploy-azure-vm.shfor testing purposes.