Skip to content

Add scripts for building, publishing images and deploying azure VM#15953

Merged
christopherco merged 8 commits intotomls/base/mainfrom
liunan/scripts
Feb 28, 2026
Merged

Add scripts for building, publishing images and deploying azure VM#15953
christopherco merged 8 commits intotomls/base/mainfrom
liunan/scripts

Conversation

@liunan-ms
Copy link
Contributor

This PR includes a couple of dev scripts. One can change the variable names in common.sh and use build-vm-images.sh to build VM images locally and publish the image to SIG via publish-sig-image.sh and deploy an Azure VM with deploy-azure-vm.sh for testing purposes.

# Set gallery configuration
export GALLERY_NAME="basevmimagegallery"
export GALLERY_IMAGE_DEFINITION="basevmimage"
export VM_NAME="azl4-vm-1" No newline at end of file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment(non-blocking): It would be good to tag the default name with $USER. Even better would be to add user and a date string or some such

}
}
tags: {}
} No newline at end of file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: please update your text editor configurations to end files with newlines. It's not an issue with this file in particular, but can be an issue if newlines are missed at the end of unix config files

@@ -0,0 +1,29 @@
#!/bin/bash
set -euxo pipefail
TARGET_DIR="/var/tmp/azl-vm-images"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: This creates a target directory in the user's /var/tmp namespace, which is not guaranteed to be safe to manipulate.

Instead, consider reworking to utilize mktemp -d invocation so you get a unique temp directory. This command also has the possible benefit, depending on how the system is configured, to create the directory in a tmpfs, which is backed by RAM for faster I/O transactions than a directory backed by a storage disk media

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the TARGET_DIR which was used for hosting the vhd outside my WSL for experiments.

./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd

# Copy the resulting VHD to Windows Downloads folder
cp ./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd "$HOME/Downloads/azl4-vm-base.x86_64-0.1.vhd"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: We don't own the user's $HOME/Downloads directory so we shouldn't copy files into this path

I suggest to remove this line

--resource-group "$RESOURCE_GROUP_NAME" \
--name "$VM_NAME" \
--size "$TEST_VM_SIZE" \
--os-disk-size-gb 60 \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: do we need to set this os disk size to 60GB? Can we successfully deploy with the default size?

storage_account_resource_id="/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME"

replicationMode="Shallow"
image_version=$VERSION
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): Consider a future change to automatically update the image_version value so users can keep uploading new images to the same SIG, but with different increasing version numbers.

--auth-mode login
fi

# Upload the image artifact to Steamboat Storage Account
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chore: remove references to steamboat

@liunan-ms liunan-ms marked this pull request as ready for review February 25, 2026 19:12
Copilot AI review requested due to automatic review settings February 25, 2026 19:12
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a small set of developer-facing scripts to build Azure Linux VM images locally, publish them to an Azure Shared Image Gallery (SIG), and deploy a test Azure VM from the published image.

Changes:

  • Add build-vm-images.sh to build the vm-base image and convert it to a fixed-size VHD.
  • Add publish-sig-image.sh plus a Bicep template to upload the VHD to Storage and create a SIG image version.
  • Add deploy-azure-vm.sh and shared common.sh configuration/helpers to deploy a VM from the SIG image version.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
scripts/build-vm-images.sh Builds the VM image via azldev and converts output VHDX → VHD.
scripts/common.sh Centralizes Azure/SIG variables and helper functions for versioning.
scripts/publish-sig-image.sh Uploads VHD to Storage and creates SIG image definition/version via Bicep deployment.
scripts/deploy-azure-vm.sh Creates an Azure VM using the SIG image version.
scripts/azure-gallery-image-base.bicep Defines the SIG image version resource created from a VHD in Storage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +4 to +16
sudo rm -rf ./base/out/images/*
sudo rm -rf ./base/build/work/vm-base/*
# Find the absolute path of the directory containing this script
SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
. "$SCRIPTS_DIR/common.sh"

# Build vm-base image using azldev
azldev image build vm-base --local-repo ./base/out --remote-repo "$REMOTE_KOJI_REPO_URL"

# Convert VHDX to VPC fixed size
qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \
./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx \
./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script deletes/builds using repo-relative paths (./base/...) before ensuring the current working directory is the repo root. If invoked from another CWD it may operate on the wrong paths. Consider deriving a REPO_ROOT from SCRIPTS_DIR (or asserting repo root like scripts/demo-build.sh) and using absolute paths.

Suggested change
sudo rm -rf ./base/out/images/*
sudo rm -rf ./base/build/work/vm-base/*
# Find the absolute path of the directory containing this script
SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
. "$SCRIPTS_DIR/common.sh"
# Build vm-base image using azldev
azldev image build vm-base --local-repo ./base/out --remote-repo "$REMOTE_KOJI_REPO_URL"
# Convert VHDX to VPC fixed size
qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \
./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx \
./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd
# Find the absolute path of the directory containing this script
SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
REPO_ROOT="$( cd "$SCRIPTS_DIR/.." &> /dev/null && pwd )"
. "$SCRIPTS_DIR/common.sh"
sudo rm -rf "$REPO_ROOT/base/out/images/"*
sudo rm -rf "$REPO_ROOT/base/build/work/vm-base/"*
# Build vm-base image using azldev
azldev image build vm-base --local-repo "$REPO_ROOT/base/out" --remote-repo "$REMOTE_KOJI_REPO_URL"
# Convert VHDX to VPC fixed size
qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \
"$REPO_ROOT/base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx" \
"$REPO_ROOT/base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd"

Copilot uses AI. Check for mistakes.
export STORAGE_CONTAINER_NAME="<your-storage-container-name>"
export PUBLISHER="<your-publisher-name>"
export OFFER="<your-offer-name>"
export storage_blob_name="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd"
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable storage_blob_name is exported in lowercase, while the rest of the configuration exports are uppercase. Consider renaming this to STORAGE_BLOB_NAME (and updating references) for consistency with the other environment-style variables.

Suggested change
export storage_blob_name="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd"
export STORAGE_BLOB_NAME="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd"
export storage_blob_name="$STORAGE_BLOB_NAME"

Copilot uses AI. Check for mistakes.
fi

# Ensure STORAGE_ACCOUNT_NAME exists and the managed identity has access
storage_account_resource_id="/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME"
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

storage_account_resource_id is redefined here even though it was already set earlier in the script. Consider defining it once and reusing it to avoid drift and reduce duplication.

Suggested change
storage_account_resource_id="/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT_NAME"

Copilot uses AI. Check for mistakes.
# Convert VHDX to VPC fixed size
qemu-img convert -f vhdx -O vpc -o subformat=fixed,force_size \
./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhdx \
./base/out/images/vm-base/azl4-vm-base.x86_64-0.1.vhd
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: I have the same observation as copilot. An alternative is to find git root git rev-parse --show-toplevel and base paths from that. It would also help to build out common prefix paths, that is a coding pattern.

export STORAGE_CONTAINER_NAME="<your-storage-container-name>"
export PUBLISHER="<your-publisher-name>"
export OFFER="<your-offer-name>"
export storage_blob_name="azl4-vm-base.x86_64-$(date +%Y%m%d-%H%M%S).vhd"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (refactor): The time string tag is used elsewhere too. You could make this available via a variable and use it for all artifacts that need a unique-id. VM name, or dir-name and such.

}

function get-image-version() {
local OP="${1:-}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (non-blocking): A getter should not have side effects even at the cost of more code. I would strongly recommend splitting this into get-image-version() and a separate increment-version() function. You already have all of the required code in hand.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (design, non-blocking, opinion): This script can be stand alone and maybe is meant to be used that way. It does not take any command line arguments but expects environment variables to be used. While that pattern works well there is a locality issue for arguments and context. i.e. any of the environment variables can be set at any moment prior to the invocation. That applies to interactive use and use in a script.

This kind of usage is very common in pipeline scripts I have seen. Unless there is a precedent or coding guideline which recommends this pattern, I would rather use explicit command line arguments for all passed-in parameters.

@@ -0,0 +1,99 @@
#!/bin/bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and other scripts, you may want '-e' even though most commands seems to be in a conditional. Safer to do that.

Copy link
Contributor

@binujp binujp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from a functionality standpoint. I would like Chris to chime in from a sig, image, vm etc. location perspective. Holding the approve for that.

… bugs

- Rename bicep params sourceDiskId/sourceDiskUrl to
  sourceStorageAccountId/sourceVhdUri to reflect actual usage
- Merge get-latest-version() and get-image-version() into a single
  get-image-version(); add separate increment-version()
- Move REPO_ROOT into common.sh so all scripts share it
- Fix variable casing (storage_blob_name → STORAGE_BLOB_NAME)
- Remove duplicate storage_account_resource_id assignment
printf "]";
}')
# Create Image Version from storage account blob
az deployment group create \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: this deployment is failing due to "The source blob is not a page blob" error

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to fix it by updating the azcopy command to add the --blob-type=PageBlob option

azcopy copy "$IMAGE_PATH" "$storage_blob_endpoint" --blob-type=PageBlob

export VM_NAME="${USER}-azl-vm-${TIME_TAG}"
export SSH_USER="<your-ssh-username>"
export SSH_PUBLIC_KEY_PATH="<path-to-your-ssh-public-key>"
export TEST_VM_SIZE="<your-test-vm-size>"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion(non-blocking): can this variable take some default value, personally I often have a hard time remembering available SKUs for different image types, and will have to search up docs/notes to figure it out

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on this suggestion. Maybe we can use "Standard_D4s_v7" as a default x86_64 VM Size and "Standard_D4ps_v6" for ARM64 VM size

liunan-ms and others added 2 commits February 28, 2026 01:11
- Replace hardcoded TEST_VM_SIZE placeholder with arch-based selection:
  Standard_D4s_v7 for x86_64, Standard_D4ps_v6 for aarch64.
- Add --blob-type=PageBlob to azcopy for VHD upload.
v6 and above SKUs need NVMe support advertised in the image definition.
While we can likely enable NVMe support advertisement in the image
defintion, let's first get a VM working. v5 SKU supports SCSI which is
what our image definition is defaulted to advertising.

Signed-off-by: Chris Co <chrco@microsoft.com>
@christopherco christopherco merged commit 20fb953 into tomls/base/main Feb 28, 2026
5 checks passed
@christopherco christopherco deleted the liunan/scripts branch February 28, 2026 07:19
binujp pushed a commit that referenced this pull request Mar 2, 2026
… VM (#15953)

This PR includes a couple of dev scripts. One can change the variable names in common.sh and use build-vm-images.sh to build VM images locally and publish the image to SIG via publish-sig-image.sh and deploy an Azure VM with deploy-azure-vm.sh for testing purposes.
binujp pushed a commit that referenced this pull request Mar 2, 2026
… VM (#15953)

This PR includes a couple of dev scripts. One can change the variable names in common.sh and use build-vm-images.sh to build VM images locally and publish the image to SIG via publish-sig-image.sh and deploy an Azure VM with deploy-azure-vm.sh for testing purposes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants