Skip to content

Commit 06a8fe5

Browse files
kvinwangkvinwang
authored
Merge pull request #589 from Dstack-TEE/worktree-vmm-registry-images
feat(vmm): OCI registry image discovery and pull
2 parents 8630143 + 99328ed commit 06a8fe5

File tree

14 files changed

+1184
-17
lines changed

14 files changed

+1184
-17
lines changed

‎Cargo.lock‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/tutorials/guest-image-setup.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,59 @@ The `image_path` should point to `/var/lib/dstack/images`.
230230

231231
If VMM isn't finding the images, verify the path in the configuration matches where you installed them.
232232

233+
## OCI Registry Setup
234+
235+
Guest images can be stored in any OCI-compatible container registry (Docker Hub, GHCR, Harbor, etc.), allowing VMM to discover and pull images directly from the web UI.
236+
237+
### Pushing Images to a Registry
238+
239+
Use the `dstack-image-oci.sh` script to package and push a guest image directory:
240+
241+
```bash
242+
# Push a standard image (auto-tags: version + sha256-hash)
243+
./scripts/dstack-image-oci.sh push /var/lib/dstack/images/dstack-0.5.8 ghcr.io/your-org/guest-image
244+
245+
# Push an nvidia variant
246+
./scripts/dstack-image-oci.sh push /var/lib/dstack/images/dstack-nvidia-0.5.8 ghcr.io/your-org/guest-image
247+
248+
# Push with a custom tag
249+
./scripts/dstack-image-oci.sh push /var/lib/dstack/images/dstack-0.5.8 ghcr.io/your-org/guest-image --tag latest
250+
251+
# List tags in the registry
252+
./scripts/dstack-image-oci.sh list ghcr.io/your-org/guest-image
253+
```
254+
255+
The script reads `metadata.json` and `digest.txt` from the image directory and auto-generates tags:
256+
257+
| Image directory | Generated tags |
258+
|---|---|
259+
| `dstack-0.5.8` | `0.5.8`, `sha256-<hash>` |
260+
| `dstack-dev-0.5.8` | `dev-0.5.8`, `sha256-<hash>` |
261+
| `dstack-nvidia-0.5.8` | `nvidia-0.5.8`, `sha256-<hash>` |
262+
263+
Prerequisites: `docker` CLI (for building), `python3`, registry login (`docker login`).
264+
265+
### Configuring VMM to Use a Registry
266+
267+
Add the `[image]` section to `vmm.toml`:
268+
269+
```toml
270+
[image]
271+
# Local image directory (default: ~/.dstack-vmm/image)
272+
# path = "/var/lib/dstack/images"
273+
274+
# OCI registry for discovering and pulling images
275+
registry = "ghcr.io/your-org/guest-image"
276+
```
277+
278+
After restarting VMM, click **Images** in the web UI to browse the registry. Click **Pull** to download an image — it will be extracted to the local image directory automatically.
279+
280+
### How It Works
281+
282+
- **Push**: The script builds a `FROM scratch` Docker image containing the guest image files (kernel, initrd, rootfs, firmware, metadata) and pushes it to the registry.
283+
- **Pull**: VMM fetches the OCI manifest via the Registry HTTP API v2, downloads each layer blob, and extracts the tar contents into the local image directory. No Docker daemon required on the VMM host.
284+
- **Discovery**: VMM queries the registry's tag list API to show available versions alongside locally installed images.
285+
233286
## Managing Multiple Image Versions
234287

235288
You can have multiple image versions installed simultaneously:

‎scripts/dstack-image-oci.sh‎

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
#!/bin/bash
2+
# SPDX-FileCopyrightText: © 2025 Phala Network <dstack@phala.network>
3+
# SPDX-License-Identifier: Apache-2.0
4+
#
5+
# dstack guest image OCI packaging tool
6+
# Pack and push dstack guest OS images to an OCI-compatible container registry.
7+
set -euo pipefail
8+
9+
usage() {
10+
cat <<EOF
11+
Usage: $0 <command> [options]
12+
13+
Commands:
14+
push <image-dir> <image-ref> [--tag <tag>] Pack and push image to registry
15+
list <image-ref> [--filter <pattern>] List available tags in registry
16+
17+
Arguments:
18+
<image-dir> Path to a dstack guest image directory (contains metadata.json)
19+
<image-ref> Full image reference (e.g., ghcr.io/org/guest-image)
20+
21+
Examples:
22+
$0 push ./dstack-0.5.8 cr.kvin.wang/dstack/guest-image
23+
$0 push ./dstack-nvidia-0.5.8 ghcr.io/dstack-tee/guest-image --tag nvidia-0.5.8
24+
$0 list cr.kvin.wang/dstack/guest-image
25+
$0 list cr.kvin.wang/dstack/guest-image --filter nvidia
26+
EOF
27+
exit 1
28+
}
29+
30+
COMMAND="${1:-}"
31+
[ -z "$COMMAND" ] && usage
32+
shift
33+
34+
# --- PUSH ---
35+
cmd_push() {
36+
local image_dir=""
37+
local image_ref=""
38+
local extra_tag=""
39+
40+
while [ $# -gt 0 ]; do
41+
case "$1" in
42+
--tag) extra_tag="$2"; shift 2 ;;
43+
-h|--help) usage ;;
44+
-*) echo "Unknown option: $1"; exit 1 ;;
45+
*)
46+
if [ -z "$image_dir" ]; then
47+
image_dir="$1"
48+
elif [ -z "$image_ref" ]; then
49+
image_ref="$1"
50+
else
51+
echo "Unexpected argument: $1"; exit 1
52+
fi
53+
shift
54+
;;
55+
esac
56+
done
57+
58+
[ -z "$image_dir" ] && { echo "Error: image directory required"; usage; }
59+
[ -z "$image_ref" ] && { echo "Error: image reference required"; usage; }
60+
[ -d "$image_dir" ] || { echo "Error: $image_dir is not a directory"; exit 1; }
61+
62+
local metadata="$image_dir/metadata.json"
63+
[ -f "$metadata" ] || { echo "Error: metadata.json not found in $image_dir"; exit 1; }
64+
65+
# Read image info
66+
local version
67+
version=$(python3 -c "import json; print(json.load(open('$metadata'))['version'])")
68+
local digest_file="$image_dir/digest.txt"
69+
local os_image_hash=""
70+
if [ -f "$digest_file" ]; then
71+
os_image_hash=$(tr -d '\n\r' < "$digest_file")
72+
fi
73+
74+
# Detect image variant from directory name
75+
local dirname
76+
dirname=$(basename "$image_dir")
77+
local variant=""
78+
if [[ "$dirname" == *-nvidia-dev-* ]]; then
79+
variant="nvidia-dev"
80+
elif [[ "$dirname" == *-nvidia-* ]]; then
81+
variant="nvidia"
82+
elif [[ "$dirname" == *-dev-* ]]; then
83+
variant="dev"
84+
elif [[ "$dirname" == *-cloud-* ]]; then
85+
variant="cloud"
86+
fi
87+
88+
# Build tag list
89+
local tags=()
90+
if [ -n "$extra_tag" ]; then
91+
tags+=("$extra_tag")
92+
else
93+
# Auto-generate tags from variant + version
94+
if [ -n "$variant" ]; then
95+
tags+=("${variant}-${version}")
96+
else
97+
tags+=("${version}")
98+
fi
99+
if [ -n "$os_image_hash" ]; then
100+
tags+=("sha256-${os_image_hash}")
101+
fi
102+
fi
103+
104+
echo "=== Packing dstack guest image ==="
105+
echo " Source: $image_dir"
106+
echo " Version: $version"
107+
echo " Variant: ${variant:-standard}"
108+
echo " Hash: ${os_image_hash:-<none>}"
109+
echo " Registry: $image_ref"
110+
echo " Tags: ${tags[*]}"
111+
echo ""
112+
113+
# Create build context in a temp directory
114+
local tmp_dir
115+
tmp_dir=$(mktemp -d)
116+
trap 'rm -rf "$tmp_dir"' EXIT
117+
118+
# Collect all files
119+
local files=()
120+
for f in "$image_dir"/*; do
121+
[ -f "$f" ] && files+=("$(basename "$f")")
122+
done
123+
124+
# Generate Dockerfile
125+
{
126+
echo "FROM scratch"
127+
for f in "${files[@]}"; do
128+
echo "COPY $f /"
129+
done
130+
echo "LABEL org.opencontainers.image.title=\"dstack-guest-image\""
131+
echo "LABEL org.opencontainers.image.version=\"$version\""
132+
echo "LABEL wang.dstack.os-image-hash=\"${os_image_hash}\""
133+
echo "LABEL wang.dstack.variant=\"${variant:-standard}\""
134+
} > "$tmp_dir/Dockerfile"
135+
136+
# Copy files to build context
137+
for f in "${files[@]}"; do
138+
cp "$image_dir/$f" "$tmp_dir/"
139+
done
140+
141+
# Build
142+
local primary_ref="${image_ref}:${tags[0]}"
143+
echo "Building: $primary_ref"
144+
docker build -t "$primary_ref" "$tmp_dir"
145+
146+
# Tag additional tags
147+
for ((i=1; i<${#tags[@]}; i++)); do
148+
local ref="${image_ref}:${tags[$i]}"
149+
echo "Tagging: $ref"
150+
docker tag "$primary_ref" "$ref"
151+
done
152+
153+
# Push all tags
154+
for tag in "${tags[@]}"; do
155+
local ref="${image_ref}:${tag}"
156+
echo "Pushing: $ref"
157+
docker push "$ref"
158+
done
159+
160+
# Build and push measurement-only image (no rootfs, for verifier)
161+
if [ -n "$os_image_hash" ]; then
162+
local mr_tag="mr-sha256-${os_image_hash}"
163+
local mr_dir
164+
mr_dir=$(mktemp -d)
165+
166+
# Read rootfs filename from metadata to exclude it
167+
local rootfs_name
168+
rootfs_name=$(python3 -c "import json; print(json.load(open('$metadata')).get('rootfs', ''))")
169+
170+
# Collect files excluding rootfs
171+
local mr_files=()
172+
for f in "${files[@]}"; do
173+
if [ "$f" != "$rootfs_name" ]; then
174+
mr_files+=("$f")
175+
cp "$image_dir/$f" "$mr_dir/"
176+
fi
177+
done
178+
179+
{
180+
echo "FROM scratch"
181+
for f in "${mr_files[@]}"; do
182+
echo "COPY $f /"
183+
done
184+
echo "LABEL org.opencontainers.image.title=\"dstack-guest-image-mr\""
185+
echo "LABEL org.opencontainers.image.version=\"$version\""
186+
echo "LABEL wang.dstack.os-image-hash=\"${os_image_hash}\""
187+
echo "LABEL wang.dstack.variant=\"${variant:-standard}\""
188+
echo "LABEL wang.dstack.measurement-only=\"true\""
189+
} > "$mr_dir/Dockerfile"
190+
191+
local mr_ref="${image_ref}:${mr_tag}"
192+
echo ""
193+
echo "Building measurement image (no rootfs): $mr_ref"
194+
echo " Files: ${mr_files[*]}"
195+
docker build -t "$mr_ref" "$mr_dir"
196+
197+
echo "Pushing: $mr_ref"
198+
docker push "$mr_ref"
199+
200+
rm -rf "$mr_dir"
201+
tags+=("$mr_tag")
202+
fi
203+
204+
echo ""
205+
echo "=== Done ==="
206+
for tag in "${tags[@]}"; do
207+
echo " ${image_ref}:${tag}"
208+
done
209+
}
210+
211+
# --- LIST ---
212+
cmd_list() {
213+
local image_ref=""
214+
local filter=""
215+
216+
while [ $# -gt 0 ]; do
217+
case "$1" in
218+
--filter) filter="$2"; shift 2 ;;
219+
-h|--help) usage ;;
220+
-*) echo "Unknown option: $1"; exit 1 ;;
221+
*)
222+
if [ -z "$image_ref" ]; then
223+
image_ref="$1"
224+
else
225+
echo "Unexpected argument: $1"; exit 1
226+
fi
227+
shift
228+
;;
229+
esac
230+
done
231+
232+
[ -z "$image_ref" ] && { echo "Error: image reference required"; usage; }
233+
234+
echo "=== Tags for ${image_ref} ==="
235+
236+
# Parse registry and repo from image_ref
237+
local registry repo
238+
registry="${image_ref%%/*}"
239+
repo="${image_ref#*/}"
240+
241+
local tags_json
242+
tags_json=$(skopeo list-tags "docker://${image_ref}" 2>/dev/null || \
243+
curl -sf "https://${registry}/v2/${repo}/tags/list" 2>/dev/null || \
244+
echo '{"tags":[]}')
245+
246+
python3 -c "
247+
import json, sys, re
248+
data = json.load(sys.stdin)
249+
tags = sorted(data.get('Tags', data.get('tags', [])))
250+
filt = '$filter'
251+
for tag in tags:
252+
if not filt or re.search(filt, tag):
253+
print(f' {tag}')
254+
" <<< "$tags_json"
255+
}
256+
257+
# Dispatch
258+
case "$COMMAND" in
259+
push) cmd_push "$@" ;;
260+
list) cmd_list "$@" ;;
261+
-h|--help) usage ;;
262+
*) echo "Unknown command: $COMMAND"; usage ;;
263+
esac

‎vmm/Cargo.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,9 @@ fatfs.workspace = true
5656
fscommon.workspace = true
5757
or-panic.workspace = true
5858
url.workspace = true
59+
reqwest.workspace = true
60+
flate2.workspace = true
61+
tar.workspace = true
5962

6063
[dev-dependencies]
6164
insta.workspace = true

0 commit comments

Comments
 (0)