Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ After that, proceed with the specific Datastore documentation you might be inter

Storage in OpenNebula is designed around the concept of datastores. A datastore is any storage medium to store disk images. OpenNebula distinguishes between three different datastore types:

* **Images Datastore**, which stores the base operating system images, persistent data volumes, CD-ROMs.
* **Images Datastore**, which stores the base operating system images, persistent data volumes, CD-ROMs and File Systems.
* **System Datastore** holds disks of running Virtual Machines. Disk are moved from/to the Images when the VMs are deployed/terminated.
* **Files & Kernels Datastore** to store plain files (not disk images), e.g. kernels, ramdisks, or contextualization files. [See details here]({{% relref "file_ds#file-ds" %}}).

Expand All @@ -44,11 +44,14 @@ Storage in OpenNebula is designed around the concept of datastores. A datastore
| [SAN - LVM]({{% relref "../lvm/lvm" %}}) | Images stored as LVs in a SAN, activated directly. | yes | raw (LV) | yes | no | poweroff | yes | KVM | **EE only** |
| [SAN - LVM<br/>(File Mode)]({{% relref "../lvm/filemode" %}}) | Images stored in frontend\*, transferred to hosts via SSH,<br/>and copied to the SAN on instantiation. | yes | raw (LV)<br/>Images: raw/qcow2 | yes** | no | poweroff/live | yes | KVM | EE/CE |
| [SAN - NetApp]({{% relref "../san_storage/netapp" %}}) | Images stored in a NetApp cabin, activated directly. | yes | raw (LUN) | yes | no | poweroff | yes | KVM | **EE only** |
| [FileSystems - VirtioFS]({{% relref "virtiofs_ds}}) | Images are filesytem paths available on the hosts | yes | filesystems (dir) | no | no | no | yes | KVM | EE/CE |

<sup>\*</sup> Additional options available by mounting remote filesystems in the frontend.

<sup>\*\*</sup> Only with LVM Thin mode enabled.

<sup>\*\*\*</sup> Images stored on the frontend just contain metadata. The filesystems to mount should be present and available on the hosts.

### Other storage options

As an admin, restrict the usage of some storage options because these give low-level access to hosts and could become a serious security risk. Please read carefully the corresponding documentation before using them:
Expand Down
101 changes: 101 additions & 0 deletions content/product/cluster_configuration/storage_system/virtiofs_ds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
title: "VirtioFS Datastore"
linkTitle: "Filesystem Storage"
date: "2026-02-18"
description:
categories:
pageintoc: "76"
tags:
weight: "10"
---

<a id="virtiofs-ds"></a>

<!--# VirtioFS Datastores -->

## Overview

OpenNebula supports sharing host directories with Virtual Machines using virtiofs. This enables high-performance, low-latency file sharing between the host and guests. The feature is exposed through a dedicated datastore configured to provide directory-based disks, which are attached to VMs and exported via virtiofs.

Unlike traditional block-based disks, these disks represent host directories. This makes them suitable for:

- Shared data volumes across VMs
- HPC / AI workloads requiring fast POSIX access
- Container-like storage semantics inside VMs

## Requirements

Before creating and using a VirtioFS Datastore, ensure the following:

- The filesystem to be mounted must already exist and be available at the same path on each compute host in the cluster where VMs will be deployed.
- The `virtiofsd` daemon and libvirt/QEMU with VirtioFS support must be installed on each hypervisor node.
- No additional transfer mechanism (`TM_MAD`) is required since the filesystems are already present on the hosts. However, a `TM_MAD` must still be defined in the datastore template (even if OpenNebula does not use it).

{{< alert title="Note" color="success" >}}
OpenNebula requires that `IMAGE_DS` share the same `TM_MAD` as `SYSTEM_DS`. Ensure that the `TM_MAD` for the VirtioFS Datastore matches the one configured for your `SYSTEM_DS`.
{{< /alert >}}

## Creating a VirtioFS Datastore


To create a new Image Datastore, define the following template parameters:

| Attribute | Values | Description |
| ----------- | ---------------- | --------------------------------------------------|
| `NAME` | | Name of the datastore |
| `TYPE` | `IMAGE_DS` | OpenNebula datastore type |
| `DS_MAD` | `virtiofs` | Datastore driver |
| `TM_MAD` | `any existing` | Transfer driver, must match a `SYSTEM_DS` |
| `DISK_TYPE` | `FILESYSTEM` | Directory-based disk type |

This can be done either in Sunstone or through the CLI. For example:

```default
$ cat fs_datastore.txt
NAME = fs_datastore
TYPE = IMAGE_DS
DISK_TYPE = FILESYSTEM
DS_MAD = virtiofs
TM_MAD = local

$ onedatastore create fs_datastore.txt
ID: 101
```

## Usage
Once the Image Datastore is created, register an image that represents a host directory. Typically, only the path needs to be defined.
```
oneimage create --name fs_data --datastore fs_datastore --persistent --path /mnt/data
```

For use cases where the same directory must be shared across multiple VMs simultaneously, create the image as non-persistent and set `READONLY="YES"`.

After the image is registered, it can be used as any other disk by adding it to the VM template using the `DISK` attribute.
```
DISK= [ IMAGE = "fs_data" ]
```

By default, OpenNebula generates:

- A mount point (`MOUNT_POINT`), in the form `/mnt/one-fs-<image_id>`
- A mount tag (`MOUNT_TAG`), in the form `one-<image_id>`

Context packages use these values to automatically mount the filesystem inside the VM via virtiofs. These values can be overridden in the `DISK` attribute to define a custom mount point (and tag) inside the guest.
```
DISK= [ IMAGE = "fs_data", MOUNT_POINT="/srv/data" ]
```

## Considerations

The VirtioFS Datastore behaves differently from other OpenNebula datastores:

- It does not support disk operations such as `snapshots`, `disk-attach`, `resize` or `save-as`.
- There is no image transfer and no support for image cloning.
- The datastore does not store VM disk data. It only stores image metadata. Reported datastore capacity (free, used, total) is always 0.
- Image size reflects the size of the directory, as reported by the `du` command.
- If the VM is configured with Hugepages, they are used as the `MemoryBacking` source with `shared` mode. Otherwise, a `memfd` source is used.
- The same image cannot be attached more than once to the same VM, nor reused with the same mount tag.

{{< alert title="Note" color="success" >}}
Access to the host directory is mapped (`UID`/`GID`) to the `oneadmin` user and group. This mapping can be adjusted globally in `vmm_exec_kvm.conf`, or per host or cluster using the same attributes defined in `vmm_exec_kvm.conf`, for example (`DISK = [ UID_MAP = "1000", GID_MAP = "1000" ]`).
{{< /alert >}}
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ The following attributes can be defined in the template:
| `PERSISTENT` | Optional | `YES`, `NO` | Persistence of the image. If omitted, the default value is `NO`. |
| `PERSISTENT_TYPE`<br/><br/><br/><br/><br/><br/> | Optional<br/><br/><br/><br/><br/><br/> | `IMMUTABLE`, `SHAREABLE`<br/><br/><br/><br/><br/><br/> | `IMMUTABLE` - A special persistent image, that will not be modified.<br/><br/><br/>`SHAREABLE` - Persistent image shareable by multiple VMs. Requires `raw` image `FORMAT` and shared datastore. [Virtualization driver]({{% relref "../opennebula_services_configuration/oned#oned-conf-virtualization-drivers" %}}) needs `SUPPORT_SHAREABLE = "yes"`<br/><br/><br/>This attribute should only be used for special storage configurations.<br/><br/> |
| `DEV_PREFIX` | Optional | Any string e.g `sd`, `hd` | Prefix for the emulated device this image will be mounted at. For instance, `hd`, `sd`, or `vd` for KVM virtio. If omitted, the default value is the one defined in [oned.conf]({{% relref "../opennebula_services_configuration/oned#oned-conf" %}}) (installation default is `hd`). |
| `TARGET` | Optional | Any string | Target for the emulated device this image will be mounted at. For instance, `hdb`, `sdc`. If omitted, it will be [assigned automatically]({{% relref "template#template-disks-device-mapping" %}}). |
| `TARGET` | Optional | Any string | Target for the emulated device this image will be mounted at. For instance, `hdb`, `sdc`. If omitted, it will be [assigned automatically]({{% relref "template#template-disks-device-mapping" %}}). |
| `FORMAT` | Optional | `raw` or `qcow2` | Format of the image backing file. |
| `FS` | Optional | File system name (e.g ext4, xfs, …) | Specific file system type. It is used for formatting datablocks and volatile disks. |
| `PATH` | Mandatory (if no SOURCE) | Any string | Path to the original file that will be copied to the image repository. If not specified for a `DATABLOCK` type image, an empty image will be created. Note that gzipped files are supported and OpenNebula will automatically decompress them. Bzip2 compressed files is also supported, but it’s strongly discouraged since OpenNebula will not calculate its size properly. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,12 @@ In OpenNenbula there are two main places where VM disk images are stored:

## Types and Persistency

OpenNebula uses three different Image types to represent VM disks. A VM can use multiple Image types simultaneously:
OpenNebula uses four different Image types to represent VM disks. A VM can use multiple Image types simultaneously:

* **Operating System** (`OS`): Main disk, the VM will start from this Image. Every VM must include an OS Image.
* **CD-ROM ISO** (`CDROM`): These Images are read-only data. Only one Image of this type can be used in a VM.
* **Data Disk** (`DATABLOCK`): A generic disk to store data. These Images can contain existing data, e.g., a database, or can be formatted as an empty drive.
* **FileSystem** (`FILESYSTEM`): Shared filesystem exposed to the VM. This type does not act as a block disk. Instead, it allows the VM to mount a filesystem avaialbe on the hosts.

Additionally, *file* Images represent plain files that can be used as:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1374,4 +1374,4 @@ The `onevm` command manages OpenNebula virtual machines. The general structure o
| `-y`, `--yaml` | Shows the resource in YAML format |
| `--yearly days` | Repeats the scheduled action on the specififed days of the year. It can be a number between 0,365 separated with commas. For example: `onevm resume 0 --schedule "09/23 14:15" --yearly 30,60`|

<a href="#onevm-command">Back to The `onevm` Command</a>
<a href="#onevm-command">Back to The `onevm` Command</a>
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ Thank you to our incredible community and partners for your continued support in
- [Storage Live Migration for LVM & file-based datastores](../../../product/cluster_configuration/storage_system/overview/#storage-portfolio) with ability to perform live migrations of Virtual Machines across LVM and file-based datastores (both shared, local and lvm thin).
- [NetApp Incremental Backup Support](../../../product/cluster_configuration/san_storage/netapp/) with improved efficiency for NetApp users with new incremental backup capabilities that reduce backup windows and storage consumption by only saving changed data blocks.
- [Support for Pure Storage FlashArray](../../../product/cluster_configuration/san_storage/purestorage/) with a native storage driver for managing the full block storage lifecycle directly through the FlashArray REST API.
- [VirtioFS](../../../product/cluster_configuration/storage_system/virtiofs_ds.md) datastores enable Virtual Machines to directly access host filesystems, providing fast, low-latency shared file access. This simplifies data sharing across VMs while improving performance for data-intensive workloads.

## Sunstone
- Integrated VM Logs in Sunstone providing real-time VM execution logs directly through the Sunstone GUI, enabling faster troubleshooting without needing CLI access.
Expand Down
Loading