Virtual machines

The Virtual machines page lists every VM and container in the cluster, wherever it runs. Containers are marked Container. Each row shows:

• Name, with its size (CPUs, memory, disk), image, VLAN, and the OS the guest reports. Notes, attached install media, USB devices, snapshots and the last backup and offsite copy show underneath.
• Node (with two or more nodes), and with three or more nodes, Replica on: the node that keeps its copy and takes it over if its node fails.
• Status: Running, Stopped, Starting, Error, Saved (stopped with its memory saved), or Importing (its disk is still being converted by an import; it cannot be started until that is done).
• Usage: CPU, network, memory and disk while it runs.
• Protection: how current its copy on another node is (Synced 40s ago, First copy in progress, Last synced … ago, Copy failing), or Not protected. While a copy is being sent to another node, Copying shows below it: for a first copy, a bar with how much of the disk has been sent, the speed and about how long is left; for later copies, which send only the changes, how much has been sent so far.
• Addresses: the IP addresses the VM has.

Each row has a Start button (or Console, when it runs) and a ⋯ menu with everything else, including Stop.

Create a VM

To bring over a VM you already have in VMware or VirtualBox, use Import instead: see Importing from VMware and VirtualBox.

1. Click New VM. Leave Virtual machine chosen at the top; Container creates a container instead.
2. Fill in Name: lowercase letters, numbers and dashes, starting with a letter, up to 63 characters.
3. Choose the Operating system:
   • Images (ready in a minute): Linux cloud images from images.linuxcontainers.org. Pick a Version below it. The image is downloaded the first time it is used, then reused.
   • Your saved images: VMs you saved with Save as image (see Images).
   • Install from an ISO: Windows or Other operating system, from an ISO on the Images page.
4. Set CPUs, Memory and Disk (GiB) (2 CPUs, 2 GiB and 20 GiB to start with; memory from 512 MiB to 32 GiB, disk at least 4 GiB). Disks are thin: unused space takes nothing on the storage pool.
5. For a cloud image, fill in Sign-in (below) and, optionally, Address.
6. Optional: Virtual TPM (for TPM-based disk encryption), VLAN, and with several nodes, Run on.
7. Leave Protect this VM ticked to keep a copy on another node and start it there if its node fails.
8. Click Create VM.

A VM from a cloud image starts on its own. A VM from an ISO is created stopped and opens in its Screen console, so you can start it there and catch the installer's boot prompt.

Sign-in for cloud images

Cloud images have no password. To sign in over SSH or at the console, set up a user here; it is created on first boot with cloud-init.

Field	What it does
User	Lowercase letters, digits, - and _, starting with a letter. Not root. The user can use sudo.
Password	At least 8 characters. Stored hashed.
SSH public key (optional)	One or more public keys. A user needs a password, a key, or both.
Packages to install (optional)	Package names separated by commas or spaces, installed on first boot.

The defaults for User, SSH keys and Packages come from Settings → New VMs (see Settings). Without a user, sign in through the Root shell tab of the console.

Only cloud-init versions can be set up this way. Versions marked no cloud-init, and other variants such as desktop, get no user, password or fixed address, and the dialog says so; use the Root shell tab.

Fixed address

By default a VM gets its address from DHCP. Tick Fixed address and give:

• Address with its network size, like 192.168.1.20/24
• Gateway, like 192.168.1.1
• DNS (optional): addresses separated by spaces or commas; the gateway if empty

The address is set on first boot. Make sure nothing else uses it. Clones, copies from a snapshot, restored backups and VMs made from a saved image get DHCP instead. For an install from an ISO, set the address inside the OS.

Windows

1. On the Images page, add the Windows installer ISO, and click Get VirtIO drivers once.
2. In New VM, choose Windows. staxmax raises the size to at least 4 CPUs, 8 GiB memory and an 80 GiB disk (64 GiB minimum), and picks the Installer ISO and Drivers ISO it finds.
3. Click Create VM. The VM opens in its Screen console, stopped. Start it and press a key when it says Press any key to boot from CD or DVD.
4. After Windows is installed, open the drivers drive and run virtio-win-guest-tools.exe. Without the VirtIO drivers, Windows has no network.
5. Choose Eject install media from the VM's menu.

Windows VMs always have a virtual TPM.

Other operating systems from an ISO

Choose Other operating system and an Installer ISO. Tick Legacy BIOS for systems that cannot boot UEFI; it turns Secure Boot off. Otherwise Secure Boot is on: if the installer doesn't boot, turn Secure Boot off under ⋯ → Edit. After installing, choose Eject install media. The Root shell and Files console tabs need the Incus agent (incus-agent) installed in the guest.

Start, stop and restart

• Start starts a stopped VM. A VM in the Saved state resumes where it was.
• ⋯ → Stop asks the VM to shut down.
• ⋯ → Restart restarts it.
• ⋯ → Force stop turns it off at once without a shutdown. Anything it has not written to disk is lost.
• ⋯ → Save memory and stop (only when memory saving is allowed, see Edit a VM) stops it with its memory saved. It stays stopped, after a reboot too, until you click Start, which resumes it exactly where it was. The saved memory is not copied to other nodes: if the VM starts on another node, after a failover for example, it boots fresh.

Stop and Force stop ask for confirmation.

Edit a VM

Choose ⋯ → Edit. Some settings change while the VM runs; others need it stopped, and the dialog says so.

Section	Settings	While running?
Resources	CPUs, Disk (GiB)	Yes. A disk can only grow; the guest sees the new space once its partition is extended (cloud images do this at the next boot).
	Memory	Stop it first.
Disk speed	Read limit (MB/s), Write limit (MB/s)	Yes. Keeps one busy VM from slowing the others and replication.
	Priority (0 lowest, 5 normal, 10 highest)	Stop it first. When the disk is busy, higher priority goes first.
Network	VLAN of eth0; Add a network card (up to three more), each with its own VLAN	Stop it first. Empty means the LAN, untagged.
Scheduled snapshots	Take one: Never, Every hour, Every day (midnight), Every week; Keep for that many hours, days or weeks (24 hours, 7 days or 4 weeks to start with)	Yes. Snapshots are only taken while the VM runs.
Start order	Order (-100 to 100) and Then wait (seconds)	Yes
More	Back up every night	Yes
	Allow saving memory and moving live	Stop it first. Needs disk space for the memory; doesn't work with USB passthrough.
	Legacy BIOS: for systems that cannot boot with UEFI; turns Secure Boot off	Stop it first.
	Virtual TPM: for disk encryption that keeps its key in the TPM (BitLocker, systemd-cryptenroll); always on for Windows	Stop it first.
	Secure Boot: only boots systems signed with Microsoft's keys, as Windows and most Linux distributions are; turn it off for others (BSDs, older or unusual Linux versions, some installers). Catalog images that can't boot with it, such as Alpine, get it off automatically.	Stop it first.
Notes	Free text shown under the VM's name, up to 2000 characters	Yes

Start order applies when a node starts several VMs at once, after a reboot or a failover: lower numbers start first, and the next ones wait the given time after this one has started. Give a DNS or database server 0 and a wait, and the VMs that depend on it 1.

With a VLAN, every node's switch port must carry that VLAN tagged, and the nodes must reach the VM's address on it for automatic failover. See Networking.

Snapshots

A snapshot is instant and takes almost no space until the disk changes. Take one before an update. Choose ⋯ → Snapshots (or click the snapshot count under the VM's name).

• Take snapshot: optionally give a Name (letters, digits, -, _ and ., not starting with stax-).
• Restore: rolls the VM back to that snapshot. Everything written since is lost. A running VM is stopped first and started again afterwards. Snapshots taken after the one you restore are deleted too, since the disk can only roll back to one point. For a protected VM, its copy is then replaced with a fresh full copy.
• Copy as VM: give the Name of the new VM and click Create VM. It is created stopped and unprotected on the same node, from the snapshot, with new MAC addresses on every network card and its own machine ID and hostname. The original is not touched.
• Delete: removes the snapshot.

Scheduled snapshots are marked scheduled and show when they will be removed.

warning

Snapshots live on the same disk as the VM (and its copy), so they protect against mistakes, not against losing the disk. Use backups for that.

Clone a VM

Choose ⋯ → Clone, give the Name of the copy, and click Clone. With two or more nodes, tick Protect the copy too to protect it.

The clone is a copy of the disk as it is now, on the same node. The original keeps running. The clone is stopped, has new MAC addresses on every network card (so it gets its own IP addresses and never clashes with the original), and takes almost no space until it changes. Snapshots and USB devices are not copied, and a fixed address is dropped in favour of DHCP.

The first time the clone runs and its Incus agent answers, staxmax gives it a new machine ID, renames the host after the clone, and restarts it once. Everything else inside the guest is the original's, so check its settings before relying on it. Without the agent (Windows, or an OS installed from an ISO without incus-agent), this doesn't happen and the clone keeps the original's hostname.

Move a VM to another node

Moving needs at least two nodes, and the target node must be online.

• ⋯ → Move to node: stops the VM, sends the latest changes, and starts it on the other node. With three or more nodes, a move to a node that doesn't hold its copy is marked (full copy) and takes longer. With two nodes, when a live move is also offered, this one reads Move to node (restart).
• ⋯ → Move live to node: the VM keeps running while it moves. Offered when the VM runs, Allow saving memory and moving live is on, it has no USB devices, and both nodes have the same CPU model.

To empty a node before a reboot, use maintenance mode instead of moving VMs one by one.

Protection

⋯ → Turn on protection keeps a copy of the VM on another node, updated at the interval set in Settings → Replication, and starts it there if its node fails. Turn off protection stops copying. See Replication and Failover.

Force start after a failure

If a node stops answering and staxmax cannot prove its VMs have stopped (no smart plug cut its power, and no majority or tiebreaker settles it), it does not start them elsewhere on its own. In that case the menu offers Force start on this node for:

• a VM whose node is down (with three or more nodes, only on the node holding its copy), or
• a VM on this node when every other node is down.

The dialog asks you to tick I have checked that node is off. The VM starts from its last synced copy.

danger

Only force start after checking that the other node is powered off or unplugged from the network. If two nodes run the same VM, the copies drift apart and changes on one of them are lost.

Preserved copies

When a node comes back and finds that another node took over one of its VMs, it keeps its own copy under a new name (<vm>-orphan-<date>), marked Preserved copy. It may hold changes made after the last sync, and can't be started. staxmax never deletes it automatically: copy anything you need out of it, then delete it.

USB devices

Choose ⋯ → USB devices to pass a USB device plugged into the VM's node through to the VM.

1. The dialog lists the devices plugged into that node, with vendor and product IDs, serial number, and the port each is plugged into.
2. Click Pass through. The VM gets the device at once, running or not, and again whenever it is plugged back in.
3. Remove takes it away. A device used by another VM shows Used by vm.

Identical devices (the same IDs and serial number, or none) are told apart by the USB port they are plugged into, so each can go to its own VM. Keep such a device in its port: plugged into another one, the VM doesn't get it. One passed through by an earlier version of staxmax goes with all the identical devices; the dialog says so, and you can remove it and pass through just the one you want.

Devices passed through but not plugged in are listed under Not plugged in.

For a container, the dialog warns under a device the node drives as a sound card, serial port, disk, camera, Bluetooth adapter or network card: a container gets only the raw USB device, so pass those through to a VM instead.

note

A USB device stays with the node it is plugged into. If the VM moves or fails over, it runs without the device there (unless the same device is plugged in there too) and gets it back when it returns.

Back up and download

• ⋯ → Back up now exports the VM to the backup storage right away, in the background (offsite instead, if there is no backup storage). See Backups.
• ⋯ → Back up offsite now, shown when offsite storage is set up, sends an encrypted copy of the VM to the offsite storage right away, in the background. See Offsite backups.
• ⋯ → Download backup has the node build an export of the VM and downloads it to your browser. This can take a while for a large disk.
• ⋯ → Download as OVA (VMs only) downloads the VM as an .ova file for VMware or VirtualBox. See Taking a VM to VMware or VirtualBox.

Save as an image

⋯ → Save as image saves the VM's disk so new VMs can start from it. See Images.

Delete a VM

Stop the VM, choose ⋯ → Delete (it reads Delete (stop it first) while the VM runs), type its name to confirm, and click Delete VM. This deletes the VM, its disk, and its copy on its replica node. You can't undo it.