Managing domains, storage, networks, and recovery with virsh

The commands and XML are generic examples and must be adapted to the host version and actual resources. Network addresses use documentation ranges; choose non-conflicting private networks for real deployments. See the relevant sections for the conditions under which image modification and device passthrough apply.

libvirt is the management abstraction layer in the KVM virtualization stack, and virsh is the command line used to communicate with it. This article focuses on the evolution of the libvirt daemon architecture, a comprehensive classification of virsh commands, the essentials of domain XML configuration, and the command-line workflows most useful in everyday administration.

1. The libvirt daemon architecture#

1.1 What is libvirt?

libvirt is a set of virtualization management APIs that provides a unified XML configuration interface and remote-management protocol for hypervisors such as KVM, QEMU, Xen, LXC, and VMware ESX. In the KVM stack, libvirt sits here:

User (virsh / virt-manager / Cockpit)
       |
       v
  libvirt API (libvirtd / modular daemons)
       |
       v
  QEMU (userspace device emulation)
       |
       v
  KVM (kernel module, CPU/memory virtualization)

libvirt's core responsibilities:

• Domain lifecycle management: start, stop, pause, resume, migrate
• Resource abstraction: Represent CPU, memory, disks, and networking in a unified XML model
• Remote management: Control hypervisors remotely over SSH, TLS, TCP, and other transports
• Security isolation: Support sVirt (SELinux), AppArmor, and cgroups resource limits

1.2 Monolithic vs. modular architecture

graph LR
    subgraph "Traditional Monolithic"
        V1["virsh / virt-manager"] --> L1["libvirtd / (All-in-One)"]
        L1 --> Q1["QEMU"]
        L1 --> N1["Network"]
        L1 --> S1["Storage"]
    end

    subgraph "Modern Modular"
        V2["virsh / virt-manager"] --> P2["virtproxyd / (Proxy)"]
        P2 --> QD["virtqemud / (QEMU VMs)"]
        P2 --> ND["virtnetworkd / (Networks)"]
        P2 --> SD["virtstoraged / (Storage Pools)"]
        P2 --> DD["virtnodedevd / (Host Devices)"]
        P2 --> SEC["virtsecretd / (Secrets)"]
        QD --> Q2["QEMU"]
        ND --> N2["dnsmasq"]
        SD --> S2["Storage Backend"]
    end

Traditional monolithic libvirtd (still used by some distributions):

• One daemon handles everything: VM management, networking, storage, devices, and secrets
• Advantage: simple, with one service and one socket
• Drawbacks: a crash in any function affects the whole daemon; coarse-grained permissions make least-privilege access difficult

Modern modular daemons (whether they are enabled depends on the distribution and installation configuration):

Benefits of the modular architecture:

• Fault isolation: Separate management processes can reduce the spread of some failures; network failures still affect VM connectivity
• Permission granularity: Users can be granted VM-management rights without network/storage-management rights
• On-demand startup: Socket activation reduces the overhead of idle daemon processes
• Independent upgrades: Each module can be restarted separately

Deployment choice: First inspect the host's actual services and sockets, then choose the architecture using the official migration procedure. The nixos-unstable libvirtd module checked for this article still uses monolithic libvirtd; a version number alone does not justify claiming that all NixOS 24.05+ systems use modular daemons. The following service commands apply only to conventional systemd distributions using that architecture. Do not mix them with monolithic sockets that remain enabled. NixOS defines services through its modules.

Enable the modular daemons (for systemd systems):

# 为 QEMU 驱动启用所有模块化守护进程
for drv in qemu interface network nodedev nwfilter secret storage
do
  sudo systemctl unmask virt${drv}d.service
  sudo systemctl unmask virt${drv}d{,-ro,-admin}.socket
  sudo systemctl enable virt${drv}d.service
  sudo systemctl enable virt${drv}d{,-ro,-admin}.socket
  sudo systemctl start virt${drv}d.socket
done

1.3 Connection URIs explained

libvirt connection URIs use a common pattern:

driver[+transport]://[username@]hostname[:port]/[path]

Local connections:

The substantive differences between system and session:

Rule of thumb: Use system for servers and session for desktop development. GPU passthrough requires system.

Remote connections:

Specify a connection URI in virsh:

# 临时指定
virsh -c qemu:///system list --all
virsh -c qemu+ssh://root@198.51.100.100/system list --all

# 设置默认 URI（shell 环境变量）
export LIBVIRT_DEFAULT_URI="qemu:///system"
virsh list --all  # 此后不再需要 -c

1.4 Key configuration files

After changing configuration, restart the corresponding daemon:

# 单体架构
sudo systemctl restart libvirtd

# 模块化架构
sudo systemctl restart virtqemud

2. A comprehensive guide to virsh commands#

virsh is libvirt's main CLI tool. It is a shell wrapper that communicates with the daemon through the libvirt RPC protocol on each invocation. All commands return meaningful exit codes, making them suitable for scripting.

2.1 Domain lifecycle management

Comparing virsh shutdown, destroy, and reset:

graph TD
    RUN[VM 运行中] -->|"virsh shutdown / (ACPI 信号, 优雅)"| STOP[正常关机]
    RUN -->|"virsh destroy / (强制断电, 拔插头)"| OFF[立即终止]
    STOP -->|"可能丢失内存数据"| OFF
    OFF -->|"virsh start"| RUN
    STOP -->|"virsh start"| RUN
    RUN -->|"virsh save <file>"| FILE[状态文件 / (保存 RAM)]
    FILE -->|"virsh restore <file>"| RUN

Recommendation: Always try virsh shutdown first; use virsh destroy only after a timeout. ACPI shutdown depends on guest ACPI support and policy. Guest-agent shutdown also requires the agent and channel; virtio-win drivers are not an inherent prerequisite for ACPI. Before forcing power-off, confirm that normal shutdown has failed and accept the risk to data.

Managing VM definitions:

Practical example:

# 批量查看所有 VM 状态
virsh list --all

# 备份 VM 配置
virsh dumpxml win11 > ~/vm-backups/win11-$(date +%F).xml

# 从备份恢复并启动
virsh define ~/vm-backups/win11-2025-01-15.xml
virsh start win11

# 彻底删除一个 VM (包括磁盘)
virsh destroy win11
virsh undefine --remove-all-storage --nvram --snapshots-metadata win11

2.2 Domain information and monitoring

Basic information

Resource monitoring

# 类 top 的实时监视器 (持续刷新)
virt-top --connect qemu:///system

# CPU 统计
virsh domstats --cpu-total <domain>

# 内存统计 (RSS, 缓存, swap-in/out)
virsh dommemstat <domain>

# 磁盘 I/O 统计 (rd_req, wr_req, rd_bytes, wr_bytes)
virsh domblkstat win11 vda

# 网卡 I/O 统计 (rx_bytes, tx_bytes, rx_packets, tx_packets)
virsh domifstat win11 vnet0

# 列出块设备
virsh domblklist <domain> --details

# 列出网络接口
virsh domiflist <domain>

# 宿主机信息
virsh nodeinfo        # CPU 型号、核心数、内存总量
virsh nodememstats    # 宿主机内存使用
virsh nodecpustats    # 宿主机 CPU 使用率（百分比）
virsh nodecpumap      # CPU topology map

vCPU analysis

Practical example:

# 获取 VM 当前资源消耗全貌
virsh dominfo win11
virsh dommemstat win11
virsh domblklist win11 --details
virsh domiflist win11

# 实时监控 (类似 top)
virt-top --connect qemu:///system

# 脚本化: 获取 VM 当前使用内存
virsh dommemstat win11 | awk '/rss/ {print $2}'

# 查看 vCPU 绑定情况
virsh vcpuinfo win11

2.3 Storage-pool and volume management

Pool/volume operations can also write to or delete real backend resources. pool-build formats resources on some backends, and vol-delete/upload should not be used on disks that are online. Identify the owning pool, file/block device, and every consumer. Simply downloading a volume that is being written to does not produce a consistent backup. Backend support varies with the libvirt build and packages; the pool type for NFS is netfs.

libvirt storage management has two layers: a storage pool is the storage backend, and a volume is an individual disk file within the pool.

Storage-pool operations

Volume operations

Practical example:

# 创建自定义存储池
virsh pool-define-as my-vms dir --target /data/libvirt/images
virsh pool-start my-vms
virsh pool-autostart my-vms

# 创建 VM 磁盘
virsh vol-create-as my-vms archlinux.qcow2 40G --format qcow2

# 查看磁盘文件的实际路径
virsh vol-path archlinux.qcow2 --pool my-vms
# => /data/libvirt/images/archlinux.qcow2

# 扩容 VM 磁盘（注意：仅扩大虚拟磁盘，分区/文件系统需 VM 内操作）
virsh vol-resize archlinux.qcow2 60G --pool my-vms

# 克隆 VM 磁盘
virsh vol-clone archlinux.qcow2 archlinux-clone.qcow2 --pool my-vms

# 备份卷到文件
virsh vol-download archlinux.qcow2 /mnt/backup/archlinux.qcow2 --pool my-vms

Common storage-pool types: dir (directory), logical (LVM), disk (physical disk), iscsi, rbd (Ceph), zfs, gluster, and netfs (NFS uses the netfs type and nfs format). The most commonly used type is dir.

2.4 Virtual-network management

Practical example:

# 查看 default 网络的 DHCP 分配情况
virsh net-dhcp-leases default

# 输出示例:
#  Expiry Time           MAC address         Protocol  IP address        Hostname
#  2025-06-20 14:30:00   <GENERATED_MAC_ADDRESS>   ipv4      192.0.2.100   ubuntu-server

Custom network XML example (isolated network):

<network>
  <name>isolated-net</name>
  <bridge name='virbr1' stp='on' delay='0'/>
  <ip address='203.0.113.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='203.0.113.10' end='203.0.113.200'/>
    </dhcp>
  </ip>
</network>

# 定义并启动自定义网络
virsh net-define isolated-net.xml
virsh net-start isolated-net
virsh net-autostart isolated-net

2.5 Snapshot management

Important: An internal snapshot stores the disk snapshot inside an image that supports it; an external snapshot switches to a new overlay file. snapshot-create reads the requested configuration from XML, while snapshot-create-as constructs it from arguments. Neither command name alone identifies the snapshot type. --disk-only means memory is not saved; it does not replace checking each disk's snapshot=external/file path and chain state. Snapshots are not independent backups. TPMs, UEFI NVRAM, VFIO, external storage, and the current software version impose limitations on snapshots, rollback, and migration.

A typical snapshot workflow:

The following illustrates a workflow with internal-snapshot support. First verify the disk format, domain features, and current snapshot support, and back up any required external state. Shutdown commands are asynchronous requests; use domstate to confirm the required state before creating, reverting, or deleting snapshots. Do not apply this workflow directly to Windows VMs with VFIO, TPMs, NVRAM, or complex external chains. Consistent online snapshots require guest quiescing/agent support and application coordination. If an operation fails, do not assume the transaction succeeded as a whole.

# 1. 做危险操作前创建快照
virsh snapshot-create-as win11 --name before-driver-update \
    --description "更新显卡驱动前"

# 2. 执行操作...出问题了

# 3. 还原快照
virsh shutdown win11
virsh snapshot-revert win11 --snapshotname before-driver-update

# 4. 确认没问题后清理快照
virsh snapshot-delete win11 --snapshotname before-driver-update

2.6 Consoles and interaction

# 获取 VM 的 IP 地址（需安装 qemu-guest-agent）
virsh domifaddr --source agent ubuntu-server

# 向 Windows VM 发送 Ctrl+Alt+Del
virsh send-key win11 KEY_LEFTCTRL KEY_LEFTALT KEY_DELETE

2.7 Other useful commands

3. Domain XML configuration essentials#

libvirt uses XML as the single source of truth for VM configuration. Understanding the domain XML structure is essential to mastering libvirt.

3.1 Top-level XML structure

<domain type='kvm'>
  <name>demo-vm</name>                        <!-- VM 名称 -->
  <!-- 首次 define 时省略 UUID，由 libvirt 生成；更新现有 VM 则保留其 UUID。 -->
  <title>My Demo VM</title>                    <!-- 可选：可读标题 -->

  <memory unit='GiB'>8</memory>               <!-- 启动时的最大内存 -->
  <currentMemory unit='GiB'>4</currentMemory> <!-- 当前实际分配的内存 -->
  <vcpu placement='static'>4</vcpu>            <!-- vCPU 数量 -->

  <os>...</os>           <!-- 启动/固件 -->
  <features>...</features>   <!-- 虚拟化特性 -->
  <cpu>...</cpu>          <!-- CPU 模型与拓扑 -->
  <clock>...</clock>      <!-- 时钟配置 -->
  <on_poweroff>...</on_poweroff>  <!-- 关机行为 -->
  <on_reboot>...</on_reboot>      <!-- 重启行为 -->
  <on_crash>...</on_crash>        <!-- 崩溃行为 -->
  <devices>...</devices>  <!-- 所有虚拟设备 -->
</domain>

3.2 Key XML elements explained

CPU configuration

Each snippet is a separate example. The topology's sockets × dies × cores × threads must match the total vCPU count. host-model does not guarantee live-migration compatibility across heterogeneous hosts. Migration requires verification of a common CPU baseline, machine type, QEMU/libvirt versions, firmware, and device support.

<!-- 方案 1: host-passthrough —— 尽量暴露宿主CPU特性；性能与迁移须实测 -->
<!-- 适合：同型号 CPU 集群，不需迁移 -->
<cpu mode='host-passthrough' check='none'>
  <topology sockets='1' dies='1' cores='4' threads='2'/>
</cpu>

<!-- 方案 2: host-model —— 兼容性好，可近迁移 -->
<!-- 适合：同架构但不同型号 CPU 间迁移 -->
<cpu mode='host-model' check='partial'>
  <topology sockets='1' dies='1' cores='4' threads='2'/>
</cpu>

<!-- 方案 3: 指定 CPU 模型 —— 最大兼容性，支持跨集群迁移 -->
<!-- 适合：异构集群统一运行时 -->
<cpu mode='custom' match='exact' check='full'>
  <model fallback='forbid'>EPYC-Milan</model>
  <topology sockets='1' cores='4' threads='2'/>
</cpu>

Selection strategy: Choose host-passthrough for a single host without migration, host-model for a cluster with the same architecture, and custom plus a vendor model for a heterogeneous cluster. NixOS users use host-passthrough by default.

Disk configuration

cache controls page-cache and write-acknowledgment semantics; io selects the I/O backend. Performance and consistency after power loss also depend on guest flushes, the host filesystem, and storage. unsafe ignores flushes and is unsuitable for workloads that require durability. Verify a block-device source using its stable /dev/disk/by-id/<DISK_ID> path and capacity; selecting the wrong device directly affects a real disk.

<!-- virtio 磁盘 (推荐，性能最佳) -->
<disk type='file' device='disk'>
  <driver name='qemu' type='qcow2' cache='writeback' discard='unmap'/>
  <source file='/var/lib/libvirt/images/vm-disk.qcow2'/>
  <target dev='vda' bus='virtio'/>
  <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</disk>

<!-- 原始块设备 (LVM / ZFS zvol) -->
<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native'/>
  <source dev='/dev/vg0/vm-disk'/>
  <target dev='vdb' bus='virtio'/>
</disk>

<!-- CD-ROM (ISO) -->
<disk type='file' device='cdrom'>
  <driver name='qemu' type='raw'/>
  <source file='/var/lib/libvirt/images/ubuntu-24.04.iso'/>
  <target dev='sdc' bus='sata'/>
  <readonly/>
</disk>

Key disk-driver attributes:

Network configuration

<!-- 方式 1: NAT 虚拟网络 (最常用) -->
<interface type='network'>
  <source network='default'/>
  <model type='virtio'/>
  <!-- 未指定 MAC，由 libvirt 生成；迁移现有 VM 应保留其地址。 -->
</interface>

<!-- 方式 2: Bridge 桥接 (VM 与宿主机同级) -->
<interface type='bridge'>
  <source bridge='br0'/>
  <model type='virtio'/>
</interface>

<!-- 方式 3: Macvtap (直通物理网卡，不需宿主机 bridge) -->
<interface type='direct'>
  <source dev='enp3s0' mode='bridge'/>
  <model type='virtio'/>
</interface>

Network-model comparison:

Recommendation: Use virtio for all Linux VMs, with <model type='virtio'/>. Use virtio for Windows VMs too after installing virtio-win drivers.

Graphics and display

<!-- SPICE (推荐, 更丰富的显示功能) -->
<graphics type='spice' autoport='yes' listen='127.0.0.1'>
  <listen type='address' address='127.0.0.1'/>
  <image compression='auto_glz'/>
  <gl enable='no'/>
</graphics>
<video>
  <model type='virtio' heads='1' primary='yes'>
    <acceleration accel3d='no'/>
  </model>
</video>

<!-- VNC (更简单, 但功能少) -->
<graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1'>
  <listen type='address' address='127.0.0.1'/>
</graphics>
<video>
  <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/>
</video>

SPICE vs VNC:

UEFI boot configuration (required for Windows 11)

<os firmware='efi'>
  <type arch='x86_64' machine='q35'>hvm</type>
  <loader readonly='yes' type='pflash'>/usr/share/edk2/x64/OVMF_CODE.fd</loader>
  <nvram template='/usr/share/edk2/x64/OVMF_VARS.fd'>
    /var/lib/libvirt/qemu/nvram/win11_VARS.fd
  </nvram>
  <boot dev='hd'/>
</os>

<features>
  <acpi/>
  <apic/>
  <hyperv mode='custom'>
    <!-- 启用 Hyper-V enlightenments 加速 Windows -->
  </hyperv>
</features>

<!-- TPM 2.0 (Windows 11 要求) -->
<tpm model='tpm-crb'>
  <backend type='emulator' version='2.0'/>
</tpm>

3.3 XML editing tips

1. Always use virsh edit <domain> rather than editing files directly. virsh edit validates the XML on exit and refuses to save malformed XML.

2. Export inactive XML before editing a persistent definition; there is no need to destroy/undefine it first:

   EXCERPTCopy code

   virsh -c qemu:///system dumpxml --inactive win11 > win11-new.xml
   # 编辑文件；保留现有 UUID、磁盘、NVRAM 和设备身份
   virt-xml-validate win11-new.xml
   virsh -c qemu:///system define win11-new.xml

   define updates the persistent definition; a running instance does not automatically receive every change. Choose live/config hotplug operations separately, or schedule a normal restart.

3. No restart needed after some changes: Certain devices, such as disks and network adapters, support hotplugging. Apply these changes to a running VM with virsh attach-device / virsh detach-device.

4. Querying and validating XML:

   EXCERPTCopy code

   # 使用 virt-xml 查询/修改单个属性（比编辑整份 XML 更安全）
   virt-xml win11 --edit --memory maxMemory=8192,currentMemory=4096
   
   # 校验 XML 语法
   virt-xml-validate win11-domain.xml

4. Common management scripts#

4.1 Backing up a VM

The following demonstrates only an offline backup of file-based disks. Shut the VM down normally first and confirm that all writers have stopped. Internal snapshots, or “taking a snapshot first,” do not automatically make an ordinary cp of a changing disk consistent. For online backups, use libvirt backup/block jobs and verify application consistency and the complete chain.

#!/usr/bin/env bash
set -euo pipefail
DOMAIN="${1:?provide domain name}"
BACKUP_DIR="/path/to/backup/vms/${DOMAIN}/$(date +%F-%H%M%S)"
URI="qemu:///system"
state=$(LC_ALL=C virsh -c "$URI" domstate "$DOMAIN")
[[ "$state" == "shut off" ]] || { echo "Guest must be shut off" >&2; exit 1; }
mkdir -p "$BACKUP_DIR"
virsh -c "$URI" dumpxml --inactive "$DOMAIN" > "$BACKUP_DIR/domain.xml"
virsh -c "$URI" domblklist --inactive "$DOMAIN" --details

# 根据 XML 和 domblklist 人工核对所有 disk 源；示例只列一块文件磁盘。
# 路径、格式、空间、权限必须确认；多盘 VM 对每块盘重复并生成恢复清单。
qemu-img info --backing-chain /path/to/vm/disk.qcow2
qemu-img convert -f qcow2 -O qcow2 /path/to/vm/disk.qcow2 "$BACKUP_DIR/disk.qcow2"
qemu-img check "$BACKUP_DIR/disk.qcow2"
echo "Backup saved to: $BACKUP_DIR"

convert flattens the selected chain into a standalone image, but the XML still points to the original path. When restoring, update paths in a copy of the XML and check UUIDs, MAC addresses, and permissions. Separately back up UEFI NVRAM, software TPM state, secure recovery material corresponding to libvirt secrets references, and disk-encryption recovery keys. Do not put real keys in public XML or logs. Backends such as block devices, RBD, and LVM need their corresponding consistent-backup workflows. Test restored copies on an isolated network to avoid identity conflicts with the original VM.

4.2 Listing IP addresses for all VMs

#!/bin/bash
# show-vm-ips.sh — 显示所有运行中 VM 的 IP（需 guest agent）

for vm in $(virsh list --name); do
  ip=$(virsh domifaddr --source agent "$vm" 2>/dev/null \
    | awk 'NR>2 {print $4}' | cut -d'/' -f1 | head -1)
  printf "%-20s %s\n" "$vm" "${ip:-No agent}"
done

4.3 Complete command to create a VM

# virt-install 一行创建 VM
virt-install --connect qemu:///system \
  --name ubuntu-server \
  --ram 4096 \
  --vcpus 4 \
  --disk path=/var/lib/libvirt/images/ubuntu-server.qcow2,size=40,format=qcow2 \
  --os-variant ubuntu24.04 \
  --network network=default \
  --graphics spice \
  --console pty,target_type=serial \
  --cdrom /var/lib/libvirt/images/ubuntu-24.04-live-server-amd64.iso \
  --boot uefi \
  --autostart \
  --noautoconsole

--os-variant is not decorative: it affects defaults such as the disk bus, network-adapter model, and USB controller. An exact match is recommended.

4.4 Cloning a VM

Before cloning, confirm that the source VM is stopped, the destination file does not exist, and enough space is available. --auto-clone can select names/storage automatically, or you can specify --file explicitly as in this example; do not mix these two approaches. A clone needs changes to guest identities such as hostname, networking, machine-id, and SSH host keys. Use virt-sysprep only on an offline copy. UEFI/TPM state and passthrough devices require separate handling.

# 克隆 VM
virt-clone \
  --original ubuntu-server \
  --name ubuntu-test \
  --file /var/lib/libvirt/images/ubuntu-test.qcow2

An administration checklist

• Shutdown: Start with shutdown and check domstate; after a timeout, use the application's recovery plan to decide whether to force power-off
• Editing: Always use virsh edit; do not edit files directly
• Backups: Persistent XML + complete, consistent disks + external state such as NVRAM/TPM, with a verified restore
• Snapshots: Choose internal/external according to backend support and recovery goals; verify the complete chain. Snapshots do not replace backups
• Monitoring: Use virt-top for a live view, domstats for collection, and event for alerts

Next steps virsh is powerful, but a GUI has its charms too. The next chapter, graphical management with virt-manager, explains how to create VMs, connect remotely with SPICE, manage snapshots, and use the hardware panel through a graphical interface. It is much more convenient than you might expect.