Declaring KVM host capabilities on NixOS

This is article 6 in the KVM virtualization series, focusing on declarative configuration and troubleshooting with NixOS as a KVM host. For general concepts (how QEMU/KVM work, libvirt architecture, and virt-manager usage), first read the general installation guide.

1. An overview of KVM configuration on NixOS#

1.1 Why NixOS is particularly well suited to KVM

NixOS’s declarative configuration model is an excellent fit for KVM virtualization, particularly in four areas:

1.2 Enable it in two steps

A usable KVM environment on NixOS requires as few as two lines:

virtualisation.libvirtd.enable = true;
programs.virt-manager.enable = true;

Also add your user to the libvirtd group:

users.users.user.extraGroups = [ "libvirtd" ];

Evaluate and build according to the project’s rules first; the administrator decides separately whether to activate or reboot. After applying the configuration, log in again for group membership to take effect, then check the libvirt URI, KVM, and networking. No rebuild or VM execution was performed for this article.

libvirtd.enable = true automatically brings in the QEMU/KVM userspace components, so there is no need to declare pkgs.qemu_kvm separately. If you need a particular QEMU version or additional QEMU functionality (such as virtiofsd), specify it explicitly through qemu.package and qemu.vhostUserPackages.

2. Declarative configuration in detail#

The following is a complete KVM host configuration for the NixOS configuration model, using the nixos-unstable module checked for this article as an interface snapshot (real projects should follow the nixpkgs pinned by flake.lock). It is grouped by function, with explanations for each group.

2.1 Kernel modules and boot parameters

{ config, pkgs, lib, ... }:
{
  # ===== 内核模块 =====
  # AMD CPU: "kvm-amd"；Intel CPU: "kvm-intel"
  # 按宿主 CPU 选择模块，下面只演示 AMD 分支
  boot.kernelModules = [ "kvm-amd" ];

  # ===== IOMMU 开启（PCI 直通前提） =====
  # AMD IOMMU 通常由固件开启后自动初始化；不要套用不存在的 amd_iommu=on 值
  # Intel: "intel_iommu=on"
  # iommu=pt 可让宿主 DMA 使用 passthrough 默认域；需评估隔离与性能，非通用默认
  boot.kernelParams = [
    "iommu.strict=1"
    # "iommu=pt"  # 仅在明确评估宿主 DMA 隔离后启用
  ];

  # ===== 嵌套虚拟化（可选） =====
  # 允许在 VM 内部再运行 KVM（比如在 VM 里跑 minikube 或测试 nested KVM）
  boot.extraModprobeConfig = ''
    options kvm_amd nested=1
  '';
}

What each parameter does:

• kvm-amd: the kernel module for AMD-V hardware virtualization. After loading it, check with lsmod | grep kvm; you should see kvm_amd and kvm.
• iommu.strict=1: controls synchronous invalidation on DMA unmap; it is not a switch for enabling AMD IOMMU. On AMD, firmware and the kernel usually initialize it automatically; verify IOMMU groups, interrupt remapping, and VFIO. SVM is the CPU virtualization switch, while AMD-Vi is the IOMMU feature.
• iommu=pt: can change the default DMA domain for host devices; it cannot be claimed to always improve performance. It also changes the isolation boundary, so choose according to your threat model.
• nested=1: allows use of svm instructions inside a VM, enabling nested virtualization.

Check whether IOMMU is active with dmesg | grep -i iommu; you should see AMD-Vi: Interrupt remapping enabled or similar output. Check IOMMU grouping with find /sys/kernel/iommu_groups/ -type l | sort -V.

2.2 Main libvirtd configuration

  virtualisation.libvirtd = {
    enable = true;

    # ===== QEMU 子配置 =====
    qemu = {
      # QEMU 包选择：pkgs.qemu_kvm 内置 KVM 加速支持
      package = pkgs.qemu_kvm;

      # system libvirt 与 QEMU 运行身份分开。此接口快照默认 true。
      # true: QEMU 以 root 运行；false: 使用 qemu-libvirtd 用户/组。
      # 迁移现有镜像时检查读写、目录遍历与安全标签；这并非整个 daemon rootless。
      runAsRoot = false;

      # Software TPM 2.0 模拟（满足 Windows 11 TPM 条件的一种后端）
      swtpm.enable = true;

      # vhost-user 后端包：virtiofsd 用于宿主机-虚拟机共享目录
      vhostUserPackages = with pkgs; [ virtiofsd ];
    };

    # ===== 宿主关机时 libvirt-guests 的 VM 处理 =====
    # "shutdown"：发送 ACPI 关机信号
    # "suspend"：尝试保存 VM 状态到磁盘；直通等域可能不支持，必须验证
    onShutdown = "shutdown";
  };

  # ===== SPICE USB 重定向 =====
  # 允许将宿主机 USB 设备热插拔到 VM 中
  virtualisation.spiceUSBRedirection.enable = true;

Important change: OVMF configuration The nixos-unstable module checked for this article has removed the old qemu.ovmf.enable/packages interface; firmware metadata comes from the selected QEMU package. For an actual migration, follow the option definitions at the project’s pinned revision. Old configuration triggers an assertion with this interface.

Firmware paths have changed as well:

Old path	New path
/run/libvirt/nix-ovmf/OVMF_CODE.fd	/run/libvirt/nix-ovmf/edk2-x86_64-code.fd
/run/libvirt/nix-ovmf/OVMF_VARS.fd	/run/libvirt/nix-ovmf/edk2-i386-vars.fd
/run/libvirt/nix-ovmf/OVMF_CODE.ms.fd	/run/libvirt/nix-ovmf/edk2-x86_64-secure-code.fd

New domains can use <os firmware="efi"> to select firmware automatically. For existing VMs, first back up NVRAM/TPM state and encryption recovery material, then check CODE/VARS compatibility. Do not simply delete <nvram> and reset existing state. The paths in the table are historical examples; verify the actual paths from current firmware metadata and domcapabilities.

2.3 GUI and client tools

  # ===== virt-manager GUI =====
  programs.virt-manager.enable = true;

  # ===== 宿主机 QEMU Guest Agent 与 SPICE Agent =====
  # qemuGuest: 宿主机作为 VM 时可被 hypervisor 控制
  # spice-vdagentd: 宿主机上的 SPICE agent daemon（剪贴板共享等）
  # 仅当这个 NixOS 系统本身是需要此集成的 guest 时启用：
  # services.qemuGuest.enable = true;
  # services.spice-vdagentd.enable = true;

  # ===== 额外系统包 =====
  environment.systemPackages = with pkgs; [
    virt-viewer          # SPICE/VNC 独立控制台
    spice-gtk            # SPICE GTK 客户端库
    win-virtio           # Windows 客户机 virtio 驱动 ISO
    win-spice            # Windows 客户机 SPICE guest tools
  ];

  # ===== 用户组 =====
  users.users.user.extraGroups = [ "libvirtd" ];

The win-virtio package provides virtio-win.iso; its actual path depends on the installation profile and package layout. For systemPackages, locate it under /run/current-system/sw/share/ or in the current store package, rather than assuming it is in the user profile. When creating a Windows VM, mount this ISO as a second CDROM to load virtio storage and network drivers during installation.

2.4 Firewall configuration

Do not treat trustedInterfaces = [ "virbr0" ] as a default requirement; it broadens guests’ inbound access to host services. First inspect the libvirt network’s nftables/iptables rules and the NixOS firewall, then allow only the services actually needed, such as DHCP/DNS. For example, where genuinely necessary:

  networking.firewall.interfaces.virbr0 = {
    allowedUDPPorts = [ 53 67 ];
    allowedTCPPorts = [ 53 ];
  };
}

This is an example for inbound DNS/DHCP traffic, not a universal fix for every forwarding/NAT issue; verify traffic against the actual backend and rules.

3. Network configuration#

3.1 Automatically activating the default NAT network

The default network’s existence, active state, and autostart state are libvirt runtime state. This interface snapshot copies the package’s default network definition on initial setup while preserving an existing file. You must still explicitly inspect net-list --all, net-info default, and subnet conflicts. For an existing persistent network, you can set net-autostart default as an authorized administrative action; this does not give Nix declarative ownership of all domain/network state.

If a service is genuinely needed to activate an existing network, the example below first checks whether it is active to avoid a duplicate net-start. It does not run net-destroy when the service stops, so rebuilds/restarts do not interrupt guest networking.

systemd.services.libvirt-default-network = {
  description = "Activate existing libvirt default NAT network";
  after = [ "libvirtd.service" ];
  requires = [ "libvirtd.service" ];
  wantedBy = [ "multi-user.target" ];
  path = [ pkgs.libvirt pkgs.gawk ];
  environment.LC_ALL = "C";
  script = ''
    active=$(virsh -c qemu:///system net-info default | awk '/^Active:/ {print $2}')
    if [ "$active" != "yes" ]; then
      virsh -c qemu:///system net-start default
    fi
  '';
  serviceConfig = { Type = "oneshot"; RemainAfterExit = true; };
};

If the project switches to modular daemons, dependencies must match the actual services; do not claim that both monolithic and modular daemons are enabled by default. If fully declarative network XML is required, evaluate NixVirt and explicitly define the scope of its takeover of existing resources.

3.2 Bridged networking (Bridged Mode)

To connect VMs directly to the host’s physical network, with IP addresses on the same network as the host, use bridged mode instead of the default NAT:

networking.bridges.br0 = {
  interfaces = [ "enp3s0" ];  # 替换为你的物理网卡名
};
networking.interfaces.br0.useDHCP = true;
networking.interfaces.enp3s0.useDHCP = false;  # 物理网卡不再独立获取 IP

Then use virsh net-define to create a libvirt network pointing to br0, or select "Bridge device" for the VM’s network interface in virt-manager and enter br0.

In bridged mode, the host no longer controls the physical interface’s IP configuration directly (the IP moves to br0). Do not make this change directly on a machine connected over remote SSH: a configuration error can disconnect you. Validate it locally first.

3.3 Network topology overview

flowchart LR
    VM_A[VM A / 192.0.2.x] --> virbr0[virbr0 NAT]
    VM_B[VM B / 192.0.2.x] --> virbr0
    virbr0 --> Host[宿主机网络栈]
    Host --> enp3s0[物理网卡 enp3s0]

    VM_C[VM C / 198.51.100.x] --> br0[br0 Bridge]
    Host --> br0
    br0 --> enp3s0

4. dconf configuration: eliminating initial connection delays#

Presetting the URI/autoconnect settings can reduce manual connection steps, but cannot guarantee the elimination of all startup delays. NixOS dconf profiles, Home Manager dconf.settings, and GUI user state have different precedence. Do not accidentally lock settings when you only intend to set defaults. Enabling lockAll below restricts user overrides; choose according to your needs.

Some users find that virt-manager’s interface remains stuck on a spinning "Connecting..." message at first startup. The reason is that virt-manager has no preset libvirt connection URI and requires qemu:///system to be entered manually before it can connect.

Pre-populating the autoconnect configuration through dconf can resolve this:

programs.dconf = {
  enable = true;
  profiles.user.databases = [
    {
      lockAll = false; # 这里只提供默认值；需要管理锁定时另行决定
      settings = with lib.gvariant; {
        "org/virt-manager/virt-manager/connections" = {
          autoconnect = [ "qemu:///system" ];
          uris = [ "qemu:///system" ];
        };
      };
    }
  ];
};

programs.dconf.profiles is a NixOS system-module interface; Home Manager uses dconf.settings to write user configuration. Their scope and precedence differ, so a system profile should not be described as Home Manager-style. Preset these settings only when an actual connection-configuration problem occurs; do not infer a fault merely from Wayland or the packaging method. A Home Manager example:

EXCERPTCopy code

# Home Manager 版
dconf.settings = {
  "org/virt-manager/virt-manager/connections" = {
    autoconnect = [ "qemu:///system" ];
    uris = [ "qemu:///system" ];
  };
};

5. AMD GPU passthrough and display options#

5.1 Host capability checklist

The configuration depends on device capabilities rather than a particular machine model. First record CPU virtualization support, firmware AMD-Vi/IOMMU support, the kernel driver, IOMMU groups, independently resettable GPU/audio functions, and an alternative display or out-of-band management path.

5.2 Basic IOMMU and VFIO configuration

boot.kernelParams = [
  "iommu.strict=1"
  "iommu=pt"
  # vfio-pci.ids 按需添加，格式：vendor:device
  # 示例占位符：VGA 与相关音频功能须从实际设备清单确认：
  # "vfio-pci.ids=<VENDOR_DEVICE_ID>,<VENDOR_DEVICE_ID>"
];

# 提前加载 VFIO 模块（在显卡驱动之前抢占设备）
boot.initrd.kernelModules = [
  "vfio_pci"
  "vfio"
  "vfio_iommu_type1"
];

Obtain the PCI device’s vendor:device ID:

EXCERPTCopy code

lspci -nn | grep -i vga    # 显卡
lspci -nn | grep -i audio  # 声卡（通常需要一起直通）

Use the placeholders <GPU_VENDOR_DEVICE_ID> and <GPU_AUDIO_VENDOR_DEVICE_ID>; do not copy someone else’s IDs. ID matching may include multiple devices of the same model. Where necessary, design binding around the BDF and driver_override.

5.3 The AMD GPU reset bug and vendor-reset

Some AMD GPUs, especially dGPUs and less often iGPUs, have a "reset bug": the GPU cannot be cleanly reset after the VM shuts down, causing errors or a lockup on the VM’s second start.

An option to consider only for supported devices: check vendor-reset’s device support list and kernel compatibility before deciding to load it:

boot.extraModulePackages = with config.boot.kernelPackages; [ vendor-reset ];
boot.kernelModules = [ "kvm-amd" "vendor-reset" ];

vendor-reset supports only specific device and kernel combinations. If a reset failure occurs, check the project’s support list first. Do not assume that a “newer GPU” is automatically supported, or that the module is guaranteed to fix reset failures.

5.4 Single-GPU passthrough (the specialisation approach)

This section is an illustrative boot configuration that requires separate validation. Binding the only GPU or blacklisting amdgpu may remove all local display output from the host. Prepare a verified recovery generation and an independent management path in advance. video=efifb:off does not guarantee coverage of modern simpledrm/firmware framebuffers; device reset and rebinding to the host must also be tested. Check specialisation merge semantics and the actual kernelParams before building.

If you want to pass an iGPU through to a Windows VM for gaming and accept that the NixOS host loses its display during passthrough, use NixOS’s specialisation feature to create a "VFIO mode" boot entry:

specialisation.VFIO.configuration = {
  system.nixos.tags = [ "VFIO" ];
  boot.kernelParams = [
    "iommu.strict=1"
    # "iommu=pt"  # 仅在明确评估宿主 DMA 隔离后启用
    "vfio-pci.ids=<VENDOR_DEVICE_ID>,<VENDOR_DEVICE_ID>"  # 替换为实际 ID
    "video=efifb:off"                     # 禁用 EFI 帧缓冲
  ];
  boot.initrd.kernelModules = [ "vfio_pci" "vfio" "vfio_iommu_type1" ];
  boot.blacklistedKernelModules = [ "amdgpu" ];
};

After rebooting, an additional "VFIO" option appears in the systemd-boot menu; select it to enter VFIO passthrough mode. A regular generation can serve as a recovery option, but external disk/firmware state does not automatically roll back with a generation.

5.5 virtio-gpu + virgl (an alternative for 3D acceleration)

If you do not want full GPU passthrough but still need 3D acceleration in a Linux guest, use virtio-gpu with virgl (OpenGL). Vulkan/Venus is a separate version/configuration path that requires its own validation:

In virt-manager, select the following for the VM’s Video device:

• Model: Virtio
• Check 3D acceleration

Or specify it directly in the domain XML:

<video>
  <model type="virtio" heads="1" primary="yes">
    <acceleration accel3d="yes"/>
  </model>
</video>

virtio-gpu/virgl and QXL/SPICE use different display paths. Validate suitability and performance against the guest desktop, drivers, GL support, and actual workload.

6. Common NixOS issues and troubleshooting#

6.1 Troubleshooting quick reference

6.2 Diagnostic command quick reference

# KVM 模块状态
lsmod | grep kvm

# IOMMU 是否启用
dmesg | grep -i "iommu\|amd-vi"

# IOMMU 分组（验证设备隔离）
find /sys/kernel/iommu_groups/ -type l | sort -V

# libvirtd 状态
systemctl status libvirtd

# 网络状态
virsh net-list --all

# VFIO 设备是否被正确绑定
dmesg | grep vfio

# AMD-V / SVM 是否可用
grep -o 'svm' /proc/cpuinfo | head -1

6.3 Special considerations for Wayland hosts

When running virt-manager and SPICE clients on a Wayland host, validate each combination against the versions of the compositor, client, guest agent, and guest desktop:

• Starting virt-manager: if a missing GTK theme prevents startup, set the cursor theme through Home Manager:
  EXCERPTCopy code

  home.pointerCursor = {
    gtk.enable = true;
    package = pkgs.vanilla-dmz;
    name = "Vanilla-DMZ";
  };

• SPICE client performance: distinguish client rendering, guest driver, and remote bandwidth issues first. Evaluate virtio-gpu/virgl for supported local combinations; SPICE GL generally requires a local Unix socket
• Clipboard sharing: integration depends on protocol support in the guest agent and compositor. Test host and guest sessions separately; do not classify every Wayland combination as unsupported or assume switching to X11 will fix every problem

7. Relationship to the NixOS VM build system#

This section discusses two complementary but non-overlapping NixOS virtualization capabilities:

How they complement each other On the KVM host configured in this article, you can run NixOS VM images built with nixos-build-vm. Both use the same qemu_kvm backend; only their management and build approaches differ. For details, see the NixOS Virtualization and Deployment MOC.

7.1 Quick example: launch a NixOS-built VM on a KVM host

# 在任意 NixOS 配置目录下构建 VM
nixos-build-vm

# 构建产物是一个可执行脚本，直接运行
./result/bin/run-nixos-vm

This command starts a QEMU VM using your current NixOS configuration and automatically sets up networking and display output.

8. Recommended reading path#

1. Within the KVM series:

   • Core Concepts and Architecture — understand the layers of KVM/QEMU/libvirt
   • General Installation and Initialization — set up KVM across distributions
   • Creating and Managing Virtual Machines — virsh / virt-install operations
   • Graphical Management — virt-manager operations
   • Storage, Networking, and GPU Passthrough — storage pools, advanced networking, and VFIO passthrough

2. Links to other topics:

   • NixOS Virtualization and Deployment MOC — building and deploying NixOS as a VM guest