The responsibilities of KVM, QEMU, and libvirt

The commands and XML are generic examples and must be adapted to the host version and available resources. Network addresses use documentation ranges; choose a non-conflicting private network for an actual deployment. See the relevant sections for the conditions under which image modification and device passthrough apply.

KVM (Kernel-based Virtual Machine) is virtualization infrastructure native to the Linux kernel, turning the Linux operating system itself into a Type-1 hypervisor. Since merging into the Linux 2.6.20 mainline kernel in 2007, KVM has become a cornerstone of global cloud infrastructure. Major platforms including Google Cloud, AWS Nitro, OpenStack, and Proxmox VE are built on KVM.

1. Virtualization fundamentals#

1.1 What is virtualization?

The core idea of virtualization is to run multiple isolated operating system instances on a single physical machine. An intermediate software layer manages physical resources (CPU, memory, disk, and network), abstracts them into virtual resources, and allocates them to individual virtual machines (guest VMs).

graph TB
    subgraph "Type-1 (Bare Metal)"
        HW1[Physical Hardware]
        H1[Hypervisor - runs directly on hardware]
        VM1[VM 1]
        VM2[VM 2]
        VM3[VM 3]
    end

    subgraph "Type-2 (Hosted)"
        HW2[Physical Hardware]
        OS2[Host OS]
        H2[Hypervisor - runs as host OS process]
        VM4[VM 1]
        VM5[VM 2]
    end

    HW1 --> H1
    H1 --> VM1
    H1 --> VM2
    H1 --> VM3

    HW2 --> OS2
    OS2 --> H2
    H2 --> VM4
    H2 --> VM5

1.2 Where KVM fits in the classification

KVM’s classification is the subject of a familiar "philosophical debate":

• Position A (KVM is Type-1): KVM is a kernel module (kvm.ko). Once loaded, the Linux kernel itself becomes a hypervisor, directly controlling hardware virtualization capabilities (VT-x/AMD-V). Guest VMs run as separate processes. The hypervisor component operates in Ring 0.
• Position B (KVM is Type-2): KVM still runs on a full Linux OS, relying on the host OS scheduler, memory management, drivers, and other facilities. KVM cannot operate without the Linux kernel.

The mainstream consensus Academia and industry generally classify KVM as a Type-1 (Bare Metal) hypervisor. The central argument is that KVM integrates the hypervisor’s core functions (CPU and memory virtualization) into kernel space (Ring 0). Guest VMs execute directly in VMX non-root mode without passing through host user space. Here, Linux serves not as a "Host OS" but as a general foundation providing low-level services: scheduling, memory management, and the I/O driver stack.

1.3 Hardware requirements

CPU flags only establish that the current environment exposes the corresponding capabilities. Firmware enablement, nested virtualization restrictions, kernel modules, /dev/kvm permissions, and the actual result of starting QEMU must be checked separately. The example output below is illustrative, not evidence from a run performed here.

BIOS/UEFI settings Many motherboards disable virtualization extensions by default. Before using KVM, enable Intel VT-x or AMD SVM in BIOS/UEFI. Enterprise servers such as Dell PowerEdge and HPE ProLiant typically enable them by default.

Verify hardware support:

# 检查 CPU 虚拟化标志（输出 > 0 表示支持）
egrep -c '(vmx|svm)' /proc/cpuinfo
# Intel 输出 vmx 数量，AMD 输出 svm 数量，等于逻辑 CPU 核数

# 更精确的检查（区分 Intel/AMD）
lscpu | grep -i virtualization
# Intel: VT-x
# AMD:   AMD-V

# 使用 kvm-ok 工具（需安装 cpu-checker 包）
kvm-ok
# INFO: /dev/kvm exists
# KVM acceleration can be used

# 验证 KVM 内核模块已加载
lsmod | grep kvm
# kvm_intel              368640  0
# kvm                   1105920  1 kvm_intel
# （AMD 平台显示 kvm_amd 而非 kvm_intel）

1.4 KVM’s key advantages

KVM became the de facto standard for Linux virtualization primarily because of its "the kernel is the hypervisor" design:

2. KVM + QEMU: cooperation between kernel and user space#

2.1 Division of responsibilities

KVM itself is responsible only for CPU and memory virtualization, the performance-critical path that must run in kernel space. Emulating peripherals such as disks, network cards, graphics cards, and USB devices requires a user-space program. In the Linux ecosystem, QEMU fills that role.

graph TD
    subgraph "Guest VM"
        G_APP[Applications]
        G_KERN[Guest Kernel]
    end

    subgraph "Host Userspace (Ring 3)"
        QEMU[QEMU Process]
        Q_DEV[Device Emulation]
        Q_MEM[Memory Management]
    end

    subgraph "Host Kernel (Ring 0)"
        KVM[KVM Kernel Module]
        K_CPU[CPU Virtualization]
        K_MMU[MMU Virtualization]
        LK[Linux Kernel Services]
    end

    subgraph "Physical Hardware"
        CPU[CPU with VT-x/AMD-V]
        MEM[RAM]
        DEV[Devices]
    end

    G_APP --> G_KERN
    G_KERN -.->|VM Entry/Exit| KVM
    QEMU -->|ioctl /dev/kvm| KVM
    QEMU --> Q_DEV
    QEMU --> Q_MEM
    KVM --> K_CPU
    KVM --> K_MMU
    KVM --> LK
    K_CPU --> CPU
    K_MMU --> MEM
    LK --> CPU
    LK --> MEM
    Q_DEV --> DEV

2.2 Communication: /dev/kvm and ioctl()

KVM exposes its user-space interface through the /dev/kvm character device. The control interface between QEMU and KVM primarily uses file descriptors and ioctl(), with mmap sharing runtime state. Modern I/O paths may also involve ioeventfd, irqfd, and vhost.

QEMU/KVM interaction flow:

1. 打开设备：
   fd_kvm = open("/dev/kvm", O_RDWR)

2. 创建虚拟机：
   fd_vm = ioctl(fd_kvm, KVM_CREATE_VM, 0)
   // 每个 fd_vm 代表一个独立的虚拟机实例

3. 分配 Guest 内存：
   ioctl(fd_vm, KVM_SET_USER_MEMORY_REGION, &mem)
   // KVM 将连片的 Host 物理内存映射到 Guest 物理地址空间

4. 创建虚拟 CPU：
   fd_vcpu = ioctl(fd_vm, KVM_CREATE_VCPU, vcpu_id)
   // 每个 VCPU 有独立的 fd_vcpu

5. 内存映射 VCPU 运行区域：
   kvm_run = mmap(NULL, size, PROT_READ|PROT_WRITE,
                  MAP_SHARED, fd_vcpu, 0)
   // 这个共享内存区用于 KVM 在 VM Exit 时写入退出原因

6. 主循环（每个 VCPU 一个线程）：
   while (running) {
       ioctl(fd_vcpu, KVM_RUN, 0)       // 阻塞线程，CPU 进入 Guest 模式
       switch (kvm_run->exit_reason) {   // VM Exit 后检查退出原因
           case KVM_EXIT_IO:              // I/O 指令模拟（磁盘/网络等）
               handle_io(kvm_run);
               break;
           case KVM_EXIT_MMIO:            // MMIO 访问
               handle_mmio(kvm_run);
               break;
           case KVM_EXIT_SHUTDOWN:        // Guest 关机
               running = false;
               break;
       }
   }

2.3 VM entry/exit flow (Intel VT-x example)

Intel VT-x introduces two CPU modes:

• VMX Root Mode: the mode in which the hypervisor (KVM) runs. It can execute VMLAUNCH/VMRESUME instructions to enter the guest.
• VMX Non-Root Mode: the mode in which guest VMs run. On a sensitive instruction or event, the CPU automatically triggers a VM Exit back to root mode.

Host User (QEMU)               Host Kernel (KVM)              Hardware (Guest)
─────────────────              ─────────────────              ────────────────
ioctl(KVM_RUN)
    │                               │
    └──────────────────────────────►│
                                    │ 上下文保存 (Host → VMCS)
                                    │ VMLAUNCH / VMRESUME
                                    │────────────────────────►  Guest Native 执行
                                    │                           (VMX Non-Root)
                                    │
                                    │  ◄────────────────────── VM Exit
                                    │     (EPT Violation /
                                    │      I/O Instruction /
                                    │      External Interrupt)
                                    │
                                    │ 检查 exit_reason
                                    │  ┌─ 可在内核处理 → 直接恢复
                                    │  └─ 需用户态处理 → 返回 QEMU
    ◄───────────────────────────────┘
    检查 kvm_run->exit_reason
    设备模拟 / I/O 处理
    → 下次 ioctl(KVM_RUN)

The VMCS (Virtual Machine Control Structure) is the central data structure of this mechanism, maintained by KVM in kernel space. It stores the complete context of host and guest state (control registers, segment registers, MSRs, and so on). QEMU in user space cannot access the VMCS directly.

2.4 Memory virtualization: EPT/NPT

There are two approaches to memory virtualization:

EPT/NPT is key to KVM’s near-native performance. It eliminates VM exits triggered by guest page-table modifications: the guest can change its own page tables as a native system would, while hardware performs two-stage address translation transparently in the background.

2.5 QEMU’s two operating modes

# 模式 1：QEMU + KVM（硬件加速，接近原生性能）
qemu-system-x86_64 \
  -enable-kvm \
  -cpu host \
  -m 4096 \
  -drive file=disk.qcow2,format=qcow2 \
  -netdev user,id=net0 \
  -device virtio-net,netdev=net0

# 模式 2：QEMU 纯模拟（无硬件加速，非常慢）
qemu-system-aarch64 \
  -M virt \
  -cpu cortex-a57 \
  -m 2048 \
  -drive file=arm64-disk.qcow2,format=qcow2
  # 注意：没有 -enable-kvm，因为这是跨架构模拟

The value of QEMU’s pure emulation QEMU’s TCG can emulate systems across instruction sets, such as emulating ARM on an x86 host. It does not use same-architecture KVM acceleration, and performance differences must be measured against actual workloads. This kind of system emulation is useful for embedded development, reverse engineering, and firmware testing.

2.6 Important kernel modules

On an x86 host, choose the module that matches the CPU; do not run both Intel and AMD examples on the same machine. Whether nested=1 is necessary depends on the kernel defaults. ignore_msrs=1 is a compatibility-debugging option that can hide MSR issues, not a general-purpose tuning default.

# 架构无关基础模块（必须）
kvm.ko           # KVM 核心框架

# 架构相关模块（二选一）
kvm_intel.ko     # Intel VT-x 支持
kvm_amd.ko       # AMD-V/SVM 支持

# 内核参数（可选调优）
modprobe kvm_intel nested=1    # 启用嵌套虚拟化（在 VM 内再跑 KVM）
# 历史兼容性调试示例；诊断未知 MSR 后才考虑，不作为默认：
# modprobe kvm ignore_msrs=1

3. libvirt: a unified management abstraction layer#

3.1 What problem does libvirt solve?

Managing VMs directly with the QEMU command line becomes seriously problematic as the deployment grows:

# 一个生产环境虚拟机可能需要 30+ 个命令行参数
qemu-system-x86_64 -name guest=prod-db-01 \
  -machine pc-q35-7.2 -accel kvm -cpu host -smp 8 \
  -m 16384 -mem-prealloc -overcommit mem-lock=on \
  -numa node,nodeid=0,cpus=0-3 -numa node,nodeid=1,cpus=4-7 \
  -drive file=/var/lib/libvirt/images/db.qcow2,format=qcow2,if=none,id=drive0 \
  -device virtio-blk-pci,drive=drive0,bootindex=1 \
  -netdev tap,id=net0,ifname=vnet0,script=no,downscript=no \
  -device virtio-net-pci,netdev=net0,mac=<GENERATED_MAC_ADDRESS> \
  -vnc 127.0.0.1:1 -monitor unix:/tmp/qemu-monitor.sock,server,nowait \
  # ... 还有更多参数

libvirt’s goal is to provide a unified management interface for different hypervisors. Its core benefits are:

3.2 Modular daemon architecture (modern libvirt)

Traditional libvirt uses a single libvirtd daemon for all functionality. libvirt supports both monolithic and modular daemon deployments. Whether a distribution uses the modular design depends on its packages and service configuration, not simply its libvirt version. In a modular deployment, each driver runs in a separate process:

Advantages of the modular architecture:

• Security isolation: each daemon has a separate attack surface. This narrows the scope of shared processes, but shared privileges, host resources, and networks can still allow effects to cross boundaries; it does not guarantee that VMs remain unaffected.
• Independent restarts: QEMU processes are independent, so restarting a management daemon usually does not shut down guests. Management operations, connections, and ongoing jobs may still be affected.
• Install only what is needed: a containerized environment can install just libvirt-daemon-driver-qemu without the full dependency set.
• Socket activation: systemd manages sockets and starts daemons on demand, reducing idle-service overhead.

Compatibility transition Many distributions still offer libvirtd as a compatibility option. Check which mode your system uses:

EXCERPTCopy code

systemctl status libvirtd virtqemud | head -4

3.3 Connection URIs

libvirt uses a URI to specify the target hypervisor to connect to:

# 本地系统级管理连接（通过授权访问；daemon 形态取决于宿主）
virsh -c qemu:///system list --all

# 本地用户会话连接（用户权限，无需 root）
virsh -c qemu:///session list --all

# 远程连接（SSH 隧道）
virsh -c qemu+ssh://user@server.example.com/system list --all

# 远程连接（TLS 加密）
virsh -c qemu+tls://server.example.com/system list --all

# 设置默认 URI（避免每次输入 -c）
export LIBVIRT_DEFAULT_URI="qemu:///system"

3.4 virsh: command-line management

# 列出所有 VM（包括未运行的）
virsh list --all

# 启动/关机
virsh start prod-db-01
virsh shutdown prod-db-01    # 优雅关机（发送 ACPI 信号）
virsh destroy prod-db-01     # 强制断电（相当于拔电源）

# 查看 VM 的 libvirt XML 配置
virsh dumpxml prod-db-01
virsh edit prod-db-01        # 在线编辑 XML（有语法校验）

# 控制台连接
virsh console prod-db-01     # 串口控制台（需要 Guest 配置 serial console）

# 资源信息
virsh dominfo prod-db-01     # VM 概览信息
virsh domstats prod-db-01    # CPU/内存/磁盘/网络统计
virsh vcpuinfo prod-db-01    # VCPU 到物理 CPU 的绑定关系

4. virt-manager: a graphical management frontend#

4.1 Overview

virt-manager is a GTK-based desktop application that manages KVM/Xen/LXC VMs through the libvirt protocol. It suits workstations and personal use, and is also useful for "emergency manual operations" on production servers.

Core features:

• VM creation wizard: graphically configure CPU, memory, disks, and networking to generate libvirt XML
• Hardware hotplug: add or remove virtual disks and network cards while running (requires guest support)
• SPICE/VNC console: built-in remote desktop connections with clipboard sharing and USB redirection
• Snapshot management: create, restore, and delete VM snapshots (internal snapshots use qcow2; support depends on disk format, firmware, TPM, and current software versions; a snapshot is not a backup)
• Performance monitoring: real-time graphs of CPU usage, memory use, and disk/network I/O

4.2 Supporting tools

4.3 Automation example: virt-install

virt-install \
  --name demo-vm \
  --vcpus 2 \
  --memory 2048 \
  --disk size=20,format=qcow2,bus=virtio \
  --network network=default,model=virtio \
  --graphics spice \
  --cdrom /path/to/ubuntu-24.04-server.iso \
  --os-variant ubuntu24.04 \
  --autostart
# 参数说明：
# --disk bus=virtio     使用 virtio 块设备，性能远优于 IDE/SATA 模拟
# --network model=virtio 使用 virtio 网络驱动，同样有更好的性能
# --os-variant           告诉 libvirt Guest OS 类型，以便自动选择最佳默认值
# --autostart            随 Host 启动时自动启动此 VM

5. Comparing virtualization options#

5.1 Hypervisor comparison

The table below summarizes design priorities. Performance, live migration, and snapshot support all depend on version, configuration, and workload. KVM’s host kernel is Linux; guest OS support in the table does not mean KVM can run as a Windows host kernel module. Containers share the host kernel, and migration mechanisms such as CRIU also have compatibility constraints.

5.2 KVM vs. containers (Docker/Podman/LXC)

This is the most common technology-selection question. The two are complementary rather than substitutes.

Selection guidelines

Use KVM when:

• You need to run different operating systems (a mix of Linux + Windows + FreeBSD)
• You need strong security isolation (multitenant public cloud environments)
• You need fully exclusive hardware access (GPU passthrough for AI training or gaming VMs)
• You need separate Docker/Podman environments running unprivileged containers (containers inside KVM)

Use containers when:

• You are deploying microservices or applications (fast startup/shutdown, version management, and CI/CD integration)
• You need very high density (hundreds of instances on one machine)
• You depend on host devices or hardware (direct use of the host’s GPU, FPGA, or RDMA)

A common combination: KVM + containers A common deployment uses VMs to provide different kernels and tenant boundaries, with applications running in containers inside the VMs. This does not mean that every public cloud uses KVM. Different platforms use different hypervisors; Azure’s virtualization foundation, for example, should not be classified as KVM.

5.3 KVM’s role in cloud computing

KVM is the dominant choice for Linux cloud infrastructure:

Summary KVM is not "a single piece of software" but a virtualization stack of cooperating layers:

• KVM kernel module: provides hardware acceleration for CPU and memory
• QEMU: provides device emulation and platform modeling
• libvirt: provides a unified management API and toolchain
• libguestfs/virt- tools*: provide image manipulation and automation

Understanding the responsibility boundaries of these four layers is the key to mastering the KVM virtualization stack.