Skip to content

Enable KVM for virt-v2v (Nested Virtualization)

Overview

vJailbreak runs virt-v2v inside a Kubernetes pod (v2v-helper) to convert VM disks and inject drivers. By default, virt-v2v uses QEMU as its conversion backend. QEMU can run in two modes:

ModeHow it worksPerformance
KVM (hardware-accelerated)Uses /dev/kvm — the host kernel’s KVM moduleFast
TCG (software emulation)Pure software fallback, no /dev/kvm required~10–20× slower

When /dev/kvm is not visible inside the pod, QEMU falls back to TCG and logs:

qemu-kvm: Could not access KVM kernel module: No such file or directory
qemu-kvm: falling back to tcg

Conversions still succeed but take significantly longer.

Performance impact

In customer benchmarking on ~100GB VMs using the Standard copy method, enabling nested virtualization alone brought cutover time down to roughly 5 minutes and total migration time to roughly 24 minutes. Combined with Storage Accelerated Copy (SAM), end-to-end migration time can drop to roughly 10–15 minutes.

Actual numbers depend on your infrastructure (CPU, storage, network) and the VM being migrated — treat these as directional, not a guarantee.

How vJailbreak exposes /dev/kvm

The v2v-helper pod already mounts the host /dev directory into the pod at /dev via a hostPath volume. This means if /dev/kvm exists on the vJailbreak VM, it is automatically visible inside the pod — no additional configuration is required.

The condition that must be met is that the vJailbreak VM itself has access to KVM, which requires nested virtualization to be enabled at every layer of the stack.

⚠️ Caution — this affects all VMs on the compute host, not just vJailbreak.

Enabling Nested Virtualization

Step 1 — Enable nested KVM on the compute host

On each compute host that will run vJailbreak VMs and agents:

cat /sys/module/kvm_intel/parameters/nested # expect Y (or 1)
# AMD: cat /sys/module/kvm_amd/parameters/nested

If disabled, enable it persistently and reload:

# Intel
echo "options kvm-intel nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
# AMD
# echo "options kvm-amd nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
sudo modprobe -r kvm_intel && sudo modprobe kvm_intel # or reboot the host

Step 2 — Set the CPU mode in nova_override.conf

Edit the [libvirt] section of /opt/pf9/etc/nova/conf.d/nova_override.conf on each compute host.

Recommended — host-passthrough (passes vmx/svm through automatically):

[libvirt]
cpu_mode = host-passthrough

Alternative — host-model (must add the flag explicitly; host-model does not expose vmx/svm by default):

[libvirt]
cpu_mode = host-model
cpu_model_extra_flags = vmx # use svm on AMD

Note: host-passthrough ties the VM to a host with a closely matching CPU, model, and microcode for live migration. For vJailbreak — a short-lived, migration-tool VM — this is rarely a concern.

Step 3 — Apply

Restart Nova on the compute host, then hard-reboot the vJailbreak VM so it picks up the new CPU definition (a soft reboot is not enough):

sudo systemctl restart pf9-ostackhost
openstack server reboot --hard <instance-uuid>

Step 4 — Verify

Inside the vJailbreak VM, confirm the flag is present:

grep -E -o 'vmx|svm' /proc/cpuinfo | sort -u # expect vmx (Intel) or svm (AMD)
ls /dev/kvm # device node should exist

If virt-v2v still falls back to TCG (no acceleration), check that /dev/kvm is present and that kvm-ok (if available) reports acceleration can be used.

Pod (automatic)

No changes are needed. The pod’s /dev hostPath mount makes /dev/kvm visible to the v2v-helper container as soon as it exists on the vJailbreak VM.

Verifying KVM is in use

After enabling nested virtualization and restarting a migration, inspect the v2v-helper pod logs. You should not see the TCG fallback warning. Instead, look for QEMU initializing with KVM:

Terminal window
kubectl -n migration-system logs <migration-name>-v2v-helper | grep -i kvm

A healthy KVM-accelerated run shows no falling back to tcg messages and noticeably faster conversion times.

Nested virtualization is not a dedicated-host requirement — it is a host-wide setting, so it can be enabled on hosts already running other workloads. That said, for the best balance of performance and isolation we recommend:

  • Enable nested virtualization only on the compute hosts that will run vJailbreak VMs, rather than cluster-wide.
  • Where possible, use a dedicated host aggregate or cluster that runs only vJailbreak VMs. This avoids changing the configuration of hosts running production workloads and keeps existing live-migration behavior on those hosts unaffected.
  • Within that aggregate/cluster, use uniform CPU hardware (same vendor, model, and microcode) across hosts if vJailbreak VMs will need to live-migrate between them.

Common questions

What is the resource overhead of enabling nested virtualization on a KVM host?

Overhead has not been formally quantified. In practice, Platform9’s own test automation runs nested virtualization on all hypervisors without issues. Nested virtualization is a host-wide setting — once enabled, every VM on that host can use it (after that VM is hard-rebooted, rebuilt, or stopped and started). A live migration onto the enabled host does not give the VM access to it — live migration preserves the guest’s existing CPU definition.

Does enabling this require restarting Nova, and is it safe to do while VMs are running?

Yes, the nova-compute (pf9-ostackhost) service must be restarted after applying the cpu_model_extra_flags / cpu_mode setting, and restarting it while VMs are running on the host is safe. The CPU configuration change only takes effect for guests the next time they are started, rebuilt, or hard-rebooted — existing running VMs keep their current CPU definition until then.

Does nested virtualization affect live migration, and does that depend on the CPU model?

Nested virtualization itself does not affect live migration. What does affect it is the cpu_mode choice:

  • host-passthrough ties the VM to hosts with a closely matching CPU model and microcode — live migration can fail or be blocked if the destination host’s hardware doesn’t match closely enough. A microcode/firmware update or a CPU vendor change across hosts can break migration under this mode.
  • host-model with cpu_model_extra_flags (see Step 2 above) is less restrictive for live migration across hosts with differing hardware.

If you need live migration to work reliably, use uniform hardware within the cluster, or prefer host-model over host-passthrough.

Are there known CVEs for nested virtualization?

There’s no CVE specific to the Nova/libvirt configuration itself — Nova only controls the guest CPU definition (cpu_mode = host-passthrough, or host-model with cpu_model_extra_flags = vmx/svm; see Step 2 above). The actual exposure is in the host kernel’s KVM code, and whether it affects you depends on your CPU vendor, kernel version, and whether nested is enabled at all (nested=0 means none of this applies).

Because open kernel CVEs in this area change over time, don’t treat “no open CVEs” as a permanent answer. Before rollout:

  • Check your host’s CPU vendor, kernel version (uname -r), and nested-virtualization state (cat /sys/module/kvm_{intel,amd}/parameters/nested).
  • Run a patched, currently-supported kernel for your distribution.
  • Re-check your distribution’s kernel security tracker for KVM nested-virtualization CVEs against that kernel version before enabling this in a customer-facing environment.
  • Mitigate exposure by enabling nested virtualization only on dedicated vJailbreak hosts (see Recommended host configuration) and disabling it on hosts where it isn’t needed.

Troubleshooting

SymptomLikely causeFix
Could not access KVM kernel module in logs/dev/kvm missing on vJailbreak VMEnable nested virt on the outer hypervisor and load the KVM module
kvm_intel/kvm_amd module fails to loadVirtualization extensions not exposed by hypervisorConfigure the outer hypervisor to pass through CPU virt flags
/dev/kvm present on VM but not in podUnlikely — the hostPath mount covers all of /devConfirm the pod spec has the /dev hostPath volume (default in vJailbreak)