Enable KVM for virt-v2v (Nested Virtualization)
Overview
vJailbreak runs virt-v2v inside a Kubernetes pod (v2v-helper) to convert VM disks and inject drivers. By default, virt-v2v uses QEMU as its conversion backend. QEMU can run in two modes:
| Mode | How it works | Performance |
|---|---|---|
| KVM (hardware-accelerated) | Uses /dev/kvm — the host kernel’s KVM module | Fast |
| TCG (software emulation) | Pure software fallback, no /dev/kvm required | ~10–20× slower |
When /dev/kvm is not visible inside the pod, QEMU falls back to TCG and logs:
qemu-kvm: Could not access KVM kernel module: No such file or directoryqemu-kvm: falling back to tcgConversions still succeed but take significantly longer.
Performance impact
In customer benchmarking on ~100GB VMs using the Standard copy method, enabling nested virtualization alone brought cutover time down to roughly 5 minutes and total migration time to roughly 24 minutes. Combined with Storage Accelerated Copy (SAM), end-to-end migration time can drop to roughly 10–15 minutes.
Actual numbers depend on your infrastructure (CPU, storage, network) and the VM being migrated — treat these as directional, not a guarantee.
How vJailbreak exposes /dev/kvm
The v2v-helper pod already mounts the host /dev directory into the pod at /dev via a hostPath volume. This means if /dev/kvm exists on the vJailbreak VM, it is automatically visible inside the pod — no additional configuration is required.
The condition that must be met is that the vJailbreak VM itself has access to KVM, which requires nested virtualization to be enabled at every layer of the stack.
⚠️ Caution — this affects all VMs on the compute host, not just vJailbreak.
Enabling Nested Virtualization
Step 1 — Enable nested KVM on the compute host
On each compute host that will run vJailbreak VMs and agents:
cat /sys/module/kvm_intel/parameters/nested # expect Y (or 1)# AMD: cat /sys/module/kvm_amd/parameters/nestedIf disabled, enable it persistently and reload:
# Intelecho "options kvm-intel nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf# AMD# echo "options kvm-amd nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf
sudo modprobe -r kvm_intel && sudo modprobe kvm_intel # or reboot the hostStep 2 — Set the CPU mode in nova_override.conf
Edit the [libvirt] section of /opt/pf9/etc/nova/conf.d/nova_override.conf on each
compute host.
Recommended — host-passthrough (passes vmx/svm through automatically):
[libvirt]cpu_mode = host-passthroughAlternative — host-model (must add the flag explicitly; host-model does
not expose vmx/svm by default):
[libvirt]cpu_mode = host-modelcpu_model_extra_flags = vmx # use svm on AMDNote:
host-passthroughties the VM to a host with a closely matching CPU, model, and microcode for live migration. For vJailbreak — a short-lived, migration-tool VM — this is rarely a concern.
Step 3 — Apply
Restart Nova on the compute host, then hard-reboot the vJailbreak VM so it picks up the new CPU definition (a soft reboot is not enough):
sudo systemctl restart pf9-ostackhostopenstack server reboot --hard <instance-uuid>Step 4 — Verify
Inside the vJailbreak VM, confirm the flag is present:
grep -E -o 'vmx|svm' /proc/cpuinfo | sort -u # expect vmx (Intel) or svm (AMD)ls /dev/kvm # device node should existIf virt-v2v still falls back to TCG (no acceleration), check that
/dev/kvm is present and that kvm-ok (if available) reports acceleration can
be used.
Pod (automatic)
No changes are needed. The pod’s /dev hostPath mount makes /dev/kvm visible to the v2v-helper container as soon as it exists on the vJailbreak VM.
Verifying KVM is in use
After enabling nested virtualization and restarting a migration, inspect the v2v-helper pod logs. You should not see the TCG fallback warning. Instead, look for QEMU initializing with KVM:
kubectl -n migration-system logs <migration-name>-v2v-helper | grep -i kvmA healthy KVM-accelerated run shows no falling back to tcg messages and noticeably faster conversion times.
Recommended host configuration
Nested virtualization is not a dedicated-host requirement — it is a host-wide setting, so it can be enabled on hosts already running other workloads. That said, for the best balance of performance and isolation we recommend:
- Enable nested virtualization only on the compute hosts that will run vJailbreak VMs, rather than cluster-wide.
- Where possible, use a dedicated host aggregate or cluster that runs only vJailbreak VMs. This avoids changing the configuration of hosts running production workloads and keeps existing live-migration behavior on those hosts unaffected.
- Within that aggregate/cluster, use uniform CPU hardware (same vendor, model, and microcode) across hosts if vJailbreak VMs will need to live-migrate between them.
Common questions
What is the resource overhead of enabling nested virtualization on a KVM host?
Overhead has not been formally quantified. In practice, Platform9’s own test automation runs nested virtualization on all hypervisors without issues. Nested virtualization is a host-wide setting — once enabled, every VM on that host can use it (after that VM is hard-rebooted, rebuilt, or stopped and started). A live migration onto the enabled host does not give the VM access to it — live migration preserves the guest’s existing CPU definition.
Does enabling this require restarting Nova, and is it safe to do while VMs are running?
Yes, the nova-compute (pf9-ostackhost) service must be restarted after applying the
cpu_model_extra_flags / cpu_mode setting, and restarting it while VMs are running on the host
is safe. The CPU configuration change only takes effect for guests the next time they are
started, rebuilt, or hard-rebooted — existing running VMs keep their current CPU definition
until then.
Does nested virtualization affect live migration, and does that depend on the CPU model?
Nested virtualization itself does not affect live migration. What does affect it is the
cpu_mode choice:
host-passthroughties the VM to hosts with a closely matching CPU model and microcode — live migration can fail or be blocked if the destination host’s hardware doesn’t match closely enough. A microcode/firmware update or a CPU vendor change across hosts can break migration under this mode.host-modelwithcpu_model_extra_flags(see Step 2 above) is less restrictive for live migration across hosts with differing hardware.
If you need live migration to work reliably, use uniform hardware within the cluster, or prefer
host-model over host-passthrough.
Are there known CVEs for nested virtualization?
There’s no CVE specific to the Nova/libvirt configuration itself — Nova only controls the guest
CPU definition (cpu_mode = host-passthrough, or host-model with cpu_model_extra_flags = vmx/svm; see Step 2 above). The actual exposure is in the host kernel’s KVM
code, and whether it affects you depends on your CPU vendor, kernel version, and whether
nested is enabled at all (nested=0 means none of this applies).
Because open kernel CVEs in this area change over time, don’t treat “no open CVEs” as a permanent answer. Before rollout:
- Check your host’s CPU vendor, kernel version (
uname -r), and nested-virtualization state (cat /sys/module/kvm_{intel,amd}/parameters/nested). - Run a patched, currently-supported kernel for your distribution.
- Re-check your distribution’s kernel security tracker for KVM nested-virtualization CVEs against that kernel version before enabling this in a customer-facing environment.
- Mitigate exposure by enabling nested virtualization only on dedicated vJailbreak hosts (see Recommended host configuration) and disabling it on hosts where it isn’t needed.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Could not access KVM kernel module in logs | /dev/kvm missing on vJailbreak VM | Enable nested virt on the outer hypervisor and load the KVM module |
kvm_intel/kvm_amd module fails to load | Virtualization extensions not exposed by hypervisor | Configure the outer hypervisor to pass through CPU virt flags |
/dev/kvm present on VM but not in pod | Unlikely — the hostPath mount covers all of /dev | Confirm the pod spec has the /dev hostPath volume (default in vJailbreak) |