[virt-tools-list] LUKS decryption slow in KVM

[Digimer]

On 13/01/14 04:49 AM, Digimer wrote:
> Hi all,
>
>    I'm trying to sort out a significant performance hit I am getting
> with a LUKS partition inside a KVM VM (host is RHEL 6.5, guest is also
> RHEL 6.5).
>
>    Inside the VM, I get ~1.1 GB/sec write speed:
>
> ====
> Inside the VM! (48 GB, 4x vCPUs, 500MB /boot, 40 GB /, 4 GB <swap>, RHEL
> 6 minimal, no selinux/iptables)
> Write to raw partition using: sync; dd if=/dev/zero of=/dev/vda5 bs=4M
> count=80000; sync
> - 335544320000 bytes (336 GB) copied, 302.885 s, 1.1 GB/s
> - 335544320000 bytes (336 GB) copied, 293.905 s, 1.1 GB/s
> - 335544320000 bytes (336 GB) copied, 289.298 s, 1.2 GB/s
>
> Write to ext4 partition using: sync; dd if=/dev/zero
> of=/mnt/data/zeros.out bs=4M count=80000; sync
> - 335544320000 bytes (336 GB) copied, 290.855 s, 1.2 GB/s
> - 335544320000 bytes (336 GB) copied, 347.901 s, 964 MB/s
> - 335544320000 bytes (336 GB) copied, 317.877 s, 1.1 GB/s
> ====
>
>    When I created the LUKS partition though, I got substantially lower
> speed:
>
> ====
> [root at vm01-ng-rhel01 ~]# cryptsetup -v status vda5
> /dev/mapper/vda5 is active.
>    type:  LUKS1
>    cipher:  aes-cbc-essiv:sha256
>    keysize: 256 bits
>    device:  /dev/vda5
>    offset:  4096 sectors
>    size:    1729118208 sectors
>    mode:    read/write
> Command successful.
>
> (killed the 'dd' when I realized how slow it was going)
>
> sync; dd if=/dev/zero of=/dev/mapper/vda5 bs=4M count=80000; sync
> 48330964992 bytes (48 GB) copied, 345.72 s, 140 MB/s
> ====
>
> I noticed that the real CPU on the host:
>
> ====
> vendor_id    : GenuineIntel
> cpu family    : 6
> model        : 62
> model name    : Intel(R) Xeon(R) CPU E5-2637 v2 @ 3.50GHz
> stepping    : 4
> cpu MHz        : 3500.087
> cache size    : 15360 KB
> physical id    : 1
> siblings    : 8
> core id        : 4
> cpu cores    : 4
> apicid        : 41
> initial apicid    : 41
> fpu        : yes
> fpu_exception    : yes
> cpuid level    : 13
> wp        : yes
> flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
> nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
> xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx
> smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt
> tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb
> xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid fsgsbase smep
> erms
> bogomips    : 6998.99
> clflush size    : 64
> cache_alignment    : 64
> address sizes    : 46 bits physical, 48 bits virtual
> power management:
> ====
>
> Has a lot more features (I see 'aes' in particular) than inside the VM:
>
> ====
> processor    : 3
> vendor_id    : GenuineIntel
> cpu family    : 6
> model        : 13
> model name    : QEMU Virtual CPU version (cpu64-rhel6)
> stepping    : 3
> cpu MHz        : 3499.998
> cache size    : 4096 KB
> fpu        : yes
> fpu_exception    : yes
> cpuid level    : 4
> wp        : yes
> flags        : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16
> hypervisor lahf_lm
> bogomips    : 6999.99
> clflush size    : 64
> cache_alignment    : 64
> address sizes    : 46 bits physical, 48 bits virtual
> power management:
> ====
>
> Should I (can I?) be passing some of these features up to the guest to
> improve LUKS performance?
>
> Thanks!

I copied the extensions of the host, and 'aes' showed in the guest's 
cpuinfo, to the VM and got somewhat better numbers;

Write to raw LUKS partition using: sync; dd if=/dev/zero 
of=/dev/mapper/vda5 bs=4M count=80000; sync
- 335544320000 bytes (336 GB) copied, 1157.27 s, 290 MB/s

Still a far cry from what the base system can do.

Thoughts?

Thanks!

-- 
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without 
access to education?