[Pulp-dev] Using nested virtualization for SELinux/FIPS CI testing on Travis & GHA

[Fabricio Aguiar]

excellent report,
thanks for working on this Mike!

Best regards,
Fabricio Aguiar
Software Engineer, Pulp Project
Red Hat Brazil - Latam <https://www.redhat.com/>
+55 11 999652368


On Tue, Feb 18, 2020 at 5:51 PM Mike DePaulo <mikedep333 at redhat.com> wrote:

> Nested hardware virtualization with KVM works on Travis, but not on GitHub
> actions.
>
> I highly recommend we use it, even if it means maintaining some CI tests
> on travis rather than GHA.
>
> I evaluated 2 other possible solutions on GitHub Actions: VirtualBox
> software virtualization, and qemu emulation.
>
> I tested by creating prototype CI for Pulplift, and after spending many
> hours, had to resort to some manual testing of the other 2 alternatives'
> final steps. They do work though; they launch virtual/emulated machines.
> https://github.com/pulp/pulplift/pull/66
>
> Test results:
>
> Nested hardware virtualization with KVM works on Travis:
> Pros:
> - Fastest
> - Very well featured
> - Works with pulplift as-is
> Cons:
> - Cannot work on GHA, which use Azure legacy BIOS VMs, without nested
> Intel VT-X / AMD-V, as of Ubuntu 18.04. (Hoping for change with 20.04).
> - May not work on Travis indefinitely. They use LXD now, but have no
> guarantees to keep using LXD with Intel VT-x / AMD-V enabled.
>
> Software Virtualization with VirtualBox:
> Pros:
> - None really, just less slow than qemu
> Cons:
> - Limited to 32-bit x86 guest OS's. CentOS 7 32-bit x86 is unofficial, and
> Fedora 30 is the last 32-bit x86 Fedora. Neither are ideal for testing
> - Slow: 1 vCPU, and slower than it was in the past (~2/3 of native CPU
> performance) back in 2006 or so when OS's used fewer hardware features.
> - Support dropped as of VirtualBox 6.1, which just came out. 6.0 is only
> supported until 2020-07.
> - Requires some changes to pulplift / box definitions to work
>
> Emulation with Qemu:
> Pros:
> - Well-Supported
> - Has all the other features of qemu-kvm, including emulated CPU count
> - Needs only 1 easily-set setting needed for pulplift to work with it
> Cons:
> - Very slow. Rule of thumb is at best, 10% of native performance.
>
> -Mike
>
> On Thu, Feb 13, 2020 at 6:14 PM Mike DePaulo <mikedep333 at redhat.com>
> wrote:
>
>> I've only tested Travis so far, but this is very promising.
>>
>> Hardware KVM virtualization appears to be working on Travis, via pulplift
>> (which uses vagrant, libvirt & KVM), without any hacks!
>>
>> My current theory is that Travis uses either OpenVZ or KVM, and that the
>> "svm" warning is a limitation of nested virtualization working properly.
>>
>> I'm going to investigate Travis a little further before trying out GHA.
>> (Or test them in parallel with same commands.) Including making 100% sure
>> it is not falling back to unaccelerated qemu emulation.
>>
>> $ uname -a
>> Linux travis-job-7dcf26ac-24c0-462e-8418-69c466817f8e 5.0.0-1026-gcp
>> #27~18.04.1-Ubuntu SMP Fri Nov 15 07:40:39 UTC 2019 x86_64 x86_64 x86_64
>> GNU/Linux
>>
>> $ sudo virt-what
>> kvm
>>
>> $ sudo qemu-system-x86_64 -machine accel=kvm -vnc 127.0.0.1:1
>> qemu-system-x86_64: warning: host doesn't support requested feature:
>> CPUID.80000001H:ECX.svm [bit 2]
>> ^C
>>
>> $ sudo vagrant ssh fedora31
>> Last login: Thu Feb 13 22:55:40 2020 from 192.168.121.1
>> $ uname -a
>> Linux localhost.localdomain 5.3.7-301.fc31.x86_64 #1 SMP Mon Oct 21
>> 19:18:58 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>> $ cat /etc/redhat-release
>> Fedora release 31 (Thirty One)
>>
>> -Mike
>>
>> --
>>
>> Mike DePaulo
>>
>> He / Him / His
>>
>> Service Reliability Engineer, Pulp
>>
>> Red Hat <https://www.redhat.com/>
>>
>> IM: mikedep333
>>
>> GPG: 51745404
>> <https://www.redhat.com/>
>>
>
>
> --
>
> Mike DePaulo
>
> He / Him / His
>
> Service Reliability Engineer, Pulp
>
> Red Hat <https://www.redhat.com/>
>
> IM: mikedep333
>
> GPG: 51745404
> <https://www.redhat.com/>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200218/e3f49ee6/attachment.htm>