[libvirt] DHCP and secure containers

Tue Sep 9 18:54:16 UTC 2014

On 09/07/2014 05:54 PM, Gene Czarcinski wrote:
> On 09/07/2014 11:03 AM, Gene Czarcinski wrote:
>> On 09/03/2014 09:42 AM, Gene Czarcinski wrote:
>>> On 09/02/2014 06:37 AM, Gene Czarcinski wrote:
>>>> OK, hopefully this mailing list is more active and I can get some 
>>>> response to my questions.
>>>> --------------------------------------------------------------------------------------------------------------------- 
>>>>
>>>> I have been "playing with" Secure Containers running a lighttpd 
>>>> server and have it up and running.  I used Adam's process 
>>>> (https://www.happyassassin.net/2014/07/23/bridged-networking-for-libvirt-with-networkmanager-2014-fedora-21/) 
>>>> for getting a bridge defined when also running NetworkManager. I 
>>>> then created a virtual network definition:
>>>>> <network>
>>>>>   <name>net18</name>
>>>>> <uuid>8d19a05b-ac85-4e2a-88bc-5ca4cbb29a33</uuid>
>>>>>   <forward mode='bridge'/>
>>>>>   <bridge name='br0'/>
>>>>> </network>
>>>> This works fine when I use static addresses such as:
>>>>> -N 
>>>>> source=net18,address=192.168.18.94/24,route=192.168.18.255%192.168.18.1 
>>>>>
>>>> but does not work when I specify using dhcp:
>>>>> -N source=net18,dhcp
>>>> I have reported this as a bug: 
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1133686
>>>>
>>>> Since there has not been much of a reaction to the BZ report, I 
>>>> decided to take a look at the source code (it sure would have been 
>>>> nice if the SRPMS were there in the F20 fedora-virt-preview but I 
>>>> get the package from development/21).
>>>>
>>>> I see that libvirt-sandbox-init-common.c has the code for starting 
>>>> dhcp and also has main() along with some runtime options for -v 
>>>> verbose and -d debug.
>>>>
>>>> OK, how do I go about turning verbose and/or debug on?
>>>>
>>>> Any suggestions on how to debug and get dhcp to work?  I not only 
>>>> want to find the problem but to fix the problem if needed.
>>> While I have not figured out how to get dhcp to work with a secure 
>>> container create by virt-sandbox-service, I have gotten a container 
>>> working with the network up and a dhcp assigned IP using the lxc-* 
>>> commands and following this procedure:
>>> https://sysadmincasts.com/episodes/24-introduction-to-containers-on-linux-using-lxc 
>>>
>>> to create a "busybox" container.  The network came up automatically.
>>>
>>> Following the procedure in this tutorial:
>>> https://major.io/2014/04/21/launch-secure-lxc-containers-on-fedora-20-using-selinux-and-svirt/ 
>>>
>>> I created and installed a test container.  I had to add ifcfg-eth0 
>>> for a simple network and then run "service network start" for the 
>>> netowrk to actually come up ... which it did with a DHCP (actually 
>>> dnsmasq) assigned IP address.  Note that this procedure explicitly 
>>> installs the dhclient package.
>>>
>>> So, what am I doing "wrong" with secure containers?  Or, is this a bug?
>>>
>> Ping!!  Hello ... anybody out there??
>>
>> To keep my sanity, would SOMEBODY PLEASE try doing a secure sandbox 
>> with a dhcp network and see if the network is started or not.  My 
>> case: static network started, dhcpnetwork is NOT started 
>> (/sbin/dhclient is not running).
>>
>> Here is what I have done so far:
>>
>> 1. "Instrumented" libvirt-sandbox-init-common.c and 
>> libvirt-sandbox-init-lxc.c by turning on debug and adding a whole 
>> bunch of fprintf(stderr,...) statements to track the initialization.  
>> These say that start_dhcp() in libvirt-sandbox-init-common.c is 
>> executed successfully. Nevertheless, for some reason, the 
>> g_spawn_async() did not result in a running /sbin/dhclient.
>>
>> 2. So, I tried running dhclient myself.  I had two networks defined: 
>> "-N <static-ip>,source=net18 -N dhcp,source=default".  After 
>> connecting top the secure container, I did:
>>       /sbin/dhclient  --no-pid  eth1
>> which resulted in the network on eth1 starting with a 192.168.122.<n> 
>> address.
>>
>> 3.  I then went a step further.  I took the start_dhcp() code from 
>> libvirt-sandbox-init-common.c and encapsulated it with a wrapper to 
>> fake what was done in init-common.c but with its own main(). Compiled 
>> this and put the binary where I could execute it after doing the 
>> connect.  Stop, start, and connect to the secure container.  The 
>> network on eth1 is not started.  Run my test_dhcp_start program and 
>> the result was the eth1 network is started and there is a dhclient 
>> running.
>>
>> Suggestions please!
>>
> This is getting really strange!  I put a bash-shell-script wrapper 
> around dhclient so that I could add a little logging when dhclient 
> started.  It is never executed!!!  And yet, once the secure container 
> has started, I can connect and manually run dhclient with no problems 
> both direct command line and via a small fake-it program which runs 
> g_spawn_async().
>
> Part of the problem is that /usr/libexec/libvirt-sandbox-init-lxc and 
> /usr/libexec/libvirt-sandbox-init-common run in the secure container 
> environment but are also part of the software which initializes the 
> secure container.  At this point, I really wish that networking was a 
> separate systemd service which was controlled by systemd.  I wonder if 
> there is some way to run gdb to help trace the execution.
>
> Next step ... convert to using g_spawn_sync() rather than 
> g_spawn_async() to see if that produces any change.  The 
> g_spawn_sync() seems to work OK running "ip" to set up the static IP NIC.
>
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
>
I found the problem and have a fix (although I am not certain why there 
is a problem).  Patches submitted.

The "fix" is to use g_spawn_sync() rather than g_spawn_async() to start 
dhclient.

Nextg, I need to get this built and running on F21.

Gene