[libvirt] DHCP and secure containers

[Gene Czarcinski]

On 09/07/2014 11:03 AM, Gene Czarcinski wrote:
> On 09/03/2014 09:42 AM, Gene Czarcinski wrote:
>> On 09/02/2014 06:37 AM, Gene Czarcinski wrote:
>>> OK, hopefully this mailing list is more active and I can get some 
>>> response to my questions.
>>> --------------------------------------------------------------------------------------------------------------------- 
>>>
>>> I have been "playing with" Secure Containers running a lighttpd 
>>> server and have it up and running.  I used Adam's process 
>>> (https://www.happyassassin.net/2014/07/23/bridged-networking-for-libvirt-with-networkmanager-2014-fedora-21/) 
>>> for getting a bridge defined when also running NetworkManager. I 
>>> then created a virtual network definition:
>>>> <network>
>>>>   <name>net18</name>
>>>> <uuid>8d19a05b-ac85-4e2a-88bc-5ca4cbb29a33</uuid>
>>>>   <forward mode='bridge'/>
>>>>   <bridge name='br0'/>
>>>> </network>
>>> This works fine when I use static addresses such as:
>>>> -N 
>>>> source=net18,address=192.168.18.94/24,route=192.168.18.255%192.168.18.1 
>>>>
>>> but does not work when I specify using dhcp:
>>>> -N source=net18,dhcp
>>> I have reported this as a bug: 
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1133686
>>>
>>> Since there has not been much of a reaction to the BZ report, I 
>>> decided to take a look at the source code (it sure would have been 
>>> nice if the SRPMS were there in the F20 fedora-virt-preview but I 
>>> get the package from development/21).
>>>
>>> I see that libvirt-sandbox-init-common.c has the code for starting 
>>> dhcp and also has main() along with some runtime options for -v 
>>> verbose and -d debug.
>>>
>>> OK, how do I go about turning verbose and/or debug on?
>>>
>>> Any suggestions on how to debug and get dhcp to work?  I not only 
>>> want to find the problem but to fix the problem if needed.
>> While I have not figured out how to get dhcp to work with a secure 
>> container create by virt-sandbox-service, I have gotten a container 
>> working with the network up and a dhcp assigned IP using the lxc-* 
>> commands and following this procedure:
>> https://sysadmincasts.com/episodes/24-introduction-to-containers-on-linux-using-lxc 
>>
>> to create a "busybox" container.  The network came up automatically.
>>
>> Following the procedure in this tutorial:
>> https://major.io/2014/04/21/launch-secure-lxc-containers-on-fedora-20-using-selinux-and-svirt/ 
>>
>> I created and installed a test container.  I had to add ifcfg-eth0 
>> for a simple network and then run "service network start" for the 
>> netowrk to actually come up ... which it did with a DHCP (actually 
>> dnsmasq) assigned IP address.  Note that this procedure explicitly 
>> installs the dhclient package.
>>
>> So, what am I doing "wrong" with secure containers?  Or, is this a bug?
>>
> Ping!!  Hello ... anybody out there??
>
> To keep my sanity, would SOMEBODY PLEASE try doing a secure sandbox 
> with a dhcp network and see if the network is started or not.  My 
> case: static network started, dhcpnetwork is NOT started 
> (/sbin/dhclient is not running).
>
> Here is what I have done so far:
>
> 1. "Instrumented" libvirt-sandbox-init-common.c and 
> libvirt-sandbox-init-lxc.c by turning on debug and adding a whole 
> bunch of fprintf(stderr,...) statements to track the initialization.  
> These say that start_dhcp() in libvirt-sandbox-init-common.c is 
> executed successfully. Nevertheless, for some reason, the 
> g_spawn_async() did not result in a running /sbin/dhclient.
>
> 2. So, I tried running dhclient myself.  I had two networks defined: 
> "-N <static-ip>,source=net18 -N dhcp,source=default".  After 
> connecting top the secure container, I did:
>       /sbin/dhclient  --no-pid  eth1
> which resulted in the network on eth1 starting with a 192.168.122.<n> 
> address.
>
> 3.  I then went a step further.  I took the start_dhcp() code from 
> libvirt-sandbox-init-common.c and encapsulated it with a wrapper to 
> fake what was done in init-common.c but with its own main(). Compiled 
> this and put the binary where I could execute it after doing the 
> connect.  Stop, start, and connect to the secure container.  The 
> network on eth1 is not started.  Run my test_dhcp_start program and 
> the result was the eth1 network is started and there is a dhclient 
> running.
>
> Suggestions please!
>
This is getting really strange!  I put a bash-shell-script wrapper 
around dhclient so that I could add a little logging when dhclient 
started.  It is never executed!!!  And yet, once the secure container 
has started, I can connect and manually run dhclient with no problems 
both direct command line and via a small fake-it program which runs 
g_spawn_async().

Part of the problem is that /usr/libexec/libvirt-sandbox-init-lxc and 
/usr/libexec/libvirt-sandbox-init-common run in the secure container 
environment but are also part of the software which initializes the 
secure container.  At this point, I really wish that networking was a 
separate systemd service which was controlled by systemd.  I wonder if 
there is some way to run gdb to help trace the execution.

Next step ... convert to using g_spawn_sync() rather than 
g_spawn_async() to see if that produces any change.  The g_spawn_sync() 
seems to work OK running "ip" to set up the static IP NIC.

Gene