[libvirt] LXC: autostart feature does set all interfaces to state up.

[Richard Weinberger]

Am 08.07.2013 05:54, schrieb Gao feng:
> On 07/05/2013 06:22 PM, Richard Weinberger wrote:
>> Am 05.07.2013 03:36, schrieb Gao feng:
>>> On 07/05/2013 04:45 AM, Richard Weinberger wrote:
>>>> Hi,
>>>>
>>>> Am 03.07.2013 12:04, schrieb Gao feng:
>>>>> Hi,
>>>>> On 07/01/2013 03:45 PM, Richard Weinberger wrote:
>>>>>> Hi!
>>>>>>
>>>>>> If you have multiple LXC containers with networking and the autostart feature enabled libvirtd fails to
>>>>>> up some veth interfaces on the host side.
>>>>>>
>>>>>> Most of the time only the first veth device is in state up, all others are down.
>>>>>>
>>>>>> Reproducing is easy.
>>>>>> 1. Define a few containers (5 in my case)
>>>>>> 2. Run "virsh autostart ..." on each one.
>>>>>> 3. stop/start libvirtd
>>>>>>
>>>>>> You'll observe that all containers are running, but "ip a" will report on the host
>>>>>> side that not all veth devices are up and are not usable within the containers.
>>>>>>
>>>>>> This is not userns related, just retested with libvirt of today.
>>>>>
>>>>> I can not reproduce this problem on my test bed...
>>>>
>>>> Strange.
>>>>
>>>>> maybe you should wait some seconds for the starting of these containers.
>>>>
>>>> Please see the attached shell script. Using it I'm able to trigger the issue on all of
>>>> my test machines.
>>>> run.sh creates six very minimal containers and enables autostart. Then it kills and restarts libvirtd.
>>>> After the script is done you'll see that only one or two veth devices are up.
>>>>
>>>> On the over hand, if I start them manually using a command like this one:
>>>> for cfg in a b c d e f ; do /opt/libvirt/bin/virsh -c lxc:/// start test-$cfg ; done
>>>> All veths are always up.
>>>>
>>>
>>>
>>> I still can not reproduce even use your script.
>>>
>>> [root at Donkey-I5 Desktop]# ./run.sh
>>> Domain test-a defined from container_a.conf
>>>
>>> Domain test-a marked as autostarted
>>>
>>> Domain test-b defined from container_b.conf
>>>
>>> Domain test-b marked as autostarted
>>>
>>> Domain test-c defined from container_c.conf
>>>
>>> Domain test-c marked as autostarted
>>>
>>> Domain test-d defined from container_d.conf
>>>
>>> Domain test-d marked as autostarted
>>>
>>> Domain test-e defined from container_e.conf
>>>
>>> Domain test-e marked as autostarted
>>>
>>> Domain test-f defined from container_f.conf
>>>
>>> Domain test-f marked as autostarted
>>>
>>> 2013-07-05 01:26:47.155+0000: 27163: info : libvirt version: 1.1.0
>>> 2013-07-05 01:26:47.155+0000: 27163: debug : virLogParseOutputs:1334 : outputs=1:file:/home/gaofeng/libvirtd.log
>>> waiting a bit....
>>> 167: veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>>> 169: veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>>> 171: veth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>>> 173: veth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>>> 175: veth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>>> 177: veth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>>>
>>>
>>> Can you post your libvirt debug log?
>>
>> Please see attached file.
>>
>> 43: veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>> 45: veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>> 47: veth2: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
>> 49: veth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>> 51: veth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UP qlen 1000
>> 53: veth5: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 100
>>
> 
> strange, I can not see veth related error message from your log.
> 
> seems like all of the veth devices of host had been up but for some reasons they become down.

I think libvirt has to do "ip link set dev vethX up".
Otherwise the device state is undefined.

> can you also show me the bridge information on your host?
> brctl show virbr0

bridge name     bridge id               STP enabled     interfaces
virbr0          8000.2e63ddca07fa       yes             veth0
                                                        veth1
                                                        veth2
                                                        veth3
                                                        veth4
                                                        veth5

> and your default net configuration
> virsh -c lxc:/// net-dumpxml default

<network>
  <name>default</name>
  <uuid>a81b9f40-d279-4e96-8cec-6a906323f162</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0' />
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>

Thanks,
//richard