<div class="zcontentRow"> i explain the case again. the net config xml of two vms is : ＜interface type='bridge'＞ ＜mac address='fa:16:3e:e1:b2:01'/＞ ＜source bridge='br100'/＞ ＜model type='virtio'/＞ ＜driver name='vhost'/＞ ＜virtualport type='openvswitch'/＞ ＜/interface＞of cause the mac of two vms is different. the bridge named br100 is created in advance.when the first vm started ,we can find the tao device named "vnet0" by virsh domiflist domid command.we can shutdow the first vm and start the second vm at or around the same time. we can find the tap device of the second vm is also named "vnet0".starting up and shutdowning vm are not <span id="w_11" high-light-id="w_4,w_11,w_12" class="" style="color: rgb(51, 51, 51); font-family: Arial, STHeiti, 宋体, 'WenQuanYi Micro Hei', sans-serif; line-height: 22px; background-color: rgb(249, 249, 249);">mutually <span id="w_12" high-light-id="w_4,w_11,w_12" class="" style="color: rgb(51, 51, 51); font-family: Arial, STHeiti, 宋体, 'WenQuanYi Micro Hei', sans-serif; line-height: 22px; background-color: rgb(249, 249, 249);">exclusive. adding port named "vnet0" to bridge and deleting port from bridge are not <span id="w_11" high-light-id="w_4,w_11,w_12" class="" style="line-height: 22px; color: rgb(51, 51, 51); font-family: Arial, STHeiti, 宋体, 'WenQuanYi Micro Hei', sans-serif; background-color: rgb(249, 249, 249);">mutually <span id="w_12" high-light-id="w_4,w_11,w_12" class="" style="line-height: 22px; color: rgb(51, 51, 51); font-family: Arial, STHeiti, 宋体, 'WenQuanYi Micro Hei', sans-serif; background-color: rgb(249, 249, 249);">exclusive.so the problem occurred. the port of the second vm added to the bridge is deteled by calling virNetDevOpenvswitchRemovePort in shutdowning the first vm . <div class="zMailSign"><div><div><div><div> 为了让您的VPlat虚拟化故障得到高效的处理，请上报故障到: $VPlat技术支持。芦志朋 luzhipeng IT开发工程师 IT Development Engineer 操作系统产品部/中心研究院/系统产品 OS Product Dept./Central R＆D Institute/System Product <table style="color: rgb(0, 0, 0); font-family: 宋体; widows: 1;"><tbody><tr class="firstRow"><td valign="top" width="100"><img id="sign-icon" src="cid:9ae3e214c17d49ed935d87c674ba3ee2" width="130" height="120"></td><td valign="top" width="500"><img id="sign-logo" src="cid:24242e5637af428891c4db731e7765ad" width="115" height="38"> 深圳市南山区科技南路55号中兴通讯研发大楼33楼 33/F, R&D Building, ZTE Corporation Hi-tech Road South, Hi-tech Industrial Park Nanshan District, Shenzhen, P.R.China, 518057 T: +86 755 xxxxxxxx F:+86 755 xxxxxxxx M: +86 xxxxxxxxxxx E: lu.zhipeng@zte.com.cn <a href="http://www.zte.com.cn/" target="_blank">www.zte.com.cn</a></td></tr></tbody></table></div></div></div></div></div><div><div class="zhistoryRow" style="display:block"><div class="zhistoryDes" style="width: 100%; height: 28px; line-height: 28px; background-color: #E0E5E9; color: #1388FF; text-align: center;" language-data="HistoryOrgTxt">原始邮件</div><div id="zwriteHistoryContainer"><div class="control-group zhistoryPanel"><div class="zhistoryHeader" style="padding: 8px; background-color: #F5F6F8;"><div>发件人： ＜joel@aenertia.net＞;</div><div>收件人： ＜laine@laine.org＞;</div><div>抄送人： ＜libvir-list@redhat.com＞;芦志朋10108272;</div><div>日期：2017年04月29日 11:21</div><div>主题：Re: [libvirt]答复: Re: [PATCH] qemu: change the name of tap device for a tapand bridge network</div></div> <div><div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">I'll try and explain the steps/situation where this is an issue. </div><div class="gmail_default" style="font-family:verdana,sans-serif">Say for example you have your router/dhcpd done inside a VM guest - you use macvtap devices and vlans off the hypervisor to get the traffic in and out (WAN and LAN). And you want the LAN side to provide connectivity to the host as well as to the network beyond the host. this is on a Ubuntu 16.04 host. </div><div class="gmail_default" style="font-family:verdana,sans-serif">1) Create and isolated virtual Network via libvirt xml - say 'LAN' this is the second network (assume default exists) creates virbr1 and a vnic1 This is addition to the WAN and LAN macvtap devices. Unbeknown-st to libvirt the LAN macvtap is going to end up bridge to the 'isolated' network by the VM. </div><div class="gmail_default" style="font-family:verdana,sans-serif">2) the virbr1 bridge needs numbering </div><div class="gmail_default" style="font-family:verdana,sans-serif">vnic1 is attached inside the VM to a dhcpd server but you still need the virbr1 interface to be numbered on the hypervisor (but you don't want the libvirt networking to take care of the numbering etc becuase that's what your fancy NFV VM is going to handle). Manually it's easy at this point you just run dhclient <a href="http://et.al" target="_blank">et.al</a> on the bridge and you are good to go. </div><div class="gmail_default" style="font-family:verdana,sans-serif">BUT you want this to happen at boot; adding any sort of etc/network or /etc/sysconfig/networking scripts refering to that virbr1 will create a race condition which will stop the vm from getting started as you mention. </div> </div><div class="gmail_extra"> <div class="gmail_quote">On 29 April 2017 at 14:45, Laine Stump ＜<a href="mailto:laine@laine..org" target="_blank">laine@laine.org</a>＞ wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 04/28/2017 07:23 PM, Joel Wirāmu Pauling wrote: ＞ Possibly related I notice race conditions caused by vnic never getting ＞ loaded if an existing bridge is already up (by OS init scripts etc) and ＞ stopping VM's from getting started. Can you explain this more exactly? In particular, by "vnic" do you mean the network device as seen by the guest? And what do you mean by the term "getting loaded"? (It doesn't make sense to me that you should mean the network device in the guest, unless "stopping VM's from getting started" just means that the guest doesn't become fully functional, rather than that the qemu process doesn't start). Beyond that, of course the bridge that the tap device will be connected to needs to exist before you can connect something to it - I don't understand how its existence could cause a failure; rather its *non*-existence would cause a failure. (As you can see by my wild suppositions that make no sense, the terms you've used are a bit too vague/open to interpretation for me to understand exactly the problem you're referring to) (Hmm - perhaps you're referring to the situation where libvirt attempts to create a bridge on the host for one of its virtual networks, but either a bridge by that name has already been created by "someone else" or another netdev already exists on the host that is on the same subnet (and thus has the same route)? Your description doesn't fit that very well, but that is a known problem and unrelated to the patch/problem we're discussing here. ＞ ＞ Often this is behavior you want ; i.e having Host Hypervisor NIC's added ＞ and up before libvirtd sets up it's nics/bridges. ＞ ＞ ＞ On 29 April 2017 at 11:15, ＜<a href="mailto:lu.zhipeng@zte.com.cn" target="_blank">lu.zhipeng@zte.com.cn</a> ＞＜mailto:<a href="mailto:lu.zhipeng@zte.com.cn" target="_blank">lu.zhipeng@zte.com.cn</a>＞＞ wrote: ＞ ＞ ＞＞On Fri, Apr 28, 2017 at 05:23:19PM +0800, ZhiPeng Lu wrote: ＞＞＞ ＞ Creating tap device and adding the device to bridge are not atomic operation. ＞＞＞ ＞ Similarly deleting tap device and removing it from bridge are not atomic operation. ＞＞＞The Problem occurs when two vms start and shutdown. When one vm with the nic ＞＞＞ ＞ named "vnet0" stopping, it deleted tap device but not removing port from bridge. ＞＞＞ ＞ At this time, another vm created the tap device named "vnet0" and added port to the ＞＞＞ ＞ same bridge. Then, the first vm deleted the tap device from the same bridge. ＞＞＞ ＞ Finally, the tap device of the second vm don't attached to the bridge. ＞＞＞ ＞ So, we can add domid to vm's nic name. For example, the vm's domid is 1 and vnet0 ＞＞＞ is renamed to vnet1.0. ＞ ＞＞Surely deleting the NIC automatically removes it from the bridge so we ＞＞can just remove the code that delets the bridge port. ＞ ＞ i have done some tests for a tap + openvswitch bridge network. i ＞ find the nic named "vnet0" don't exsit before calling ＞ the virNetDevOpenvswitchRemovePort. ＞ ＞ i think the tap is deleted not by removingport from bridge. i ＞ think hotpluging net has the same problem. ＞ ＞ ＞ by the way: ＞ ＞ my company's e-mail does not support thread posting and has ＞ some other problems. i believe that my colleagues can soon resloved ＞ them. ＞ ＞ ＞ ＞ ＞ ＞ ＞ *为了让您的VPlat虚拟化故障得到高效的处理，请上报故障到: $VPlat技术支 ＞持。* ＞ ＞芦志朋 luzhipeng ＞ ＞ ＞ IT开发工程师 IT Development Engineer ＞操作系统产品部/中心研究院/系统产品 OS Product Dept./Central R＆D ＞ Institute/System Product ＞ ＞ ＞ ＞深圳市南山区科技南路55号中兴通讯研发大楼33楼 ＞ 33/F, R&D Building, ZTE Corporation Hi-tech Road South, ＞ Hi-tech Industrial Park Nanshan District, Shenzhen, P.R.China, 518057 ＞ T: +86 755 xxxxxxxx F:+86 755 xxxxxxxx ＞ M: +86 xxxxxxxxxxx ＞ E: <a href="mailto:lu.zhipeng@zte.com.cn" target="_blank">lu.zhipeng@zte.com.cn</a> ＜mailto:<a href="mailto:lu.zhipeng@zte.com.cn" target="_blank">lu.zhipeng@zte.com.cn</a>＞ ＞ <a href="http://www.zte.com.cn" rel="noreferrer" target="_blank">www.zte.com.cn</a> ＜<a href="http://www.zte.com.cn/" rel="noreferrer" target="_blank">http://www.zte.com.cn/</a>＞ ＞ ＞原始邮件 ＞ *发件人：*＜<a href="mailto:berrange@redhat.com" target="_blank">berrange@redhat.com</a> ＜mailto:<a href="mailto:berrange@redhat.com" target="_blank">berrange@redhat.com</a>＞＞; ＞ *收件人：*芦志朋10108272; ＞ *抄送人：*＜<a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a> ＜mailto:<a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a>＞＞; ＞ *日期：*2017年04月28日 19:27 ＞ *主题：**Re: [libvirt] [PATCH] qemu: change the name of tap device ＞ for a tapand bridge network* ＞ ＞ ＞ On Fri, Apr 28, 2017 at 05:23:19PM +0800, ZhiPeng Lu wrote: ＞＞ Creating tap device and adding the device to bridge are not atomic operation. ＞＞ Similarly deleting tap device and removing it from bridge are not atomic operation. ＞＞ The Problem occurs when two vms start and shutdown. When one vm with the nic ＞＞ named "vnet0" stopping, it deleted tap device but not removing port from bridge. ＞＞ At this time, another vm created the tap device named "vnet0" and added port to the ＞＞ same bridge. Then, the first vm deleted the tap device from the same bridge. ＞＞ Finally, the tap device of the second vm don't attached to the bridge. ＞＞ So, we can add domid to vm's nic name.. For example, the vm's domid is 1 and vnet0 ＞＞ is renamed to vnet1.0. ＞ ＞ Surely deleting the NIC automatically removes it from the bridge so we ＞ can just remove the code that delets the bridge port. ＞ ＞ ＞ Regards, ＞ Daniel ＞ -- ＞ |: <a href="https://berrange.com" rel="noreferrer" target="_blank">https://berrange.com</a> -o- <a href="https://www.flickr.com/photos/dberrange" rel="noreferrer" target="_blank">https://www.flickr.com/photos/dberrange</a> ＞＜<a href="https://www.flickr.com/photos/dberrange" rel="noreferrer" target="_blank">https://www.flickr.com/photos/dberrange</a>＞ :| ＞ |: <a href="https://libvirt.org" rel="noreferrer" target="_blank">https://libvirt.org</a> -o- <a href="https://fstop138..berrange.com" rel="noreferrer" target="_blank">https://fstop138.berrange.com</a> ＞＜<a href="https://fstop138.berrange.com" rel="noreferrer" target="_blank">https://fstop138.berrange.com</a>＞ :| ＞ |: <a href="https://entangle-photo.org" rel="noreferrer" target="_blank">https://entangle-photo.org</a> -o- <a href="https://www.instagram.com/dberrange" rel="noreferrer" target="_blank">https://www.instagram.com/dberrange</a> ＞＜<a href="https://www.instagram.com/dberrange" rel="noreferrer" target="_blank">https://www.instagram.com/dberrange</a>＞ :| ＞ ＞ -- ＞ libvir-list mailing list ＞ <a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a> ＜mailto:<a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a>＞ ＞ <a href="https://www.redhat.com/mailman/listinfo/libvir-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a> ＞＜<a href="https://www.redhat.com/mailman/listinfo/libvir-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a>＞ ＞ ＞ ＞ ＞ ＞ -- ＞ libvir-list mailing list ＞ <a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a> ＜mailto:<a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a>＞ ＞ <a href="https://www.redhat.com/mailman/listinfo/libvir-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a> <div class="HOEnZb"><div class="h5">＞＜<a href="https://www.redhat.com/mailman/listinfo/libvir-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a>＞ ＞ ＞ ＞ ＞ ＞ -- ＞ libvir-list mailing list ＞ <a href="mailto:libvir-list@redhat.com" target="_blank">libvir-list@redhat.com</a> ＞ <a href="https://www.redhat.com/mailman/listinfo/libvir-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/libvir-list</a> ＞ </div></div></blockquote></div> </div></div> </div></div></div></div> </div>