From arindam at live.com Fri Apr 12 11:13:18 2013 From: arindam at live.com (Arindam Choudhury) Date: Fri, 12 Apr 2013 13:13:18 +0200 Subject: [Rdo-list] problem installing quantum in fedora 18 Message-ID: Hi, I am new to openstack and I am trying to install openstack folsom on fedora 18. any help will be highly appreciated. while installing quantum I got this error: [(keystone_admin)]$ quantum net-create --tenant-id 112a75ab04224fa3b44109a6c4859c28 net1 [Errno 110] Connection timed out /var/log/quantum/server.log 2013-04-12 12:43:53 INFO [quantum.common.config] Logging enabled! 2013-04-12 12:43:53 INFO [quantum.common.config] Config paste file: /etc/quantum/api-paste.ini 2013-04-12 12:43:53 INFO [quantum.manager] Loading Plugin: quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 2013-04-12 12:43:53 INFO [quantum.plugins.openvswitch.ovs_quantum_plugin] Network VLAN ranges: {} 2013-04-12 12:43:53 INFO [quantum.openstack.common.rpc.impl_qpid] Connected to AMQP server on 158.109.65.21:5672 2013-04-12 12:43:53 INFO [quantum.api.extensions] Initializing extension manager. 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: __init__.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: extensions.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: _quotav2_model.py 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: l3.py 2013-04-12 12:43:53 WARNING [quantum.api.extensions] Loaded extension: router 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: _quotav2_model.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: _quotav2_driver.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: quotasv2.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: providernet.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: flavor.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: l3.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: quotasv2.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: l3.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: extensions.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: providernet.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: quotasv2.py 2013-04-12 12:43:53 WARNING [quantum.api.extensions] Exception loading extension: Invalid extension environment: quota driver quantum.extensions._quotav2_driver.DbQuotaDriver is needed. 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: extensions.py 2013-04-12 12:43:53 WARNING [quantum.api.extensions] Did not find expected name "Extensions" in /usr/lib/python2.7/site-packages/quantum/extensions/extensions.py 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: __init__.py 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: flavor.py 2013-04-12 12:43:53 WARNING [quantum.api.extensions] extension flavor not supported by plugin 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: __init__.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: _quotav2_driver.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: flavor.pyc 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: providernet.py 2013-04-12 12:43:53 WARNING [quantum.api.extensions] Loaded extension: provider 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: _quotav2_model.pyo 2013-04-12 12:43:53 INFO [quantum.api.extensions] Loading extension file: _quotav2_driver.py 2013-04-12 12:43:53 INFO [keystone.middleware.auth_token] Starting keystone auth_token middleware 2013-04-12 12:43:53 INFO [keystone.middleware.auth_token] Using /var/lib/quantum/keystone-signing as cache directory for signing certificate 2013-04-12 13:08:07 INFO [quantum.openstack.common.rpc.impl_qpid] Connected to AMQP server on 158.109.65.21:5672 2013-04-12 13:09:05 INFO [quantum.common.config] Logging enabled! 2013-04-12 13:09:05 INFO [quantum.common.config] Config paste file: /etc/quantum/api-paste.ini 2013-04-12 13:09:05 INFO [quantum.manager] Loading Plugin: quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 2013-04-12 13:09:05 INFO [quantum.plugins.openvswitch.ovs_quantum_plugin] Network VLAN ranges: {} 2013-04-12 13:09:05 INFO [quantum.openstack.common.rpc.impl_qpid] Connected to AMQP server on 158.109.65.21:5672 2013-04-12 13:09:05 INFO [quantum.api.extensions] Initializing extension manager. 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: __init__.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: extensions.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: _quotav2_model.py 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: l3.py 2013-04-12 13:09:05 WARNING [quantum.api.extensions] Loaded extension: router 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: _quotav2_model.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: _quotav2_driver.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: quotasv2.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: providernet.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: flavor.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: l3.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: quotasv2.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: l3.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: extensions.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: providernet.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: quotasv2.py 2013-04-12 13:09:05 WARNING [quantum.api.extensions] Exception loading extension: Invalid extension environment: quota driver quantum.extensions._quotav2_driver.DbQuotaDriver is needed. 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: extensions.py 2013-04-12 13:09:05 WARNING [quantum.api.extensions] Did not find expected name "Extensions" in /usr/lib/python2.7/site-packages/quantum/extensions/extensions.py 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: __init__.py 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: flavor.py 2013-04-12 13:09:05 WARNING [quantum.api.extensions] extension flavor not supported by plugin 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: __init__.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: _quotav2_driver.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: flavor.pyc 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: providernet.py 2013-04-12 13:09:05 WARNING [quantum.api.extensions] Loaded extension: provider 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: _quotav2_model.pyo 2013-04-12 13:09:05 INFO [quantum.api.extensions] Loading extension file: _quotav2_driver.py 2013-04-12 13:09:05 INFO [keystone.middleware.auth_token] Starting keystone auth_token middleware 2013-04-12 13:09:05 INFO [keystone.middleware.auth_token] Using /var/lib/quantum/keystone-signing as cache directory for signing certificate -------------- next part -------------- An HTML attachment was scrubbed... URL: From dneary at redhat.com Wed Apr 17 19:20:39 2013 From: dneary at redhat.com (Dave Neary) Date: Wed, 17 Apr 2013 12:20:39 -0700 Subject: [Rdo-list] Announcing RDO Message-ID: <516EF607.2030506@redhat.com> Hi everyone, The RDO community site is now live! RDO is Red Hat's community-supported distribution of OpenStack for Red Hat Enterprise Linux and its clones, and for Fedora. The site is now online at: http://openstack.redhat.com What we've announced is two things: * We are providing well integrated, easy to install packages of OpenStack Grizzly for Red Hat Enterprise Linux 6.4, and equivalent versions of CentOS, Scientific Linux, etc, and for Fedora 18. * We have released a website at openstack.redhat.com to grow a community of OpenStack users on Red Hat platforms If you are interested in trying out OpenStack Grizzly on RHEL, or other Enterprise Linux distributions, then you are welcome to install it, join our forums and share your experiences. For those who prefer mailing lists to forums, we also have a mailing list, rdo-list: https://www.redhat.com/mailman/listinfo/rdo-list What does this mean for Red Hat OpenStack users, and subscribers to rhos-list? The short answer is that this adds a new option for you. If you would like to install a community supported OpenStack Grizzly distribution on Red Hat Enterprise Linux, CentOS or Scientific Linux in anticipation of a future Red Hat supported Grizzly-based product, then RDO is a good choice. If you are interested in deploying enterprise-hardened Folsom on Red Hat Enterprise Linux, then Red Hat OpenStack early adopter Edition is a great choice, and rhos-list is the best place to get help with that. You can read more about the RDO announcement at http://www.redhat.com/about/news/press-archive/2013/4/red-hat-advances-its-openstack-enterprise-and-community-technologies-and-roadmap Thanks, Dave. -- Dave Neary - Community Action and Impact Open Source and Standards, Red Hat - http://community.redhat.com Ph: +33 9 50 71 55 62 / Cell: +33 6 77 01 92 13 From dennisml at conversis.de Thu Apr 18 14:41:32 2013 From: dennisml at conversis.de (Dennis Jacobfeuerborn) Date: Thu, 18 Apr 2013 16:41:32 +0200 Subject: [Rdo-list] Dashboard error Message-ID: <5170061C.1080204@conversis.de> Hi, I just deployed an openstack controller using the stackforge puppet modules combined with the RDO package repository. The good news is that I got no errors and everything seems to be up and running. The bad news is that the dashboard doesn't seem to work and spits out the following error when run with debugging enabled: TypeError at /admin/ __init__() got an unexpected keyword argument 'http_log_debug' Request Method: GET Request URL: http://10.16.171.3/dashboard/admin/ Django Version: 1.4.5 Exception Type: TypeError Exception Value: __init__() got an unexpected keyword argument 'http_log_debug' Exception Location: /usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/cinder.py in cinderclient, line 59 Python Executable: /usr/bin/python Python Version: 2.6.6 Python Path: ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', '/usr/lib64/python26.zip', '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old', '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages', '/usr/lib/python2.6/site-packages', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/usr/share/openstack-dashboard/openstack_dashboard'] The horizon log file doesn't show any errors: novaclient connection created using token "5c94a3f98ee1464caf918db60f556646" and url "http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f" REQ: curl -i http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-simple-tenant-usage?start=2013-04-01T00:00:00&end=2013-04-18T14:06:54.316095&detailed=1 -X GET -H "X-Auth-Project-Id: e6a5d2cc19a841e1b50208422742814f" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 5c94a3f98ee1464caf918db60f556646" RESP:{'status': '200', 'content-length': '21', 'content-location': u'http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-simple-tenant-usage?start=2013-04-01T00:00:00&end=2013-04-18T14:06:54.316095&detailed=1', 'x-compute-request-id': 'req-b839be8f-dd65-424c-8b85-2394f1817ccb', 'date': 'Thu, 18 Apr 2013 14:06:54 GMT', 'content-type': 'application/json'} {"tenant_usages": []} novaclient connection created using token "5c94a3f98ee1464caf918db60f556646" and url "http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f" REQ: curl -i http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-quota-sets/e6a5d2cc19a841e1b50208422742814f -X GET -H "X-Auth-Project-Id: e6a5d2cc19a841e1b50208422742814f" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 5c94a3f98ee1464caf918db60f556646" RESP:{'status': '200', 'content-length': '322', 'content-location': u'http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-quota-sets/e6a5d2cc19a841e1b50208422742814f', 'x-compute-request-id': 'req-95c8b946-1e31-47bc-9e94-d913f9cf9ddc', 'date': 'Thu, 18 Apr 2013 14:06:54 GMT', 'content-type': 'application/json'} {"quota_set": {"injected_file_content_bytes": 10240, "metadata_items": 128, "ram": 51200, "floating_ips": 10, "key_pairs": 100, "id": "e6a5d2cc19a841e1b50208422742814f", "instances": 10, "security_group_rules": 20, "injected_files": 5, "cores": 20, "fixed_ips": -1, "injected_file_path_bytes": 255, "security_groups": 10}} cinderclient connection created using token "5c94a3f98ee1464caf918db60f556646" and url "http://10.16.171.3:8776/v1/e6a5d2cc19a841e1b50208422742814f" I seems the login has succeeded but the /admin URL doesn't seem to work. Any ideas on what could be the cause of this? Regards, Dennis From dennisml at conversis.de Thu Apr 18 16:53:29 2013 From: dennisml at conversis.de (Dennis Jacobfeuerborn) Date: Thu, 18 Apr 2013 18:53:29 +0200 Subject: [Rdo-list] Dashboard error In-Reply-To: <5170061C.1080204@conversis.de> References: <5170061C.1080204@conversis.de> Message-ID: <51702509.5000306@conversis.de> Never mind. It turns out that while I updated the openstack-* packages from epel to the rdo versions I forgot to do the same for the python client packages. After that things work as expected. Regards, Dennis On 18.04.2013 16:41, Dennis Jacobfeuerborn wrote: > Hi, > I just deployed an openstack controller using the stackforge puppet > modules combined with the RDO package repository. The good news is that > I got no errors and everything seems to be up and running. The bad news > is that the dashboard doesn't seem to work and spits out the following > error when run with debugging enabled: > > TypeError at /admin/ > > __init__() got an unexpected keyword argument 'http_log_debug' > > Request Method: GET > Request URL: http://10.16.171.3/dashboard/admin/ > Django Version: 1.4.5 > Exception Type: TypeError > Exception Value: > > __init__() got an unexpected keyword argument 'http_log_debug' > > Exception Location: > /usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/cinder.py > in cinderclient, line 59 > Python Executable: /usr/bin/python > Python Version: 2.6.6 > Python Path: > > ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', > '/usr/lib64/python26.zip', > '/usr/lib64/python2.6', > '/usr/lib64/python2.6/plat-linux2', > '/usr/lib64/python2.6/lib-tk', > '/usr/lib64/python2.6/lib-old', > '/usr/lib64/python2.6/lib-dynload', > '/usr/lib64/python2.6/site-packages', > '/usr/lib/python2.6/site-packages', > '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', > '/usr/share/openstack-dashboard/openstack_dashboard'] > > The horizon log file doesn't show any errors: > > novaclient connection created using token > "5c94a3f98ee1464caf918db60f556646" and url > "http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f" > > REQ: curl -i > http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-simple-tenant-usage?start=2013-04-01T00:00:00&end=2013-04-18T14:06:54.316095&detailed=1 > -X GET -H "X-Auth-Project-Id: e6a5d2cc19a841e1b50208422742814f" -H > "User-Agent: python-novaclient" -H "Accept: application/json" -H > "X-Auth-Token: 5c94a3f98ee1464caf918db60f556646" > > RESP:{'status': '200', 'content-length': '21', 'content-location': > u'http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-simple-tenant-usage?start=2013-04-01T00:00:00&end=2013-04-18T14:06:54.316095&detailed=1', > 'x-compute-request-id': 'req-b839be8f-dd65-424c-8b85-2394f1817ccb', > 'date': 'Thu, 18 Apr 2013 14:06:54 GMT', 'content-type': > 'application/json'} {"tenant_usages": []} > > novaclient connection created using token > "5c94a3f98ee1464caf918db60f556646" and url > "http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f" > > REQ: curl -i > http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-quota-sets/e6a5d2cc19a841e1b50208422742814f > -X GET -H "X-Auth-Project-Id: e6a5d2cc19a841e1b50208422742814f" -H > "User-Agent: python-novaclient" -H "Accept: application/json" -H > "X-Auth-Token: 5c94a3f98ee1464caf918db60f556646" > > RESP:{'status': '200', 'content-length': '322', 'content-location': > u'http://10.16.171.3:8774/v2/e6a5d2cc19a841e1b50208422742814f/os-quota-sets/e6a5d2cc19a841e1b50208422742814f', > 'x-compute-request-id': 'req-95c8b946-1e31-47bc-9e94-d913f9cf9ddc', > 'date': 'Thu, 18 Apr 2013 14:06:54 GMT', 'content-type': > 'application/json'} {"quota_set": {"injected_file_content_bytes": 10240, > "metadata_items": 128, "ram": 51200, "floating_ips": 10, "key_pairs": > 100, "id": "e6a5d2cc19a841e1b50208422742814f", "instances": 10, > "security_group_rules": 20, "injected_files": 5, "cores": 20, > "fixed_ips": -1, "injected_file_path_bytes": 255, "security_groups": 10}} > > cinderclient connection created using token > "5c94a3f98ee1464caf918db60f556646" and url > "http://10.16.171.3:8776/v1/e6a5d2cc19a841e1b50208422742814f" > > I seems the login has succeeded but the /admin URL doesn't seem to work. > Any ideas on what could be the cause of this? > > Regards, > Dennis > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list From dneary at redhat.com Thu Apr 18 21:25:43 2013 From: dneary at redhat.com (Dave Neary) Date: Thu, 18 Apr 2013 14:25:43 -0700 Subject: [Rdo-list] Dashboard error In-Reply-To: <51702509.5000306@conversis.de> References: <5170061C.1080204@conversis.de> <51702509.5000306@conversis.de> Message-ID: <517064D7.1090609@redhat.com> On 04/18/2013 09:53 AM, Dennis Jacobfeuerborn wrote: > Never mind. It turns out that while I updated the openstack-* packages > from epel to the rdo versions I forgot to do the same for the python > client packages. After that things work as expected. Thanks for letting us know! Cheers, Dave. -- Dave Neary - Community Action and Impact Open Source and Standards, Red Hat - http://community.redhat.com Ph: +33 9 50 71 55 62 / Cell: +33 6 77 01 92 13 From dneary at redhat.com Fri Apr 19 01:30:25 2013 From: dneary at redhat.com (Dave Neary) Date: Thu, 18 Apr 2013 18:30:25 -0700 Subject: [Rdo-list] Dashboard error In-Reply-To: <51702509.5000306@conversis.de> References: <5170061C.1080204@conversis.de> <51702509.5000306@conversis.de> Message-ID: <51709E31.2000801@redhat.com> On 04/18/2013 09:53 AM, Dennis Jacobfeuerborn wrote: > Never mind. It turns out that while I updated the openstack-* packages > from epel to the rdo versions I forgot to do the same for the python > client packages. After that things work as expected. Thanks for letting us know! Cheers, Dave. -- Dave Neary - Community Action and Impact Open Source and Standards, Red Hat - http://community.redhat.com Ph: +33 9 50 71 55 62 / Cell: +33 6 77 01 92 13 From thomas.oulevey at cern.ch Mon Apr 22 12:04:15 2013 From: thomas.oulevey at cern.ch (Thomas Oulevey) Date: Mon, 22 Apr 2013 14:04:15 +0200 Subject: [Rdo-list] Create el6-based glance image. Message-ID: <5175273F.5000801@cern.ch> Hello RDO users, I have little tool to create small el6 based Glance images, I like to share. (no more cirros ;) It is called: create-glance-image-el6.sh usage: $0 OPTIONS -u This script create a small(-ish) el6-based image for Glance and can import it. OPTIONS: -a make image public in Glance -e extras packages to install e.g: "ipmitool mypkg2 mypkg3" ; -g execute glance command instead of printing it ; -h print this help ; -n nameserver ; -p set root password / default: toor ; -s image size. (number of block of 1024) / default: 2097152 -u release rpm url or file ; I tested it with 3 different distributions: centos : sudo create-glance-image-el6.sh -u http://mirror.switch.ch/ftp/mirror/centos/6.4/os/x86_64/Packages/centos-release-6-4.el6.centos.10.x86_64.rpm mycentos64 scientific linux : sudo create-glance-image-el6.sh -u http://ftp.scientificlinux.org/linux/scientific/6.4/x86_64/os/Packages/sl-release-6.4-1.x86_64.rpm mysl64 slc : sudo create-glance-image-el6.sh -u http://linuxsoft.cern.ch/cern/slc6X/x86_64/Packages/sl-release-6.4-1.slc6.x86_64.rpm myslc64 Please use different mirrors. Please not you need to use -g to automate the Glance upload. Please test on development server : NO WARRANTIES Code & Download : https://github.com/alphacc/openstack-tools Feedback is welcome ! Thomas. From shake.chen at gmail.com Mon Apr 22 15:29:22 2013 From: shake.chen at gmail.com (Shake Chen) Date: Mon, 22 Apr 2013 23:29:22 +0800 Subject: [Rdo-list] Create el6-based glance image. In-Reply-To: <5175273F.5000801@cern.ch> References: <5175273F.5000801@cern.ch> Message-ID: Great tool. now I am testing and seem working and have no any problem. On Mon, Apr 22, 2013 at 8:04 PM, Thomas Oulevey wrote: > Hello RDO users, > > I have little tool to create small el6 based Glance images, I like to > share. (no more cirros ;) > > It is called: create-glance-image-el6.sh > > usage: $0 OPTIONS -u > This script create a small(-ish) el6-based image for Glance and can import > it. > OPTIONS: > -a make image public in Glance > -e extras packages to install e.g: "ipmitool mypkg2 mypkg3" ; > -g execute glance command instead of printing it ; > -h print this help ; > -n nameserver ; > -p set root password / default: toor ; > -s image size. (number of block of 1024) / default: 2097152 > -u release rpm url or file ; > > > I tested it with 3 different distributions: > > centos : sudo create-glance-image-el6.sh -u http://mirror.switch.ch/ftp/** > mirror/centos/6.4/os/x86_64/**Packages/centos-release-6-4.** > el6.centos.10.x86_64.rpmmycentos64 > > scientific linux : sudo create-glance-image-el6.sh -u > http://ftp.scientificlinux.**org/linux/scientific/6.4/x86_** > 64/os/Packages/sl-release-6.4-**1.x86_64.rpmmysl64 > > slc : sudo create-glance-image-el6.sh -u http://linuxsoft.cern.ch/cern/** > slc6X/x86_64/Packages/sl-**release-6.4-1.slc6.x86_64.rpmmyslc64 > > > Please use different mirrors. > Please not you need to use -g to automate the Glance upload. > Please test on development server : NO WARRANTIES > > Code & Download : https://github.com/alphacc/**openstack-tools > > Feedback is welcome ! > > Thomas. > > ______________________________**_________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/**mailman/listinfo/rdo-list > -- Shake Chen -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 06:38:18 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 06:38:18 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Message-ID: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> When I start VM instance, the VM can't get IP address. Could someone help me on this ? 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. - Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) - Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network - Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network - Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Thank you! Regards, Kimi -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 06:50:12 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 09:50:12 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> Message-ID: <517CC6A4.9040201@redhat.com> Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > converted from rtf > When I start VM instance, the VM can't get IP address. Could someone > help me on this ? I will try > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > * Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > Network: bond0(10.68.125.11 for O&M) > > * Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. > * Compute node: > > Services: nove-compute and quantum-openvswitch-agent > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > * Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) > of network and compute nodes. > > 1. Quantum.conf: > > [DEFAULT] > debug = True > verbose = True > lock_path = $state_path/lock > bind_host = 0.0.0.0 > bind_port = 9696 > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > api_paste_config = api-paste.ini > rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? > control_exchange = quantum > rabbit_host = 10.68.125.11 > notification_driver = quantum.openstack.common.notifier.rpc_notifier > default_notification_level = INFO > notification_topics = notifications > [QUOTAS] > [DEFAULT_SERVICETYPE] > [AGENT] > polling_interval = 2 > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > [keystone_authtoken] > auth_host = 10.68.125.11 > auth_port = 35357 > auth_protocol = http > signing_dir = /var/lib/quantum/keystone-signing > admin_tenant_name = service > admin_user = quantum > admin_password = password > > 2. ovs_quantum_plugin.ini > > [DATABASE] > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > reconnect_interval = 2 > [OVS] > tenant_network_type = vlan > network_vlan_ranges = physnet1:1000:2999 > bridge_mappings = physnet1:br-p3p1 > [AGENT] > polling_interval = 2 > [SECURITYGROUP] > > 3. nova.conf > > [DEFAULT] > verbose=true > logdir = /var/log/nova > state_path = /var/lib/nova > lock_path = /var/lib/nova/tmp > volumes_dir = /etc/nova/volumes > dhcpbridge = /usr/bin/nova-dhcpbridge > dhcpbridge_flagfile = /etc/nova/nova.conf > force_dhcp_release = True > injected_network_template = /usr/share/nova/interfaces.template > libvirt_nonblocking = True > libvirt_inject_partition = -1 > network_manager = nova.network.manager.FlatDHCPManager > iscsi_helper = tgtadm > compute_driver = libvirt.LibvirtDriver > libvirt_type=kvm > libvirt_ovs_bridge=br-int > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > manager=nova.conductor.manager.ConductorManager > rpc_backend = nova.openstack.common.rpc.impl_kombu > rabbit_host = 10.68.125.11 > rootwrap_config = /etc/nova/rootwrap.conf > use_deprecated_auth=false > auth_strategy=keystone > glance_api_servers=10.68.125.11:9292 > image_service=nova.image.glance.GlanceImageService > novnc_enabled=true > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > novncproxy_port=6080 > vncserver_proxyclient_address=10.68.125.16 > vncserver_listen=0.0.0.0 > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > libvirt_use_virtio_for_bridges=True > network_api_class=nova.network.quantumv2.api.API > quantum_url=http://10.68.125.11:9696 > quantum_auth_strategy=keystone > quantum_admin_tenant_name=service > quantum_admin_username=quantum > quantum_admin_password=password > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > libvirt_vif_type=ethernet > service_quantum_metadata_proxy = True > quantum_metadata_proxy_shared_secret = helloOpenStack > metadata_host = 10.68.125.11 > metadata_listen = 0.0.0.0 > metadata_listen_port = 8775 > [keystone_authtoken] > admin_tenant_name = service > admin_user = nova > admin_password = password > auth_host = 10.68.125.11 > auth_port = 35357 > auth_protocol = http > signing_dir = /tmp/keystone-signing-nova > > 4. ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > Bridge "br-p3p1" > Port "p3p1" > Interface "p3p1" > Port "phy-br-p3p1" > Interface "phy-br-p3p1" > Port "br-p3p1" > Interface "br-p3p1" > type: internal > Bridge br-ex > Port br-ex > Interface br-ex > type: internal > Port "qg-a83c0abd-f4" > Interface "qg-a83c0abd-f4" > type: internal > Port "p3p2" > Interface "p3p2" > Bridge br-int > Port br-int > Interface br-int > type: internal > Port "int-br-p3p1" > Interface "int-br-p3p1" > Port "tap1f386a2a-12" > tag: 1 > Interface "tap1f386a2a-12" > type: internal > ovs_version: "1.9.0" > > 5. ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > Bridge "br-p3p1" > Port "br-p3p1" > Interface "br-p3p1" > type: internal > Port "phy-br-p3p1" > Interface "phy-br-p3p1" > Port "p3p1" > Interface "p3p1" > Bridge br-int > Port "qvo56a4572c-dc" > tag: 2 > Interface "qvo56a4572c-dc" > Port "int-br-p3p1" > Interface "int-br-p3p1" > Port br-int > Interface br-int > type: internal > ovs_version: "1.9.0" > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? > Thank you! > Regards, > Kimi > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 06:54:27 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 06:54:27 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CC6A4.9040201@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> Hi, Gary Yes, I'm aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this... Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can't get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. * Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) * Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. * Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network * Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 06:58:50 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 06:58:50 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CC6A4.9040201@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC088046FA8@SGSIMBX001.nsn-intra.net> Sorry, the answers of your questions: Are you using rabbit or qpid? Rabbitmq Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. dhcp_agent.ini [DEFAULT] verbose = true interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver dhcp_driver = quantum.agent.linux.dhcp.Dnsmasq auth_url = http://10.68.125.11:35357/v2.0 auth_region = RegionOne admin_tenant_name = service admin_user = quantum admin_password = password use_namespaces = False [DEFAULT] debug = True verbose = True interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver use_namespaces = False router_id = bd532b18-a67c-4adf-abf4-872a5afa8358 auth_url = http://10.68.125.11:35357/v2.0 auth_region = RegionOne admin_tenant_name = service admin_user = quantum admin_password = password I understand RHEL6.4 does not support namespace, so yes, usage of them is set to False. Are there any log errors? No log errors actually... Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can't get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. * Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) * Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. * Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network * Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 07:01:08 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 10:01:08 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> Message-ID: <517CC934.4070809@redhat.com> On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > Yes, I'm aware of that packstack does not support quantum yet. The > whole setup was installed manually. > > I did run quantum-server-setup and quantum-host-setup, I tried > linuxbridge plugin too, it has no issue for VM to get IP address, but > openvswitch has issues on this... > ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list > Regards, > > Kimi > > *From:*rdo-list-bounces at redhat.com > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 2:50 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi Kimi, > Thanks for the mail. Please see the inline comments below. Please note > that at the moment we do not have packstack support for Quantum so > there is a little manual plumbing that needs to be done (not sure if > you have done this already). > On the host where the quantum service is running you need to run > quantum-server-setup and on the compute nodes you need to run > quantum-host-setup (please note that the relevant keystone credentials > need to be set too). > Thanks > Gary > > On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > converted from rtf > > When I start VM instance, the VM can't get IP address. Could someone > help me on this ? > > > I will try > > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > ?Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > > Network: bond0(10.68.125.11 for O&M) > > ?Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network > > > Please note that RHEL currently does not support namespaces so there > are a number of limitations. We are addressing this at the moment. If > namespaces are not used then it is suggested that one does not run the > DHCP agent and the L3 agent on the same host. The reason for this is > that there is no network isolation. > > > ?Compute node: > > Services: nove-compute and quantum-openvswitch-agent > > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of > network and compute nodes. > > 1.Quantum.conf: > > [DEFAULT] > > debug = True > > verbose = True > > lock_path = $state_path/lock > > bind_host = 0.0.0.0 > > bind_port = 9696 > > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > > api_paste_config = api-paste.ini > > rpc_backend = quantum.openstack.common.rpc.impl_kombu > > > Are you using rabbit or qpid? > > > control_exchange = quantum > > rabbit_host = 10.68.125.11 > > notification_driver = quantum.openstack.common.notifier.rpc_notifier > > default_notification_level = INFO > > notification_topics = notifications > > [QUOTAS] > > [DEFAULT_SERVICETYPE] > > [AGENT] > > polling_interval = 2 > > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > > [keystone_authtoken] > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /var/lib/quantum/keystone-signing > > admin_tenant_name = service > > admin_user = quantum > > admin_password = password > > 2.ovs_quantum_plugin.ini > > [DATABASE] > > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > > > reconnect_interval = 2 > > [OVS] > > tenant_network_type = vlan > > network_vlan_ranges = physnet1:1000:2999 > > bridge_mappings = physnet1:br-p3p1 > > [AGENT] > > polling_interval = 2 > > [SECURITYGROUP] > > 3.nova.conf > > [DEFAULT] > > verbose=true > > logdir = /var/log/nova > > state_path = /var/lib/nova > > lock_path = /var/lib/nova/tmp > > volumes_dir = /etc/nova/volumes > > dhcpbridge = /usr/bin/nova-dhcpbridge > > dhcpbridge_flagfile = /etc/nova/nova.conf > > force_dhcp_release = True > > injected_network_template = /usr/share/nova/interfaces.template > > libvirt_nonblocking = True > > libvirt_inject_partition = -1 > > network_manager = nova.network.manager.FlatDHCPManager > > iscsi_helper = tgtadm > > compute_driver = libvirt.LibvirtDriver > > libvirt_type=kvm > > libvirt_ovs_bridge=br-int > > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > > manager=nova.conductor.manager.ConductorManager > > rpc_backend = nova.openstack.common.rpc.impl_kombu > > rabbit_host = 10.68.125.11 > > rootwrap_config = /etc/nova/rootwrap.conf > > use_deprecated_auth=false > > auth_strategy=keystone > > glance_api_servers=10.68.125.11:9292 > > image_service=nova.image.glance.GlanceImageService > > novnc_enabled=true > > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > > novncproxy_port=6080 > > vncserver_proxyclient_address=10.68.125.16 > > vncserver_listen=0.0.0.0 > > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > > libvirt_use_virtio_for_bridges=True > > network_api_class=nova.network.quantumv2.api.API > > quantum_url=http://10.68.125.11:9696 > > quantum_auth_strategy=keystone > > quantum_admin_tenant_name=service > > quantum_admin_username=quantum > > quantum_admin_password=password > > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > > libvirt_vif_type=ethernet > > service_quantum_metadata_proxy = True > > quantum_metadata_proxy_shared_secret = helloOpenStack > > metadata_host = 10.68.125.11 > > metadata_listen = 0.0.0.0 > > metadata_listen_port = 8775 > > [keystone_authtoken] > > admin_tenant_name = service > > admin_user = nova > > admin_password = password > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /tmp/keystone-signing-nova > > 4.ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > > Bridge "br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Bridge br-ex > > Port br-ex > > Interface br-ex > > type: internal > > Port "qg-a83c0abd-f4" > > Interface "qg-a83c0abd-f4" > > type: internal > > Port "p3p2" > > Interface "p3p2" > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "tap1f386a2a-12" > > tag: 1 > > Interface "tap1f386a2a-12" > > type: internal > > ovs_version: "1.9.0" > > 5.ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > > Bridge "br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Bridge br-int > > Port "qvo56a4572c-dc" > > tag: 2 > > Interface "qvo56a4572c-dc" > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port br-int > > Interface br-int > > type: internal > > ovs_version: "1.9.0" > > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. > > > Any chance to get the DHCP and the L3 agent configuration files? > Please check that use_namespaces = False in both of these files. > > Are there any log errors? > > > Thank you! > > Regards, > > Kimi > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 07:04:10 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 07:04:10 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CC934.4070809@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> I tried that too, no lucky. >From tcpdump ,it seems br-int does not forward any packet to interfaces connect to br-p3p1, which connects to physical network... Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:01 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary Yes, I'm aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this... ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can't get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. * Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) * Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. * Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network * Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 07:07:58 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 10:07:58 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> Message-ID: <517CCACE.8000203@redhat.com> On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > I tried that too, no lucky. > > From tcpdump ,it seems br-int does not forward any packet to > interfaces connect to br-p3p1, which connects to physical network... > There could be a number of issues here: 1. The iptables are dropping the traffic (I am in the process of getting a setup up and running) 2. The network connectivity In order to ensure that it is not the first one can you try and see which iptables rules are matched or disable the iptables? > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:01 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > Yes, I'm aware of that packstack does not support quantum yet. The > whole setup was installed manually. > > I did run quantum-server-setup and quantum-host-setup, I tried > linuxbridge plugin too, it has no issue for VM to get IP address, but > openvswitch has issues on this... > > > ok. > > if you configure and IP address manually on the VM are you able to > ping the port of the DHCP agent? > > you can get the IP from quantum port-list > > > > Regards, > > Kimi > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 2:50 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi Kimi, > Thanks for the mail. Please see the inline comments below. Please note > that at the moment we do not have packstack support for Quantum so > there is a little manual plumbing that needs to be done (not sure if > you have done this already). > On the host where the quantum service is running you need to run > quantum-server-setup and on the compute nodes you need to run > quantum-host-setup (please note that the relevant keystone credentials > need to be set too). > Thanks > Gary > > On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > converted from rtf > > When I start VM instance, the VM can't get IP address. Could someone > help me on this ? > > > I will try > > > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > ?Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > > Network: bond0(10.68.125.11 for O&M) > > ?Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network > > > Please note that RHEL currently does not support namespaces so there > are a number of limitations. We are addressing this at the moment. If > namespaces are not used then it is suggested that one does not run the > DHCP agent and the L3 agent on the same host. The reason for this is > that there is no network isolation. > > > > ?Compute node: > > Services: nove-compute and quantum-openvswitch-agent > > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of > network and compute nodes. > > 1.Quantum.conf: > > [DEFAULT] > > debug = True > > verbose = True > > lock_path = $state_path/lock > > bind_host = 0.0.0.0 > > bind_port = 9696 > > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > > api_paste_config = api-paste.ini > > rpc_backend = quantum.openstack.common.rpc.impl_kombu > > > Are you using rabbit or qpid? > > > > control_exchange = quantum > > rabbit_host = 10.68.125.11 > > notification_driver = quantum.openstack.common.notifier.rpc_notifier > > default_notification_level = INFO > > notification_topics = notifications > > [QUOTAS] > > [DEFAULT_SERVICETYPE] > > [AGENT] > > polling_interval = 2 > > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > > [keystone_authtoken] > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /var/lib/quantum/keystone-signing > > admin_tenant_name = service > > admin_user = quantum > > admin_password = password > > 2.ovs_quantum_plugin.ini > > [DATABASE] > > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > > > reconnect_interval = 2 > > [OVS] > > tenant_network_type = vlan > > network_vlan_ranges = physnet1:1000:2999 > > bridge_mappings = physnet1:br-p3p1 > > [AGENT] > > polling_interval = 2 > > [SECURITYGROUP] > > 3.nova.conf > > [DEFAULT] > > verbose=true > > logdir = /var/log/nova > > state_path = /var/lib/nova > > lock_path = /var/lib/nova/tmp > > volumes_dir = /etc/nova/volumes > > dhcpbridge = /usr/bin/nova-dhcpbridge > > dhcpbridge_flagfile = /etc/nova/nova.conf > > force_dhcp_release = True > > injected_network_template = /usr/share/nova/interfaces.template > > libvirt_nonblocking = True > > libvirt_inject_partition = -1 > > network_manager = nova.network.manager.FlatDHCPManager > > iscsi_helper = tgtadm > > compute_driver = libvirt.LibvirtDriver > > libvirt_type=kvm > > libvirt_ovs_bridge=br-int > > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > > manager=nova.conductor.manager.ConductorManager > > rpc_backend = nova.openstack.common.rpc.impl_kombu > > rabbit_host = 10.68.125.11 > > rootwrap_config = /etc/nova/rootwrap.conf > > use_deprecated_auth=false > > auth_strategy=keystone > > glance_api_servers=10.68.125.11:9292 > > image_service=nova.image.glance.GlanceImageService > > novnc_enabled=true > > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > > novncproxy_port=6080 > > vncserver_proxyclient_address=10.68.125.16 > > vncserver_listen=0.0.0.0 > > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > > libvirt_use_virtio_for_bridges=True > > network_api_class=nova.network.quantumv2.api.API > > quantum_url=http://10.68.125.11:9696 > > quantum_auth_strategy=keystone > > quantum_admin_tenant_name=service > > quantum_admin_username=quantum > > quantum_admin_password=password > > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > > libvirt_vif_type=ethernet > > service_quantum_metadata_proxy = True > > quantum_metadata_proxy_shared_secret = helloOpenStack > > metadata_host = 10.68.125.11 > > metadata_listen = 0.0.0.0 > > metadata_listen_port = 8775 > > [keystone_authtoken] > > admin_tenant_name = service > > admin_user = nova > > admin_password = password > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /tmp/keystone-signing-nova > > 4.ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > > Bridge "br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Bridge br-ex > > Port br-ex > > Interface br-ex > > type: internal > > Port "qg-a83c0abd-f4" > > Interface "qg-a83c0abd-f4" > > type: internal > > Port "p3p2" > > Interface "p3p2" > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "tap1f386a2a-12" > > tag: 1 > > Interface "tap1f386a2a-12" > > type: internal > > ovs_version: "1.9.0" > > 5.ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > > Bridge "br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Bridge br-int > > Port "qvo56a4572c-dc" > > tag: 2 > > Interface "qvo56a4572c-dc" > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port br-int > > Interface br-int > > type: internal > > ovs_version: "1.9.0" > > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. > > > Any chance to get the DHCP and the L3 agent configuration files? > Please check that use_namespaces = False in both of these files. > > Are there any log errors? > > > > Thank you! > > Regards, > > Kimi > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 07:16:39 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 07:16:39 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CCACE.8000203@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> Hi, Gary I tried to disable iptables on both network and compute nodes, still does not work out :( >From quantum openvswitch agent logs, following messages keeps coming out repeatly every 2-3 seconds, not sure if they matter or not? 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making synchronous call on q-plugin ... 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:08 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: I tried that too, no lucky. >From tcpdump ,it seems br-int does not forward any packet to interfaces connect to br-p3p1, which connects to physical network... There could be a number of issues here: 1. The iptables are dropping the traffic (I am in the process of getting a setup up and running) 2. The network connectivity In order to ensure that it is not the first one can you try and see which iptables rules are matched or disable the iptables? Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:01 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary Yes, I'm aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this... ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can't get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. * Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) * Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. * Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network * Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 07:21:56 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 10:21:56 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> Message-ID: <517CCE14.2040105@redhat.com> On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > I tried to disable iptables on both network and compute nodes, still > does not work out L > Can you please look at https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing When using the OVS there are a number of devices. Would it be possible that you try and capture on each device so that we can try and see where the packet is discarded. I will have a setup ready in about an hour. > From quantum openvswitch agent logs, following messages keeps coming > out repeatly every 2-3 seconds, not sure if they matter or not? > The messages below are OK - this is how the OVS agent works. It polls the OVS every interval to check if new ports are created. > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] > Making synchronous call on q-plugin ... > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] > MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] > UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running > command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running > command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running > command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running > command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running > command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running > command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:08 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > I tried that too, no lucky. > > From tcpdump ,it seems br-int does not forward any packet to > interfaces connect to br-p3p1, which connects to physical network... > > > There could be a number of issues here: > 1. The iptables are dropping the traffic (I am in the process of > getting a setup up and running) > 2. The network connectivity > > In order to ensure that it is not the first one can you try and see > which iptables rules are matched or disable the iptables? > > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:01 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > Yes, I'm aware of that packstack does not support quantum yet. The > whole setup was installed manually. > > I did run quantum-server-setup and quantum-host-setup, I tried > linuxbridge plugin too, it has no issue for VM to get IP address, but > openvswitch has issues on this... > > > ok. > > if you configure and IP address manually on the VM are you able to > ping the port of the DHCP agent? > > you can get the IP from quantum port-list > > > > > Regards, > > Kimi > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 2:50 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi Kimi, > Thanks for the mail. Please see the inline comments below. Please note > that at the moment we do not have packstack support for Quantum so > there is a little manual plumbing that needs to be done (not sure if > you have done this already). > On the host where the quantum service is running you need to run > quantum-server-setup and on the compute nodes you need to run > quantum-host-setup (please note that the relevant keystone credentials > need to be set too). > Thanks > Gary > > On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > converted from rtf > > When I start VM instance, the VM can't get IP address. Could someone > help me on this ? > > > I will try > > > > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > ?Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > > Network: bond0(10.68.125.11 for O&M) > > ?Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network > > > Please note that RHEL currently does not support namespaces so there > are a number of limitations. We are addressing this at the moment. If > namespaces are not used then it is suggested that one does not run the > DHCP agent and the L3 agent on the same host. The reason for this is > that there is no network isolation. > > > > > ?Compute node: > > Services: nove-compute and quantum-openvswitch-agent > > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of > network and compute nodes. > > 1.Quantum.conf: > > [DEFAULT] > > debug = True > > verbose = True > > lock_path = $state_path/lock > > bind_host = 0.0.0.0 > > bind_port = 9696 > > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > > api_paste_config = api-paste.ini > > rpc_backend = quantum.openstack.common.rpc.impl_kombu > > > Are you using rabbit or qpid? > > > > > control_exchange = quantum > > rabbit_host = 10.68.125.11 > > notification_driver = quantum.openstack.common.notifier.rpc_notifier > > default_notification_level = INFO > > notification_topics = notifications > > [QUOTAS] > > [DEFAULT_SERVICETYPE] > > [AGENT] > > polling_interval = 2 > > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > > [keystone_authtoken] > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /var/lib/quantum/keystone-signing > > admin_tenant_name = service > > admin_user = quantum > > admin_password = password > > 2.ovs_quantum_plugin.ini > > [DATABASE] > > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > > > reconnect_interval = 2 > > [OVS] > > tenant_network_type = vlan > > network_vlan_ranges = physnet1:1000:2999 > > bridge_mappings = physnet1:br-p3p1 > > [AGENT] > > polling_interval = 2 > > [SECURITYGROUP] > > 3.nova.conf > > [DEFAULT] > > verbose=true > > logdir = /var/log/nova > > state_path = /var/lib/nova > > lock_path = /var/lib/nova/tmp > > volumes_dir = /etc/nova/volumes > > dhcpbridge = /usr/bin/nova-dhcpbridge > > dhcpbridge_flagfile = /etc/nova/nova.conf > > force_dhcp_release = True > > injected_network_template = /usr/share/nova/interfaces.template > > libvirt_nonblocking = True > > libvirt_inject_partition = -1 > > network_manager = nova.network.manager.FlatDHCPManager > > iscsi_helper = tgtadm > > compute_driver = libvirt.LibvirtDriver > > libvirt_type=kvm > > libvirt_ovs_bridge=br-int > > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > > manager=nova.conductor.manager.ConductorManager > > rpc_backend = nova.openstack.common.rpc.impl_kombu > > rabbit_host = 10.68.125.11 > > rootwrap_config = /etc/nova/rootwrap.conf > > use_deprecated_auth=false > > auth_strategy=keystone > > glance_api_servers=10.68.125.11:9292 > > image_service=nova.image.glance.GlanceImageService > > novnc_enabled=true > > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > > novncproxy_port=6080 > > vncserver_proxyclient_address=10.68.125.16 > > vncserver_listen=0.0.0.0 > > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > > libvirt_use_virtio_for_bridges=True > > network_api_class=nova.network.quantumv2.api.API > > quantum_url=http://10.68.125.11:9696 > > quantum_auth_strategy=keystone > > quantum_admin_tenant_name=service > > quantum_admin_username=quantum > > quantum_admin_password=password > > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > > libvirt_vif_type=ethernet > > service_quantum_metadata_proxy = True > > quantum_metadata_proxy_shared_secret = helloOpenStack > > metadata_host = 10.68.125.11 > > metadata_listen = 0.0.0.0 > > metadata_listen_port = 8775 > > [keystone_authtoken] > > admin_tenant_name = service > > admin_user = nova > > admin_password = password > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /tmp/keystone-signing-nova > > 4.ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > > Bridge "br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Bridge br-ex > > Port br-ex > > Interface br-ex > > type: internal > > Port "qg-a83c0abd-f4" > > Interface "qg-a83c0abd-f4" > > type: internal > > Port "p3p2" > > Interface "p3p2" > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "tap1f386a2a-12" > > tag: 1 > > Interface "tap1f386a2a-12" > > type: internal > > ovs_version: "1.9.0" > > 5.ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > > Bridge "br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Bridge br-int > > Port "qvo56a4572c-dc" > > tag: 2 > > Interface "qvo56a4572c-dc" > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port br-int > > Interface br-int > > type: internal > > ovs_version: "1.9.0" > > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. > > > Any chance to get the DHCP and the L3 agent configuration files? > Please check that use_namespaces = False in both of these files. > > Are there any log errors? > > > > > Thank you! > > Regards, > > Kimi > > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 07:32:44 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 10:32:44 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CCE14.2040105@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> Message-ID: <517CD09C.9080609@redhat.com> Hi, Can you also please check that firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is configured in plugin.ini file.And security_group_api = quantum is set in nova.conf Thanks Gary On 04/28/2013 10:21 AM, Gary Kotton wrote: > On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi, Gary >> >> I tried to disable iptables on both network and compute nodes, still >> does not work out L >> > > Can you please look at > https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing > > When using the OVS there are a number of devices. Would it be possible > that you try and capture on each device so that we can try and see > where the packet is discarded. > > I will have a setup ready in about an hour. > >> From quantum openvswitch agent logs, following messages keeps coming >> out repeatly every 2-3 seconds, not sure if they matter or not? >> > > The messages below are OK - this is how the OVS agent works. It polls > the OVS every interval to check if new ports are created. > >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> Making synchronous call on q-plugin ... >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >> *Sent:* Sunday, April 28, 2013 3:08 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> I tried that too, no lucky. >> >> From tcpdump ,it seems br-int does not forward any packet to >> interfaces connect to br-p3p1, which connects to physical network... >> >> >> There could be a number of issues here: >> 1. The iptables are dropping the traffic (I am in the process of >> getting a setup up and running) >> 2. The network connectivity >> >> In order to ensure that it is not the first one can you try and see >> which iptables rules are matched or disable the iptables? >> >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >> *Sent:* Sunday, April 28, 2013 3:01 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi, Gary >> >> Yes, I'm aware of that packstack does not support quantum yet. The >> whole setup was installed manually. >> >> I did run quantum-server-setup and quantum-host-setup, I tried >> linuxbridge plugin too, it has no issue for VM to get IP address, but >> openvswitch has issues on this... >> >> >> ok. >> >> if you configure and IP address manually on the VM are you able to >> ping the port of the DHCP agent? >> >> you can get the IP from quantum port-list >> >> >> >> >> Regards, >> >> Kimi >> >> *From:*rdo-list-bounces at redhat.com >> >> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton >> *Sent:* Sunday, April 28, 2013 2:50 PM >> *To:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi Kimi, >> Thanks for the mail. Please see the inline comments below. Please >> note that at the moment we do not have packstack support for Quantum >> so there is a little manual plumbing that needs to be done (not sure >> if you have done this already). >> On the host where the quantum service is running you need to run >> quantum-server-setup and on the compute nodes you need to run >> quantum-host-setup (please note that the relevant keystone >> credentials need to be set too). >> Thanks >> Gary >> >> On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> converted from rtf >> >> When I start VM instance, the VM can't get IP address. Could someone >> help me on this ? >> >> >> I will try >> >> >> >> 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. >> >> ?Controller node: >> >> Services: Keystone+Glance+Cinder+Quantum server + Nova services >> >> Network: bond0(10.68.125.11 for O&M) >> >> ?Network node: >> >> Services: quantum-openvswitch-agent, quantum-l3-agent, >> quantum-dhcp-agent, quantum-metadata-agent >> >> Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, >> p3p2 for external network >> >> >> Please note that RHEL currently does not support namespaces so there >> are a number of limitations. We are addressing this at the moment. If >> namespaces are not used then it is suggested that one does not run >> the DHCP agent and the L3 agent on the same host. The reason for this >> is that there is no network isolation. >> >> >> >> >> ?Compute node: >> >> Services: nove-compute and quantum-openvswitch-agent >> >> Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network >> >> ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) >> of network and compute nodes. >> >> 1.Quantum.conf: >> >> [DEFAULT] >> >> debug = True >> >> verbose = True >> >> lock_path = $state_path/lock >> >> bind_host = 0.0.0.0 >> >> bind_port = 9696 >> >> core_plugin = >> quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 >> >> api_paste_config = api-paste.ini >> >> rpc_backend = quantum.openstack.common.rpc.impl_kombu >> >> >> Are you using rabbit or qpid? >> >> >> >> >> control_exchange = quantum >> >> rabbit_host = 10.68.125.11 >> >> notification_driver = quantum.openstack.common.notifier.rpc_notifier >> >> default_notification_level = INFO >> >> notification_topics = notifications >> >> [QUOTAS] >> >> [DEFAULT_SERVICETYPE] >> >> [AGENT] >> >> polling_interval = 2 >> >> root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf >> >> [keystone_authtoken] >> >> auth_host = 10.68.125.11 >> >> auth_port = 35357 >> >> auth_protocol = http >> >> signing_dir = /var/lib/quantum/keystone-signing >> >> admin_tenant_name = service >> >> admin_user = quantum >> >> admin_password = password >> >> 2.ovs_quantum_plugin.ini >> >> [DATABASE] >> >> sql_connection = >> mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum >> >> >> reconnect_interval = 2 >> >> [OVS] >> >> tenant_network_type = vlan >> >> network_vlan_ranges = physnet1:1000:2999 >> >> bridge_mappings = physnet1:br-p3p1 >> >> [AGENT] >> >> polling_interval = 2 >> >> [SECURITYGROUP] >> >> 3.nova.conf >> >> [DEFAULT] >> >> verbose=true >> >> logdir = /var/log/nova >> >> state_path = /var/lib/nova >> >> lock_path = /var/lib/nova/tmp >> >> volumes_dir = /etc/nova/volumes >> >> dhcpbridge = /usr/bin/nova-dhcpbridge >> >> dhcpbridge_flagfile = /etc/nova/nova.conf >> >> force_dhcp_release = True >> >> injected_network_template = /usr/share/nova/interfaces.template >> >> libvirt_nonblocking = True >> >> libvirt_inject_partition = -1 >> >> network_manager = nova.network.manager.FlatDHCPManager >> >> iscsi_helper = tgtadm >> >> compute_driver = libvirt.LibvirtDriver >> >> libvirt_type=kvm >> >> libvirt_ovs_bridge=br-int >> >> firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver >> >> manager=nova.conductor.manager.ConductorManager >> >> rpc_backend = nova.openstack.common.rpc.impl_kombu >> >> rabbit_host = 10.68.125.11 >> >> rootwrap_config = /etc/nova/rootwrap.conf >> >> use_deprecated_auth=false >> >> auth_strategy=keystone >> >> glance_api_servers=10.68.125.11:9292 >> >> image_service=nova.image.glance.GlanceImageService >> >> novnc_enabled=true >> >> novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html >> >> novncproxy_port=6080 >> >> vncserver_proxyclient_address=10.68.125.16 >> >> vncserver_listen=0.0.0.0 >> >> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver >> >> libvirt_use_virtio_for_bridges=True >> >> network_api_class=nova.network.quantumv2.api.API >> >> quantum_url=http://10.68.125.11:9696 >> >> quantum_auth_strategy=keystone >> >> quantum_admin_tenant_name=service >> >> quantum_admin_username=quantum >> >> quantum_admin_password=password >> >> quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 >> >> linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver >> >> libvirt_vif_type=ethernet >> >> service_quantum_metadata_proxy = True >> >> quantum_metadata_proxy_shared_secret = helloOpenStack >> >> metadata_host = 10.68.125.11 >> >> metadata_listen = 0.0.0.0 >> >> metadata_listen_port = 8775 >> >> [keystone_authtoken] >> >> admin_tenant_name = service >> >> admin_user = nova >> >> admin_password = password >> >> auth_host = 10.68.125.11 >> >> auth_port = 35357 >> >> auth_protocol = http >> >> signing_dir = /tmp/keystone-signing-nova >> >> 4.ovs-vsctl show on network node: >> >> aeeb6cf7-271b-405a-aa17-1b95bcd9e301 >> >> Bridge "br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> Bridge br-ex >> >> Port br-ex >> >> Interface br-ex >> >> type: internal >> >> Port "qg-a83c0abd-f4" >> >> Interface "qg-a83c0abd-f4" >> >> type: internal >> >> Port "p3p2" >> >> Interface "p3p2" >> >> Bridge br-int >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port "tap1f386a2a-12" >> >> tag: 1 >> >> Interface "tap1f386a2a-12" >> >> type: internal >> >> ovs_version: "1.9.0" >> >> 5.ovs-vsctl show on compute node: >> >> 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 >> >> Bridge "br-p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Bridge br-int >> >> Port "qvo56a4572c-dc" >> >> tag: 2 >> >> Interface "qvo56a4572c-dc" >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> ovs_version: "1.9.0" >> >> On compute node, I can see dhcp request packet from tcpdump on >> qvo56a4572c-dc, but it seems the packet is not forwarded out since I >> can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. >> >> >> Any chance to get the DHCP and the L3 agent configuration files? >> Please check that use_namespaces = False in both of these files. >> >> Are there any log errors? >> >> >> >> >> Thank you! >> >> Regards, >> >> Kimi >> >> >> >> >> >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rdo-list >> > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 07:40:06 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 07:40:06 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CD09C.9080609@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> Very nice pic, I am going to try to capture packet on each port. I did not configure to use quantum to manage firewall , just leave it to nova-compute, will try your configs later. Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 3:33 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi, Can you also please check that firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is configured in plugin.ini file.And security_group_api = quantum is set in nova.conf Thanks Gary On 04/28/2013 10:21 AM, Gary Kotton wrote: On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary I tried to disable iptables on both network and compute nodes, still does not work out :( Can you please look at https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing When using the OVS there are a number of devices. Would it be possible that you try and capture on each device so that we can try and see where the packet is discarded. I will have a setup ready in about an hour. >From quantum openvswitch agent logs, following messages keeps coming out repeatly every 2-3 seconds, not sure if they matter or not? The messages below are OK - this is how the OVS agent works. It polls the OVS every interval to check if new ports are created. 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making synchronous call on q-plugin ... 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:08 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: I tried that too, no lucky. >From tcpdump ,it seems br-int does not forward any packet to interfaces connect to br-p3p1, which connects to physical network... There could be a number of issues here: 1. The iptables are dropping the traffic (I am in the process of getting a setup up and running) 2. The network connectivity In order to ensure that it is not the first one can you try and see which iptables rules are matched or disable the iptables? Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:01 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary Yes, I'm aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this... ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can't get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. * Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) * Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. * Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network * Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 08:17:00 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 08:17:00 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> Message-ID: <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> Hi? Gary I tried capture packet while keeping VM to restart it?s network. I can see dhcp request broadcast packet on tap, qbr, qvb and qvo interfaces. Failed to see packet on int-br-p3p1 on bridge br-int. Not sure if it has something to do with openflow setting? I attach some ovs-ofctl outputs I have not seen ?veth? port anywhere? ---Record--- [root at computer-2 ~]# brctl show bridge name bridge id STP enabled interfaces qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec tap39242f22-ec virbr0 8000.525400c47f62 yes virbr0-nic [root at computer-2 ~]# ovs-vsctl show 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "qvo39242f22-ec" tag: 1 Interface "qvo39242f22-ec" Bridge "br-p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal ovs_version: "1.9.0" [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 1 packets captured 1 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 1 packets captured 1 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel ---output of ovs-ofctl--- [root at computer-2 ~]# ovs-ofctl show br-int OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 n_tables:255, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max LOCAL(br-int): addr:86:40:18:20:f1:42 config: PORT_DOWN state: LINK_DOWN speed: 100 Mbps now, 100 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 [root at computer-2 ~]# [root at computer-2 ~]# ovs-ofctl show br-p3p1 OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 n_tables:255, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 1(p3p1): addr:a0:36:9f:15:d4:24 config: 0 state: 0 current: 10GB-FD advertised: 10GB-FD FIBER supported: 10GB-FD FIBER speed: 10000 Mbps now, 10000 Mbps max 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 config: PORT_DOWN state: LINK_DOWN speed: 100 Mbps now, 100 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 [root at computer-2 ~]# ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4125.444s, table=0, n_packets=1707, n_bytes=90606, idle_age=12, priority=1 actions=NORMAL cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, idle_age=3349, priority=3,in_port=1,dl_vlan=1001 actions=mod_vlan_vid:1,NORMAL [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4129.629s, table=0, n_packets=2175, n_bytes=138652, idle_age=0, priority=1 actions=NORMAL cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, idle_age=1045, priority=2,in_port=2 actions=drop cookie=0x0, duration=3354.578s, table=0, n_packets=1697, n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:1001,NORMAL Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Zhang, Kimi (NSN - CN/Cheng Du) Sent: Sunday, April 28, 2013 3:40 PM To: gkotton at redhat.com; rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Very nice pic, I am going to try to capture packet on each port. I did not configure to use quantum to manage firewall , just leave it to nova-compute, will try your configs later. Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 3:33 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi, Can you also please check that firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is configured in plugin.ini file.And security_group_api = quantum is set in nova.conf Thanks Gary On 04/28/2013 10:21 AM, Gary Kotton wrote: On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary I tried to disable iptables on both network and compute nodes, still does not work out :( Can you please look at https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing When using the OVS there are a number of devices. Would it be possible that you try and capture on each device so that we can try and see where the packet is discarded. I will have a setup ready in about an hour. From quantum openvswitch agent logs, following messages keeps coming out repeatly every 2-3 seconds, not sure if they matter or not? The messages below are OK - this is how the OVS agent works. It polls the OVS every interval to check if new ports are created. 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making synchronous call on q-plugin ... 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:08 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: I tried that too, no lucky. From tcpdump ,it seems br-int does not forward any packet to interfaces connect to br-p3p1, which connects to physical network? There could be a number of issues here: 1. The iptables are dropping the traffic (I am in the process of getting a setup up and running) 2. The network connectivity In order to ensure that it is not the first one can you try and see which iptables rules are matched or disable the iptables? Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:01 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary Yes, I?m aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this? ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can?t get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. ? Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) ? Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. ? Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network ? Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can?t see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 08:30:44 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 11:30:44 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> Message-ID: <517CDE34.1080506@redhat.com> Hi, I have a few questions (please be patient with me): 1. On the compute node, which services are running? 2. Can you please print the iptables on the compute node? 3. Can you please print the flow table rules (ovs-dpctl dump-flows br-int)? Thanks Gary On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi?Gary > > I tried capture packet while keeping VM to restart it?s network. > > I can see dhcp request broadcast packet on tap, qbr, qvb and qvo > interfaces. > > Failed to see packet on int-br-p3p1 on bridge br-int. > > Not sure if it has something to do with openflow setting? I attach > some ovs-ofctl outputs > > I have not seen ?veth? port anywhere? > > ---Record--- > > [root at computer-2 ~]# brctl show > > bridge name bridge id STP enabled interfaces > > qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec > > tap39242f22-ec > > virbr0 8000.525400c47f62 yes virbr0-nic > > [root at computer-2 ~]# ovs-vsctl show > > 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "qvo39242f22-ec" > > tag: 1 > > Interface "qvo39242f22-ec" > > Bridge "br-p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > ovs_version: "1.9.0" > > [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 > > tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 2 packets captured > > 2 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 > > tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 1 packets captured > > 1 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 > > tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 1 packets captured > > 1 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 > > tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 2 packets captured > > 2 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 > > tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size > 65535 bytes > > ^C > > 0 packets captured > > 0 packets received by filter > > 0 packets dropped by kernel > > ---output of ovs-ofctl--- > > [root at computer-2 ~]# ovs-ofctl show br-int > > OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 > > n_tables:255, n_buffers:256 > > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC > SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > > 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > LOCAL(br-int): addr:86:40:18:20:f1:42 > > config: PORT_DOWN > > state: LINK_DOWN > > speed: 100 Mbps now, 100 Mbps max > > OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 > > [root at computer-2 ~]# > > [root at computer-2 ~]# ovs-ofctl show br-p3p1 > > OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 > > n_tables:255, n_buffers:256 > > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC > SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > > 1(p3p1): addr:a0:36:9f:15:d4:24 > > config: 0 > > state: 0 > > current: 10GB-FD > > advertised: 10GB-FD FIBER > > supported: 10GB-FD FIBER > > speed: 10000 Mbps now, 10000 Mbps max > > 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 > > config: PORT_DOWN > > state: LINK_DOWN > > speed: 100 Mbps now, 100 Mbps max > > OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 > > [root at computer-2 ~]# ovs-ofctl dump-flows br-int > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4125.444s, table=0, n_packets=1707, > n_bytes=90606, idle_age=12, priority=1 actions=NORMAL > > cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, > idle_age=20, priority=2,in_port=1 actions=drop > > cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, > idle_age=3349, priority=3,in_port=1,dl_vlan=1001 > actions=mod_vlan_vid:1,NORMAL > > [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4129.629s, table=0, n_packets=2175, > n_bytes=138652, idle_age=0, priority=1 actions=NORMAL > > cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, > idle_age=1045, priority=2,in_port=2 actions=drop > > cookie=0x0, duration=3354.578s, table=0, n_packets=1697, > n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 > actions=mod_vlan_vid:1001,NORMAL > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*rdo-list-bounces at redhat.com > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Zhang, Kimi > (NSN - CN/Cheng Du) > *Sent:* Sunday, April 28, 2013 3:40 PM > *To:* gkotton at redhat.com; rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Very nice pic, I am going to try to capture packet on each port. > > I did not configure to use quantum to manage firewall , just leave it > to nova-compute, will try your configs later. > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 3:33 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi, > Can you also please check that firewall_driver = > quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver > is configured in plugin.ini file.And security_group_api = quantum is > set in nova.conf > Thanks > Gary > > On 04/28/2013 10:21 AM, Gary Kotton wrote: > > On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > I tried to disable iptables on both network and compute nodes, still > does not work out L > > > Can you please look at > https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing > > When using the OVS there are a number of devices. Would it be possible > that you try and capture on each device so that we can try and see > where the packet is discarded. > > I will have a setup ready in about an hour. > > From quantum openvswitch agent logs, following messages keeps coming > out repeatly every 2-3 seconds, not sure if they matter or not? > > > The messages below are OK - this is how the OVS agent works. It polls > the OVS every interval to check if new ports are created. > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making > synchronous call on q-plugin ... > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID > is 92f4e83cf92c46f1b9304c879f9b7a41 > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] > UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:08 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > I tried that too, no lucky. > > From tcpdump ,it seems br-int does not forward any packet to > interfaces connect to br-p3p1, which connects to physical network? > > > There could be a number of issues here: > 1. The iptables are dropping the traffic (I am in the process of > getting a setup up and running) > 2. The network connectivity > > In order to ensure that it is not the first one can you try and see > which iptables rules are matched or disable the iptables? > > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:01 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > Yes, I?m aware of that packstack does not support quantum yet. The > whole setup was installed manually. > > I did run quantum-server-setup and quantum-host-setup, I tried > linuxbridge plugin too, it has no issue for VM to get IP address, but > openvswitch has issues on this? > > > ok. > > if you configure and IP address manually on the VM are you able to > ping the port of the DHCP agent? > > you can get the IP from quantum port-list > > > > > Regards, > > Kimi > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 2:50 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi Kimi, > Thanks for the mail. Please see the inline comments below. Please note > that at the moment we do not have packstack support for Quantum so > there is a little manual plumbing that needs to be done (not sure if > you have done this already). > On the host where the quantum service is running you need to run > quantum-server-setup and on the compute nodes you need to run > quantum-host-setup (please note that the relevant keystone credentials > need to be set too). > Thanks > Gary > > On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > converted from rtf > > When I start VM instance, the VM can?t get IP address. Could someone > help me on this ? > > > I will try > > > > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > ?Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > > Network: bond0(10.68.125.11 for O&M) > > ?Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network > > > Please note that RHEL currently does not support namespaces so there > are a number of limitations. We are addressing this at the moment. If > namespaces are not used then it is suggested that one does not run the > DHCP agent and the L3 agent on the same host. The reason for this is > that there is no network isolation. > > > > > ?Compute node: > > Services: nove-compute and quantum-openvswitch-agent > > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of > network and compute nodes. > > 1.Quantum.conf: > > [DEFAULT] > > debug = True > > verbose = True > > lock_path = $state_path/lock > > bind_host = 0.0.0.0 > > bind_port = 9696 > > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > > api_paste_config = api-paste.ini > > rpc_backend = quantum.openstack.common.rpc.impl_kombu > > > Are you using rabbit or qpid? > > > > > control_exchange = quantum > > rabbit_host = 10.68.125.11 > > notification_driver = quantum.openstack.common.notifier.rpc_notifier > > default_notification_level = INFO > > notification_topics = notifications > > [QUOTAS] > > [DEFAULT_SERVICETYPE] > > [AGENT] > > polling_interval = 2 > > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > > [keystone_authtoken] > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /var/lib/quantum/keystone-signing > > admin_tenant_name = service > > admin_user = quantum > > admin_password = password > > 2.ovs_quantum_plugin.ini > > [DATABASE] > > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > > > reconnect_interval = 2 > > [OVS] > > tenant_network_type = vlan > > network_vlan_ranges = physnet1:1000:2999 > > bridge_mappings = physnet1:br-p3p1 > > [AGENT] > > polling_interval = 2 > > [SECURITYGROUP] > > 3.nova.conf > > [DEFAULT] > > verbose=true > > logdir = /var/log/nova > > state_path = /var/lib/nova > > lock_path = /var/lib/nova/tmp > > volumes_dir = /etc/nova/volumes > > dhcpbridge = /usr/bin/nova-dhcpbridge > > dhcpbridge_flagfile = /etc/nova/nova.conf > > force_dhcp_release = True > > injected_network_template = /usr/share/nova/interfaces.template > > libvirt_nonblocking = True > > libvirt_inject_partition = -1 > > network_manager = nova.network.manager.FlatDHCPManager > > iscsi_helper = tgtadm > > compute_driver = libvirt.LibvirtDriver > > libvirt_type=kvm > > libvirt_ovs_bridge=br-int > > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > > manager=nova.conductor.manager.ConductorManager > > rpc_backend = nova.openstack.common.rpc.impl_kombu > > rabbit_host = 10.68.125.11 > > rootwrap_config = /etc/nova/rootwrap.conf > > use_deprecated_auth=false > > auth_strategy=keystone > > glance_api_servers=10.68.125.11:9292 > > image_service=nova.image.glance.GlanceImageService > > novnc_enabled=true > > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > > novncproxy_port=6080 > > vncserver_proxyclient_address=10.68.125.16 > > vncserver_listen=0.0.0.0 > > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > > libvirt_use_virtio_for_bridges=True > > network_api_class=nova.network.quantumv2.api.API > > quantum_url=http://10.68.125.11:9696 > > quantum_auth_strategy=keystone > > quantum_admin_tenant_name=service > > quantum_admin_username=quantum > > quantum_admin_password=password > > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > > libvirt_vif_type=ethernet > > service_quantum_metadata_proxy = True > > quantum_metadata_proxy_shared_secret = helloOpenStack > > metadata_host = 10.68.125.11 > > metadata_listen = 0.0.0.0 > > metadata_listen_port = 8775 > > [keystone_authtoken] > > admin_tenant_name = service > > admin_user = nova > > admin_password = password > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /tmp/keystone-signing-nova > > 4.ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > > Bridge "br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Bridge br-ex > > Port br-ex > > Interface br-ex > > type: internal > > Port "qg-a83c0abd-f4" > > Interface "qg-a83c0abd-f4" > > type: internal > > Port "p3p2" > > Interface "p3p2" > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "tap1f386a2a-12" > > tag: 1 > > Interface "tap1f386a2a-12" > > type: internal > > ovs_version: "1.9.0" > > 5.ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > > Bridge "br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Bridge br-int > > Port "qvo56a4572c-dc" > > tag: 2 > > Interface "qvo56a4572c-dc" > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port br-int > > Interface br-int > > type: internal > > ovs_version: "1.9.0" > > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can?t see packet from int-br-p3p1 on br-int or any port from br-p3p1. > > > Any chance to get the DHCP and the L3 agent configuration files? > Please check that use_namespaces = False in both of these files. > > Are there any log errors? > > > > > Thank you! > > Regards, > > Kimi > > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 08:41:59 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 08:41:59 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CDE34.1080506@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> <517CDE34.1080506@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> Sure, my answers below. :) Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 4:31 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi, I have a few questions (please be patient with me): 1. On the compute node, which services are running? nova-compute, nova-novncproxy, quantum-openvswitch-agent, openvswitch 2. Can you please print the iptables on the compute node? I disabled it already, here's output before I do it. [root at computer-2 ~]# iptables-save # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 *filter :INPUT ACCEPT [22634:3487580] :FORWARD ACCEPT [22:704] :OUTPUT ACCEPT [22619:5860198] :nova-compute-FORWARD - [0:0] :nova-compute-INPUT - [0:0] :nova-compute-OUTPUT - [0:0] :nova-compute-inst-26 - [0:0] :nova-compute-local - [0:0] :nova-compute-provider - [0:0] :nova-compute-sg-fallback - [0:0] :nova-filter-top - [0:0] -A INPUT -j nova-compute-INPUT -A FORWARD -j nova-filter-top -A FORWARD -j nova-compute-FORWARD -A OUTPUT -j nova-filter-top -A OUTPUT -j nova-compute-OUTPUT -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A nova-compute-inst-26 -m state --state INVALID -j DROP -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT -A nova-compute-inst-26 -j nova-compute-provider -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT -A nova-compute-inst-26 -p icmp -j ACCEPT -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT -A nova-compute-inst-26 -j nova-compute-sg-fallback -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 -A nova-compute-sg-fallback -j DROP -A nova-filter-top -j nova-compute-local COMMIT # Completed on Sun Apr 28 16:37:18 2013 # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 *mangle :PREROUTING ACCEPT [22733:3519752] :INPUT ACCEPT [22733:3519752] :FORWARD ACCEPT [175:50468] :OUTPUT ACCEPT [22705:5868566] :POSTROUTING ACCEPT [22880:5919034] :nova-compute-POSTROUTING - [0:0] -A POSTROUTING -j nova-compute-POSTROUTING COMMIT # Completed on Sun Apr 28 16:37:18 2013 # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 *nat :PREROUTING ACCEPT [16:14570] :POSTROUTING ACCEPT [338:22855] :OUTPUT ACCEPT [331:20579] :nova-compute-OUTPUT - [0:0] :nova-compute-POSTROUTING - [0:0] :nova-compute-PREROUTING - [0:0] :nova-compute-float-snat - [0:0] :nova-compute-snat - [0:0] :nova-postrouting-bottom - [0:0] -A PREROUTING -j nova-compute-PREROUTING -A POSTROUTING -j nova-compute-POSTROUTING -A POSTROUTING -j nova-postrouting-bottom -A OUTPUT -j nova-compute-OUTPUT -A nova-compute-snat -j nova-compute-float-snat -A nova-postrouting-bottom -j nova-compute-snat COMMIT # Completed on Sun Apr 28 16:37:18 2013 3. Can you please print the flow table rules (ovs-dpctl dump-flows br-int)? I suppose you mean ovs-ofctl dump-flows br-int ? [root at computer-2 ~]# ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4125.444s, table=0, n_packets=1707, n_bytes=90606, idle_age=12, priority=1 actions=NORMAL cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, idle_age=3349, priority=3,in_port=1,dl_vlan=1001 actions=mod_vlan_vid:1,NORMAL Here?s also ovs-dpctl show: [root at computer-2 ~]# ovs-dpctl show system at br-p3p1: lookups: hit:3967 missed:314 lost:0 flows: 1 port 0: br-p3p1 (internal) port 1: p3p1 port 2: phy-br-p3p1 system at br-int: lookups: hit:1575 missed:302 lost:0 flows: 0 port 0: br-int (internal) port 1: int-br-p3p1 port 4: qvo39242f22-ec Thanks Gary On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi? Gary I tried capture packet while keeping VM to restart it?s network. I can see dhcp request broadcast packet on tap, qbr, qvb and qvo interfaces. Failed to see packet on int-br-p3p1 on bridge br-int. Not sure if it has something to do with openflow setting? I attach some ovs-ofctl outputs I have not seen ?veth? port anywhere? ---Record--- [root at computer-2 ~]# brctl show bridge name bridge id STP enabled interfaces qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec tap39242f22-ec virbr0 8000.525400c47f62 yes virbr0-nic [root at computer-2 ~]# ovs-vsctl show 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "qvo39242f22-ec" tag: 1 Interface "qvo39242f22-ec" Bridge "br-p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal ovs_version: "1.9.0" [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 1 packets captured 1 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 1 packets captured 1 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel ---output of ovs-ofctl--- [root at computer-2 ~]# ovs-ofctl show br-int OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 n_tables:255, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max LOCAL(br-int): addr:86:40:18:20:f1:42 config: PORT_DOWN state: LINK_DOWN speed: 100 Mbps now, 100 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 [root at computer-2 ~]# [root at computer-2 ~]# ovs-ofctl show br-p3p1 OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 n_tables:255, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 1(p3p1): addr:a0:36:9f:15:d4:24 config: 0 state: 0 current: 10GB-FD advertised: 10GB-FD FIBER supported: 10GB-FD FIBER speed: 10000 Mbps now, 10000 Mbps max 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 config: PORT_DOWN state: LINK_DOWN speed: 100 Mbps now, 100 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 [root at computer-2 ~]# ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4125.444s, table=0, n_packets=1707, n_bytes=90606, idle_age=12, priority=1 actions=NORMAL cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, idle_age=3349, priority=3,in_port=1,dl_vlan=1001 actions=mod_vlan_vid:1,NORMAL [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4129.629s, table=0, n_packets=2175, n_bytes=138652, idle_age=0, priority=1 actions=NORMAL cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, idle_age=1045, priority=2,in_port=2 actions=drop cookie=0x0, duration=3354.578s, table=0, n_packets=1697, n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:1001,NORMAL Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Zhang, Kimi (NSN - CN/Cheng Du) Sent: Sunday, April 28, 2013 3:40 PM To: gkotton at redhat.com; rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Very nice pic, I am going to try to capture packet on each port. I did not configure to use quantum to manage firewall , just leave it to nova-compute, will try your configs later. Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 3:33 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi, Can you also please check that firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is configured in plugin.ini file.And security_group_api = quantum is set in nova.conf Thanks Gary On 04/28/2013 10:21 AM, Gary Kotton wrote: On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary I tried to disable iptables on both network and compute nodes, still does not work out :( Can you please look at https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing When using the OVS there are a number of devices. Would it be possible that you try and capture on each device so that we can try and see where the packet is discarded. I will have a setup ready in about an hour. From quantum openvswitch agent logs, following messages keeps coming out repeatly every 2-3 seconds, not sure if they matter or not? The messages below are OK - this is how the OVS agent works. It polls the OVS every interval to check if new ports are created. 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making synchronous call on q-plugin ... 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:08 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: I tried that too, no lucky. From tcpdump ,it seems br-int does not forward any packet to interfaces connect to br-p3p1, which connects to physical network? There could be a number of issues here: 1. The iptables are dropping the traffic (I am in the process of getting a setup up and running) 2. The network connectivity In order to ensure that it is not the first one can you try and see which iptables rules are matched or disable the iptables? Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:01 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary Yes, I?m aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this? ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can?t get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. ? Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) ? Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. ? Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network ? Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can?t see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 08:47:14 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 11:47:14 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> <517CDE34.1080506@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> Message-ID: <517CE212.2030000@redhat.com> Thanks. One more question - on the network node, did you run quantum-dhcp-setup? I am nearly ready with my setup. Hopefully I'll have a reproduction or some additional questions. Thanks Gary On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Sure, my answers below. :) > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 4:31 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi, > I have a few questions (please be patient with me): > 1. On the compute node, which services are running? > > nova-compute, nova-novncproxy, quantum-openvswitch-agent, openvswitch > > > 2. Can you please print the iptables on the compute node? > > I disabled it already, here's output before I do it. > > [root at computer-2 ~]# iptables-save > > # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 > > *filter > > :INPUT ACCEPT [22634:3487580] > > :FORWARD ACCEPT [22:704] > > :OUTPUT ACCEPT [22619:5860198] > > :nova-compute-FORWARD - [0:0] > > :nova-compute-INPUT - [0:0] > > :nova-compute-OUTPUT - [0:0] > > :nova-compute-inst-26 - [0:0] > > :nova-compute-local - [0:0] > > :nova-compute-provider - [0:0] > > :nova-compute-sg-fallback - [0:0] > > :nova-filter-top - [0:0] > > -A INPUT -j nova-compute-INPUT > > -A FORWARD -j nova-filter-top > > -A FORWARD -j nova-compute-FORWARD > > -A OUTPUT -j nova-filter-top > > -A OUTPUT -j nova-compute-OUTPUT > > -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m > udp --sport 68 --dport 67 -j ACCEPT > > -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m > udp --sport 68 --dport 67 -j ACCEPT > > -A nova-compute-inst-26 -m state --state INVALID -j DROP > > -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT > > -A nova-compute-inst-26 -j nova-compute-provider > > -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 > --dport 68 -j ACCEPT > > -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT > > -A nova-compute-inst-26 -p icmp -j ACCEPT > > -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT > > -A nova-compute-inst-26 -j nova-compute-sg-fallback > > -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 > > -A nova-compute-sg-fallback -j DROP > > -A nova-filter-top -j nova-compute-local > > COMMIT > > # Completed on Sun Apr 28 16:37:18 2013 > > # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 > > *mangle > > :PREROUTING ACCEPT [22733:3519752] > > :INPUT ACCEPT [22733:3519752] > > :FORWARD ACCEPT [175:50468] > > :OUTPUT ACCEPT [22705:5868566] > > :POSTROUTING ACCEPT [22880:5919034] > > :nova-compute-POSTROUTING - [0:0] > > -A POSTROUTING -j nova-compute-POSTROUTING > > COMMIT > > # Completed on Sun Apr 28 16:37:18 2013 > > # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 > > *nat > > :PREROUTING ACCEPT [16:14570] > > :POSTROUTING ACCEPT [338:22855] > > :OUTPUT ACCEPT [331:20579] > > :nova-compute-OUTPUT - [0:0] > > :nova-compute-POSTROUTING - [0:0] > > :nova-compute-PREROUTING - [0:0] > > :nova-compute-float-snat - [0:0] > > :nova-compute-snat - [0:0] > > :nova-postrouting-bottom - [0:0] > > -A PREROUTING -j nova-compute-PREROUTING > > -A POSTROUTING -j nova-compute-POSTROUTING > > -A POSTROUTING -j nova-postrouting-bottom > > -A OUTPUT -j nova-compute-OUTPUT > > -A nova-compute-snat -j nova-compute-float-snat > > -A nova-postrouting-bottom -j nova-compute-snat > > COMMIT > > # Completed on Sun Apr 28 16:37:18 2013 > > > 3. Can you please print the flow table rules (ovs-dpctl dump-flows > br-int)? > > I suppose you mean ovs-ofctl dump-flows br-int ? > > [root at computer-2 ~]# ovs-ofctl dump-flows br-int > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4125.444s, table=0, n_packets=1707, > n_bytes=90606, idle_age=12, priority=1 actions=NORMAL > > cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, > idle_age=20, priority=2,in_port=1 actions=drop > > cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, > idle_age=3349, priority=3,in_port=1,dl_vlan=1001 > actions=mod_vlan_vid:1,NORMAL > > Here?s also ovs-dpctl show: > > [root at computer-2 ~]# ovs-dpctl show > > system at br-p3p1: > > lookups: hit:3967 missed:314 lost:0 > > flows: 1 > > port 0: br-p3p1 (internal) > > port 1: p3p1 > > port 2: phy-br-p3p1 > > system at br-int: > > lookups: hit:1575 missed:302 lost:0 > > flows: 0 > > port 0: br-int (internal) > > port 1: int-br-p3p1 > > port 4: qvo39242f22-ec > > > Thanks > Gary > > On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi?Gary > > I tried capture packet while keeping VM to restart it?s network. > > I can see dhcp request broadcast packet on tap, qbr, qvb and qvo > interfaces. > > Failed to see packet on int-br-p3p1 on bridge br-int. > > Not sure if it has something to do with openflow setting? I attach > some ovs-ofctl outputs > > I have not seen ?veth? port anywhere? > > ---Record--- > > [root at computer-2 ~]# brctl show > > bridge name bridge id STP enabled interfaces > > qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec > > tap39242f22-ec > > virbr0 8000.525400c47f62 yes virbr0-nic > > [root at computer-2 ~]# ovs-vsctl show > > 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "qvo39242f22-ec" > > tag: 1 > > Interface "qvo39242f22-ec" > > Bridge "br-p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > ovs_version: "1.9.0" > > [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 > > tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 2 packets captured > > 2 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 > > tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 1 packets captured > > 1 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 > > tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 1 packets captured > > 1 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 > > tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 2 packets captured > > 2 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 > > tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size > 65535 bytes > > ^C > > 0 packets captured > > 0 packets received by filter > > 0 packets dropped by kernel > > ---output of ovs-ofctl--- > > [root at computer-2 ~]# ovs-ofctl show br-int > > OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 > > n_tables:255, n_buffers:256 > > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC > SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > > 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > LOCAL(br-int): addr:86:40:18:20:f1:42 > > config: PORT_DOWN > > state: LINK_DOWN > > speed: 100 Mbps now, 100 Mbps max > > OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 > > [root at computer-2 ~]# > > [root at computer-2 ~]# ovs-ofctl show br-p3p1 > > OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 > > n_tables:255, n_buffers:256 > > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC > SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > > 1(p3p1): addr:a0:36:9f:15:d4:24 > > config: 0 > > state: 0 > > current: 10GB-FD > > advertised: 10GB-FD FIBER > > supported: 10GB-FD FIBER > > speed: 10000 Mbps now, 10000 Mbps max > > 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 > > config: PORT_DOWN > > state: LINK_DOWN > > speed: 100 Mbps now, 100 Mbps max > > OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 > > [root at computer-2 ~]# ovs-ofctl dump-flows br-int > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4125.444s, table=0, n_packets=1707, > n_bytes=90606, idle_age=12, priority=1 actions=NORMAL > > cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, > idle_age=20, priority=2,in_port=1 actions=drop > > cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, > idle_age=3349, priority=3,in_port=1,dl_vlan=1001 > actions=mod_vlan_vid:1,NORMAL > > [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4129.629s, table=0, n_packets=2175, > n_bytes=138652, idle_age=0, priority=1 actions=NORMAL > > cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, > idle_age=1045, priority=2,in_port=2 actions=drop > > cookie=0x0, duration=3354.578s, table=0, n_packets=1697, > n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 > actions=mod_vlan_vid:1001,NORMAL > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Zhang, Kimi > (NSN - CN/Cheng Du) > *Sent:* Sunday, April 28, 2013 3:40 PM > *To:* gkotton at redhat.com ; > rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Very nice pic, I am going to try to capture packet on each port. > > I did not configure to use quantum to manage firewall , just leave it > to nova-compute, will try your configs later. > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 3:33 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi, > Can you also please check that firewall_driver = > quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver > is configured in plugin.ini file.And security_group_api = quantum is > set in nova.conf > Thanks > Gary > > On 04/28/2013 10:21 AM, Gary Kotton wrote: > > On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > I tried to disable iptables on both network and compute nodes, still > does not work out L > > > Can you please look at > https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing > > When using the OVS there are a number of devices. Would it be possible > that you try and capture on each device so that we can try and see > where the packet is discarded. > > I will have a setup ready in about an hour. > > > From quantum openvswitch agent logs, following messages keeps coming > out repeatly every 2-3 seconds, not sure if they matter or not? > > > The messages below are OK - this is how the OVS agent works. It polls > the OVS every interval to check if new ports are created. > > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making > synchronous call on q-plugin ... > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID > is 92f4e83cf92c46f1b9304c879f9b7a41 > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] > UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:08 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > I tried that too, no lucky. > > From tcpdump ,it seems br-int does not forward any packet to > interfaces connect to br-p3p1, which connects to physical network? > > > There could be a number of issues here: > 1. The iptables are dropping the traffic (I am in the process of > getting a setup up and running) > 2. The network connectivity > > In order to ensure that it is not the first one can you try and see > which iptables rules are matched or disable the iptables? > > > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:01 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > Yes, I?m aware of that packstack does not support quantum yet. The > whole setup was installed manually. > > I did run quantum-server-setup and quantum-host-setup, I tried > linuxbridge plugin too, it has no issue for VM to get IP address, but > openvswitch has issues on this? > > > ok. > > if you configure and IP address manually on the VM are you able to > ping the port of the DHCP agent? > > you can get the IP from quantum port-list > > > > > > Regards, > > Kimi > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 2:50 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi Kimi, > Thanks for the mail. Please see the inline comments below. Please note > that at the moment we do not have packstack support for Quantum so > there is a little manual plumbing that needs to be done (not sure if > you have done this already). > On the host where the quantum service is running you need to run > quantum-server-setup and on the compute nodes you need to run > quantum-host-setup (please note that the relevant keystone credentials > need to be set too). > Thanks > Gary > > On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > converted from rtf > > When I start VM instance, the VM can?t get IP address. Could someone > help me on this ? > > > I will try > > > > > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > ?Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > > Network: bond0(10.68.125.11 for O&M) > > ?Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network > > > Please note that RHEL currently does not support namespaces so there > are a number of limitations. We are addressing this at the moment. If > namespaces are not used then it is suggested that one does not run the > DHCP agent and the L3 agent on the same host. The reason for this is > that there is no network isolation. > > > > > > ?Compute node: > > Services: nove-compute and quantum-openvswitch-agent > > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of > network and compute nodes. > > 1.Quantum.conf: > > [DEFAULT] > > debug = True > > verbose = True > > lock_path = $state_path/lock > > bind_host = 0.0.0.0 > > bind_port = 9696 > > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > > api_paste_config = api-paste.ini > > rpc_backend = quantum.openstack.common.rpc.impl_kombu > > > Are you using rabbit or qpid? > > > > > > control_exchange = quantum > > rabbit_host = 10.68.125.11 > > notification_driver = quantum.openstack.common.notifier.rpc_notifier > > default_notification_level = INFO > > notification_topics = notifications > > [QUOTAS] > > [DEFAULT_SERVICETYPE] > > [AGENT] > > polling_interval = 2 > > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > > [keystone_authtoken] > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /var/lib/quantum/keystone-signing > > admin_tenant_name = service > > admin_user = quantum > > admin_password = password > > 2.ovs_quantum_plugin.ini > > [DATABASE] > > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > > > reconnect_interval = 2 > > [OVS] > > tenant_network_type = vlan > > network_vlan_ranges = physnet1:1000:2999 > > bridge_mappings = physnet1:br-p3p1 > > [AGENT] > > polling_interval = 2 > > [SECURITYGROUP] > > 3.nova.conf > > [DEFAULT] > > verbose=true > > logdir = /var/log/nova > > state_path = /var/lib/nova > > lock_path = /var/lib/nova/tmp > > volumes_dir = /etc/nova/volumes > > dhcpbridge = /usr/bin/nova-dhcpbridge > > dhcpbridge_flagfile = /etc/nova/nova.conf > > force_dhcp_release = True > > injected_network_template = /usr/share/nova/interfaces.template > > libvirt_nonblocking = True > > libvirt_inject_partition = -1 > > network_manager = nova.network.manager.FlatDHCPManager > > iscsi_helper = tgtadm > > compute_driver = libvirt.LibvirtDriver > > libvirt_type=kvm > > libvirt_ovs_bridge=br-int > > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > > manager=nova.conductor.manager.ConductorManager > > rpc_backend = nova.openstack.common.rpc.impl_kombu > > rabbit_host = 10.68.125.11 > > rootwrap_config = /etc/nova/rootwrap.conf > > use_deprecated_auth=false > > auth_strategy=keystone > > glance_api_servers=10.68.125.11:9292 > > image_service=nova.image.glance.GlanceImageService > > novnc_enabled=true > > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > > novncproxy_port=6080 > > vncserver_proxyclient_address=10.68.125.16 > > vncserver_listen=0.0.0.0 > > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > > libvirt_use_virtio_for_bridges=True > > network_api_class=nova.network.quantumv2.api.API > > quantum_url=http://10.68.125.11:9696 > > quantum_auth_strategy=keystone > > quantum_admin_tenant_name=service > > quantum_admin_username=quantum > > quantum_admin_password=password > > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > > libvirt_vif_type=ethernet > > service_quantum_metadata_proxy = True > > quantum_metadata_proxy_shared_secret = helloOpenStack > > metadata_host = 10.68.125.11 > > metadata_listen = 0.0.0.0 > > metadata_listen_port = 8775 > > [keystone_authtoken] > > admin_tenant_name = service > > admin_user = nova > > admin_password = password > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /tmp/keystone-signing-nova > > 4.ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > > Bridge "br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Bridge br-ex > > Port br-ex > > Interface br-ex > > type: internal > > Port "qg-a83c0abd-f4" > > Interface "qg-a83c0abd-f4" > > type: internal > > Port "p3p2" > > Interface "p3p2" > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "tap1f386a2a-12" > > tag: 1 > > Interface "tap1f386a2a-12" > > type: internal > > ovs_version: "1.9.0" > > 5.ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > > Bridge "br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Bridge br-int > > Port "qvo56a4572c-dc" > > tag: 2 > > Interface "qvo56a4572c-dc" > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port br-int > > Interface br-int > > type: internal > > ovs_version: "1.9.0" > > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can?t see packet from int-br-p3p1 on br-int or any port from br-p3p1. > > > Any chance to get the DHCP and the L3 agent configuration files? > Please check that use_namespaces = False in both of these files. > > Are there any log errors? > > > > > > Thank you! > > Regards, > > Kimi > > > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kimi.zhang at nsn.com Sun Apr 28 08:56:40 2013 From: kimi.zhang at nsn.com (Zhang, Kimi (NSN - CN/Cheng Du)) Date: Sun, 28 Apr 2013 08:56:40 +0000 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CE212.2030000@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> <517CDE34.1080506@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> <517CE212.2030000@redhat.com> Message-ID: <90CF2062F86FD8498897037C7FBBC0880470FB@SGSIMBX001.nsn-intra.net> Yes, I did run quantum-dhcp-setup on network node. Thanks, good luck there. Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 4:47 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Thanks. One more question - on the network node, did you run quantum-dhcp-setup? I am nearly ready with my setup. Hopefully I'll have a reproduction or some additional questions. Thanks Gary On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Sure, my answers below. :) Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 4:31 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi, I have a few questions (please be patient with me): 1. On the compute node, which services are running? nova-compute, nova-novncproxy, quantum-openvswitch-agent, openvswitch 2. Can you please print the iptables on the compute node? I disabled it already, here's output before I do it. [root at computer-2 ~]# iptables-save # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 *filter :INPUT ACCEPT [22634:3487580] :FORWARD ACCEPT [22:704] :OUTPUT ACCEPT [22619:5860198] :nova-compute-FORWARD - [0:0] :nova-compute-INPUT - [0:0] :nova-compute-OUTPUT - [0:0] :nova-compute-inst-26 - [0:0] :nova-compute-local - [0:0] :nova-compute-provider - [0:0] :nova-compute-sg-fallback - [0:0] :nova-filter-top - [0:0] -A INPUT -j nova-compute-INPUT -A FORWARD -j nova-filter-top -A FORWARD -j nova-compute-FORWARD -A OUTPUT -j nova-filter-top -A OUTPUT -j nova-compute-OUTPUT -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A nova-compute-inst-26 -m state --state INVALID -j DROP -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT -A nova-compute-inst-26 -j nova-compute-provider -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT -A nova-compute-inst-26 -p icmp -j ACCEPT -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT -A nova-compute-inst-26 -j nova-compute-sg-fallback -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 -A nova-compute-sg-fallback -j DROP -A nova-filter-top -j nova-compute-local COMMIT # Completed on Sun Apr 28 16:37:18 2013 # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 *mangle :PREROUTING ACCEPT [22733:3519752] :INPUT ACCEPT [22733:3519752] :FORWARD ACCEPT [175:50468] :OUTPUT ACCEPT [22705:5868566] :POSTROUTING ACCEPT [22880:5919034] :nova-compute-POSTROUTING - [0:0] -A POSTROUTING -j nova-compute-POSTROUTING COMMIT # Completed on Sun Apr 28 16:37:18 2013 # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 *nat :PREROUTING ACCEPT [16:14570] :POSTROUTING ACCEPT [338:22855] :OUTPUT ACCEPT [331:20579] :nova-compute-OUTPUT - [0:0] :nova-compute-POSTROUTING - [0:0] :nova-compute-PREROUTING - [0:0] :nova-compute-float-snat - [0:0] :nova-compute-snat - [0:0] :nova-postrouting-bottom - [0:0] -A PREROUTING -j nova-compute-PREROUTING -A POSTROUTING -j nova-compute-POSTROUTING -A POSTROUTING -j nova-postrouting-bottom -A OUTPUT -j nova-compute-OUTPUT -A nova-compute-snat -j nova-compute-float-snat -A nova-postrouting-bottom -j nova-compute-snat COMMIT # Completed on Sun Apr 28 16:37:18 2013 3. Can you please print the flow table rules (ovs-dpctl dump-flows br-int)? I suppose you mean ovs-ofctl dump-flows br-int ? [root at computer-2 ~]# ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4125.444s, table=0, n_packets=1707, n_bytes=90606, idle_age=12, priority=1 actions=NORMAL cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, idle_age=3349, priority=3,in_port=1,dl_vlan=1001 actions=mod_vlan_vid:1,NORMAL Here?s also ovs-dpctl show: [root at computer-2 ~]# ovs-dpctl show system at br-p3p1: lookups: hit:3967 missed:314 lost:0 flows: 1 port 0: br-p3p1 (internal) port 1: p3p1 port 2: phy-br-p3p1 system at br-int: lookups: hit:1575 missed:302 lost:0 flows: 0 port 0: br-int (internal) port 1: int-br-p3p1 port 4: qvo39242f22-ec Thanks Gary On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi? Gary I tried capture packet while keeping VM to restart it?s network. I can see dhcp request broadcast packet on tap, qbr, qvb and qvo interfaces. Failed to see packet on int-br-p3p1 on bridge br-int. Not sure if it has something to do with openflow setting? I attach some ovs-ofctl outputs I have not seen ?veth? port anywhere? ---Record--- [root at computer-2 ~]# brctl show bridge name bridge id STP enabled interfaces qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec tap39242f22-ec virbr0 8000.525400c47f62 yes virbr0-nic [root at computer-2 ~]# ovs-vsctl show 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "qvo39242f22-ec" tag: 1 Interface "qvo39242f22-ec" Bridge "br-p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal ovs_version: "1.9.0" [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 1 packets captured 1 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 1 packets captured 1 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size 65535 bytes 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel ---output of ovs-ofctl--- [root at computer-2 ~]# ovs-ofctl show br-int OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 n_tables:255, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max LOCAL(br-int): addr:86:40:18:20:f1:42 config: PORT_DOWN state: LINK_DOWN speed: 100 Mbps now, 100 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 [root at computer-2 ~]# [root at computer-2 ~]# ovs-ofctl show br-p3p1 OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 n_tables:255, n_buffers:256 capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE 1(p3p1): addr:a0:36:9f:15:d4:24 config: 0 state: 0 current: 10GB-FD advertised: 10GB-FD FIBER supported: 10GB-FD FIBER speed: 10000 Mbps now, 10000 Mbps max 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 config: 0 state: 0 current: 10GB-FD COPPER speed: 10000 Mbps now, 100 Mbps max LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 config: PORT_DOWN state: LINK_DOWN speed: 100 Mbps now, 100 Mbps max OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 [root at computer-2 ~]# ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4125.444s, table=0, n_packets=1707, n_bytes=90606, idle_age=12, priority=1 actions=NORMAL cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, idle_age=3349, priority=3,in_port=1,dl_vlan=1001 actions=mod_vlan_vid:1,NORMAL [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=4129.629s, table=0, n_packets=2175, n_bytes=138652, idle_age=0, priority=1 actions=NORMAL cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, idle_age=1045, priority=2,in_port=2 actions=drop cookie=0x0, duration=3354.578s, table=0, n_packets=1697, n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:1001,NORMAL Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Zhang, Kimi (NSN - CN/Cheng Du) Sent: Sunday, April 28, 2013 3:40 PM To: gkotton at redhat.com; rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Very nice pic, I am going to try to capture packet on each port. I did not configure to use quantum to manage firewall , just leave it to nova-compute, will try your configs later. Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 3:33 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi, Can you also please check that firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is configured in plugin.ini file.And security_group_api = quantum is set in nova.conf Thanks Gary On 04/28/2013 10:21 AM, Gary Kotton wrote: On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary I tried to disable iptables on both network and compute nodes, still does not work out :( Can you please look at https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing When using the OVS there are a number of devices. Would it be possible that you try and capture on each device so that we can try and see where the packet is discarded. I will have a setup ready in about an hour. From quantum openvswitch agent logs, following messages keeps coming out repeatly every 2-3 seconds, not sure if they matter or not? The messages below are OK - this is how the OVS agent works. It polls the OVS every interval to check if new ports are created. 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making synchronous call on q-plugin ... 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] Exit code: 0 Stdout: 'int-br-p3p1\n' Stderr: '' 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', 'external_ids'] Exit code: 0 Stdout: '{}\n' Stderr: '' Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:08 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: I tried that too, no lucky. From tcpdump ,it seems br-int does not forward any packet to interfaces connect to br-p3p1, which connects to physical network? There could be a number of issues here: 1. The iptables are dropping the traffic (I am in the process of getting a setup up and running) 2. The network connectivity In order to ensure that it is not the first one can you try and see which iptables rules are matched or disable the iptables? Regards, Kimi Zhang MP: +86 186 0800 8182 Call me(NCS): sip:+86018608008182 From: ext Gary Kotton [mailto:gkotton at redhat.com] Sent: Sunday, April 28, 2013 3:01 PM To: Zhang, Kimi (NSN - CN/Cheng Du) Cc: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: Hi, Gary Yes, I?m aware of that packstack does not support quantum yet. The whole setup was installed manually. I did run quantum-server-setup and quantum-host-setup, I tried linuxbridge plugin too, it has no issue for VM to get IP address, but openvswitch has issues on this? ok. if you configure and IP address manually on the VM are you able to ping the port of the DHCP agent? you can get the IP from quantum port-list Regards, Kimi From: rdo-list-bounces at redhat.com [mailto:rdo-list-bounces at redhat.com] On Behalf Of ext Gary Kotton Sent: Sunday, April 28, 2013 2:50 PM To: rdo-list at redhat.com Subject: Re: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan Hi Kimi, Thanks for the mail. Please see the inline comments below. Please note that at the moment we do not have packstack support for Quantum so there is a little manual plumbing that needs to be done (not sure if you have done this already). On the host where the quantum service is running you need to run quantum-server-setup and on the compute nodes you need to run quantum-host-setup (please note that the relevant keystone credentials need to be set too). Thanks Gary On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: converted from rtf When I start VM instance, the VM can?t get IP address. Could someone help me on this ? I will try 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. ? Controller node: Services: Keystone+Glance+Cinder+Quantum server + Nova services Network: bond0(10.68.125.11 for O&M) ? Network node: Services: quantum-openvswitch-agent, quantum-l3-agent, quantum-dhcp-agent, quantum-metadata-agent Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, p3p2 for external network Please note that RHEL currently does not support namespaces so there are a number of limitations. We are addressing this at the moment. If namespaces are not used then it is suggested that one does not run the DHCP agent and the L3 agent on the same host. The reason for this is that there is no network isolation. ? Compute node: Services: nove-compute and quantum-openvswitch-agent Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network ? Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of network and compute nodes. 1. Quantum.conf: [DEFAULT] debug = True verbose = True lock_path = $state_path/lock bind_host = 0.0.0.0 bind_port = 9696 core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 api_paste_config = api-paste.ini rpc_backend = quantum.openstack.common.rpc.impl_kombu Are you using rabbit or qpid? control_exchange = quantum rabbit_host = 10.68.125.11 notification_driver = quantum.openstack.common.notifier.rpc_notifier default_notification_level = INFO notification_topics = notifications [QUOTAS] [DEFAULT_SERVICETYPE] [AGENT] polling_interval = 2 root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf [keystone_authtoken] auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /var/lib/quantum/keystone-signing admin_tenant_name = service admin_user = quantum admin_password = password 2. ovs_quantum_plugin.ini [DATABASE] sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum reconnect_interval = 2 [OVS] tenant_network_type = vlan network_vlan_ranges = physnet1:1000:2999 bridge_mappings = physnet1:br-p3p1 [AGENT] polling_interval = 2 [SECURITYGROUP] 3. nova.conf [DEFAULT] verbose=true logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm compute_driver = libvirt.LibvirtDriver libvirt_type=kvm libvirt_ovs_bridge=br-int firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver manager=nova.conductor.manager.ConductorManager rpc_backend = nova.openstack.common.rpc.impl_kombu rabbit_host = 10.68.125.11 rootwrap_config = /etc/nova/rootwrap.conf use_deprecated_auth=false auth_strategy=keystone glance_api_servers=10.68.125.11:9292 image_service=nova.image.glance.GlanceImageService novnc_enabled=true novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=10.68.125.16 vncserver_listen=0.0.0.0 libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver libvirt_use_virtio_for_bridges=True network_api_class=nova.network.quantumv2.api.API quantum_url=http://10.68.125.11:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=password quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver libvirt_vif_type=ethernet service_quantum_metadata_proxy = True quantum_metadata_proxy_shared_secret = helloOpenStack metadata_host = 10.68.125.11 metadata_listen = 0.0.0.0 metadata_listen_port = 8775 [keystone_authtoken] admin_tenant_name = service admin_user = nova admin_password = password auth_host = 10.68.125.11 auth_port = 35357 auth_protocol = http signing_dir = /tmp/keystone-signing-nova 4. ovs-vsctl show on network node: aeeb6cf7-271b-405a-aa17-1b95bcd9e301 Bridge "br-p3p1" Port "p3p1" Interface "p3p1" Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port "qg-a83c0abd-f4" Interface "qg-a83c0abd-f4" type: internal Port "p3p2" Interface "p3p2" Bridge br-int Port br-int Interface br-int type: internal Port "int-br-p3p1" Interface "int-br-p3p1" Port "tap1f386a2a-12" tag: 1 Interface "tap1f386a2a-12" type: internal ovs_version: "1.9.0" 5. ovs-vsctl show on compute node: 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 Bridge "br-p3p1" Port "br-p3p1" Interface "br-p3p1" type: internal Port "phy-br-p3p1" Interface "phy-br-p3p1" Port "p3p1" Interface "p3p1" Bridge br-int Port "qvo56a4572c-dc" tag: 2 Interface "qvo56a4572c-dc" Port "int-br-p3p1" Interface "int-br-p3p1" Port br-int Interface br-int type: internal ovs_version: "1.9.0" On compute node, I can see dhcp request packet from tcpdump on qvo56a4572c-dc, but it seems the packet is not forwarded out since I can?t see packet from int-br-p3p1 on br-int or any port from br-p3p1. Any chance to get the DHCP and the L3 agent configuration files? Please check that use_namespaces = False in both of these files. Are there any log errors? Thank you! Regards, Kimi _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list _______________________________________________ Rdo-list mailing list Rdo-list at redhat.com https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 09:26:26 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 12:26:26 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <90CF2062F86FD8498897037C7FBBC0880470FB@SGSIMBX001.nsn-intra.net> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> <517CDE34.1080506@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> <517CE212.2030000@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470FB@SGSIMBX001.nsn-intra.net> Message-ID: <517CEB42.1010005@redhat.com> Hi, I have been able to reproduce the problem. I'll get back to you as soon as I have any information. Thanks Gary On 04/28/2013 11:56 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Yes, I did run quantum-dhcp-setup on network node. > > Thanks, good luck there. > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 4:47 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Thanks. > One more question - on the network node, did you run quantum-dhcp-setup? > I am nearly ready with my setup. Hopefully I'll have a reproduction or > some additional questions. > Thanks > Gary > > On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Sure, my answers below. :) > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 4:31 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi, > I have a few questions (please be patient with me): > 1. On the compute node, which services are running? > > nova-compute, nova-novncproxy, quantum-openvswitch-agent, openvswitch > > > 2. Can you please print the iptables on the compute node? > > I disabled it already, here's output before I do it. > > [root at computer-2 ~]# iptables-save > > # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 > > *filter > > :INPUT ACCEPT [22634:3487580] > > :FORWARD ACCEPT [22:704] > > :OUTPUT ACCEPT [22619:5860198] > > :nova-compute-FORWARD - [0:0] > > :nova-compute-INPUT - [0:0] > > :nova-compute-OUTPUT - [0:0] > > :nova-compute-inst-26 - [0:0] > > :nova-compute-local - [0:0] > > :nova-compute-provider - [0:0] > > :nova-compute-sg-fallback - [0:0] > > :nova-filter-top - [0:0] > > -A INPUT -j nova-compute-INPUT > > -A FORWARD -j nova-filter-top > > -A FORWARD -j nova-compute-FORWARD > > -A OUTPUT -j nova-filter-top > > -A OUTPUT -j nova-compute-OUTPUT > > -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m > udp --sport 68 --dport 67 -j ACCEPT > > -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m > udp --sport 68 --dport 67 -j ACCEPT > > -A nova-compute-inst-26 -m state --state INVALID -j DROP > > -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT > > -A nova-compute-inst-26 -j nova-compute-provider > > -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 > --dport 68 -j ACCEPT > > -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT > > -A nova-compute-inst-26 -p icmp -j ACCEPT > > -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT > > -A nova-compute-inst-26 -j nova-compute-sg-fallback > > -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 > > -A nova-compute-sg-fallback -j DROP > > -A nova-filter-top -j nova-compute-local > > COMMIT > > # Completed on Sun Apr 28 16:37:18 2013 > > # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 > > *mangle > > :PREROUTING ACCEPT [22733:3519752] > > :INPUT ACCEPT [22733:3519752] > > :FORWARD ACCEPT [175:50468] > > :OUTPUT ACCEPT [22705:5868566] > > :POSTROUTING ACCEPT [22880:5919034] > > :nova-compute-POSTROUTING - [0:0] > > -A POSTROUTING -j nova-compute-POSTROUTING > > COMMIT > > # Completed on Sun Apr 28 16:37:18 2013 > > # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 > > *nat > > :PREROUTING ACCEPT [16:14570] > > :POSTROUTING ACCEPT [338:22855] > > :OUTPUT ACCEPT [331:20579] > > :nova-compute-OUTPUT - [0:0] > > :nova-compute-POSTROUTING - [0:0] > > :nova-compute-PREROUTING - [0:0] > > :nova-compute-float-snat - [0:0] > > :nova-compute-snat - [0:0] > > :nova-postrouting-bottom - [0:0] > > -A PREROUTING -j nova-compute-PREROUTING > > -A POSTROUTING -j nova-compute-POSTROUTING > > -A POSTROUTING -j nova-postrouting-bottom > > -A OUTPUT -j nova-compute-OUTPUT > > -A nova-compute-snat -j nova-compute-float-snat > > -A nova-postrouting-bottom -j nova-compute-snat > > COMMIT > > # Completed on Sun Apr 28 16:37:18 2013 > > > 3. Can you please print the flow table rules (ovs-dpctl dump-flows > br-int)? > > I suppose you mean ovs-ofctl dump-flows br-int ? > > [root at computer-2 ~]# ovs-ofctl dump-flows br-int > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4125.444s, table=0, n_packets=1707, > n_bytes=90606, idle_age=12, priority=1 actions=NORMAL > > cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, > idle_age=20, priority=2,in_port=1 actions=drop > > cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, > idle_age=3349, priority=3,in_port=1,dl_vlan=1001 > actions=mod_vlan_vid:1,NORMAL > > Here?s also ovs-dpctl show: > > [root at computer-2 ~]# ovs-dpctl show > > system at br-p3p1: > > lookups: hit:3967 missed:314 lost:0 > > flows: 1 > > port 0: br-p3p1 (internal) > > port 1: p3p1 > > port 2: phy-br-p3p1 > > system at br-int: > > lookups: hit:1575 missed:302 lost:0 > > flows: 0 > > port 0: br-int (internal) > > port 1: int-br-p3p1 > > port 4: qvo39242f22-ec > > > Thanks > Gary > > On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi?Gary > > I tried capture packet while keeping VM to restart it?s network. > > I can see dhcp request broadcast packet on tap, qbr, qvb and qvo > interfaces. > > Failed to see packet on int-br-p3p1 on bridge br-int. > > Not sure if it has something to do with openflow setting? I attach > some ovs-ofctl outputs > > I have not seen ?veth? port anywhere? > > ---Record--- > > [root at computer-2 ~]# brctl show > > bridge name bridge id STP enabled interfaces > > qbr39242f22-ec 8000.c6f95e6a859a no qvb39242f22-ec > > tap39242f22-ec > > virbr0 8000.525400c47f62 yes virbr0-nic > > [root at computer-2 ~]# ovs-vsctl show > > 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "qvo39242f22-ec" > > tag: 1 > > Interface "qvo39242f22-ec" > > Bridge "br-p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > ovs_version: "1.9.0" > > [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 > > tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 2 packets captured > > 2 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 > > tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 1 packets captured > > 1 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 > > tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 1 packets captured > > 1 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 > > tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture size > 65535 bytes > > 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: > BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 > > ^C > > 2 packets captured > > 2 packets received by filter > > 0 packets dropped by kernel > > [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 > > tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size > 65535 bytes > > ^C > > 0 packets captured > > 0 packets received by filter > > 0 packets dropped by kernel > > ---output of ovs-ofctl--- > > [root at computer-2 ~]# ovs-ofctl show br-int > > OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 > > n_tables:255, n_buffers:256 > > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC > SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > > 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > LOCAL(br-int): addr:86:40:18:20:f1:42 > > config: PORT_DOWN > > state: LINK_DOWN > > speed: 100 Mbps now, 100 Mbps max > > OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 > > [root at computer-2 ~]# > > [root at computer-2 ~]# ovs-ofctl show br-p3p1 > > OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 > > n_tables:255, n_buffers:256 > > capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP > > actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC > SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE > > 1(p3p1): addr:a0:36:9f:15:d4:24 > > config: 0 > > state: 0 > > current: 10GB-FD > > advertised: 10GB-FD FIBER > > supported: 10GB-FD FIBER > > speed: 10000 Mbps now, 10000 Mbps max > > 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 > > config: 0 > > state: 0 > > current: 10GB-FD COPPER > > speed: 10000 Mbps now, 100 Mbps max > > LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 > > config: PORT_DOWN > > state: LINK_DOWN > > speed: 100 Mbps now, 100 Mbps max > > OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 > > [root at computer-2 ~]# ovs-ofctl dump-flows br-int > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4125.444s, table=0, n_packets=1707, > n_bytes=90606, idle_age=12, priority=1 actions=NORMAL > > cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, > idle_age=20, priority=2,in_port=1 actions=drop > > cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, > idle_age=3349, priority=3,in_port=1,dl_vlan=1001 > actions=mod_vlan_vid:1,NORMAL > > [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 > > NXST_FLOW reply (xid=0x4): > > cookie=0x0, duration=4129.629s, table=0, n_packets=2175, > n_bytes=138652, idle_age=0, priority=1 actions=NORMAL > > cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, > idle_age=1045, priority=2,in_port=2 actions=drop > > cookie=0x0, duration=3354.578s, table=0, n_packets=1697, > n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 > actions=mod_vlan_vid:1001,NORMAL > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Zhang, Kimi > (NSN - CN/Cheng Du) > *Sent:* Sunday, April 28, 2013 3:40 PM > *To:* gkotton at redhat.com ; > rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Very nice pic, I am going to try to capture packet on each port. > > I did not configure to use quantum to manage firewall , just leave it > to nova-compute, will try your configs later. > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 3:33 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi, > Can you also please check that firewall_driver = > quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver > is configured in plugin.ini file.And security_group_api = quantum is > set in nova.conf > Thanks > Gary > > On 04/28/2013 10:21 AM, Gary Kotton wrote: > > On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > I tried to disable iptables on both network and compute nodes, still > does not work out L > > > Can you please look at > https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing > > When using the OVS there are a number of devices. Would it be possible > that you try and capture on each device so that we can try and see > where the packet is discarded. > > I will have a setup ready in about an hour. > > > > From quantum openvswitch agent logs, following messages keeps coming > out repeatly every 2-3 seconds, not sure if they matter or not? > > > The messages below are OK - this is how the OVS agent works. It polls > the OVS every interval to check if new ports are created. > > > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] Making > synchronous call on q-plugin ... > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] MSG_ID > is 92f4e83cf92c46f1b9304c879f9b7a41 > > 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] > UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] > > Exit code: 0 > > Stdout: 'int-br-p3p1\n' > > Stderr: '' > > 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running command: > ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] > > Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', > 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', > 'external_ids'] > > Exit code: 0 > > Stdout: '{}\n' > > Stderr: '' > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:08 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > I tried that too, no lucky. > > From tcpdump ,it seems br-int does not forward any packet to > interfaces connect to br-p3p1, which connects to physical network? > > > There could be a number of issues here: > 1. The iptables are dropping the traffic (I am in the process of > getting a setup up and running) > 2. The network connectivity > > In order to ensure that it is not the first one can you try and see > which iptables rules are matched or disable the iptables? > > > > > Regards, > > Kimi Zhang > > MP: +86 186 0800 8182 > > Call me(NCS): sip:+86018608008182 > > *From:*ext Gary Kotton [mailto:gkotton at redhat.com] > *Sent:* Sunday, April 28, 2013 3:01 PM > *To:* Zhang, Kimi (NSN - CN/Cheng Du) > *Cc:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > Hi, Gary > > Yes, I?m aware of that packstack does not support quantum yet. The > whole setup was installed manually. > > I did run quantum-server-setup and quantum-host-setup, I tried > linuxbridge plugin too, it has no issue for VM to get IP address, but > openvswitch has issues on this? > > > ok. > > if you configure and IP address manually on the VM are you able to > ping the port of the DHCP agent? > > you can get the IP from quantum port-list > > > > > > > Regards, > > Kimi > > *From:*rdo-list-bounces at redhat.com > > [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton > *Sent:* Sunday, April 28, 2013 2:50 PM > *To:* rdo-list at redhat.com > *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + > Openvswitch + Vlan > > Hi Kimi, > Thanks for the mail. Please see the inline comments below. Please note > that at the moment we do not have packstack support for Quantum so > there is a little manual plumbing that needs to be done (not sure if > you have done this already). > On the host where the quantum service is running you need to run > quantum-server-setup and on the compute nodes you need to run > quantum-host-setup (please note that the relevant keystone credentials > need to be set too). > Thanks > Gary > > On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: > > converted from rtf > > When I start VM instance, the VM can?t get IP address. Could someone > help me on this ? > > > I will try > > > > > > 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. > > ?Controller node: > > Services: Keystone+Glance+Cinder+Quantum server + Nova services > > Network: bond0(10.68.125.11 for O&M) > > ?Network node: > > Services: quantum-openvswitch-agent, quantum-l3-agent, > quantum-dhcp-agent, quantum-metadata-agent > > Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, > p3p2 for external network > > > Please note that RHEL currently does not support namespaces so there > are a number of limitations. We are addressing this at the moment. If > namespaces are not used then it is suggested that one does not run the > DHCP agent and the L3 agent on the same host. The reason for this is > that there is no network isolation. > > > > > > > ?Compute node: > > Services: nove-compute and quantum-openvswitch-agent > > Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network > > ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) of > network and compute nodes. > > 1.Quantum.conf: > > [DEFAULT] > > debug = True > > verbose = True > > lock_path = $state_path/lock > > bind_host = 0.0.0.0 > > bind_port = 9696 > > core_plugin = > quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 > > api_paste_config = api-paste.ini > > rpc_backend = quantum.openstack.common.rpc.impl_kombu > > > Are you using rabbit or qpid? > > > > > > > control_exchange = quantum > > rabbit_host = 10.68.125.11 > > notification_driver = quantum.openstack.common.notifier.rpc_notifier > > default_notification_level = INFO > > notification_topics = notifications > > [QUOTAS] > > [DEFAULT_SERVICETYPE] > > [AGENT] > > polling_interval = 2 > > root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf > > [keystone_authtoken] > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /var/lib/quantum/keystone-signing > > admin_tenant_name = service > > admin_user = quantum > > admin_password = password > > 2.ovs_quantum_plugin.ini > > [DATABASE] > > sql_connection = mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum > > > reconnect_interval = 2 > > [OVS] > > tenant_network_type = vlan > > network_vlan_ranges = physnet1:1000:2999 > > bridge_mappings = physnet1:br-p3p1 > > [AGENT] > > polling_interval = 2 > > [SECURITYGROUP] > > 3.nova.conf > > [DEFAULT] > > verbose=true > > logdir = /var/log/nova > > state_path = /var/lib/nova > > lock_path = /var/lib/nova/tmp > > volumes_dir = /etc/nova/volumes > > dhcpbridge = /usr/bin/nova-dhcpbridge > > dhcpbridge_flagfile = /etc/nova/nova.conf > > force_dhcp_release = True > > injected_network_template = /usr/share/nova/interfaces.template > > libvirt_nonblocking = True > > libvirt_inject_partition = -1 > > network_manager = nova.network.manager.FlatDHCPManager > > iscsi_helper = tgtadm > > compute_driver = libvirt.LibvirtDriver > > libvirt_type=kvm > > libvirt_ovs_bridge=br-int > > firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver > > manager=nova.conductor.manager.ConductorManager > > rpc_backend = nova.openstack.common.rpc.impl_kombu > > rabbit_host = 10.68.125.11 > > rootwrap_config = /etc/nova/rootwrap.conf > > use_deprecated_auth=false > > auth_strategy=keystone > > glance_api_servers=10.68.125.11:9292 > > image_service=nova.image.glance.GlanceImageService > > novnc_enabled=true > > novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html > > novncproxy_port=6080 > > vncserver_proxyclient_address=10.68.125.16 > > vncserver_listen=0.0.0.0 > > libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver > > libvirt_use_virtio_for_bridges=True > > network_api_class=nova.network.quantumv2.api.API > > quantum_url=http://10.68.125.11:9696 > > quantum_auth_strategy=keystone > > quantum_admin_tenant_name=service > > quantum_admin_username=quantum > > quantum_admin_password=password > > quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 > > linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver > > libvirt_vif_type=ethernet > > service_quantum_metadata_proxy = True > > quantum_metadata_proxy_shared_secret = helloOpenStack > > metadata_host = 10.68.125.11 > > metadata_listen = 0.0.0.0 > > metadata_listen_port = 8775 > > [keystone_authtoken] > > admin_tenant_name = service > > admin_user = nova > > admin_password = password > > auth_host = 10.68.125.11 > > auth_port = 35357 > > auth_protocol = http > > signing_dir = /tmp/keystone-signing-nova > > 4.ovs-vsctl show on network node: > > aeeb6cf7-271b-405a-aa17-1b95bcd9e301 > > Bridge "br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Bridge br-ex > > Port br-ex > > Interface br-ex > > type: internal > > Port "qg-a83c0abd-f4" > > Interface "qg-a83c0abd-f4" > > type: internal > > Port "p3p2" > > Interface "p3p2" > > Bridge br-int > > Port br-int > > Interface br-int > > type: internal > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port "tap1f386a2a-12" > > tag: 1 > > Interface "tap1f386a2a-12" > > type: internal > > ovs_version: "1.9.0" > > 5.ovs-vsctl show on compute node: > > 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 > > Bridge "br-p3p1" > > Port "br-p3p1" > > Interface "br-p3p1" > > type: internal > > Port "phy-br-p3p1" > > Interface "phy-br-p3p1" > > Port "p3p1" > > Interface "p3p1" > > Bridge br-int > > Port "qvo56a4572c-dc" > > tag: 2 > > Interface "qvo56a4572c-dc" > > Port "int-br-p3p1" > > Interface "int-br-p3p1" > > Port br-int > > Interface br-int > > type: internal > > ovs_version: "1.9.0" > > On compute node, I can see dhcp request packet from tcpdump on > qvo56a4572c-dc, but it seems the packet is not forwarded out since I > can?t see packet from int-br-p3p1 on br-int or any port from br-p3p1. > > > Any chance to get the DHCP and the L3 agent configuration files? > Please check that use_namespaces = False in both of these files. > > Are there any log errors? > > > > > > > Thank you! > > Regards, > > Kimi > > > > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > > > > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 10:55:55 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 13:55:55 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517CEB42.1010005@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> <517CDE34.1080506@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> <517CE212.2030000@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470FB@SGSIMBX001.nsn-intra.net> <517CEB42.1010005@redhat.com> Message-ID: <517D003B.6030802@redhat.com> Hi, I have found a few problems and hopefully one or more may be related to the case that you have experienced: 1. When using OVS it is important you run the service ovs-quantum-cleanup when the host boots. This is due to the fact that OVS will store all tap device. This causes havoc when restarting hosts (in particular ones that have dhcp and l3 agents). So please make sure you have run "chkconfig quantum-ovs-cleanup on" on all hosts that are running the OVS. You can verify if this is the case by checking of the DHCP agent has created an IP address on the host. [Please note that we have a problem here - in the file /etc/init.d/quantum-ovs-cleanup "--config-file /usr/share/$proj/$proj-dist.conf" needs to be removed]. 2. Which dnsmasq version are you using? If this is 2.48 then there is a problem with the DHCP agent running. We are in the process of resolving this. If you make use of a version with tag support then this will work. 3. The quantum client needs to be updated to support the security groups. Hopefully we will have solutions for all of the above ASAP. Thanks Gary On 04/28/2013 12:26 PM, Gary Kotton wrote: > Hi, > I have been able to reproduce the problem. I'll get back to you as > soon as I have any information. > Thanks > Gary > > On 04/28/2013 11:56 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Yes, I did run quantum-dhcp-setup on network node. >> >> Thanks, good luck there. >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >> *Sent:* Sunday, April 28, 2013 4:47 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Thanks. >> One more question - on the network node, did you run quantum-dhcp-setup? >> I am nearly ready with my setup. Hopefully I'll have a reproduction >> or some additional questions. >> Thanks >> Gary >> >> On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Sure, my answers below. :) >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >> *Sent:* Sunday, April 28, 2013 4:31 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi, >> I have a few questions (please be patient with me): >> 1. On the compute node, which services are running? >> >> nova-compute, nova-novncproxy, quantum-openvswitch-agent, openvswitch >> >> >> 2. Can you please print the iptables on the compute node? >> >> I disabled it already, here's output before I do it. >> >> [root at computer-2 ~]# iptables-save >> >> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >> >> *filter >> >> :INPUT ACCEPT [22634:3487580] >> >> :FORWARD ACCEPT [22:704] >> >> :OUTPUT ACCEPT [22619:5860198] >> >> :nova-compute-FORWARD - [0:0] >> >> :nova-compute-INPUT - [0:0] >> >> :nova-compute-OUTPUT - [0:0] >> >> :nova-compute-inst-26 - [0:0] >> >> :nova-compute-local - [0:0] >> >> :nova-compute-provider - [0:0] >> >> :nova-compute-sg-fallback - [0:0] >> >> :nova-filter-top - [0:0] >> >> -A INPUT -j nova-compute-INPUT >> >> -A FORWARD -j nova-filter-top >> >> -A FORWARD -j nova-compute-FORWARD >> >> -A OUTPUT -j nova-filter-top >> >> -A OUTPUT -j nova-compute-OUTPUT >> >> -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m >> udp --sport 68 --dport 67 -j ACCEPT >> >> -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m >> udp --sport 68 --dport 67 -j ACCEPT >> >> -A nova-compute-inst-26 -m state --state INVALID -j DROP >> >> -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT >> >> -A nova-compute-inst-26 -j nova-compute-provider >> >> -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 >> --dport 68 -j ACCEPT >> >> -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT >> >> -A nova-compute-inst-26 -p icmp -j ACCEPT >> >> -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT >> >> -A nova-compute-inst-26 -j nova-compute-sg-fallback >> >> -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 >> >> -A nova-compute-sg-fallback -j DROP >> >> -A nova-filter-top -j nova-compute-local >> >> COMMIT >> >> # Completed on Sun Apr 28 16:37:18 2013 >> >> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >> >> *mangle >> >> :PREROUTING ACCEPT [22733:3519752] >> >> :INPUT ACCEPT [22733:3519752] >> >> :FORWARD ACCEPT [175:50468] >> >> :OUTPUT ACCEPT [22705:5868566] >> >> :POSTROUTING ACCEPT [22880:5919034] >> >> :nova-compute-POSTROUTING - [0:0] >> >> -A POSTROUTING -j nova-compute-POSTROUTING >> >> COMMIT >> >> # Completed on Sun Apr 28 16:37:18 2013 >> >> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >> >> *nat >> >> :PREROUTING ACCEPT [16:14570] >> >> :POSTROUTING ACCEPT [338:22855] >> >> :OUTPUT ACCEPT [331:20579] >> >> :nova-compute-OUTPUT - [0:0] >> >> :nova-compute-POSTROUTING - [0:0] >> >> :nova-compute-PREROUTING - [0:0] >> >> :nova-compute-float-snat - [0:0] >> >> :nova-compute-snat - [0:0] >> >> :nova-postrouting-bottom - [0:0] >> >> -A PREROUTING -j nova-compute-PREROUTING >> >> -A POSTROUTING -j nova-compute-POSTROUTING >> >> -A POSTROUTING -j nova-postrouting-bottom >> >> -A OUTPUT -j nova-compute-OUTPUT >> >> -A nova-compute-snat -j nova-compute-float-snat >> >> -A nova-postrouting-bottom -j nova-compute-snat >> >> COMMIT >> >> # Completed on Sun Apr 28 16:37:18 2013 >> >> >> 3. Can you please print the flow table rules (ovs-dpctl dump-flows >> br-int)? >> >> I suppose you mean ovs-ofctl dump-flows br-int ? >> >> [root at computer-2 ~]# ovs-ofctl dump-flows br-int >> >> NXST_FLOW reply (xid=0x4): >> >> cookie=0x0, duration=4125.444s, table=0, n_packets=1707, >> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL >> >> cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, >> idle_age=20, priority=2,in_port=1 actions=drop >> >> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, >> idle_age=3349, priority=3,in_port=1,dl_vlan=1001 >> actions=mod_vlan_vid:1,NORMAL >> >> Here's also ovs-dpctl show: >> >> [root at computer-2 ~]# ovs-dpctl show >> >> system at br-p3p1: >> >> lookups: hit:3967 missed:314 lost:0 >> >> flows: 1 >> >> port 0: br-p3p1 (internal) >> >> port 1: p3p1 >> >> port 2: phy-br-p3p1 >> >> system at br-int: >> >> lookups: hit:1575 missed:302 lost:0 >> >> flows: 0 >> >> port 0: br-int (internal) >> >> port 1: int-br-p3p1 >> >> port 4: qvo39242f22-ec >> >> >> Thanks >> Gary >> >> On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi,Gary >> >> I tried capture packet while keeping VM to restart it's network. >> >> I can see dhcp request broadcast packet on tap, qbr, qvb and qvo >> interfaces. >> >> Failed to see packet on int-br-p3p1 on bridge br-int. >> >> Not sure if it has something to do with openflow setting? I attach >> some ovs-ofctl outputs >> >> I have not seen "veth" port anywhere... >> >> ---Record--- >> >> [root at computer-2 ~]# brctl show >> >> bridge name bridge id STP enabled interfaces >> >> qbr39242f22-ec 8000.c6f95e6a859a no >> qvb39242f22-ec >> >> tap39242f22-ec >> >> virbr0 8000.525400c47f62 yes virbr0-nic >> >> [root at computer-2 ~]# ovs-vsctl show >> >> 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b >> >> Bridge br-int >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port "qvo39242f22-ec" >> >> tag: 1 >> >> Interface "qvo39242f22-ec" >> >> Bridge "br-p3p1" >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> ovs_version: "1.9.0" >> >> [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 >> >> tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 2 packets captured >> >> 2 packets received by filter >> >> 0 packets dropped by kernel >> >> [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 >> >> tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 1 packets captured >> >> 1 packets received by filter >> >> 0 packets dropped by kernel >> >> [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 >> >> tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 1 packets captured >> >> 1 packets received by filter >> >> 0 packets dropped by kernel >> >> [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 >> >> tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture >> size 65535 bytes >> >> 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >> >> ^C >> >> 2 packets captured >> >> 2 packets received by filter >> >> 0 packets dropped by kernel >> >> [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 >> >> tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned >> >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> >> listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size >> 65535 bytes >> >> ^C >> >> 0 packets captured >> >> 0 packets received by filter >> >> 0 packets dropped by kernel >> >> ---output of ovs-ofctl--- >> >> [root at computer-2 ~]# ovs-ofctl show br-int >> >> OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 >> >> n_tables:255, n_buffers:256 >> >> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >> >> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC >> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE >> >> 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD COPPER >> >> speed: 10000 Mbps now, 100 Mbps max >> >> 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD COPPER >> >> speed: 10000 Mbps now, 100 Mbps max >> >> LOCAL(br-int): addr:86:40:18:20:f1:42 >> >> config: PORT_DOWN >> >> state: LINK_DOWN >> >> speed: 100 Mbps now, 100 Mbps max >> >> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 >> >> [root at computer-2 ~]# >> >> [root at computer-2 ~]# ovs-ofctl show br-p3p1 >> >> OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 >> >> n_tables:255, n_buffers:256 >> >> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >> >> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC >> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE >> >> 1(p3p1): addr:a0:36:9f:15:d4:24 >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD >> >> advertised: 10GB-FD FIBER >> >> supported: 10GB-FD FIBER >> >> speed: 10000 Mbps now, 10000 Mbps max >> >> 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 >> >> config: 0 >> >> state: 0 >> >> current: 10GB-FD COPPER >> >> speed: 10000 Mbps now, 100 Mbps max >> >> LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 >> >> config: PORT_DOWN >> >> state: LINK_DOWN >> >> speed: 100 Mbps now, 100 Mbps max >> >> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 >> >> [root at computer-2 ~]# ovs-ofctl dump-flows br-int >> >> NXST_FLOW reply (xid=0x4): >> >> cookie=0x0, duration=4125.444s, table=0, n_packets=1707, >> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL >> >> cookie=0x0, duration=4123.006s, table=0, n_packets=143, n_bytes=8688, >> idle_age=20, priority=2,in_port=1 actions=drop >> >> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, >> idle_age=3349, priority=3,in_port=1,dl_vlan=1001 >> actions=mod_vlan_vid:1,NORMAL >> >> [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 >> >> NXST_FLOW reply (xid=0x4): >> >> cookie=0x0, duration=4129.629s, table=0, n_packets=2175, >> n_bytes=138652, idle_age=0, priority=1 actions=NORMAL >> >> cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, >> idle_age=1045, priority=2,in_port=2 actions=drop >> >> cookie=0x0, duration=3354.578s, table=0, n_packets=1697, >> n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 >> actions=mod_vlan_vid:1001,NORMAL >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*rdo-list-bounces at redhat.com >> >> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Zhang, Kimi >> (NSN - CN/Cheng Du) >> *Sent:* Sunday, April 28, 2013 3:40 PM >> *To:* gkotton at redhat.com ; >> rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Very nice pic, I am going to try to capture packet on each port. >> >> I did not configure to use quantum to manage firewall , just leave it >> to nova-compute, will try your configs later. >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*rdo-list-bounces at redhat.com >> >> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton >> *Sent:* Sunday, April 28, 2013 3:33 PM >> *To:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi, >> Can you also please check that firewall_driver = >> quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver >> is configured in plugin.ini file.And security_group_api = quantum is >> set in nova.conf >> Thanks >> Gary >> >> On 04/28/2013 10:21 AM, Gary Kotton wrote: >> >> On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi, Gary >> >> I tried to disable iptables on both network and compute nodes, still >> does not work out L >> >> >> Can you please look at >> https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing >> >> When using the OVS there are a number of devices. Would it be >> possible that you try and capture on each device so that we can try >> and see where the packet is discarded. >> >> I will have a setup ready in about an hour. >> >> >> >> From quantum openvswitch agent logs, following messages keeps coming >> out repeatly every 2-3 seconds, not sure if they matter or not? >> >> >> The messages below are OK - this is how the OVS agent works. It polls >> the OVS every interval to check if new ports are created. >> >> >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> Making synchronous call on q-plugin ... >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 >> >> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >> UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >> >> Exit code: 0 >> >> Stdout: 'int-br-p3p1\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >> >> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >> 'external_ids'] >> >> Exit code: 0 >> >> Stdout: '{}\n' >> >> Stderr: '' >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >> *Sent:* Sunday, April 28, 2013 3:08 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> I tried that too, no lucky. >> >> From tcpdump ,it seems br-int does not forward any packet to >> interfaces connect to br-p3p1, which connects to physical network... >> >> >> There could be a number of issues here: >> 1. The iptables are dropping the traffic (I am in the process of >> getting a setup up and running) >> 2. The network connectivity >> >> In order to ensure that it is not the first one can you try and see >> which iptables rules are matched or disable the iptables? >> >> >> >> >> Regards, >> >> Kimi Zhang >> >> MP: +86 186 0800 8182 >> >> Call me(NCS): sip:+86018608008182 >> >> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >> *Sent:* Sunday, April 28, 2013 3:01 PM >> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >> *Cc:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> Hi, Gary >> >> Yes, I'm aware of that packstack does not support quantum yet. The >> whole setup was installed manually. >> >> I did run quantum-server-setup and quantum-host-setup, I tried >> linuxbridge plugin too, it has no issue for VM to get IP address, but >> openvswitch has issues on this... >> >> >> ok. >> >> if you configure and IP address manually on the VM are you able to >> ping the port of the DHCP agent? >> >> you can get the IP from quantum port-list >> >> >> >> >> >> >> Regards, >> >> Kimi >> >> *From:*rdo-list-bounces at redhat.com >> >> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton >> *Sent:* Sunday, April 28, 2013 2:50 PM >> *To:* rdo-list at redhat.com >> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >> Openvswitch + Vlan >> >> Hi Kimi, >> Thanks for the mail. Please see the inline comments below. Please >> note that at the moment we do not have packstack support for Quantum >> so there is a little manual plumbing that needs to be done (not sure >> if you have done this already). >> On the host where the quantum service is running you need to run >> quantum-server-setup and on the compute nodes you need to run >> quantum-host-setup (please note that the relevant keystone >> credentials need to be set too). >> Thanks >> Gary >> >> On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >> >> converted from rtf >> >> When I start VM instance, the VM can't get IP address. Could someone >> help me on this ? >> >> >> I will try >> >> >> >> >> >> 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. >> >> ?Controller node: >> >> Services: Keystone+Glance+Cinder+Quantum server + Nova services >> >> Network: bond0(10.68.125.11 for O&M) >> >> ?Network node: >> >> Services: quantum-openvswitch-agent, quantum-l3-agent, >> quantum-dhcp-agent, quantum-metadata-agent >> >> Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, >> p3p2 for external network >> >> >> Please note that RHEL currently does not support namespaces so there >> are a number of limitations. We are addressing this at the moment. If >> namespaces are not used then it is suggested that one does not run >> the DHCP agent and the L3 agent on the same host. The reason for this >> is that there is no network isolation. >> >> >> >> >> >> >> ?Compute node: >> >> Services: nove-compute and quantum-openvswitch-agent >> >> Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network >> >> ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) >> of network and compute nodes. >> >> 1.Quantum.conf: >> >> [DEFAULT] >> >> debug = True >> >> verbose = True >> >> lock_path = $state_path/lock >> >> bind_host = 0.0.0.0 >> >> bind_port = 9696 >> >> core_plugin = >> quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 >> >> api_paste_config = api-paste.ini >> >> rpc_backend = quantum.openstack.common.rpc.impl_kombu >> >> >> Are you using rabbit or qpid? >> >> >> >> >> >> >> control_exchange = quantum >> >> rabbit_host = 10.68.125.11 >> >> notification_driver = quantum.openstack.common.notifier.rpc_notifier >> >> default_notification_level = INFO >> >> notification_topics = notifications >> >> [QUOTAS] >> >> [DEFAULT_SERVICETYPE] >> >> [AGENT] >> >> polling_interval = 2 >> >> root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf >> >> [keystone_authtoken] >> >> auth_host = 10.68.125.11 >> >> auth_port = 35357 >> >> auth_protocol = http >> >> signing_dir = /var/lib/quantum/keystone-signing >> >> admin_tenant_name = service >> >> admin_user = quantum >> >> admin_password = password >> >> 2.ovs_quantum_plugin.ini >> >> [DATABASE] >> >> sql_connection = >> mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum >> >> >> reconnect_interval = 2 >> >> [OVS] >> >> tenant_network_type = vlan >> >> network_vlan_ranges = physnet1:1000:2999 >> >> bridge_mappings = physnet1:br-p3p1 >> >> [AGENT] >> >> polling_interval = 2 >> >> [SECURITYGROUP] >> >> 3.nova.conf >> >> [DEFAULT] >> >> verbose=true >> >> logdir = /var/log/nova >> >> state_path = /var/lib/nova >> >> lock_path = /var/lib/nova/tmp >> >> volumes_dir = /etc/nova/volumes >> >> dhcpbridge = /usr/bin/nova-dhcpbridge >> >> dhcpbridge_flagfile = /etc/nova/nova.conf >> >> force_dhcp_release = True >> >> injected_network_template = /usr/share/nova/interfaces.template >> >> libvirt_nonblocking = True >> >> libvirt_inject_partition = -1 >> >> network_manager = nova.network.manager.FlatDHCPManager >> >> iscsi_helper = tgtadm >> >> compute_driver = libvirt.LibvirtDriver >> >> libvirt_type=kvm >> >> libvirt_ovs_bridge=br-int >> >> firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver >> >> manager=nova.conductor.manager.ConductorManager >> >> rpc_backend = nova.openstack.common.rpc.impl_kombu >> >> rabbit_host = 10.68.125.11 >> >> rootwrap_config = /etc/nova/rootwrap.conf >> >> use_deprecated_auth=false >> >> auth_strategy=keystone >> >> glance_api_servers=10.68.125.11:9292 >> >> image_service=nova.image.glance.GlanceImageService >> >> novnc_enabled=true >> >> novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html >> >> novncproxy_port=6080 >> >> vncserver_proxyclient_address=10.68.125.16 >> >> vncserver_listen=0.0.0.0 >> >> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver >> >> libvirt_use_virtio_for_bridges=True >> >> network_api_class=nova.network.quantumv2.api.API >> >> quantum_url=http://10.68.125.11:9696 >> >> quantum_auth_strategy=keystone >> >> quantum_admin_tenant_name=service >> >> quantum_admin_username=quantum >> >> quantum_admin_password=password >> >> quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 >> >> linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver >> >> libvirt_vif_type=ethernet >> >> service_quantum_metadata_proxy = True >> >> quantum_metadata_proxy_shared_secret = helloOpenStack >> >> metadata_host = 10.68.125.11 >> >> metadata_listen = 0.0.0.0 >> >> metadata_listen_port = 8775 >> >> [keystone_authtoken] >> >> admin_tenant_name = service >> >> admin_user = nova >> >> admin_password = password >> >> auth_host = 10.68.125.11 >> >> auth_port = 35357 >> >> auth_protocol = http >> >> signing_dir = /tmp/keystone-signing-nova >> >> 4.ovs-vsctl show on network node: >> >> aeeb6cf7-271b-405a-aa17-1b95bcd9e301 >> >> Bridge "br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> Bridge br-ex >> >> Port br-ex >> >> Interface br-ex >> >> type: internal >> >> Port "qg-a83c0abd-f4" >> >> Interface "qg-a83c0abd-f4" >> >> type: internal >> >> Port "p3p2" >> >> Interface "p3p2" >> >> Bridge br-int >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port "tap1f386a2a-12" >> >> tag: 1 >> >> Interface "tap1f386a2a-12" >> >> type: internal >> >> ovs_version: "1.9.0" >> >> 5.ovs-vsctl show on compute node: >> >> 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 >> >> Bridge "br-p3p1" >> >> Port "br-p3p1" >> >> Interface "br-p3p1" >> >> type: internal >> >> Port "phy-br-p3p1" >> >> Interface "phy-br-p3p1" >> >> Port "p3p1" >> >> Interface "p3p1" >> >> Bridge br-int >> >> Port "qvo56a4572c-dc" >> >> tag: 2 >> >> Interface "qvo56a4572c-dc" >> >> Port "int-br-p3p1" >> >> Interface "int-br-p3p1" >> >> Port br-int >> >> Interface br-int >> >> type: internal >> >> ovs_version: "1.9.0" >> >> On compute node, I can see dhcp request packet from tcpdump on >> qvo56a4572c-dc, but it seems the packet is not forwarded out since I >> can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. >> >> >> Any chance to get the DHCP and the L3 agent configuration files? >> Please check that use_namespaces = False in both of these files. >> >> Are there any log errors? >> >> >> >> >> >> >> Thank you! >> >> Regards, >> >> Kimi >> >> >> >> >> >> >> >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rdo-list >> >> >> >> >> >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rdo-list >> > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From gkotton at redhat.com Sun Apr 28 13:47:30 2013 From: gkotton at redhat.com (Gary Kotton) Date: Sun, 28 Apr 2013 16:47:30 +0300 Subject: [Rdo-list] [Grizzly] Network problem with Quantum + Openvswitch + Vlan In-Reply-To: <517D003B.6030802@redhat.com> References: <90CF2062F86FD8498897037C7FBBC088046F52@SGSIMBX001.nsn-intra.net> <517CC6A4.9040201@redhat.com> <90CF2062F86FD8498897037C7FBBC088046F91@SGSIMBX001.nsn-intra.net> <517CC934.4070809@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FC9@SGSIMBX001.nsn-intra.net> <517CCACE.8000203@redhat.com> <90CF2062F86FD8498897037C7FBBC088046FEE@SGSIMBX001.nsn-intra.net> <517CCE14.2040105@redhat.com> <517CD09C.9080609@redhat.com> <90CF2062F86FD8498897037C7FBBC08804705E@SGSIMBX001.nsn-intra.net> <90CF2062F86FD8498897037C7FBBC0880470B0@SGSIMBX001.nsn-intra.net> <517CDE34.1080506@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470D8@SGSIMBX001.nsn-intra.net> <517CE212.2030000@redhat.com> <90CF2062F86FD8498897037C7FBBC0880470FB@SGSIMBX001.nsn-intra.net> <517CEB42.1010005@redhat.com> <517D003B.6030802@redhat.com> Message-ID: <517D2872.9000401@redhat.com> Hi, In addition to that I have discovered that there are the following: [root at dhcp-4-126 ~]# ps aux |grep dns nobody 2320 0.0 0.0 12888 576 ? S 09:31 0:00 /usr/sbin/dnsmasq --strict-order --local=// --domain-needed --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --bind-interfaces --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override --dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile --addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts nobody 2718 0.0 0.0 12884 600 ? S 09:32 0:00 /usr/sbin/dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tapd0ed5836-38 --except-interface=lo --pid-file=/var/lib/quantum/dhcp/45f9b635-c996-4230-89df-b8c6ac1adb71/pid --dhcp-hostsfile=/var/lib/quantum/dhcp/45f9b635-c996-4230-89df-b8c6ac1adb71/host --dhcp-optsfile=/var/lib/quantum/dhcp/45f9b635-c996-4230-89df-b8c6ac1adb71/opts --dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s --conf-file= --domain=openstacklocal root 2719 0.0 0.0 12884 208 ? S 09:32 0:00 /usr/sbin/dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tapd0ed5836-38 --except-interface=lo --pid-file=/var/lib/quantum/dhcp/45f9b635-c996-4230-89df-b8c6ac1adb71/pid --dhcp-hostsfile=/var/lib/quantum/dhcp/45f9b635-c996-4230-89df-b8c6ac1adb71/host --dhcp-optsfile=/var/lib/quantum/dhcp/45f9b635-c996-4230-89df-b8c6ac1adb71/opts --dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s --conf-file= --domain=openstacklocal root 6054 0.0 0.0 103248 840 pts/0 S+ 09:39 0:00 grep dns When the process 2320 is killed the VM receives its address. So in short we have some hardening to do :) With patches for the issues below and shutting down the aforementioned process I have a VM getting a address. Thanks Gary On 04/28/2013 01:55 PM, Gary Kotton wrote: > Hi, > I have found a few problems and hopefully one or more may be related > to the case that you have experienced: > 1. When using OVS it is important you run the service > ovs-quantum-cleanup when the host boots. This is due to the fact that > OVS will store all tap device. This causes havoc when restarting hosts > (in particular ones that have dhcp and l3 agents). So please make sure > you have run "chkconfig quantum-ovs-cleanup on" on all hosts that are > running the OVS. You can verify if this is the case by checking of the > DHCP agent has created an IP address on the host. [Please note that we > have a problem here - in the file /etc/init.d/quantum-ovs-cleanup > "--config-file /usr/share/$proj/$proj-dist.conf" needs to be removed]. > 2. Which dnsmasq version are you using? If this is 2.48 then there is > a problem with the DHCP agent running. We are in the process of > resolving this. If you make use of a version with tag support then > this will work. > 3. The quantum client needs to be updated to support the security groups. > Hopefully we will have solutions for all of the above ASAP. > Thanks > Gary > > > > On 04/28/2013 12:26 PM, Gary Kotton wrote: >> Hi, >> I have been able to reproduce the problem. I'll get back to you as >> soon as I have any information. >> Thanks >> Gary >> >> On 04/28/2013 11:56 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> Yes, I did run quantum-dhcp-setup on network node. >>> >>> Thanks, good luck there. >>> >>> Regards, >>> >>> Kimi Zhang >>> >>> MP: +86 186 0800 8182 >>> >>> Call me(NCS): sip:+86018608008182 >>> >>> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >>> *Sent:* Sunday, April 28, 2013 4:47 PM >>> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >>> *Cc:* rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> Thanks. >>> One more question - on the network node, did you run quantum-dhcp-setup? >>> I am nearly ready with my setup. Hopefully I'll have a reproduction >>> or some additional questions. >>> Thanks >>> Gary >>> >>> On 04/28/2013 11:41 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> Sure, my answers below. :) >>> >>> Regards, >>> >>> Kimi Zhang >>> >>> MP: +86 186 0800 8182 >>> >>> Call me(NCS): sip:+86018608008182 >>> >>> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >>> *Sent:* Sunday, April 28, 2013 4:31 PM >>> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >>> *Cc:* rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> Hi, >>> I have a few questions (please be patient with me): >>> 1. On the compute node, which services are running? >>> >>> nova-compute, nova-novncproxy, quantum-openvswitch-agent, >>> openvswitch >>> >>> >>> 2. Can you please print the iptables on the compute node? >>> >>> I disabled it already, here's output before I do it. >>> >>> [root at computer-2 ~]# iptables-save >>> >>> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >>> >>> *filter >>> >>> :INPUT ACCEPT [22634:3487580] >>> >>> :FORWARD ACCEPT [22:704] >>> >>> :OUTPUT ACCEPT [22619:5860198] >>> >>> :nova-compute-FORWARD - [0:0] >>> >>> :nova-compute-INPUT - [0:0] >>> >>> :nova-compute-OUTPUT - [0:0] >>> >>> :nova-compute-inst-26 - [0:0] >>> >>> :nova-compute-local - [0:0] >>> >>> :nova-compute-provider - [0:0] >>> >>> :nova-compute-sg-fallback - [0:0] >>> >>> :nova-filter-top - [0:0] >>> >>> -A INPUT -j nova-compute-INPUT >>> >>> -A FORWARD -j nova-filter-top >>> >>> -A FORWARD -j nova-compute-FORWARD >>> >>> -A OUTPUT -j nova-filter-top >>> >>> -A OUTPUT -j nova-compute-OUTPUT >>> >>> -A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp >>> -m udp --sport 68 --dport 67 -j ACCEPT >>> >>> -A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m >>> udp --sport 68 --dport 67 -j ACCEPT >>> >>> -A nova-compute-inst-26 -m state --state INVALID -j DROP >>> >>> -A nova-compute-inst-26 -m state --state RELATED,ESTABLISHED -j ACCEPT >>> >>> -A nova-compute-inst-26 -j nova-compute-provider >>> >>> -A nova-compute-inst-26 -s 172.1.1.3/32 -p udp -m udp --sport 67 >>> --dport 68 -j ACCEPT >>> >>> -A nova-compute-inst-26 -s 172.1.1.0/24 -j ACCEPT >>> >>> -A nova-compute-inst-26 -p icmp -j ACCEPT >>> >>> -A nova-compute-inst-26 -p tcp -m tcp --dport 22 -j ACCEPT >>> >>> -A nova-compute-inst-26 -j nova-compute-sg-fallback >>> >>> -A nova-compute-local -d 172.1.1.5/32 -j nova-compute-inst-26 >>> >>> -A nova-compute-sg-fallback -j DROP >>> >>> -A nova-filter-top -j nova-compute-local >>> >>> COMMIT >>> >>> # Completed on Sun Apr 28 16:37:18 2013 >>> >>> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >>> >>> *mangle >>> >>> :PREROUTING ACCEPT [22733:3519752] >>> >>> :INPUT ACCEPT [22733:3519752] >>> >>> :FORWARD ACCEPT [175:50468] >>> >>> :OUTPUT ACCEPT [22705:5868566] >>> >>> :POSTROUTING ACCEPT [22880:5919034] >>> >>> :nova-compute-POSTROUTING - [0:0] >>> >>> -A POSTROUTING -j nova-compute-POSTROUTING >>> >>> COMMIT >>> >>> # Completed on Sun Apr 28 16:37:18 2013 >>> >>> # Generated by iptables-save v1.4.7 on Sun Apr 28 16:37:18 2013 >>> >>> *nat >>> >>> :PREROUTING ACCEPT [16:14570] >>> >>> :POSTROUTING ACCEPT [338:22855] >>> >>> :OUTPUT ACCEPT [331:20579] >>> >>> :nova-compute-OUTPUT - [0:0] >>> >>> :nova-compute-POSTROUTING - [0:0] >>> >>> :nova-compute-PREROUTING - [0:0] >>> >>> :nova-compute-float-snat - [0:0] >>> >>> :nova-compute-snat - [0:0] >>> >>> :nova-postrouting-bottom - [0:0] >>> >>> -A PREROUTING -j nova-compute-PREROUTING >>> >>> -A POSTROUTING -j nova-compute-POSTROUTING >>> >>> -A POSTROUTING -j nova-postrouting-bottom >>> >>> -A OUTPUT -j nova-compute-OUTPUT >>> >>> -A nova-compute-snat -j nova-compute-float-snat >>> >>> -A nova-postrouting-bottom -j nova-compute-snat >>> >>> COMMIT >>> >>> # Completed on Sun Apr 28 16:37:18 2013 >>> >>> >>> 3. Can you please print the flow table rules (ovs-dpctl dump-flows >>> br-int)? >>> >>> I suppose you mean ovs-ofctl dump-flows br-int ? >>> >>> [root at computer-2 ~]# ovs-ofctl dump-flows br-int >>> >>> NXST_FLOW reply (xid=0x4): >>> >>> cookie=0x0, duration=4125.444s, table=0, n_packets=1707, >>> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL >>> >>> cookie=0x0, duration=4123.006s, table=0, n_packets=143, >>> n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop >>> >>> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, >>> idle_age=3349, priority=3,in_port=1,dl_vlan=1001 >>> actions=mod_vlan_vid:1,NORMAL >>> >>> Here's also ovs-dpctl show: >>> >>> [root at computer-2 ~]# ovs-dpctl show >>> >>> system at br-p3p1: >>> >>> lookups: hit:3967 missed:314 lost:0 >>> >>> flows: 1 >>> >>> port 0: br-p3p1 (internal) >>> >>> port 1: p3p1 >>> >>> port 2: phy-br-p3p1 >>> >>> system at br-int: >>> >>> lookups: hit:1575 missed:302 lost:0 >>> >>> flows: 0 >>> >>> port 0: br-int (internal) >>> >>> port 1: int-br-p3p1 >>> >>> port 4: qvo39242f22-ec >>> >>> >>> Thanks >>> Gary >>> >>> On 04/28/2013 11:17 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> Hi,Gary >>> >>> I tried capture packet while keeping VM to restart it's network. >>> >>> I can see dhcp request broadcast packet on tap, qbr, qvb and qvo >>> interfaces. >>> >>> Failed to see packet on int-br-p3p1 on bridge br-int. >>> >>> Not sure if it has something to do with openflow setting? I attach >>> some ovs-ofctl outputs >>> >>> I have not seen "veth" port anywhere... >>> >>> ---Record--- >>> >>> [root at computer-2 ~]# brctl show >>> >>> bridge name bridge id STP enabled interfaces >>> >>> qbr39242f22-ec 8000.c6f95e6a859a no >>> qvb39242f22-ec >>> >>> tap39242f22-ec >>> >>> virbr0 8000.525400c47f62 yes virbr0-nic >>> >>> [root at computer-2 ~]# ovs-vsctl show >>> >>> 5660d1b5-1f26-46fc-bcb7-0ccfd06fe57b >>> >>> Bridge br-int >>> >>> Port br-int >>> >>> Interface br-int >>> >>> type: internal >>> >>> Port "int-br-p3p1" >>> >>> Interface "int-br-p3p1" >>> >>> Port "qvo39242f22-ec" >>> >>> tag: 1 >>> >>> Interface "qvo39242f22-ec" >>> >>> Bridge "br-p3p1" >>> >>> Port "phy-br-p3p1" >>> >>> Interface "phy-br-p3p1" >>> >>> Port "p3p1" >>> >>> Interface "p3p1" >>> >>> Port "br-p3p1" >>> >>> Interface "br-p3p1" >>> >>> type: internal >>> >>> ovs_version: "1.9.0" >>> >>> [root at computer-2 ~]# tcpdump -i tap39242f22-ec port 67 >>> >>> tcpdump: WARNING: tap39242f22-ec: no IPv4 address assigned >>> >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> >>> listening on tap39242f22-ec, link-type EN10MB (Ethernet), capture >>> size 65535 bytes >>> >>> 16:12:21.455212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >>> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >>> >>> 16:12:21.455289 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >>> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >>> >>> ^C >>> >>> 2 packets captured >>> >>> 2 packets received by filter >>> >>> 0 packets dropped by kernel >>> >>> [root at computer-2 ~]# tcpdump -i qbr39242f22-ec port 67 >>> >>> tcpdump: WARNING: qbr39242f22-ec: no IPv4 address assigned >>> >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> >>> listening on qbr39242f22-ec, link-type EN10MB (Ethernet), capture >>> size 65535 bytes >>> >>> 16:12:34.456228 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >>> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >>> >>> ^C >>> >>> 1 packets captured >>> >>> 1 packets received by filter >>> >>> 0 packets dropped by kernel >>> >>> [root at computer-2 ~]# tcpdump -i qvb39242f22-ec port 67 >>> >>> tcpdump: WARNING: qvb39242f22-ec: no IPv4 address assigned >>> >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> >>> listening on qvb39242f22-ec, link-type EN10MB (Ethernet), capture >>> size 65535 bytes >>> >>> 16:12:43.460251 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >>> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >>> >>> ^C >>> >>> 1 packets captured >>> >>> 1 packets received by filter >>> >>> 0 packets dropped by kernel >>> >>> [root at computer-2 ~]# tcpdump -i qvo39242f22-ec port 67 >>> >>> tcpdump: WARNING: qvo39242f22-ec: no IPv4 address assigned >>> >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> >>> listening on qvo39242f22-ec, link-type EN10MB (Ethernet), capture >>> size 65535 bytes >>> >>> 16:13:03.712272 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >>> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >>> >>> 16:13:08.455932 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: >>> BOOTP/DHCP, Request from fa:16:3e:15:82:82 (oui Unknown), length 300 >>> >>> ^C >>> >>> 2 packets captured >>> >>> 2 packets received by filter >>> >>> 0 packets dropped by kernel >>> >>> [root at computer-2 ~]# tcpdump -i int-br-p3p1 port 67 >>> >>> tcpdump: WARNING: int-br-p3p1: no IPv4 address assigned >>> >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> >>> listening on int-br-p3p1, link-type EN10MB (Ethernet), capture size >>> 65535 bytes >>> >>> ^C >>> >>> 0 packets captured >>> >>> 0 packets received by filter >>> >>> 0 packets dropped by kernel >>> >>> ---output of ovs-ofctl--- >>> >>> [root at computer-2 ~]# ovs-ofctl show br-int >>> >>> OFPT_FEATURES_REPLY (xid=0x1): dpid:000086401820f142 >>> >>> n_tables:255, n_buffers:256 >>> >>> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >>> >>> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC >>> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST >>> ENQUEUE >>> >>> 1(int-br-p3p1): addr:de:42:e4:9d:b7:1d >>> >>> config: 0 >>> >>> state: 0 >>> >>> current: 10GB-FD COPPER >>> >>> speed: 10000 Mbps now, 100 Mbps max >>> >>> 4(qvo39242f22-ec): addr:ea:5d:b8:7e:4a:78 >>> >>> config: 0 >>> >>> state: 0 >>> >>> current: 10GB-FD COPPER >>> >>> speed: 10000 Mbps now, 100 Mbps max >>> >>> LOCAL(br-int): addr:86:40:18:20:f1:42 >>> >>> config: PORT_DOWN >>> >>> state: LINK_DOWN >>> >>> speed: 100 Mbps now, 100 Mbps max >>> >>> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 >>> >>> [root at computer-2 ~]# >>> >>> [root at computer-2 ~]# ovs-ofctl show br-p3p1 >>> >>> OFPT_FEATURES_REPLY (xid=0x1): dpid:0000a0369f15d424 >>> >>> n_tables:255, n_buffers:256 >>> >>> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP >>> >>> actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC >>> SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST >>> ENQUEUE >>> >>> 1(p3p1): addr:a0:36:9f:15:d4:24 >>> >>> config: 0 >>> >>> state: 0 >>> >>> current: 10GB-FD >>> >>> advertised: 10GB-FD FIBER >>> >>> supported: 10GB-FD FIBER >>> >>> speed: 10000 Mbps now, 10000 Mbps max >>> >>> 2(phy-br-p3p1): addr:be:3c:f9:8d:d9:d0 >>> >>> config: 0 >>> >>> state: 0 >>> >>> current: 10GB-FD COPPER >>> >>> speed: 10000 Mbps now, 100 Mbps max >>> >>> LOCAL(br-p3p1): addr:a0:36:9f:15:d4:24 >>> >>> config: PORT_DOWN >>> >>> state: LINK_DOWN >>> >>> speed: 100 Mbps now, 100 Mbps max >>> >>> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0 >>> >>> [root at computer-2 ~]# ovs-ofctl dump-flows br-int >>> >>> NXST_FLOW reply (xid=0x4): >>> >>> cookie=0x0, duration=4125.444s, table=0, n_packets=1707, >>> n_bytes=90606, idle_age=12, priority=1 actions=NORMAL >>> >>> cookie=0x0, duration=4123.006s, table=0, n_packets=143, >>> n_bytes=8688, idle_age=20, priority=2,in_port=1 actions=drop >>> >>> cookie=0x0, duration=3349.566s, table=0, n_packets=0, n_bytes=0, >>> idle_age=3349, priority=3,in_port=1,dl_vlan=1001 >>> actions=mod_vlan_vid:1,NORMAL >>> >>> [root at computer-2 ~]# ovs-ofctl dump-flows br-p3p1 >>> >>> NXST_FLOW reply (xid=0x4): >>> >>> cookie=0x0, duration=4129.629s, table=0, n_packets=2175, >>> n_bytes=138652, idle_age=0, priority=1 actions=NORMAL >>> >>> cookie=0x0, duration=4127.415s, table=0, n_packets=16, n_bytes=1224, >>> idle_age=1045, priority=2,in_port=2 actions=drop >>> >>> cookie=0x0, duration=3354.578s, table=0, n_packets=1697, >>> n_bytes=96638, idle_age=17, priority=4,in_port=2,dl_vlan=1 >>> actions=mod_vlan_vid:1001,NORMAL >>> >>> Regards, >>> >>> Kimi Zhang >>> >>> MP: +86 186 0800 8182 >>> >>> Call me(NCS): sip:+86018608008182 >>> >>> *From:*rdo-list-bounces at redhat.com >>> >>> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Zhang, Kimi >>> (NSN - CN/Cheng Du) >>> *Sent:* Sunday, April 28, 2013 3:40 PM >>> *To:* gkotton at redhat.com ; >>> rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> Very nice pic, I am going to try to capture packet on each port. >>> >>> I did not configure to use quantum to manage firewall , just leave >>> it to nova-compute, will try your configs later. >>> >>> Regards, >>> >>> Kimi Zhang >>> >>> MP: +86 186 0800 8182 >>> >>> Call me(NCS): sip:+86018608008182 >>> >>> *From:*rdo-list-bounces at redhat.com >>> >>> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton >>> *Sent:* Sunday, April 28, 2013 3:33 PM >>> *To:* rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> Hi, >>> Can you also please check that firewall_driver = >>> quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is >>> configured in plugin.ini file.And security_group_api = quantum is >>> set in nova.conf >>> Thanks >>> Gary >>> >>> On 04/28/2013 10:21 AM, Gary Kotton wrote: >>> >>> On 04/28/2013 10:16 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> Hi, Gary >>> >>> I tried to disable iptables on both network and compute nodes, still >>> does not work out L >>> >>> >>> Can you please look at >>> https://docs.google.com/drawings/d/1wax2Nlk-LRJeOXwF_6X9L05cAf9HKl2FI_0B51rG4XE/edit?usp=sharing >>> >>> When using the OVS there are a number of devices. Would it be >>> possible that you try and capture on each device so that we can try >>> and see where the packet is discarded. >>> >>> I will have a setup ready in about an hour. >>> >>> >>> >>> From quantum openvswitch agent logs, following messages keeps coming >>> out repeatly every 2-3 seconds, not sure if they matter or not? >>> >>> >>> The messages below are OK - this is how the OVS agent works. It >>> polls the OVS every interval to check if new ports are created. >>> >>> >>> >>> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >>> Making synchronous call on q-plugin ... >>> >>> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >>> MSG_ID is 92f4e83cf92c46f1b9304c879f9b7a41 >>> >>> 2013-04-28 15:15:39 DEBUG [quantum.openstack.common.rpc.amqp] >>> UNIQUE_ID is b27f9545ca9d4745961ac574abdc103b. >>> >>> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >>> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >>> >>> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] >>> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >>> >>> Exit code: 0 >>> >>> Stdout: 'int-br-p3p1\n' >>> >>> Stderr: '' >>> >>> 2013-04-28 15:15:40 DEBUG [quantum.agent.linux.utils] Running >>> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >>> 'external_ids'] >>> >>> 2013-04-28 15:15:41 DEBUG [quantum.agent.linux.utils] >>> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >>> 'external_ids'] >>> >>> Exit code: 0 >>> >>> Stdout: '{}\n' >>> >>> Stderr: '' >>> >>> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >>> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >>> >>> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >>> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >>> >>> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >>> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >>> >>> Exit code: 0 >>> >>> Stdout: 'int-br-p3p1\n' >>> >>> Stderr: '' >>> >>> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >>> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >>> 'external_ids'] >>> >>> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] >>> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int'] >>> >>> Exit code: 0 >>> >>> Stdout: 'int-br-p3p1\n' >>> >>> Stderr: '' >>> >>> 2013-04-28 15:15:42 DEBUG [quantum.agent.linux.utils] Running >>> command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >>> 'external_ids'] >>> >>> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >>> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >>> 'external_ids'] >>> >>> Exit code: 0 >>> >>> Stdout: '{}\n' >>> >>> Stderr: '' >>> >>> 2013-04-28 15:15:43 DEBUG [quantum.agent.linux.utils] >>> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'int-br-p3p1', >>> 'external_ids'] >>> >>> Exit code: 0 >>> >>> Stdout: '{}\n' >>> >>> Stderr: '' >>> >>> Regards, >>> >>> Kimi Zhang >>> >>> MP: +86 186 0800 8182 >>> >>> Call me(NCS): sip:+86018608008182 >>> >>> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >>> *Sent:* Sunday, April 28, 2013 3:08 PM >>> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >>> *Cc:* rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> On 04/28/2013 10:04 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> I tried that too, no lucky. >>> >>> From tcpdump ,it seems br-int does not forward any packet to >>> interfaces connect to br-p3p1, which connects to physical network... >>> >>> >>> There could be a number of issues here: >>> 1. The iptables are dropping the traffic (I am in the process of >>> getting a setup up and running) >>> 2. The network connectivity >>> >>> In order to ensure that it is not the first one can you try and see >>> which iptables rules are matched or disable the iptables? >>> >>> >>> >>> >>> Regards, >>> >>> Kimi Zhang >>> >>> MP: +86 186 0800 8182 >>> >>> Call me(NCS): sip:+86018608008182 >>> >>> *From:*ext Gary Kotton [mailto:gkotton at redhat.com] >>> *Sent:* Sunday, April 28, 2013 3:01 PM >>> *To:* Zhang, Kimi (NSN - CN/Cheng Du) >>> *Cc:* rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> On 04/28/2013 09:54 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> Hi, Gary >>> >>> Yes, I'm aware of that packstack does not support quantum yet. The >>> whole setup was installed manually. >>> >>> I did run quantum-server-setup and quantum-host-setup, I tried >>> linuxbridge plugin too, it has no issue for VM to get IP address, >>> but openvswitch has issues on this... >>> >>> >>> ok. >>> >>> if you configure and IP address manually on the VM are you able to >>> ping the port of the DHCP agent? >>> >>> you can get the IP from quantum port-list >>> >>> >>> >>> >>> >>> >>> Regards, >>> >>> Kimi >>> >>> *From:*rdo-list-bounces at redhat.com >>> >>> [mailto:rdo-list-bounces at redhat.com] *On Behalf Of *ext Gary Kotton >>> *Sent:* Sunday, April 28, 2013 2:50 PM >>> *To:* rdo-list at redhat.com >>> *Subject:* Re: [Rdo-list] [Grizzly] Network problem with Quantum + >>> Openvswitch + Vlan >>> >>> Hi Kimi, >>> Thanks for the mail. Please see the inline comments below. Please >>> note that at the moment we do not have packstack support for Quantum >>> so there is a little manual plumbing that needs to be done (not sure >>> if you have done this already). >>> On the host where the quantum service is running you need to run >>> quantum-server-setup and on the compute nodes you need to run >>> quantum-host-setup (please note that the relevant keystone >>> credentials need to be set too). >>> Thanks >>> Gary >>> >>> On 04/28/2013 09:38 AM, Zhang, Kimi (NSN - CN/Cheng Du) wrote: >>> >>> converted from rtf >>> >>> When I start VM instance, the VM can't get IP address. Could someone >>> help me on this ? >>> >>> >>> I will try >>> >>> >>> >>> >>> >>> 3 nodes Setup with RHEL 6.4 OS + rdo grizzly repository. >>> >>> ?Controller node: >>> >>> Services: Keystone+Glance+Cinder+Quantum server + Nova services >>> >>> Network: bond0(10.68.125.11 for O&M) >>> >>> ?Network node: >>> >>> Services: quantum-openvswitch-agent, quantum-l3-agent, >>> quantum-dhcp-agent, quantum-metadata-agent >>> >>> Network: bond0(10.68.125.15 for O&M) , p3p1 for VM internal network, >>> p3p2 for external network >>> >>> >>> Please note that RHEL currently does not support namespaces so there >>> are a number of limitations. We are addressing this at the moment. >>> If namespaces are not used then it is suggested that one does not >>> run the DHCP agent and the L3 agent on the same host. The reason for >>> this is that there is no network isolation. >>> >>> >>> >>> >>> >>> >>> ?Compute node: >>> >>> Services: nove-compute and quantum-openvswitch-agent >>> >>> Network: bond0(10.68.125.16 for O&M), p3p1 for VM internal network >>> >>> ?Switch setup tagging for vlan 1000-2999 for p3p1 ports(VM network) >>> of network and compute nodes. >>> >>> 1.Quantum.conf: >>> >>> [DEFAULT] >>> >>> debug = True >>> >>> verbose = True >>> >>> lock_path = $state_path/lock >>> >>> bind_host = 0.0.0.0 >>> >>> bind_port = 9696 >>> >>> core_plugin = >>> quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 >>> >>> api_paste_config = api-paste.ini >>> >>> rpc_backend = quantum.openstack.common.rpc.impl_kombu >>> >>> >>> Are you using rabbit or qpid? >>> >>> >>> >>> >>> >>> >>> control_exchange = quantum >>> >>> rabbit_host = 10.68.125.11 >>> >>> notification_driver = quantum.openstack.common.notifier.rpc_notifier >>> >>> default_notification_level = INFO >>> >>> notification_topics = notifications >>> >>> [QUOTAS] >>> >>> [DEFAULT_SERVICETYPE] >>> >>> [AGENT] >>> >>> polling_interval = 2 >>> >>> root_helper = sudo quantum-rootwrap /etc/quantum/rootwrap.conf >>> >>> [keystone_authtoken] >>> >>> auth_host = 10.68.125.11 >>> >>> auth_port = 35357 >>> >>> auth_protocol = http >>> >>> signing_dir = /var/lib/quantum/keystone-signing >>> >>> admin_tenant_name = service >>> >>> admin_user = quantum >>> >>> admin_password = password >>> >>> 2.ovs_quantum_plugin.ini >>> >>> [DATABASE] >>> >>> sql_connection = >>> mysql://quantum:quantum at 10.68.125.11:3306/ovs_quantum >>> >>> >>> reconnect_interval = 2 >>> >>> [OVS] >>> >>> tenant_network_type = vlan >>> >>> network_vlan_ranges = physnet1:1000:2999 >>> >>> bridge_mappings = physnet1:br-p3p1 >>> >>> [AGENT] >>> >>> polling_interval = 2 >>> >>> [SECURITYGROUP] >>> >>> 3.nova.conf >>> >>> [DEFAULT] >>> >>> verbose=true >>> >>> logdir = /var/log/nova >>> >>> state_path = /var/lib/nova >>> >>> lock_path = /var/lib/nova/tmp >>> >>> volumes_dir = /etc/nova/volumes >>> >>> dhcpbridge = /usr/bin/nova-dhcpbridge >>> >>> dhcpbridge_flagfile = /etc/nova/nova.conf >>> >>> force_dhcp_release = True >>> >>> injected_network_template = /usr/share/nova/interfaces.template >>> >>> libvirt_nonblocking = True >>> >>> libvirt_inject_partition = -1 >>> >>> network_manager = nova.network.manager.FlatDHCPManager >>> >>> iscsi_helper = tgtadm >>> >>> compute_driver = libvirt.LibvirtDriver >>> >>> libvirt_type=kvm >>> >>> libvirt_ovs_bridge=br-int >>> >>> firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver >>> >>> manager=nova.conductor.manager.ConductorManager >>> >>> rpc_backend = nova.openstack.common.rpc.impl_kombu >>> >>> rabbit_host = 10.68.125.11 >>> >>> rootwrap_config = /etc/nova/rootwrap.conf >>> >>> use_deprecated_auth=false >>> >>> auth_strategy=keystone >>> >>> glance_api_servers=10.68.125.11:9292 >>> >>> image_service=nova.image.glance.GlanceImageService >>> >>> novnc_enabled=true >>> >>> novncproxy_base_url=http://10.68.125.11:6080/vnc_auto.html >>> >>> novncproxy_port=6080 >>> >>> vncserver_proxyclient_address=10.68.125.16 >>> >>> vncserver_listen=0.0.0.0 >>> >>> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver >>> >>> libvirt_use_virtio_for_bridges=True >>> >>> network_api_class=nova.network.quantumv2.api.API >>> >>> quantum_url=http://10.68.125.11:9696 >>> >>> quantum_auth_strategy=keystone >>> >>> quantum_admin_tenant_name=service >>> >>> quantum_admin_username=quantum >>> >>> quantum_admin_password=password >>> >>> quantum_admin_auth_url=http://10.68.125.11:35357/v2.0 >>> >>> linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver >>> >>> libvirt_vif_type=ethernet >>> >>> service_quantum_metadata_proxy = True >>> >>> quantum_metadata_proxy_shared_secret = helloOpenStack >>> >>> metadata_host = 10.68.125.11 >>> >>> metadata_listen = 0.0.0.0 >>> >>> metadata_listen_port = 8775 >>> >>> [keystone_authtoken] >>> >>> admin_tenant_name = service >>> >>> admin_user = nova >>> >>> admin_password = password >>> >>> auth_host = 10.68.125.11 >>> >>> auth_port = 35357 >>> >>> auth_protocol = http >>> >>> signing_dir = /tmp/keystone-signing-nova >>> >>> 4.ovs-vsctl show on network node: >>> >>> aeeb6cf7-271b-405a-aa17-1b95bcd9e301 >>> >>> Bridge "br-p3p1" >>> >>> Port "p3p1" >>> >>> Interface "p3p1" >>> >>> Port "phy-br-p3p1" >>> >>> Interface "phy-br-p3p1" >>> >>> Port "br-p3p1" >>> >>> Interface "br-p3p1" >>> >>> type: internal >>> >>> Bridge br-ex >>> >>> Port br-ex >>> >>> Interface br-ex >>> >>> type: internal >>> >>> Port "qg-a83c0abd-f4" >>> >>> Interface "qg-a83c0abd-f4" >>> >>> type: internal >>> >>> Port "p3p2" >>> >>> Interface "p3p2" >>> >>> Bridge br-int >>> >>> Port br-int >>> >>> Interface br-int >>> >>> type: internal >>> >>> Port "int-br-p3p1" >>> >>> Interface "int-br-p3p1" >>> >>> Port "tap1f386a2a-12" >>> >>> tag: 1 >>> >>> Interface "tap1f386a2a-12" >>> >>> type: internal >>> >>> ovs_version: "1.9.0" >>> >>> 5.ovs-vsctl show on compute node: >>> >>> 8d6c2637-ff69-4a2d-a7db-e4f181273bc0 >>> >>> Bridge "br-p3p1" >>> >>> Port "br-p3p1" >>> >>> Interface "br-p3p1" >>> >>> type: internal >>> >>> Port "phy-br-p3p1" >>> >>> Interface "phy-br-p3p1" >>> >>> Port "p3p1" >>> >>> Interface "p3p1" >>> >>> Bridge br-int >>> >>> Port "qvo56a4572c-dc" >>> >>> tag: 2 >>> >>> Interface "qvo56a4572c-dc" >>> >>> Port "int-br-p3p1" >>> >>> Interface "int-br-p3p1" >>> >>> Port br-int >>> >>> Interface br-int >>> >>> type: internal >>> >>> ovs_version: "1.9.0" >>> >>> On compute node, I can see dhcp request packet from tcpdump on >>> qvo56a4572c-dc, but it seems the packet is not forwarded out since I >>> can't see packet from int-br-p3p1 on br-int or any port from br-p3p1. >>> >>> >>> Any chance to get the DHCP and the L3 agent configuration files? >>> Please check that use_namespaces = False in both of these files. >>> >>> Are there any log errors? >>> >>> >>> >>> >>> >>> >>> Thank you! >>> >>> Regards, >>> >>> Kimi >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Rdo-list mailing list >>> Rdo-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/rdo-list >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Rdo-list mailing list >>> Rdo-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/rdo-list >>> >> >> >> >> _______________________________________________ >> Rdo-list mailing list >> Rdo-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rdo-list > > > > _______________________________________________ > Rdo-list mailing list > Rdo-list at redhat.com > https://www.redhat.com/mailman/listinfo/rdo-list -------------- next part -------------- An HTML attachment was scrubbed... URL: