From pmyers at redhat.com Fri Nov 1 11:56:47 2013 From: pmyers at redhat.com (Perry Myers) Date: Fri, 01 Nov 2013 07:56:47 -0400 Subject: [rhos-list] nova compute not starting In-Reply-To: <18CF1869BE7AB04DB1E4CC93FD43702A1B7310F1@P-EXMB2-DC21.corp.sgi.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B7310F1@P-EXMB2-DC21.corp.sgi.com> Message-ID: <527396FF.8050000@redhat.com> On 10/31/2013 04:46 PM, David Raddatz wrote: > Hello, > > > > I?m hoping someone here can help me with this please? > > > > I ran into a problem launching an instance. Looking around, I > determined it's because nova compute was not started. In > /var/log/nova/compute.log, I saw 5 errors like this: > > > > 2013-10-31 15:09:24.930 ERROR nova.compute.manager > [req-06e4a986-c134-40ae-9420-2d21170dff60 None None] Instance > 51003341-a898-cd52-8d89-75780ea8be2d found in the hypervisor, but not in > the database > > > > And then a CRITICAL error with: > > > > 2013-10-31 15:09:27.529 271623 CRITICAL nova [-] Unexpected error while > running command. > > Command: env LC_ALL=C LANG=C qemu-img info > /root/work/openstack-testing/images/basevm.img > > Exit code: 1 > > Stdout: '' > > Stderr: "qemu-img: Could not open > '/root/work/openstack-testing/images/basevm.img'\n" > > > > That basevm.img image is VM image I was working on and is not related to > OpenStack (and it does exist in the /root/work/openstack-testing/images > directory). Note that if I run virsh list --all, I get: I wonder if there are permissions or selinux reasons why the Nova process cannot access that image? I think what Nova tries to do when it starts is 'find all of the running vms' so that it can determine what the state of the system is and compare it to the database. I wonder if Nova just needs to be made more resilient so that if it finds a VM when it starts that it can't access, it just ignores it and says "Well, if I can't accecss this VM, it must be because I'm not it's master" and then continues on. Russell/Nikola/Dan, you guys have any thoughts? > > > Id Name State > > ---------------------------------------------------- > > - basevm shut off > > - default16cvm shut off > > - instance-00000037 shut off > > - kvmnode1 shut off > > - numa16cvm shut off > > - old-kvmtest-default8cvm shut off > > > > Only the instance-00000037 VM is an OpenStack VM and the other 5 (note > it is 5 - same as the number of error messages above) are not OpenStack > VMs/instances. The instance that was created and result in > instance-00000037 no longer exists if I run a nova list command. > > > > So, any ideas on how to recover from this? If I try restarting the > openstack-nova-compute service I just get the same errors that I > mentioned above. Fix the permissions or destroy the VM and see if the error goes away as a short term hack? Perry From pmyers at redhat.com Fri Nov 1 11:58:08 2013 From: pmyers at redhat.com (Perry Myers) Date: Fri, 01 Nov 2013 07:58:08 -0400 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> Message-ID: <52739750.7060207@redhat.com> On 10/31/2013 04:49 PM, David Raddatz wrote: > A quick post-mortem on this issue and I wanted to share what we learned (or "I" learned anyway) in case it avoids problems for others... > > While I was apparently following the docs by "just install cloud-init on your VM image", while working with Lars we discovered if you reboot your image after installing cloud-init, it will create some files/directories in /var/lib/cloud which will then prevent the instance from allowing ssh to work (my Permission denied issue I was seeing). > > For example, doing the following: > - create VM and install RHEL 6.4 and set up with other files/software > - install/configure cloud-init on VM > - reboot VM (just to make sure it still boots) > - shutdown VM > - run virt-sysprep on image (per recommendation in docs) > - upload to glance and launch instance using the defined keypair > - trying to use ssh -i with the same keypair results in Permission denied > > Whereas, doing the following: > - create VM and install RHEL 6.4 and set up with other files/software > - install/configure cloud-init on VM > - shutdown VM > - run virt-sysprep on image (per recommendation in docs) > - upload to glance and launch instance using the defined keypair > - trying to use ssh -i with the same keypair Works! No password prompt (as expected) > > Not rebooting the VM allowed the ssh to work as the /var/lib/cloud directory was empty when it was shutdown. > > Not sure if this is a doc issue (to warn people NOT to reboot the VM after cloud-init installation/configuration) or a bug (which Lars was going to investigate a little) but thought it was worth a quick email to warn folks about. > > Thanks again to Lars for his assistance on this, Interesting. Lars/Steve, is this a bug or just a docs issue do you think? Perry From lars at redhat.com Fri Nov 1 13:05:30 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Fri, 1 Nov 2013 09:05:30 -0400 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <52739750.7060207@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> Message-ID: <20131101130530.GB24084@redhat.com> On Fri, Nov 01, 2013 at 07:58:08AM -0400, Perry Myers wrote: > Interesting. Lars/Steve, is this a bug or just a docs issue do you think? I suspect a bug somewhere. Something was creating a *directory* /var/lib/cloud/instance, which is normally a symlink to /var/lib/cloud/instances/. It appears that when this directory exists cloud-init falls over. I suspect that cloud-init may be doing this when booted without a valid data source, but I'm not sure. I've opened a bz here to investigate and possibly correctly things: https://bugzilla.redhat.com/show_bug.cgi?id=1025749 -- Lars Kellogg-Stedman -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From shardy at redhat.com Fri Nov 1 13:42:59 2013 From: shardy at redhat.com (Steven Hardy) Date: Fri, 1 Nov 2013 13:42:59 +0000 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <52739750.7060207@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> Message-ID: <20131101134258.GA2865@t430slt.redhat.com> On Fri, Nov 01, 2013 at 07:58:08AM -0400, Perry Myers wrote: > On 10/31/2013 04:49 PM, David Raddatz wrote: > > A quick post-mortem on this issue and I wanted to share what we learned (or "I" learned anyway) in case it avoids problems for others... > > > > While I was apparently following the docs by "just install cloud-init on your VM image", while working with Lars we discovered if you reboot your image after installing cloud-init, it will create some files/directories in /var/lib/cloud which will then prevent the instance from allowing ssh to work (my Permission denied issue I was seeing). > > > > For example, doing the following: > > - create VM and install RHEL 6.4 and set up with other files/software > > - install/configure cloud-init on VM > > - reboot VM (just to make sure it still boots) > > - shutdown VM > > - run virt-sysprep on image (per recommendation in docs) > > - upload to glance and launch instance using the defined keypair > > - trying to use ssh -i with the same keypair results in Permission denied > > > > Whereas, doing the following: > > - create VM and install RHEL 6.4 and set up with other files/software > > - install/configure cloud-init on VM > > - shutdown VM > > - run virt-sysprep on image (per recommendation in docs) > > - upload to glance and launch instance using the defined keypair > > - trying to use ssh -i with the same keypair Works! No password prompt (as expected) > > > > Not rebooting the VM allowed the ssh to work as the /var/lib/cloud directory was empty when it was shutdown. > > > > Not sure if this is a doc issue (to warn people NOT to reboot the VM after cloud-init installation/configuration) or a bug (which Lars was going to investigate a little) but thought it was worth a quick email to warn folks about. > > > > Thanks again to Lars for his assistance on this, > > Interesting. Lars/Steve, is this a bug or just a docs issue do you think? Yes, interesting - I've not observed this specific failure pattern, but I did see various issues related to SSH key installation with the current 0.7.1 cloud-init in RHEL (and on Fedora, before it was updated to 0.7.2). It's these sorts of issues which prompted the proposed update to 0.7.2: https://bugzilla.redhat.com/show_bug.cgi?id=968246 I'll be interested to see if the failure mode described above can be reproduced with cloud-init 0.7.2. Steve From draddatz at sgi.com Fri Nov 1 14:11:24 2013 From: draddatz at sgi.com (David Raddatz) Date: Fri, 1 Nov 2013 14:11:24 +0000 Subject: [rhos-list] nova compute not starting In-Reply-To: <527396FF.8050000@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B7310F1@P-EXMB2-DC21.corp.sgi.com> <527396FF.8050000@redhat.com> Message-ID: <18CF1869BE7AB04DB1E4CC93FD43702A1B7311E5@P-EXMB2-DC21.corp.sgi.com> Hi, Perry, This morning I tried the following: - Had a thought about instance-00000037 that it shouldn't be around as I deleted it using nova delete (it doesn't show up in nova list, but shows up in virsh list --all). Deleting the VM instance-00000037 in virt-manager did nothing (somewhat as expected) - nova-compute would not restart still. - After seeing your comments about permissions I looked and noticed that basevm.img has 600 permissions while the others had 644. I changed basevm.img to 644 and tried restarting the nova-compute service and it worked! I still see similar errors about my non-OpenStack VMs not being in the database (somewhat expected as those aren't OpenStack VMs) but I don't see the CRITICAL error about basevm.img. Now, I just see an error about a different image (old-kvmtest-default8cvm.img) that no longer exists in the same directory. I had done some image cleaning and moved that one. Moving it back and restarting nova-compute now gives me just the "not found in database errors" and no other errors. It seems kind of odd since the old-kvmtest-default8cvm.img also only has 600 permissions but it didn't cause the critical error after I moved it back to the images directory. It is listed last in virsh list --all whereas the basevm.img was the first one listed (if that matters). Anyway, nova-compute now has a smiley face in the nova-manage service list output and I can launch my instances again. Thanks, Dave > -----Original Message----- > From: Perry Myers [mailto:pmyers at redhat.com] > Sent: Friday, November 01, 2013 6:57 AM > To: David Raddatz; rhos-list at redhat.com; Russell Bryant; Dan Smith; Nikola > ?ipanov; Dave Allan > Subject: Re: [rhos-list] nova compute not starting > > On 10/31/2013 04:46 PM, David Raddatz wrote: > > Hello, > > > > > > > > I?m hoping someone here can help me with this please? > > > > > > > > I ran into a problem launching an instance. Looking around, I > > determined it's because nova compute was not started. In > > /var/log/nova/compute.log, I saw 5 errors like this: > > > > > > > > 2013-10-31 15:09:24.930 ERROR nova.compute.manager > > [req-06e4a986-c134-40ae-9420-2d21170dff60 None None] Instance > > 51003341-a898-cd52-8d89-75780ea8be2d found in the hypervisor, but not > > in the database > > > > > > > > And then a CRITICAL error with: > > > > > > > > 2013-10-31 15:09:27.529 271623 CRITICAL nova [-] Unexpected error > > while running command. > > > > Command: env LC_ALL=C LANG=C qemu-img info > > /root/work/openstack-testing/images/basevm.img > > > > Exit code: 1 > > > > Stdout: '' > > > > Stderr: "qemu-img: Could not open > > '/root/work/openstack-testing/images/basevm.img'\n" > > > > > > > > That basevm.img image is VM image I was working on and is not related > > to OpenStack (and it does exist in the > > /root/work/openstack-testing/images > > directory). Note that if I run virsh list --all, I get: > > I wonder if there are permissions or selinux reasons why the Nova process > cannot access that image? I think what Nova tries to do when it starts is 'find > all of the running vms' so that it can determine what the state of the system > is and compare it to the database. > > I wonder if Nova just needs to be made more resilient so that if it finds a VM > when it starts that it can't access, it just ignores it and says "Well, if I can't > accecss this VM, it must be because I'm not it's master" and then continues > on. > > Russell/Nikola/Dan, you guys have any thoughts? > > > > > > > Id Name State > > > > ---------------------------------------------------- > > > > - basevm shut off > > > > - default16cvm shut off > > > > - instance-00000037 shut off > > > > - kvmnode1 shut off > > > > - numa16cvm shut off > > > > - old-kvmtest-default8cvm shut off > > > > > > > > Only the instance-00000037 VM is an OpenStack VM and the other 5 (note > > it is 5 - same as the number of error messages above) are not > > OpenStack VMs/instances. The instance that was created and result in > > instance-00000037 no longer exists if I run a nova list command. > > > > > > > > So, any ideas on how to recover from this? If I try restarting the > > openstack-nova-compute service I just get the same errors that I > > mentioned above. > > Fix the permissions or destroy the VM and see if the error goes away as a > short term hack? > > Perry From dansmith at redhat.com Fri Nov 1 14:26:31 2013 From: dansmith at redhat.com (Dan Smith) Date: Fri, 01 Nov 2013 07:26:31 -0700 Subject: [rhos-list] nova compute not starting In-Reply-To: <18CF1869BE7AB04DB1E4CC93FD43702A1B7311E5@P-EXMB2-DC21.corp.sgi.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B7310F1@P-EXMB2-DC21.corp.sgi.com> <527396FF.8050000@redhat.com> <18CF1869BE7AB04DB1E4CC93FD43702A1B7311E5@P-EXMB2-DC21.corp.sgi.com> Message-ID: <5273BA17.2040801@redhat.com> > Moving it back and restarting nova-compute now gives me just the "not > found in database errors" and no other errors. Nova expects to own the system, yeah. However, it _is_ tolerant of other VMs on the system if you configure it to be (otherwise it will delete them when it finds them). The problem you experienced before was probably due to the fact that nova uses libvirt, running under its user. When it went to list or examine the VMs on the system, libvirt failed to access some of those files and nova died, expectedly, as a result. You can leave the other VMs on the system, but you will get (harmless) warnings from Nova about them being there. --Dan From vkarani1 at in.ibm.com Fri Nov 1 16:46:25 2013 From: vkarani1 at in.ibm.com (Velayutham Karani1) Date: Fri, 1 Nov 2013 22:16:25 +0530 Subject: [rhos-list] AUTO: Velayutham Karani1 is out of the office (returning 11-11-2013) Message-ID: I am out of the office until 11-11-2013. I will be on official trip out of country till 11 November 2013 with limited access email and no access to mobile. I will return to office by w12 Nov 2013. I can reply to your mail as and when possible. If urgent response needed, please contact my manager Shanmugam Raman at shanmugam at in.ibm.com. Thanks, Velu. Note: This is an automated response to your message "rhos-list Digest, Vol 16, Issue 1" sent on 01/11/2013 17:28:09. This is the only notification you will receive while this person is away. From john.haller at alcatel-lucent.com Mon Nov 4 22:23:42 2013 From: john.haller at alcatel-lucent.com (Haller, John H (John)) Date: Mon, 4 Nov 2013 22:23:42 +0000 Subject: [rhos-list] Installing Havana RDO interaction with NetworkManager Message-ID: <7C1824C61EE769448FCE74CD83F0CB4F57DC4FA1@US70TWXCHMBA11.zam.alcatel-lucent.com> This was on a fresh CentOS install from a Live DVD, with a full yum update as of last Friday. Unfortunately, I had eth0 configured via NetworkManager, and the init-script "network" also was not running by default. The packstack action for disabling NetworkManager is inadequate, as it really needs to be sure the network init script is set to run, and after NetworkManger deconfigures the interface, the interface needs to be brought back up with ifup, none of which works over a ssh connection. If packstack is going to disable the NetworkManager settings, it needs to kill the NetworkManager process first, so that it doesn't de-configure the network port. It should also ensure that the network init script is set to automatically run on boot, so that the interface will automatically be enabled. Otherwise, it should just abort if it detects NetworkManger is in charge of the network, and leave this problem to documentation. This is a new problem with Havana, Grizzly didn't have this issue, although there were other network issues I never investigated, such as not recovering after the network cable was disconnected. Regards, John Haller -------------- next part -------------- An HTML attachment was scrubbed... URL: From twilson at redhat.com Tue Nov 5 05:52:46 2013 From: twilson at redhat.com (Terry Wilson) Date: Tue, 5 Nov 2013 00:52:46 -0500 (EST) Subject: [rhos-list] Installing Havana RDO interaction with NetworkManager In-Reply-To: <7C1824C61EE769448FCE74CD83F0CB4F57DC4FA1@US70TWXCHMBA11.zam.alcatel-lucent.com> References: <7C1824C61EE769448FCE74CD83F0CB4F57DC4FA1@US70TWXCHMBA11.zam.alcatel-lucent.com> Message-ID: <176338291.18336265.1383630766083.JavaMail.root@redhat.com> ----- Original Message ----- > > > This was on a fresh CentOS install from a Live DVD, with a full yum update as > of last Friday. > > > > Unfortunately, I had eth0 configured via NetworkManager, and the init-script > > "network" also was not running by default. The packstack action for disabling > NetworkManager > > is inadequate, as it really needs to be sure the network init script is set > to > > run, and after NetworkManger deconfigures the interface, the interface needs > to > > be brought back up with ifup, none of which works over a ssh connection. > > > > If packstack is going to disable the NetworkManager settings, it needs to > kill the > > NetworkManager process first, so that it doesn't de-configure the network > port. It > > should also ensure that the network init script is set to automatically run > on boot, > > so that the interface will automatically be enabled. Otherwise, it should > just abort > > if it detects NetworkManger is in charge of the network, and leave this > problem to > > documentation. > > > > This is a new problem with Havana, Grizzly didn't have this issue, although > there > > were other network issues I never investigated, such as not recovering after > the > > network cable was disconnected. I have posted a comment linking this thread on the issue at https://bugzilla.redhat.com/show_bug.cgi?id=967369. You should be able to track the issue/communicate with the developer working on the issue there. Thanks for the report! Terry From tgraf at redhat.com Tue Nov 5 08:13:09 2013 From: tgraf at redhat.com (Thomas Graf) Date: Tue, 05 Nov 2013 09:13:09 +0100 Subject: [rhos-list] Installing Havana RDO interaction with NetworkManager In-Reply-To: <176338291.18336265.1383630766083.JavaMail.root@redhat.com> References: <7C1824C61EE769448FCE74CD83F0CB4F57DC4FA1@US70TWXCHMBA11.zam.alcatel-lucent.com> <176338291.18336265.1383630766083.JavaMail.root@redhat.com> Message-ID: <5278A895.9020201@redhat.com> CC'ing NM developers On 11/05/2013 06:52 AM, Terry Wilson wrote: > ----- Original Message ----- >> >> >> This was on a fresh CentOS install from a Live DVD, with a full yum update as >> of last Friday. >> >> >> >> Unfortunately, I had eth0 configured via NetworkManager, and the init-script >> >> "network" also was not running by default. The packstack action for disabling >> NetworkManager >> >> is inadequate, as it really needs to be sure the network init script is set >> to >> >> run, and after NetworkManger deconfigures the interface, the interface needs >> to >> >> be brought back up with ifup, none of which works over a ssh connection. >> >> >> >> If packstack is going to disable the NetworkManager settings, it needs to >> kill the >> >> NetworkManager process first, so that it doesn't de-configure the network >> port. It >> >> should also ensure that the network init script is set to automatically run >> on boot, >> >> so that the interface will automatically be enabled. Otherwise, it should >> just abort >> >> if it detects NetworkManger is in charge of the network, and leave this >> problem to >> >> documentation. >> >> >> >> This is a new problem with Havana, Grizzly didn't have this issue, although >> there >> >> were other network issues I never investigated, such as not recovering after >> the >> >> network cable was disconnected. > > > I have posted a comment linking this thread on the issue at https://bugzilla.redhat.com/show_bug.cgi?id=967369. You should be able to track the issue/communicate with the developer working on the issue there. Thanks for the report! > > Terry > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From xzhao at bnl.gov Tue Nov 5 19:42:38 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Tue, 05 Nov 2013 14:42:38 -0500 Subject: [rhos-list] system panic after starting OVS agent Message-ID: <52794A2E.1080508@bnl.gov> Hello, On my grizzly quantum/OVS network node, after I start the quantum-openvswitch-agent, the system log shows errors as below, and it repeats every second since then... and the panic messages continue on even after I stop all openstack daemons, only a system reboot can clear it out. Nov 5 14:13:58 cldnet01 kernel: qg-581539d2-ac: hw csum failure. Nov 5 14:13:58 cldnet01 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-358.123.2.openstack.el6.x86_64 #1 Nov 5 14:13:58 cldnet01 kernel: Call Trace: Nov 5 14:13:58 cldnet01 kernel: [] ? netdev_rx_csum_fault+0x42/0x50 Nov 5 14:13:58 cldnet01 kernel: [] ? __skb_checksum_complete_head+0x60/0x70 Nov 5 14:13:58 cldnet01 kernel: [] ? __skb_checksum_complete+0x11/0x20 Nov 5 14:13:58 cldnet01 kernel: [] ? nf_ip_checksum+0x5d/0x130 Nov 5 14:13:58 cldnet01 kernel: [] ? udp_error+0xb1/0x1e0 [nf_conntrack] Nov 5 14:13:58 cldnet01 kernel: [] ? nf_conntrack_in+0x138/0xa00 [nf_conntrack] Nov 5 14:13:58 cldnet01 kernel: [] ? alloc_null_binding+0x5b/0xa0 [iptable_nat] Nov 5 14:13:58 cldnet01 kernel: [] ? nf_nat_fn+0x91/0x260 [iptable_nat] Nov 5 14:13:58 cldnet01 kernel: [] ? ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] Nov 5 14:13:58 cldnet01 kernel: [] ? nf_iterate+0x69/0xb0 Nov 5 14:13:58 cldnet01 kernel: [] ? ip_rcv_finish+0x199/0x440 Nov 5 14:13:58 cldnet01 kernel: [] ? ip_rcv_finish+0x0/0x440 Nov 5 14:13:58 cldnet01 kernel: [] ? nf_hook_slow+0x74/0x110 Nov 5 14:13:58 cldnet01 kernel: [] ? ip_rcv_finish+0x0/0x440 Nov 5 14:13:58 cldnet01 kernel: [] ? ip_rcv+0x264/0x350 Nov 5 14:13:58 cldnet01 kernel: [] ? ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] Nov 5 14:13:58 cldnet01 kernel: [] ? __netif_receive_skb+0x4ab/0x750 Nov 5 14:13:58 cldnet01 kernel: [] ? process_backlog+0x9a/0x100 Nov 5 14:13:58 cldnet01 kernel: [] ? net_rx_action+0x103/0x2f0 Nov 5 14:13:58 cldnet01 kernel: [] ? __do_softirq+0xc1/0x1e0 Nov 5 14:13:58 cldnet01 kernel: [] ? handle_IRQ_event+0x60/0x170 Nov 5 14:13:58 cldnet01 kernel: [] ? call_softirq+0x1c/0x30 Nov 5 14:13:58 cldnet01 kernel: [] ? do_softirq+0x65/0xa0 Nov 5 14:13:58 cldnet01 kernel: [] ? irq_exit+0x85/0x90 Nov 5 14:13:58 cldnet01 kernel: [] ? do_IRQ+0x75/0xf0 Nov 5 14:13:58 cldnet01 kernel: [] ? ret_from_intr+0x0/0x11 Nov 5 14:13:58 cldnet01 kernel: [] ? mwait_idle+0x77/0xd0 Nov 5 14:13:58 cldnet01 kernel: [] ? atomic_notifier_call_chain+0x1a/0x20 Nov 5 14:13:58 cldnet01 kernel: [] ? cpu_idle+0xb6/0x110 Nov 5 14:13:58 cldnet01 kernel: [] ? start_secondary+0x2ac/0x2ef The only other message in the syslog that's related to CSUM is the following: Nov 5 14:10:44 cldnet01 kernel: lo: Dropping TSO features since no CSUM feature. Nov 5 14:10:44 cldnet01 kernel: lo: Disabled Privacy Extensions (this message appears after starting the l3-agent) The network host is RHEL6.4, kernel is 2.6.32-358.123.2.openstack.el6.x86_64 All the daemons appear to being running, an instance can start, but network doesn't work for the instance. Any wisdom on what's going on? Thanks, Xin From xzhao at bnl.gov Tue Nov 5 21:43:23 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Tue, 05 Nov 2013 16:43:23 -0500 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <52794A2E.1080508@bnl.gov> References: <52794A2E.1080508@bnl.gov> Message-ID: <5279667B.3000104@bnl.gov> Add the general Openstack list, sorry for folks who are on both lists... On 11/5/2013 2:42 PM, Xin Zhao wrote: > Hello, > > On my grizzly quantum/OVS network node, after I start the > quantum-openvswitch-agent, the system log shows errors as below, > and it repeats every second since then... and the panic messages > continue on even after I stop all openstack daemons, only a system reboot > can clear it out. > > Nov 5 14:13:58 cldnet01 kernel: qg-581539d2-ac: hw csum failure. > Nov 5 14:13:58 cldnet01 kernel: Pid: 0, comm: swapper Not tainted > 2.6.32-358.123.2.openstack.el6.x86_64 #1 > Nov 5 14:13:58 cldnet01 kernel: Call Trace: > Nov 5 14:13:58 cldnet01 kernel: [] ? > netdev_rx_csum_fault+0x42/0x50 > Nov 5 14:13:58 cldnet01 kernel: [] ? > __skb_checksum_complete_head+0x60/0x70 > Nov 5 14:13:58 cldnet01 kernel: [] ? > __skb_checksum_complete+0x11/0x20 > Nov 5 14:13:58 cldnet01 kernel: [] ? > nf_ip_checksum+0x5d/0x130 > Nov 5 14:13:58 cldnet01 kernel: [] ? > udp_error+0xb1/0x1e0 [nf_conntrack] > Nov 5 14:13:58 cldnet01 kernel: [] ? > nf_conntrack_in+0x138/0xa00 [nf_conntrack] > Nov 5 14:13:58 cldnet01 kernel: [] ? > alloc_null_binding+0x5b/0xa0 [iptable_nat] > Nov 5 14:13:58 cldnet01 kernel: [] ? > nf_nat_fn+0x91/0x260 [iptable_nat] > Nov 5 14:13:58 cldnet01 kernel: [] ? > ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] > Nov 5 14:13:58 cldnet01 kernel: [] ? > nf_iterate+0x69/0xb0 > Nov 5 14:13:58 cldnet01 kernel: [] ? > ip_rcv_finish+0x199/0x440 > Nov 5 14:13:58 cldnet01 kernel: [] ? > ip_rcv_finish+0x0/0x440 > Nov 5 14:13:58 cldnet01 kernel: [] ? > nf_hook_slow+0x74/0x110 > Nov 5 14:13:58 cldnet01 kernel: [] ? > ip_rcv_finish+0x0/0x440 > Nov 5 14:13:58 cldnet01 kernel: [] ? > ip_rcv+0x264/0x350 > Nov 5 14:13:58 cldnet01 kernel: [] ? > ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] > Nov 5 14:13:58 cldnet01 kernel: [] ? > __netif_receive_skb+0x4ab/0x750 > Nov 5 14:13:58 cldnet01 kernel: [] ? > process_backlog+0x9a/0x100 > Nov 5 14:13:58 cldnet01 kernel: [] ? > net_rx_action+0x103/0x2f0 > Nov 5 14:13:58 cldnet01 kernel: [] ? > __do_softirq+0xc1/0x1e0 > Nov 5 14:13:58 cldnet01 kernel: [] ? > handle_IRQ_event+0x60/0x170 > Nov 5 14:13:58 cldnet01 kernel: [] ? > call_softirq+0x1c/0x30 > Nov 5 14:13:58 cldnet01 kernel: [] ? > do_softirq+0x65/0xa0 > Nov 5 14:13:58 cldnet01 kernel: [] ? > irq_exit+0x85/0x90 > Nov 5 14:13:58 cldnet01 kernel: [] ? do_IRQ+0x75/0xf0 > Nov 5 14:13:58 cldnet01 kernel: [] ? > ret_from_intr+0x0/0x11 > Nov 5 14:13:58 cldnet01 kernel: [] ? > mwait_idle+0x77/0xd0 > Nov 5 14:13:58 cldnet01 kernel: [] ? > atomic_notifier_call_chain+0x1a/0x20 > Nov 5 14:13:58 cldnet01 kernel: [] ? > cpu_idle+0xb6/0x110 > Nov 5 14:13:58 cldnet01 kernel: [] ? > start_secondary+0x2ac/0x2ef > > The only other message in the syslog that's related to CSUM is the > following: > Nov 5 14:10:44 cldnet01 kernel: lo: Dropping TSO features since no > CSUM feature. > Nov 5 14:10:44 cldnet01 kernel: lo: Disabled Privacy Extensions > (this message appears after starting the l3-agent) > > The network host is RHEL6.4, kernel is > 2.6.32-358.123.2.openstack.el6.x86_64 > > All the daemons appear to being running, an instance can start, but > network doesn't work for the instance. > > Any wisdom on what's going on? > > Thanks, > Xin > > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From tgraf at redhat.com Tue Nov 5 21:55:46 2013 From: tgraf at redhat.com (Thomas Graf) Date: Tue, 05 Nov 2013 22:55:46 +0100 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <5279667B.3000104@bnl.gov> References: <52794A2E.1080508@bnl.gov> <5279667B.3000104@bnl.gov> Message-ID: <52796962.1090906@redhat.com> On 11/05/2013 10:43 PM, Xin Zhao wrote: > Add the general Openstack list, sorry for folks who are on both lists... > > On 11/5/2013 2:42 PM, Xin Zhao wrote: >> Hello, >> >> On my grizzly quantum/OVS network node, after I start the >> quantum-openvswitch-agent, the system log shows errors as below, >> and it repeats every second since then... and the panic messages >> continue on even after I stop all openstack daemons, only a system reboot >> can clear it out. >> >> Nov 5 14:13:58 cldnet01 kernel: qg-581539d2-ac: hw csum failure. >> Nov 5 14:13:58 cldnet01 kernel: Pid: 0, comm: swapper Not tainted >> 2.6.32-358.123.2.openstack.el6.x86_64 #1 >> Nov 5 14:13:58 cldnet01 kernel: Call Trace: >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> netdev_rx_csum_fault+0x42/0x50 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> __skb_checksum_complete_head+0x60/0x70 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> __skb_checksum_complete+0x11/0x20 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> nf_ip_checksum+0x5d/0x130 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> udp_error+0xb1/0x1e0 [nf_conntrack] >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> nf_conntrack_in+0x138/0xa00 [nf_conntrack] >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> alloc_null_binding+0x5b/0xa0 [iptable_nat] >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> nf_nat_fn+0x91/0x260 [iptable_nat] >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> nf_iterate+0x69/0xb0 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ip_rcv_finish+0x199/0x440 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ip_rcv_finish+0x0/0x440 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> nf_hook_slow+0x74/0x110 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ip_rcv_finish+0x0/0x440 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ip_rcv+0x264/0x350 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> __netif_receive_skb+0x4ab/0x750 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> process_backlog+0x9a/0x100 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> net_rx_action+0x103/0x2f0 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> __do_softirq+0xc1/0x1e0 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> handle_IRQ_event+0x60/0x170 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> call_softirq+0x1c/0x30 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> do_softirq+0x65/0xa0 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> irq_exit+0x85/0x90 >> Nov 5 14:13:58 cldnet01 kernel: [] ? do_IRQ+0x75/0xf0 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> ret_from_intr+0x0/0x11 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> mwait_idle+0x77/0xd0 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> atomic_notifier_call_chain+0x1a/0x20 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> cpu_idle+0xb6/0x110 >> Nov 5 14:13:58 cldnet01 kernel: [] ? >> start_secondary+0x2ac/0x2ef >> >> The only other message in the syslog that's related to CSUM is the >> following: >> Nov 5 14:10:44 cldnet01 kernel: lo: Dropping TSO features since no >> CSUM feature. >> Nov 5 14:10:44 cldnet01 kernel: lo: Disabled Privacy Extensions >> (this message appears after starting the l3-agent) >> >> The network host is RHEL6.4, kernel is >> 2.6.32-358.123.2.openstack.el6.x86_64 >> >> All the daemons appear to being running, an instance can start, but >> network doesn't work for the instance. >> >> Any wisdom on what's going on? Most likely a driver issue, what NIC model and driver are you using? ehttool -i $DEV will help. From xzhao at bnl.gov Tue Nov 5 22:31:51 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Tue, 05 Nov 2013 17:31:51 -0500 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <52796962.1090906@redhat.com> References: <52794A2E.1080508@bnl.gov> <5279667B.3000104@bnl.gov> <52796962.1090906@redhat.com> Message-ID: <527971D7.1030805@bnl.gov> Hi Thomas, Thanks for the reply, here is the info of the drivers: 1) NIC for the VM network connection: $> ethtool -i eth1 driver: e1000e version: 2.1.4-k firmware-version: 1.6-12 bus-info: 0000:04:00.1 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: no 2) NIC for the external internet connection: $> ethtool -i eth2 driver: myri10ge version: 1.5.1-1.451 firmware-version: 1.4.52 -- 2010/10/28 21:27:06 m bus-info: 0000:0b:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no Xin On 11/5/2013 4:55 PM, Thomas Graf wrote: > On 11/05/2013 10:43 PM, Xin Zhao wrote: >> Add the general Openstack list, sorry for folks who are on both lists... >> >> On 11/5/2013 2:42 PM, Xin Zhao wrote: >>> Hello, >>> >>> On my grizzly quantum/OVS network node, after I start the >>> quantum-openvswitch-agent, the system log shows errors as below, >>> and it repeats every second since then... and the panic messages >>> continue on even after I stop all openstack daemons, only a system >>> reboot >>> can clear it out. >>> >>> Nov 5 14:13:58 cldnet01 kernel: qg-581539d2-ac: hw csum failure. >>> Nov 5 14:13:58 cldnet01 kernel: Pid: 0, comm: swapper Not tainted >>> 2.6.32-358.123.2.openstack.el6.x86_64 #1 >>> Nov 5 14:13:58 cldnet01 kernel: Call Trace: >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> netdev_rx_csum_fault+0x42/0x50 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> __skb_checksum_complete_head+0x60/0x70 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> __skb_checksum_complete+0x11/0x20 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> nf_ip_checksum+0x5d/0x130 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> udp_error+0xb1/0x1e0 [nf_conntrack] >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> nf_conntrack_in+0x138/0xa00 [nf_conntrack] >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> alloc_null_binding+0x5b/0xa0 [iptable_nat] >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> nf_nat_fn+0x91/0x260 [iptable_nat] >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> nf_iterate+0x69/0xb0 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ip_rcv_finish+0x199/0x440 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ip_rcv_finish+0x0/0x440 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> nf_hook_slow+0x74/0x110 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ip_rcv_finish+0x0/0x440 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ip_rcv+0x264/0x350 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> __netif_receive_skb+0x4ab/0x750 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> process_backlog+0x9a/0x100 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> net_rx_action+0x103/0x2f0 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> __do_softirq+0xc1/0x1e0 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> handle_IRQ_event+0x60/0x170 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> call_softirq+0x1c/0x30 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> do_softirq+0x65/0xa0 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> irq_exit+0x85/0x90 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> do_IRQ+0x75/0xf0 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> ret_from_intr+0x0/0x11 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> mwait_idle+0x77/0xd0 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> atomic_notifier_call_chain+0x1a/0x20 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> cpu_idle+0xb6/0x110 >>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>> start_secondary+0x2ac/0x2ef >>> >>> The only other message in the syslog that's related to CSUM is the >>> following: >>> Nov 5 14:10:44 cldnet01 kernel: lo: Dropping TSO features since no >>> CSUM feature. >>> Nov 5 14:10:44 cldnet01 kernel: lo: Disabled Privacy Extensions >>> (this message appears after starting the l3-agent) >>> >>> The network host is RHEL6.4, kernel is >>> 2.6.32-358.123.2.openstack.el6.x86_64 >>> >>> All the daemons appear to being running, an instance can start, but >>> network doesn't work for the instance. >>> >>> Any wisdom on what's going on? > > Most likely a driver issue, what NIC model and driver are you using? > ehttool -i $DEV will help. From prmarino1 at gmail.com Wed Nov 6 01:29:58 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Tue, 05 Nov 2013 20:29:58 -0500 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <527971D7.1030805@bnl.gov> Message-ID: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> An HTML attachment was scrubbed... URL: From mrunge at redhat.com Wed Nov 6 02:51:02 2013 From: mrunge at redhat.com (Matthias Runge) Date: Wed, 06 Nov 2013 03:51:02 +0100 Subject: [rhos-list] Installing Havana RDO interaction with NetworkManager In-Reply-To: <176338291.18336265.1383630766083.JavaMail.root@redhat.com> References: <7C1824C61EE769448FCE74CD83F0CB4F57DC4FA1@US70TWXCHMBA11.zam.alcatel-lucent.com> <176338291.18336265.1383630766083.JavaMail.root@redhat.com> Message-ID: <5279AE96.3050302@redhat.com> >> network cable was disconnected. > > > I have posted a comment linking this thread on the issue at > https://bugzilla.redhat.com/show_bug.cgi?id=967369. You should be > able to track the issue/communicate with the developer working on the > issue there. Thanks for the report! > > Terry Additional to that, a few days earlier, I filed https://bugzilla.redhat.com/show_bug.cgi?id=1024292 The situation is currently: when just diabling NetworkManager, noone might be able to install OpenStack via packstack, because no package can be fetched anymore. Matthias From xzhao at bnl.gov Wed Nov 6 14:42:44 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Wed, 06 Nov 2013 09:42:44 -0500 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> References: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> Message-ID: <527A5564.3010003@bnl.gov> Hi Paul, The kernel is 2.6.32-358.123.2.openstack.el6.x86_64, it is from RDO: name=OpenStack Grizzly Repository baseurl=http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/ Thanks, Xin On 11/5/2013 8:29 PM, Paul Robert Marino wrote: > Which kernel are you running > If you are running the stock RedHat kernel not the patched version > that comes with RHOS or RDO that might explain it. > > > > -- Sent from my HP Pre3 > > ------------------------------------------------------------------------ > On Nov 5, 2013 17:33, Xin Zhao wrote: > > Hi Thomas, > > Thanks for the reply, here is the info of the drivers: > > 1) NIC for the VM network connection: > > $> ethtool -i eth1 > driver: e1000e > version: 2.1.4-k > firmware-version: 1.6-12 > bus-info: 0000:04:00.1 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: no > > 2) NIC for the external internet connection: > > $> ethtool -i eth2 > driver: myri10ge > version: 1.5.1-1.451 > firmware-version: 1.4.52 -- 2010/10/28 21:27:06 m > bus-info: 0000:0b:00.0 > supports-statistics: yes > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: no > > Xin > > > On 11/5/2013 4:55 PM, Thomas Graf wrote: > > On 11/05/2013 10:43 PM, Xin Zhao wrote: > >> Add the general Openstack list, sorry for folks who are on both > lists... > >> > >> On 11/5/2013 2:42 PM, Xin Zhao wrote: > >>> Hello, > >>> > >>> On my grizzly quantum/OVS network node, after I start the > >>> quantum-openvswitch-agent, the system log shows errors as below, > >>> and it repeats every second since then... and the panic messages > >>> continue on even after I stop all openstack daemons, only a system > >>> reboot > >>> can clear it out. > >>> > >>> Nov 5 14:13:58 cldnet01 kernel: qg-581539d2-ac: hw csum failure. > >>> Nov 5 14:13:58 cldnet01 kernel: Pid: 0, comm: swapper Not tainted > >>> 2.6.32-358.123.2.openstack.el6.x86_64 #1 > >>> Nov 5 14:13:58 cldnet01 kernel: Call Trace: > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> netdev_rx_csum_fault+0x42/0x50 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> __skb_checksum_complete_head+0x60/0x70 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> __skb_checksum_complete+0x11/0x20 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> nf_ip_checksum+0x5d/0x130 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> udp_error+0xb1/0x1e0 [nf_conntrack] > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> nf_conntrack_in+0x138/0xa00 [nf_conntrack] > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> alloc_null_binding+0x5b/0xa0 [iptable_nat] > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> nf_nat_fn+0x91/0x260 [iptable_nat] > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> nf_iterate+0x69/0xb0 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ip_rcv_finish+0x199/0x440 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ip_rcv_finish+0x0/0x440 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> nf_hook_slow+0x74/0x110 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ip_rcv_finish+0x0/0x440 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ip_rcv+0x264/0x350 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> __netif_receive_skb+0x4ab/0x750 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> process_backlog+0x9a/0x100 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> net_rx_action+0x103/0x2f0 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> __do_softirq+0xc1/0x1e0 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> handle_IRQ_event+0x60/0x170 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> call_softirq+0x1c/0x30 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> do_softirq+0x65/0xa0 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> irq_exit+0x85/0x90 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> do_IRQ+0x75/0xf0 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> ret_from_intr+0x0/0x11 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> mwait_idle+0x77/0xd0 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> atomic_notifier_call_chain+0x1a/0x20 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> cpu_idle+0xb6/0x110 > >>> Nov 5 14:13:58 cldnet01 kernel: [] ? > >>> start_secondary+0x2ac/0x2ef > >>> > >>> The only other message in the syslog that's related to CSUM is the > >>> following: > >>> Nov 5 14:10:44 cldnet01 kernel: lo: Dropping TSO features since no > >>> CSUM feature. > >>> Nov 5 14:10:44 cldnet01 kernel: lo: Disabled Privacy Extensions > >>> (this message appears after starting the l3-agent) > >>> > >>> The network host is RHEL6.4, kernel is > >>> 2.6.32-358.123.2.openstack.el6.x86_64 > >>> > >>> All the daemons appear to being running, an instance can start, but > >>> network doesn't work for the instance. > >>> > >>> Any wisdom on what's going on? > > > > Most likely a driver issue, what NIC model and driver are you using? > > ehttool -i $DEV will help. > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey at xdel.ru Wed Nov 6 14:53:14 2013 From: andrey at xdel.ru (Andrey Korolyov) Date: Wed, 6 Nov 2013 18:53:14 +0400 Subject: [rhos-list] iproute2 fix proposal Message-ID: Hello, Can anyone please add make-ip-netns-delete-more-likely-to-succeed.patch from https://launchpad.net/ubuntu/+source/iproute/20111117-1ubuntu2.1 to the iproute-2.6.32-130.el6ost.netns.2 in the current OpenStack repo? This fix seems to be necessary for silent namespace deletion when a) processes are running inside NS and quantum tries to delete it, b) one side of veth pair is inside NS in same situation. From andrey at xdel.ru Wed Nov 6 15:20:15 2013 From: andrey at xdel.ru (Andrey Korolyov) Date: Wed, 6 Nov 2013 19:20:15 +0400 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <527A5564.3010003@bnl.gov> References: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> <527A5564.3010003@bnl.gov> Message-ID: Problem seems to be presented only with virtio drivers for the machines and harmless in the general means. My guess is that the missing checksum offload in the virtio driver causing skb to scream. On Wed, Nov 6, 2013 at 6:42 PM, Xin Zhao wrote: > Hi Paul, > > The kernel is 2.6.32-358.123.2.openstack.el6.x86_64, it is from RDO: > > name=OpenStack Grizzly Repository > baseurl=http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/ > > Thanks, > Xin > > > On 11/5/2013 8:29 PM, Paul Robert Marino wrote: > > Which kernel are you running > If you are running the stock RedHat kernel not the patched version that > comes with RHOS or RDO that might explain it. > > > > -- Sent from my HP Pre3 > > ________________________________ > On Nov 5, 2013 17:33, Xin Zhao wrote: > > Hi Thomas, > > Thanks for the reply, here is the info of the drivers: > > 1) NIC for the VM network connection: > > $> ethtool -i eth1 > driver: e1000e > version: 2.1.4-k > firmware-version: 1.6-12 > bus-info: 0000:04:00.1 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: no > > 2) NIC for the external internet connection: > > $> ethtool -i eth2 > driver: myri10ge > version: 1.5.1-1.451 > firmware-version: 1.4.52 -- 2010/10/28 21:27:06 m > bus-info: 0000:0b:00.0 > supports-statistics: yes > supports-test: no > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: no > > Xin > > > On 11/5/2013 4:55 PM, Thomas Graf wrote: >> On 11/05/2013 10:43 PM, Xin Zhao wrote: >>> Add the general Openstack list, sorry for folks who are on both lists... >>> >>> On 11/5/2013 2:42 PM, Xin Zhao wrote: >>>> Hello, >>>> >>>> On my grizzly quantum/OVS network node, after I start the >>>> quantum-openvswitch-agent, the system log shows errors as below, >>>> and it repeats every second since then... and the panic messages >>>> continue on even after I stop all openstack daemons, only a system >>>> reboot >>>> can clear it out. >>>> >>>> Nov 5 14:13:58 cldnet01 kernel: qg-581539d2-ac: hw csum failure. >>>> Nov 5 14:13:58 cldnet01 kernel: Pid: 0, comm: swapper Not tainted >>>> 2.6.32-358.123.2.openstack.el6.x86_64 #1 >>>> Nov 5 14:13:58 cldnet01 kernel: Call Trace: >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> netdev_rx_csum_fault+0x42/0x50 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> __skb_checksum_complete_head+0x60/0x70 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> __skb_checksum_complete+0x11/0x20 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> nf_ip_checksum+0x5d/0x130 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> udp_error+0xb1/0x1e0 [nf_conntrack] >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> nf_conntrack_in+0x138/0xa00 [nf_conntrack] >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> alloc_null_binding+0x5b/0xa0 [iptable_nat] >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> nf_nat_fn+0x91/0x260 [iptable_nat] >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> nf_iterate+0x69/0xb0 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ip_rcv_finish+0x199/0x440 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ip_rcv_finish+0x0/0x440 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> nf_hook_slow+0x74/0x110 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ip_rcv_finish+0x0/0x440 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ip_rcv+0x264/0x350 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> __netif_receive_skb+0x4ab/0x750 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> process_backlog+0x9a/0x100 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> net_rx_action+0x103/0x2f0 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> __do_softirq+0xc1/0x1e0 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> handle_IRQ_event+0x60/0x170 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> call_softirq+0x1c/0x30 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> do_softirq+0x65/0xa0 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> irq_exit+0x85/0x90 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> do_IRQ+0x75/0xf0 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> ret_from_intr+0x0/0x11 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> mwait_idle+0x77/0xd0 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> atomic_notifier_call_chain+0x1a/0x20 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> cpu_idle+0xb6/0x110 >>>> Nov 5 14:13:58 cldnet01 kernel: [] ? >>>> start_secondary+0x2ac/0x2ef >>>> >>>> The only other message in the syslog that's related to CSUM is the >>>> following: >>>> Nov 5 14:10:44 cldnet01 kernel: lo: Dropping TSO features since no >>>> CSUM feature. >>>> Nov 5 14:10:44 cldnet01 kernel: lo: Disabled Privacy Extensions >>>> (this message appears after starting the l3-agent) >>>> >>>> The network host is RHEL6.4, kernel is >>>> 2.6.32-358.123.2.openstack.el6.x86_64 >>>> >>>> All the daemons appear to being running, an instance can start, but >>>> network doesn't work for the instance. >>>> >>>> Any wisdom on what's going on? >> >> Most likely a driver issue, what NIC model and driver are you using? >> ehttool -i $DEV will help. > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From dfv at eurotux.com Thu Nov 7 19:12:33 2013 From: dfv at eurotux.com (Diogo Vieira) Date: Thu, 7 Nov 2013 19:12:33 +0000 Subject: [rhos-list] Problems starting l3 and openvswitch agents: "sudo: no tty present and no askpass program specified" Message-ID: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> Hi, I've been having problems starting a l3-agent and an openvswitch-agent on one of my nodes. Can someone help me? /var/log/openvswitch-agent.log: 2013-11-07 18:17:08.994 17604 INFO neutron.common.config [-] Logging enabled! 2013-11-07 18:17:09.052 17604 INFO neutron.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.10.4.6:5672 2013-11-07 18:17:09.091 17604 INFO neutron.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.10.4.6:5672 2013-11-07 18:17:09.094 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-port', 'br-int', 'patch-tun']. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-port', 'br-int', 'patch-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 18:17:09.107 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'del-flows', 'br-int']. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'del-flows', 'br-int'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 18:17:09.122 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=1,actions=normal']. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=1,actions=normal'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 18:17:09.136 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to retrieve bridges. Exception: Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-br'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Traceback (most recent call last): 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ovs_lib.py", line 369, in get_bridges 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib return utils.execute(args, root_helper=root_helper).strip().split("\n") 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 62, in execute 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib raise RuntimeError(m) 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib RuntimeError: 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-br'] 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Exit code: 1 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Stdout: '' 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib 2013-11-07 18:17:09.137 17604 CRITICAL neutron [-] 'br-int' 2013-11-07 18:17:09.137 17604 TRACE neutron Traceback (most recent call last): 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/bin/neutron-openvswitch-agent", line 10, in 2013-11-07 18:17:09.137 17604 TRACE neutron sys.exit(main()) 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1196, in main 2013-11-07 18:17:09.137 17604 TRACE neutron plugin = OVSNeutronAgent(**agent_config) 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 213, in __init__ 2013-11-07 18:17:09.137 17604 TRACE neutron self.ancillary_brs = self.setup_ancillary_bridges(integ_br, tun_br) 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 658, in setup_ancillary_bridges 2013-11-07 18:17:09.137 17604 TRACE neutron ovs_bridges.remove(integ_br) 2013-11-07 18:17:09.137 17604 TRACE neutron KeyError: 'br-int' 2013-11-07 18:17:09.137 17604 TRACE neutron /var/log/neutron/l3-agent.log: 2013-11-07 17:23:30.563 16396 INFO neutron.common.config [-] Logging enabled! 2013-11-07 17:23:30.564 16396 ERROR neutron.common.legacy [-] Skipping unknown group key: firewall_driver 2013-11-07 17:23:30.582 16396 CRITICAL neutron [-] Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'list'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 17:23:30.582 16396 TRACE neutron Traceback (most recent call last): 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/bin/neutron-l3-agent", line 10, in 2013-11-07 17:23:30.582 16396 TRACE neutron sys.exit(main()) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 876, in main 2013-11-07 17:23:30.582 16396 TRACE neutron manager=manager) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 204, in create 2013-11-07 17:23:30.582 16396 TRACE neutron periodic_fuzzy_delay=periodic_fuzzy_delay) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 137, in __init__ 2013-11-07 17:23:30.582 16396 TRACE neutron self.manager = manager_class(host=host, *args, **kwargs) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 797, in __init__ 2013-11-07 17:23:30.582 16396 TRACE neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 217, in __init__ 2013-11-07 17:23:30.582 16396 TRACE neutron self._destroy_router_namespaces(self.conf.router_id) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 249, in _destroy_router_namespaces 2013-11-07 17:23:30.582 16396 TRACE neutron for ns in root_ip.get_namespaces(self.root_helper): 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 174, in get_namespaces 2013-11-07 17:23:30.582 16396 TRACE neutron output = cls._execute('', 'netns', ('list',), root_helper=root_helper) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 76, in _execute 2013-11-07 17:23:30.582 16396 TRACE neutron root_helper=root_helper) 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 62, in execute 2013-11-07 17:23:30.582 16396 TRACE neutron raise RuntimeError(m) 2013-11-07 17:23:30.582 16396 TRACE neutron RuntimeError: 2013-11-07 17:23:30.582 16396 TRACE neutron Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'list'] 2013-11-07 17:23:30.582 16396 TRACE neutron Exit code: 1 2013-11-07 17:23:30.582 16396 TRACE neutron Stdout: '' 2013-11-07 17:23:30.582 16396 TRACE neutron Stderr: 'sudo: no tty present and no askpass program specified\n' 2013-11-07 17:23:30.582 16396 TRACE neutron Thank you very much, Diogo Vieira From prmarino1 at gmail.com Thu Nov 7 19:47:10 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 7 Nov 2013 14:47:10 -0500 Subject: [rhos-list] Problems starting l3 and openvswitch agents: "sudo: no tty present and no askpass program specified" In-Reply-To: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> References: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> Message-ID: there is something missing from your sudoers file Im still running quantum so it will be slightly different for you but here is the line that corrects it in /etc/sudoers.d/quantum " Defaults:quantum !requiretty " just replace quantum with the neutron user On Thu, Nov 7, 2013 at 2:12 PM, Diogo Vieira wrote: > Hi, > > I've been having problems starting a l3-agent and an openvswitch-agent on one of my nodes. Can someone help me? > > /var/log/openvswitch-agent.log: > > 2013-11-07 18:17:08.994 17604 INFO neutron.common.config [-] Logging enabled! > 2013-11-07 18:17:09.052 17604 INFO neutron.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.10.4.6:5672 > 2013-11-07 18:17:09.091 17604 INFO neutron.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.10.4.6:5672 > 2013-11-07 18:17:09.094 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-port', 'br-int', 'patch-tun']. Exception: > Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-port', 'br-int', 'patch-tun'] > Exit code: 1 > Stdout: '' > Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 18:17:09.107 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'del-flows', 'br-int']. Exception: > Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'del-flows', 'br-int'] > Exit code: 1 > Stdout: '' > Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 18:17:09.122 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=1,actions=normal']. Exception: > Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=1,actions=normal'] > Exit code: 1 > Stdout: '' > Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 18:17:09.136 17604 ERROR neutron.agent.linux.ovs_lib [-] Unable to retrieve bridges. Exception: > Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-br'] > Exit code: 1 > Stdout: '' > Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Traceback (most recent call last): > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ovs_lib.py", line 369, in get_bridges > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib return utils.execute(args, root_helper=root_helper).strip().split("\n") > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 62, in execute > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib raise RuntimeError(m) > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib RuntimeError: > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'list-br'] > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Exit code: 1 > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Stdout: '' > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 18:17:09.136 17604 TRACE neutron.agent.linux.ovs_lib > 2013-11-07 18:17:09.137 17604 CRITICAL neutron [-] 'br-int' > 2013-11-07 18:17:09.137 17604 TRACE neutron Traceback (most recent call last): > 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/bin/neutron-openvswitch-agent", line 10, in > 2013-11-07 18:17:09.137 17604 TRACE neutron sys.exit(main()) > 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1196, in main > 2013-11-07 18:17:09.137 17604 TRACE neutron plugin = OVSNeutronAgent(**agent_config) > 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 213, in __init__ > 2013-11-07 18:17:09.137 17604 TRACE neutron self.ancillary_brs = self.setup_ancillary_bridges(integ_br, tun_br) > 2013-11-07 18:17:09.137 17604 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 658, in setup_ancillary_bridges > 2013-11-07 18:17:09.137 17604 TRACE neutron ovs_bridges.remove(integ_br) > 2013-11-07 18:17:09.137 17604 TRACE neutron KeyError: 'br-int' > 2013-11-07 18:17:09.137 17604 TRACE neutron > > /var/log/neutron/l3-agent.log: > > 2013-11-07 17:23:30.563 16396 INFO neutron.common.config [-] Logging enabled! > 2013-11-07 17:23:30.564 16396 ERROR neutron.common.legacy [-] Skipping unknown group key: firewall_driver > 2013-11-07 17:23:30.582 16396 CRITICAL neutron [-] > Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'list'] > Exit code: 1 > Stdout: '' > Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 17:23:30.582 16396 TRACE neutron Traceback (most recent call last): > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/bin/neutron-l3-agent", line 10, in > 2013-11-07 17:23:30.582 16396 TRACE neutron sys.exit(main()) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 876, in main > 2013-11-07 17:23:30.582 16396 TRACE neutron manager=manager) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 204, in create > 2013-11-07 17:23:30.582 16396 TRACE neutron periodic_fuzzy_delay=periodic_fuzzy_delay) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/service.py", line 137, in __init__ > 2013-11-07 17:23:30.582 16396 TRACE neutron self.manager = manager_class(host=host, *args, **kwargs) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 797, in __init__ > 2013-11-07 17:23:30.582 16396 TRACE neutron super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 217, in __init__ > 2013-11-07 17:23:30.582 16396 TRACE neutron self._destroy_router_namespaces(self.conf.router_id) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/l3_agent.py", line 249, in _destroy_router_namespaces > 2013-11-07 17:23:30.582 16396 TRACE neutron for ns in root_ip.get_namespaces(self.root_helper): > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 174, in get_namespaces > 2013-11-07 17:23:30.582 16396 TRACE neutron output = cls._execute('', 'netns', ('list',), root_helper=root_helper) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 76, in _execute > 2013-11-07 17:23:30.582 16396 TRACE neutron root_helper=root_helper) > 2013-11-07 17:23:30.582 16396 TRACE neutron File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 62, in execute > 2013-11-07 17:23:30.582 16396 TRACE neutron raise RuntimeError(m) > 2013-11-07 17:23:30.582 16396 TRACE neutron RuntimeError: > 2013-11-07 17:23:30.582 16396 TRACE neutron Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'list'] > 2013-11-07 17:23:30.582 16396 TRACE neutron Exit code: 1 > 2013-11-07 17:23:30.582 16396 TRACE neutron Stdout: '' > 2013-11-07 17:23:30.582 16396 TRACE neutron Stderr: 'sudo: no tty present and no askpass program specified\n' > 2013-11-07 17:23:30.582 16396 TRACE neutron > > Thank you very much, > Diogo Vieira > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From lars at redhat.com Thu Nov 7 20:58:33 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Thu, 7 Nov 2013 15:58:33 -0500 Subject: [rhos-list] Problems starting l3 and openvswitch agents: "sudo: no tty present and no askpass program specified" In-Reply-To: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> References: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> Message-ID: <20131107205833.GC27494@redhat.com> On Thu, Nov 07, 2013 at 07:12:33PM +0000, Diogo Vieira wrote: > Hi, > > I've been having problems starting a l3-agent and an > openvswitch-agent on one of my nodes. Can someone help me? > > /var/log/openvswitch-agent.log: > > [...] > Stderr: 'sudo: no tty present and no askpass program specified\n' This error generally indicates that as a result of an upgrade from quantum to neutron you have both users "quantum" and "neutron" on your system, but only a sudoers file for "neutron". Because the "quantum" and "neutron" users have the same userid, when neutron runs a command via "sudo", sudo evaluates the sudoers file as if the user were named "quantum". Simply deleting the user "quantum" should get things working. -- Lars Kellogg-Stedman | larsks @ irc Cloud Engineering / OpenStack | " " @ twitter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From twilson at redhat.com Thu Nov 7 23:20:28 2013 From: twilson at redhat.com (Terry Wilson) Date: Thu, 7 Nov 2013 18:20:28 -0500 (EST) Subject: [rhos-list] Problems starting l3 and openvswitch agents: "sudo: no tty present and no askpass program specified" In-Reply-To: <20131107205833.GC27494@redhat.com> References: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> <20131107205833.GC27494@redhat.com> Message-ID: <2029341400.20244156.1383866428624.JavaMail.root@redhat.com> ----- Original Message ----- > On Thu, Nov 07, 2013 at 07:12:33PM +0000, Diogo Vieira wrote: > > Hi, > > > > I've been having problems starting a l3-agent and an > > openvswitch-agent on one of my nodes. Can someone help me? > > > > /var/log/openvswitch-agent.log: > > > > [...] > > Stderr: 'sudo: no tty present and no askpass program specified\n' > > This error generally indicates that as a result of an upgrade from > quantum to neutron you have both users "quantum" and "neutron" on your > system, but only a sudoers file for "neutron". Because the "quantum" > and "neutron" users have the same userid, when neutron runs a command > via "sudo", sudo evaluates the sudoers file as if the user were named > "quantum". > > Simply deleting the user "quantum" should get things working. > > -- > Lars Kellogg-Stedman | larsks @ irc > Cloud Engineering / OpenStack | " " @ twitter In addition, this should be fixed as of build openstack-neutron-2013.2-4.el6ost as it no longer attempts to share the quantum/neutron uid/gids. Of course, if you already have the neutron user/group with the re-used uid/gid, you will need to delete the quantum user (or create a sudoers.d/quantum file) as a workaround. Terry From dfv at eurotux.com Fri Nov 8 10:10:09 2013 From: dfv at eurotux.com (Diogo Vieira) Date: Fri, 8 Nov 2013 10:10:09 +0000 Subject: [rhos-list] Problems starting l3 and openvswitch agents: "sudo: no tty present and no askpass program specified" In-Reply-To: <20131107205833.GC27494@redhat.com> References: <39A44B32-CB67-4A9A-A139-F26CB9EFE8C0@eurotux.com> <20131107205833.GC27494@redhat.com> Message-ID: This seems to be the problem. Removing the user quantum worked! On Nov 7, 2013, at 8:58 PM, Lars Kellogg-Stedman wrote: > > This error generally indicates that as a result of an upgrade from > quantum to neutron you have both users "quantum" and "neutron" on your > system, but only a sudoers file for "neutron". Because the "quantum" > and "neutron" users have the same userid, when neutron runs a command > via "sudo", sudo evaluates the sudoers file as if the user were named > "quantum". > > Simply deleting the user "quantum" should get things working. > Thank you very much for all the answers, Diogo Vieira From tgraf at redhat.com Fri Nov 8 13:52:25 2013 From: tgraf at redhat.com (Thomas Graf) Date: Fri, 08 Nov 2013 14:52:25 +0100 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> References: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> Message-ID: <527CEC99.607@redhat.com> On 11/06/2013 02:29 AM, Paul Robert Marino wrote: > Which kernel are you running > If you are running the stock RedHat kernel not the patched version that > comes with RHOS or RDO that might explain it. > > > > -- Sent from my HP Pre3 > > ------------------------------------------------------------------------ > On Nov 5, 2013 17:33, Xin Zhao wrote: > > Hi Thomas, > > Thanks for the reply, here is the info of the drivers: > > 1) NIC for the VM network connection: > > $> ethtool -i eth1 > driver: e1000e > version: 2.1.4-k > firmware-version: 1.6-12 > bus-info: 0000:04:00.1 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: no e1000e is a driver that is known to work. The error would indicate a silicon or firmware issue. Are you seeing this issue on multiple machines? From xzhao at bnl.gov Fri Nov 8 16:39:45 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Fri, 08 Nov 2013 11:39:45 -0500 Subject: [rhos-list] system panic after starting OVS agent In-Reply-To: <527CEC99.607@redhat.com> References: <52799b97.c46cec0a.3acf.ffff97f9@mx.google.com> <527CEC99.607@redhat.com> Message-ID: <527D13D1.7030603@bnl.gov> Hi Thomas, I didn't see similar error message from the compute node, where OVS agent/libvirt/Nova-compute daemons are running. For the network host, I did several things in the last couple of days to address the issue: 1) switch to a new physical machine for the network host, the VM network NIC info is now: [root at cldnet01 bin]# ethtool -i em2 driver: bnx2 version: 2.2.3 firmware-version: 6.2.14 bc 5.2.3 NCSI 2.0.11 bus-info: 0000:01:00.1 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: no 2) installed the latest version of Openstack kernel/packages from RDO on the network host 3) noticed vlan tagging module was not loaded, so I load the 8021q module on both the network and compute nodes 4) set use_namespace to True on both the L3 and DHCP agent config file on network host 5) recreated the bridges (br-int/br-em2/br-ex) on the network host Now if I only start the openvswitch daemon on the network host, the syslog shows similar errors (and repeating it...), but after the quantum-openvswitch-agent daemon starts, the error message stops. Below after my signature is the snippet from the syslog. Now if I start an instance, its eth0 still can't get an IP. Any wisdom on what's going on ? Thanks, Xin (start openvswitch service) Nov 8 11:21:01 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.1.0 Nov 8 11:21:01 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=1.11.0 "external-ids:system-id=\"9161c89c-da3f-43da-a42d-712465fbf71c\"" "system-type=\"unknown\"" "system-version=\"unknown\"" Nov 8 11:21:04 cldnet01 kernel: br-int: hw csum failure. Nov 8 11:21:04 cldnet01 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-358.123.2.openstack.el6.x86_64 #1 Nov 8 11:21:04 cldnet01 kernel: Call Trace: Nov 8 11:21:04 cldnet01 kernel: [] ? netdev_rx_csum_fault+0x42/0x50 Nov 8 11:21:04 cldnet01 kernel: [] ? __skb_checksum_complete_head+0x60/0x70 Nov 8 11:21:04 cldnet01 kernel: [] ? __skb_checksum_complete+0x11/0x20 Nov 8 11:21:04 cldnet01 kernel: [] ? nf_ip_checksum+0x5d/0x130 Nov 8 11:21:04 cldnet01 kernel: [] ? udp_error+0xb1/0x1e0 [nf_conntrack] Nov 8 11:21:04 cldnet01 kernel: [] ? ovs_vport_send+0x22/0x90 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? __skb_clone+0x2e/0x120 Nov 8 11:21:04 cldnet01 kernel: [] ? nf_conntrack_in+0x138/0xa00 [nf_conntrack] Nov 8 11:21:04 cldnet01 kernel: [] ? find_bucket+0x66/0x70 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? ovs_flow_tbl_lookup+0x51/0xb0 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] Nov 8 11:21:04 cldnet01 kernel: [] ? nf_iterate+0x69/0xb0 Nov 8 11:21:04 cldnet01 kernel: [] ? ip_rcv_finish+0x0/0x440 Nov 8 11:21:04 cldnet01 kernel: [] ? nf_hook_slow+0x74/0x110 Nov 8 11:21:04 cldnet01 kernel: [] ? ip_rcv_finish+0x0/0x440 Nov 8 11:21:04 cldnet01 kernel: [] ? ip_rcv+0x264/0x350 Nov 8 11:21:04 cldnet01 kernel: [] ? ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? __netif_receive_skb+0x4ab/0x750 Nov 8 11:21:04 cldnet01 kernel: [] ? process_backlog+0x9a/0x100 Nov 8 11:21:04 cldnet01 kernel: [] ? net_rx_action+0x103/0x2f0 Nov 8 11:21:04 cldnet01 kernel: [] ? __do_softirq+0xc1/0x1e0 Nov 8 11:21:04 cldnet01 kernel: [] ? handle_IRQ_event+0x60/0x170 Nov 8 11:21:04 cldnet01 kernel: [] ? call_softirq+0x1c/0x30 Nov 8 11:21:04 cldnet01 kernel: [] ? do_softirq+0x65/0xa0 Nov 8 11:21:04 cldnet01 kernel: [] ? irq_exit+0x85/0x90 Nov 8 11:21:04 cldnet01 kernel: [] ? do_IRQ+0x75/0xf0 Nov 8 11:21:04 cldnet01 kernel: [] ? ret_from_intr+0x0/0x11 Nov 8 11:21:04 cldnet01 kernel: [] ? intel_idle+0xde/0x170 Nov 8 11:21:04 cldnet01 kernel: [] ? intel_idle+0xc1/0x170 Nov 8 11:21:04 cldnet01 kernel: [] ? cpuidle_idle_call+0xa7/0x140 Nov 8 11:21:04 cldnet01 kernel: [] ? cpu_idle+0xb6/0x110 Nov 8 11:21:04 cldnet01 kernel: [] ? start_secondary+0x2ac/0x2ef Nov 8 11:21:04 cldnet01 kernel: br-em2: hw csum failure. Nov 8 11:21:04 cldnet01 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-358.123.2.openstack.el6.x86_64 #1 Nov 8 11:21:04 cldnet01 kernel: Call Trace: Nov 8 11:21:04 cldnet01 kernel: [] ? netdev_rx_csum_fault+0x42/0x50 Nov 8 11:21:04 cldnet01 kernel: [] ? __skb_checksum_complete_head+0x60/0x70 Nov 8 11:21:04 cldnet01 kernel: [] ? __skb_checksum_complete+0x11/0x20 Nov 8 11:21:04 cldnet01 kernel: [] ? nf_ip_checksum+0x5d/0x130 Nov 8 11:21:04 cldnet01 kernel: [] ? udp_error+0xb1/0x1e0 [nf_conntrack] Nov 8 11:21:04 cldnet01 kernel: [] ? ovs_vport_send+0x22/0x90 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? __skb_clone+0x2e/0x120 Nov 8 11:21:04 cldnet01 kernel: [] ? nf_conntrack_in+0x138/0xa00 [nf_conntrack] Nov 8 11:21:04 cldnet01 kernel: [] ? find_bucket+0x66/0x70 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? ovs_flow_tbl_lookup+0x51/0xb0 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? ipv4_conntrack_in+0x21/0x30 [nf_conntrack_ipv4] Nov 8 11:21:04 cldnet01 kernel: [] ? nf_iterate+0x69/0xb0 Nov 8 11:21:04 cldnet01 kernel: [] ? ip_rcv_finish+0x0/0x440 Nov 8 11:21:04 cldnet01 kernel: [] ? nf_hook_slow+0x74/0x110 Nov 8 11:21:04 cldnet01 kernel: [] ? ip_rcv_finish+0x0/0x440 Nov 8 11:21:04 cldnet01 kernel: [] ? ip_rcv+0x264/0x350 Nov 8 11:21:04 cldnet01 kernel: [] ? ovs_netdev_frame_hook+0xb3/0x110 [openvswitch] Nov 8 11:21:04 cldnet01 kernel: [] ? __netif_receive_skb+0x4ab/0x750 Nov 8 11:21:04 cldnet01 kernel: [] ? process_backlog+0x9a/0x100 Nov 8 11:21:04 cldnet01 kernel: [] ? net_rx_action+0x103/0x2f0 Nov 8 11:21:04 cldnet01 kernel: [] ? __do_softirq+0xc1/0x1e0 Nov 8 11:21:04 cldnet01 kernel: [] ? handle_IRQ_event+0x60/0x170 Nov 8 11:21:04 cldnet01 kernel: [] ? call_softirq+0x1c/0x30 Nov 8 11:21:04 cldnet01 kernel: [] ? do_softirq+0x65/0xa0 Nov 8 11:21:04 cldnet01 kernel: [] ? irq_exit+0x85/0x90 Nov 8 11:21:04 cldnet01 kernel: [] ? do_IRQ+0x75/0xf0 Nov 8 11:21:04 cldnet01 kernel: [] ? ret_from_intr+0x0/0x11 Nov 8 11:21:04 cldnet01 kernel: [] ? intel_idle+0xde/0x170 Nov 8 11:21:04 cldnet01 kernel: [] ? intel_idle+0xc1/0x170 Nov 8 11:21:04 cldnet01 kernel: [] ? cpuidle_idle_call+0xa7/0x140 Nov 8 11:21:04 cldnet01 kernel: [] ? cpu_idle+0xb6/0x110 Nov 8 11:21:04 cldnet01 kernel: [] ? start_secondary+0x2ac/0x2ef ... ... ... (start quantum-openvswitch-agent service) Nov 8 11:21:13 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-int patch-tun Nov 8 11:21:13 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-int int-br-ex Nov 8 11:21:13 cldnet01 kernel: device int-br-ex left promiscuous mode Nov 8 11:21:13 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-ex phy-br-ex Nov 8 11:21:13 cldnet01 kernel: device phy-br-ex left promiscuous mode Nov 8 11:21:14 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --may-exist add-port br-int int-br-ex Nov 8 11:21:14 cldnet01 kernel: device int-br-ex entered promiscuous mode Nov 8 11:21:14 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --may-exist add-port br-ex phy-br-ex Nov 8 11:21:14 cldnet01 kernel: device phy-br-ex entered promiscuous mode Nov 8 11:21:14 cldnet01 kernel: ADDRCONF(NETDEV_UP): int-br-ex: link is not ready Nov 8 11:21:14 cldnet01 kernel: ADDRCONF(NETDEV_CHANGE): int-br-ex: link becomes ready Nov 8 11:21:14 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-int int-br-em2 Nov 8 11:21:14 cldnet01 kernel: device int-br-em2 left promiscuous mode Nov 8 11:21:14 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-em2 phy-br-em2 Nov 8 11:21:14 cldnet01 kernel: device phy-br-em2 left promiscuous mode Nov 8 11:21:16 cldnet01 ntpd[2295]: Deleting interface #26 int-br-ex, fe80::68e7:66ff:fef7:f28b#123, interface stats: received=0, sent=0, dropped=0, active_time=1853 secs Nov 8 11:21:16 cldnet01 ntpd[2295]: Deleting interface #27 phy-br-ex, fe80::94dd:f8ff:fe75:e19b#123, interface stats: received=0, sent=0, dropped=0, active_time=1853 secs Nov 8 11:21:16 cldnet01 ntpd[2295]: Deleting interface #28 int-br-em2, fe80::bc7e:9aff:fe93:2a0c#123, interface stats: received=0, sent=0, dropped=0, active_time=1851 secs Nov 8 11:21:16 cldnet01 ntpd[2295]: Deleting interface #29 phy-br-em2, fe80::20b0:56ff:fe3e:7019#123, interface stats: received=0, sent=0, dropped=0, active_time=1851 secs Nov 8 11:21:16 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --may-exist add-port br-int int-br-em2 Nov 8 11:21:16 cldnet01 kernel: device int-br-em2 entered promiscuous mode Nov 8 11:21:16 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --may-exist add-port br-em2 phy-br-em2 Nov 8 11:21:16 cldnet01 kernel: device phy-br-em2 entered promiscuous mode Nov 8 11:21:16 cldnet01 kernel: ADDRCONF(NETDEV_UP): int-br-em2: link is not ready Nov 8 11:21:16 cldnet01 kernel: ADDRCONF(NETDEV_CHANGE): int-br-em2: link becomes ready Nov 8 11:21:17 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qr-ed2e07cf-db tag=1 Nov 8 11:21:17 cldnet01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port tap4aafbf9c-49 tag=1 Nov 8 11:21:18 cldnet01 ntpd[2295]: Listening on interface #30 int-br-ex, fe80::14d5:daff:feda:2941#123 Enabled Nov 8 11:21:18 cldnet01 ntpd[2295]: Listening on interface #31 phy-br-ex, fe80::5452:6bff:fe41:27bc#123 Enabled Nov 8 11:21:20 cldnet01 ntpd[2295]: Listening on interface #32 int-br-em2, fe80::4067:b1ff:fe50:194c#123 Enabled Nov 8 11:21:20 cldnet01 ntpd[2295]: Listening on interface #33 phy-br-em2, fe80::bc54:5dff:fedd:64fe#123 Enabled (no more "hw csum failure" errors from here on) On 11/8/2013 8:52 AM, Thomas Graf wrote: > On 11/06/2013 02:29 AM, Paul Robert Marino wrote: >> Which kernel are you running >> If you are running the stock RedHat kernel not the patched version that >> comes with RHOS or RDO that might explain it. >> >> >> >> -- Sent from my HP Pre3 >> >> ------------------------------------------------------------------------ >> On Nov 5, 2013 17:33, Xin Zhao wrote: >> >> Hi Thomas, >> >> Thanks for the reply, here is the info of the drivers: >> >> 1) NIC for the VM network connection: >> >> $> ethtool -i eth1 >> driver: e1000e >> version: 2.1.4-k >> firmware-version: 1.6-12 >> bus-info: 0000:04:00.1 >> supports-statistics: yes >> supports-test: yes >> supports-eeprom-access: yes >> supports-register-dump: yes >> supports-priv-flags: no > > e1000e is a driver that is known to work. The error would indicate a > silicon or firmware issue. Are you seeing this issue on multiple > machines? From vijaykakkars at gmail.com Fri Nov 8 17:27:20 2013 From: vijaykakkars at gmail.com (Vijay Kakkar) Date: Fri, 8 Nov 2013 22:57:20 +0530 Subject: [rhos-list] OpenStack Network Issues in All In One Setup Message-ID: Hi Everyone, I am testing All In One setup of RHOS ( Grizzly ) on one VM which is hosted on my Fedora19 laptop.I have RHEL6.4 64bit installed of the VM and have configured the RHOS using packstack.The setup is working smooth,i am able to launch the instances but the challenge is the OpenStack Networking. Following is my network setup 1. I have Linux Bridge br100 configured on my fedora19 which i am using for all the external communication and have shared the same device to my VM for external communication. 2.VM has br-ex (OVSBridge) being setup for external communication and has same range of IP address as on br100 of fedora19. Please help/suggest what i have made the error. -- Regards Vijay Kakkar -------------- next part -------------- An HTML attachment was scrubbed... URL: From twilson at redhat.com Fri Nov 8 18:53:11 2013 From: twilson at redhat.com (Terry Wilson) Date: Fri, 8 Nov 2013 13:53:11 -0500 (EST) Subject: [rhos-list] OpenStack Network Issues in All In One Setup In-Reply-To: References: Message-ID: <1165896042.20864092.1383936791736.JavaMail.root@redhat.com> ----- Original Message ----- > Hi Everyone, > > I am testing All In One setup of RHOS ( Grizzly ) on one VM which is hosted > on my Fedora19 laptop.I have RHEL6.4 64bit installed of the VM and have > configured the RHOS using packstack.The setup is working smooth,i am able to > launch the instances but the challenge is the OpenStack Networking. > > Following is my network setup > > 1. I have Linux Bridge br100 configured on my fedora19 which i am using for > all the external communication and have shared the same device to my VM for > external communication. > > 2.VM has br-ex (OVSBridge) being setup for external communication and has > same range of IP address as on br100 of fedora19. > > > Please help/suggest what i have made the error. Not enough info for me to know what exactly is going wrong, but I'd read over http://openstack.redhat.com/Neutron_with_existing_external_network for getting an allinone install working with an existing external network. It was written for havana, but I think it should work for grizzly by substituting quantum for neutron. Terry From sdake at redhat.com Fri Nov 8 19:08:15 2013 From: sdake at redhat.com (Steven Dake) Date: Fri, 08 Nov 2013 12:08:15 -0700 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <52739750.7060207@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> Message-ID: <527D369F.4010104@redhat.com> On 11/01/2013 04:58 AM, Perry Myers wrote: > On 10/31/2013 04:49 PM, David Raddatz wrote: >> A quick post-mortem on this issue and I wanted to share what we learned (or "I" learned anyway) in case it avoids problems for others... >> >> While I was apparently following the docs by "just install cloud-init on your VM image", while working with Lars we discovered if you reboot your image after installing cloud-init, it will create some files/directories in /var/lib/cloud which will then prevent the instance from allowing ssh to work (my Permission denied issue I was seeing). >> >> For example, doing the following: >> - create VM and install RHEL 6.4 and set up with other files/software >> - install/configure cloud-init on VM >> - reboot VM (just to make sure it still boots) >> - shutdown VM >> - run virt-sysprep on image (per recommendation in docs) >> - upload to glance and launch instance using the defined keypair >> - trying to use ssh -i with the same keypair results in Permission denied >> >> Whereas, doing the following: >> - create VM and install RHEL 6.4 and set up with other files/software >> - install/configure cloud-init on VM >> - shutdown VM >> - run virt-sysprep on image (per recommendation in docs) >> - upload to glance and launch instance using the defined keypair >> - trying to use ssh -i with the same keypair Works! No password prompt (as expected) >> >> Not rebooting the VM allowed the ssh to work as the /var/lib/cloud directory was empty when it was shutdown. >> >> Not sure if this is a doc issue (to warn people NOT to reboot the VM after cloud-init installation/configuration) or a bug (which Lars was going to investigate a little) but thought it was worth a quick email to warn folks about. >> >> Thanks again to Lars for his assistance on this, > Interesting. Lars/Steve, is this a bug or just a docs issue do you think? > > Perry Perry, The tl;dr version is the code is working as intended. Recommend more docs. The cloud-init code is designed to only run the cloud config modules once by default. Modules can be configured to "run-always" with an optional override configurable per module. Cloud-init places locks in the /var/lib/cloud/* directories to prevent modules that have already run from running on the next boot. If the boot process fails in some way (eg the metadata service is unavailable) the modules still run, and locks are still created. The rationale is that if you have physical storage connected to the virtual machine, it doesn't make sense to run the configuration that the cloud config provides on each fresh reboot of the vm since it could at worse confuse/break the operating system's existing storage or at best result in slower boot cycles. One possible improvement that could be considered upstream is to record those locks only on successful module executions. I believe that would solve the particular problem Lars pointed out in a followup email. Another possibility is to specify run-always in the cloudconfig, but this may result in unintended consequences as described previously. Regards -steve > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From lars at redhat.com Fri Nov 8 19:16:02 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Fri, 8 Nov 2013 14:16:02 -0500 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <527D369F.4010104@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> Message-ID: <20131108191602.GC16571@redhat.com> On Fri, Nov 08, 2013 at 12:08:15PM -0700, Steven Dake wrote: > The tl;dr version is the code is working as intended. Recommend more docs. I think the actual situation was something different. In the filesystem image Dave sent to me, something had created a *directory* called /var/lib/cloud/instance. Normally this is a symlink into instances/, and having this a directory was preventing cloud-init from creating the symlink and was making it fall over. Removing this directory allowed cloud-init to proceed as intended. I was not able to reproduce this failure mode in my own testing, so I'm not sure how that directory go there. > The cloud-init code is designed to only run the cloud config modules once by > default. This wasn't an issue with the per-instance semaphores. -- Lars Kellogg-Stedman | larsks @ irc Cloud Engineering / OpenStack | " " @ twitter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From lars at redhat.com Fri Nov 8 19:16:44 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Fri, 8 Nov 2013 14:16:44 -0500 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <527D369F.4010104@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> Message-ID: <20131108191644.GD16571@redhat.com> On Fri, Nov 08, 2013 at 12:08:15PM -0700, Steven Dake wrote: > Recommend more docs. I absolutely agree on this. The upstream docs aren't bad but people aren't always looking upstream for this information. -- Lars Kellogg-Stedman | larsks @ irc Cloud Engineering / OpenStack | " " @ twitter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From sdake at redhat.com Fri Nov 8 19:36:21 2013 From: sdake at redhat.com (Steven Dake) Date: Fri, 08 Nov 2013 12:36:21 -0700 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <20131108191602.GC16571@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> Message-ID: <527D3D35.2070200@redhat.com> On 11/08/2013 12:16 PM, Lars Kellogg-Stedman wrote: > On Fri, Nov 08, 2013 at 12:08:15PM -0700, Steven Dake wrote: >> The tl;dr version is the code is working as intended. Recommend more docs. > I think the actual situation was something different. > > In the filesystem image Dave sent to me, something had created a > *directory* called /var/lib/cloud/instance. Normally this is a > symlink into instances/, and having this a directory was > preventing cloud-init from creating the symlink and was making it fall > over. > > Removing this directory allowed cloud-init to proceed as intended. I > was not able to reproduce this failure mode in my own testing, so I'm > not sure how that directory go there. I see. If cloud-init spits out some kind of odd error like "couldn't create symlink, giving up", then that sounds like a bug that needs root-cause analysis. IIRC the lock files are stored in that instance directory, but they may actually have been in the symlinked instance-id dirs. In recommend opening a bug so we can track logs/progress/image contents as opposed to using a mailing list to triage bugs. If we can figure out how the image was created, it is something that can be reproduced. But without a reproducer, the problem may go unsolved. Regards -steve >> The cloud-init code is designed to only run the cloud config modules once by >> default. > This wasn't an issue with the per-instance semaphores. > From lars at redhat.com Fri Nov 8 20:26:18 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Fri, 8 Nov 2013 15:26:18 -0500 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <527D3D35.2070200@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> Message-ID: <20131108202618.GF16571@redhat.com> On Fri, Nov 08, 2013 at 12:36:21PM -0700, Steven Dake wrote: > In recommend opening a bug so we can track logs/progress/image contents as > opposed to using a mailing list to triage bugs. If we can figure out how > the image was created, it is something that can be reproduced. I opened https://bugzilla.redhat.com/show_bug.cgi?id=1025749 on this issue several days ago, and I spent some time trying to reproduce the failure with various permutations of removing /var/lib/cloud in whole or in part and booting with or without an available metadataserver and I wasn't able to reproduce the problem. Dave spent a lot of time fiddling with his images trying to get them working so it's entirely possible this was ultimately a user-induced failure. Unless someone else is seeing this behavior I don't think it's worth pursuing. -- Lars Kellogg-Stedman | larsks @ irc Cloud Engineering / OpenStack | " " @ twitter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From draddatz at sgi.com Fri Nov 8 20:27:17 2013 From: draddatz at sgi.com (David Raddatz) Date: Fri, 8 Nov 2013 20:27:17 +0000 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <527D3D35.2070200@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> Message-ID: <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> Hi, Guys, Thanks for following up on this... Regarding the instance directory that was created... Here's what I did to recreate the error (if it helps with any debug): It's been awhile but I'm pretty sure that /var/lib/cloud/instance directory was getting created after I rebooted my VM image right after installing cloud-init but BEFORE I uploaded it to glance (I was rebooting the VM image just to make sure it would still boot after installing cloud-init). Here are the steps I did to recreate the problem after working with Lars - create VM and install RHEL 6.4 and set up with other files/software - install/configure cloud-init on VM If you check /var/lib/cloud, it'll be empty. At this point, ssh to the image works fine. - reboot VM (my paranoid reboot thinking I needed to check that the VM would still reboot) If you check /var/lib/cloud now it will have the instance directory and some other files/directories Now at this point, ssh to the image is broken even before preparing and uploading it to glance - I get the Permission denied message. - shutdown VM - run virt-sysprep on image (per recommendation in docs) - upload to glance and launch instance using the defined keypair - trying to use ssh -i with the same keypair results in Permission denied same as above I'm fairly certain the image I sent to Lars was an image created by using these steps. So the filesystem I sent to Lars was one that had the /var/lib/cloud/instance directory because I had rebooted it and it was the problem image that I needed help with. Hope that helps, Dave > -----Original Message----- > From: rhos-list-bounces at redhat.com [mailto:rhos-list- > bounces at redhat.com] On Behalf Of Steven Dake > Sent: Friday, November 08, 2013 1:36 PM > To: Lars Kellogg-Stedman > Cc: Perry Myers; rhos-list at redhat.com > Subject: Re: [rhos-list] cloud-init configuration for ssh access > > On 11/08/2013 12:16 PM, Lars Kellogg-Stedman wrote: > > On Fri, Nov 08, 2013 at 12:08:15PM -0700, Steven Dake wrote: > >> The tl;dr version is the code is working as intended. Recommend more > docs. > > I think the actual situation was something different. > > > > In the filesystem image Dave sent to me, something had created a > > *directory* called /var/lib/cloud/instance. Normally this is a > > symlink into instances/, and having this a directory was > > preventing cloud-init from creating the symlink and was making it fall > > over. > > > > Removing this directory allowed cloud-init to proceed as intended. I > > was not able to reproduce this failure mode in my own testing, so I'm > > not sure how that directory go there. > I see. If cloud-init spits out some kind of odd error like "couldn't create > symlink, giving up", then that sounds like a bug that needs root-cause > analysis. IIRC the lock files are stored in that instance directory, but they may > actually have been in the symlinked instance-id dirs. > > In recommend opening a bug so we can track logs/progress/image contents > as opposed to using a mailing list to triage bugs. If we can figure out how the > image was created, it is something that can be reproduced. > > But without a reproducer, the problem may go unsolved. > > Regards > -steve > > >> The cloud-init code is designed to only run the cloud config modules > >> once by default. > > This wasn't an issue with the per-instance semaphores. > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From draddatz at sgi.com Fri Nov 8 20:48:46 2013 From: draddatz at sgi.com (David Raddatz) Date: Fri, 8 Nov 2013 20:48:46 +0000 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <20131108202618.GF16571@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> <20131108202618.GF16571@redhat.com> Message-ID: <18CF1869BE7AB04DB1E4CC93FD43702A1B731DA1@P-EXMB2-DC21.corp.sgi.com> Hi, Lars, Please see my note that pass through the wires while you sent your email below. It has how to recreate the issue (assuming one starts from a fresh VM installation). I was able to recreate it pretty easily and I think it's repeatable/reproducible. I can provide more details offline if needed (or if I had access to the bugzilla, my id didn't have access). Dave > -----Original Message----- > From: rhos-list-bounces at redhat.com [mailto:rhos-list- > bounces at redhat.com] On Behalf Of Lars Kellogg-Stedman > Sent: Friday, November 08, 2013 2:26 PM > To: Steven Dake > Cc: Perry Myers; rhos-list at redhat.com > Subject: Re: [rhos-list] cloud-init configuration for ssh access > > On Fri, Nov 08, 2013 at 12:36:21PM -0700, Steven Dake wrote: > > In recommend opening a bug so we can track logs/progress/image > > contents as opposed to using a mailing list to triage bugs. If we can > > figure out how the image was created, it is something that can be > reproduced. > > I opened https://bugzilla.redhat.com/show_bug.cgi?id=1025749 on this issue > several days ago, and I spent some time trying to reproduce the failure with > various permutations of removing /var/lib/cloud in whole or in part and > booting with or without an available metadataserver and I wasn't able to > reproduce the problem. > > Dave spent a lot of time fiddling with his images trying to get them working > so it's entirely possible this was ultimately a user-induced failure. > > Unless someone else is seeing this behavior I don't think it's worth pursuing. > > -- > Lars Kellogg-Stedman | larsks @ irc > Cloud Engineering / OpenStack | " " @ twitter From lars at redhat.com Fri Nov 8 20:54:49 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Fri, 8 Nov 2013 15:54:49 -0500 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> Message-ID: <20131108205449.GG16571@redhat.com> On Fri, Nov 08, 2013 at 08:27:17PM +0000, David Raddatz wrote: > - install/configure cloud-init on VM > If you check /var/lib/cloud, it'll be empty. At this point, ssh to the image works fine. > - reboot VM (my paranoid reboot thinking I needed to check that the VM would still reboot) > If you check /var/lib/cloud now it will have the instance directory and some other files/directories This is the part I can't replicate. In every scenario I've tried, cloud-init is correctly creating the /var/lib/instance *symlink*. Can you verify which version of cloud-init was installed when you ran into this problem? I think some of these issues you encountered may be solvable with better documentation (maybe some sort of "how to cloud-init-ify an image"). -- Lars Kellogg-Stedman | larsks @ irc Cloud Engineering / OpenStack | " " @ twitter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From sdake at redhat.com Fri Nov 8 21:03:15 2013 From: sdake at redhat.com (Steven Dake) Date: Fri, 08 Nov 2013 14:03:15 -0700 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> Message-ID: <527D5193.7030800@redhat.com> On 11/08/2013 01:27 PM, David Raddatz wrote: > Hi, Guys, > > Thanks for following up on this... Regarding the instance directory that was created... Here's what I did to recreate the error (if it helps with any debug): > > It's been awhile but I'm pretty sure that /var/lib/cloud/instance directory was getting created after I rebooted my VM image right after installing cloud-init but BEFORE I uploaded it to glance (I was rebooting the VM image just to make sure it would still boot after installing cloud-init). > > Here are the steps I did to recreate the problem after working with Lars > - create VM and install RHEL 6.4 and set up with other files/software > - install/configure cloud-init on VM > If you check /var/lib/cloud, it'll be empty. At this point, ssh to the image works fine. > - reboot VM (my paranoid reboot thinking I needed to check that the VM would still reboot) > If you check /var/lib/cloud now it will have the instance directory and some other files/directories > Now at this point, ssh to the image is broken even before preparing and uploading it to glance - I get the Permission denied message. > - shutdown VM > - run virt-sysprep on image (per recommendation in docs) > - upload to glance and launch instance using the defined keypair > - trying to use ssh -i with the same keypair results in Permission denied same as above > > I'm fairly certain the image I sent to Lars was an image created by using these steps. So the filesystem I sent to Lars was one that had the /var/lib/cloud/instance directory because I had rebooted it and it was the problem image that I needed help with. > > Hope that helps, > Dave Dave, Can you add your reproducer to: https://bugzilla.redhat.com/show_bug.cgi?id=1025749 Even if Lars can't reproduce it, better to have the reproducer in bugzilla then lost in a mailing list :) Regards -steve >> -----Original Message----- >> From: rhos-list-bounces at redhat.com [mailto:rhos-list- >> bounces at redhat.com] On Behalf Of Steven Dake >> Sent: Friday, November 08, 2013 1:36 PM >> To: Lars Kellogg-Stedman >> Cc: Perry Myers; rhos-list at redhat.com >> Subject: Re: [rhos-list] cloud-init configuration for ssh access >> >> On 11/08/2013 12:16 PM, Lars Kellogg-Stedman wrote: >>> On Fri, Nov 08, 2013 at 12:08:15PM -0700, Steven Dake wrote: >>>> The tl;dr version is the code is working as intended. Recommend more >> docs. >>> I think the actual situation was something different. >>> >>> In the filesystem image Dave sent to me, something had created a >>> *directory* called /var/lib/cloud/instance. Normally this is a >>> symlink into instances/, and having this a directory was >>> preventing cloud-init from creating the symlink and was making it fall >>> over. >>> >>> Removing this directory allowed cloud-init to proceed as intended. I >>> was not able to reproduce this failure mode in my own testing, so I'm >>> not sure how that directory go there. >> I see. If cloud-init spits out some kind of odd error like "couldn't create >> symlink, giving up", then that sounds like a bug that needs root-cause >> analysis. IIRC the lock files are stored in that instance directory, but they may >> actually have been in the symlinked instance-id dirs. >> >> In recommend opening a bug so we can track logs/progress/image contents >> as opposed to using a mailing list to triage bugs. If we can figure out how the >> image was created, it is something that can be reproduced. >> >> But without a reproducer, the problem may go unsolved. >> >> Regards >> -steve >> >>>> The cloud-init code is designed to only run the cloud config modules >>>> once by default. >>> This wasn't an issue with the per-instance semaphores. >>> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list From draddatz at sgi.com Fri Nov 8 21:49:59 2013 From: draddatz at sgi.com (David Raddatz) Date: Fri, 8 Nov 2013 21:49:59 +0000 Subject: [rhos-list] cloud-init configuration for ssh access Message-ID: <18CF1869BE7AB04DB1E4CC93FD43702A1B731DC8@P-EXMB2-DC21.corp.sgi.com> Hi, Lars, Please see below for my responses... Thanks, Dave > -----Original Message----- > From: Lars Kellogg-Stedman [mailto:lars at redhat.com] > Sent: Friday, November 08, 2013 2:55 PM > To: David Raddatz > Cc: Steven Dake; Perry Myers; rhos-list at redhat.com > Subject: Re: [rhos-list] cloud-init configuration for ssh access > > On Fri, Nov 08, 2013 at 08:27:17PM +0000, David Raddatz wrote: > > - install/configure cloud-init on VM > > If you check /var/lib/cloud, it'll be empty. At this point, ssh to the image > works fine. > > - reboot VM (my paranoid reboot thinking I needed to check that the VM > would still reboot) > > If you check /var/lib/cloud now it will have the instance directory > > and some other files/directories > > This is the part I can't replicate. In every scenario I've tried, cloud-init is > correctly creating the /var/lib/instance *symlink*. Interesting - I just re-created the issue again and the instance directory is not a symlink to anything but a directory containing a boot-finished file. The /var/lib/cloud directory contains the following directories: data handlers instance instances scripts seed sem All I do is install RHEL6.4 on a new VM, install cloud-init, reboot. When the VM is booting after the cloud-init install there are lots of http errors. None of this is using anything from OpenStack really - doing this from the VM's console in virt-manager. If I ssh to the VM before I reboot the VM, it works. If I try to ssh to the VM after the reboot, I get the "Permission denied (publickey,gssapi-keyex,gssapi-with-mic)" error (which then prevents ssh from working with the rhos keypair in openstack). Obviously, there must be something in my environment that is causing this (or a difference between our environments). > > Can you verify which version of cloud-init was installed when you ran into this > problem? I have this version of cloud-init: cloud-init-0.7.1-2.el6.noarch. > > I think some of these issues you encountered may be solvable with better > documentation (maybe some sort of "how to cloud-init-ify an image"). Yeah - the main thing that got me working was not rebooting my VM image after installing cloud-init. Then I could sysprep it, upload to glance and the ssh with keypair worked on my instances. Dave > > -- > Lars Kellogg-Stedman | larsks @ irc > Cloud Engineering / OpenStack | " " @ twitter From draddatz at sgi.com Fri Nov 8 21:51:15 2013 From: draddatz at sgi.com (David Raddatz) Date: Fri, 8 Nov 2013 21:51:15 +0000 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <527D5193.7030800@redhat.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> <527D5193.7030800@redhat.com> Message-ID: <18CF1869BE7AB04DB1E4CC93FD43702A1B731DD7@P-EXMB2-DC21.corp.sgi.com> > -----Original Message----- > From: Steven Dake [mailto:sdake at redhat.com] > Sent: Friday, November 08, 2013 3:03 PM > To: David Raddatz; Lars Kellogg-Stedman > Cc: Perry Myers; rhos-list at redhat.com > Subject: Re: [rhos-list] cloud-init configuration for ssh access > > On 11/08/2013 01:27 PM, David Raddatz wrote: > > Hi, Guys, > > > > Thanks for following up on this... Regarding the instance directory that was > created... Here's what I did to recreate the error (if it helps with any debug): > > > > It's been awhile but I'm pretty sure that /var/lib/cloud/instance directory > was getting created after I rebooted my VM image right after installing cloud- > init but BEFORE I uploaded it to glance (I was rebooting the VM image just to > make sure it would still boot after installing cloud-init). > > > > Here are the steps I did to recreate the problem after working with > > Lars > > - create VM and install RHEL 6.4 and set up with other files/software > > - install/configure cloud-init on VM > > If you check /var/lib/cloud, it'll be empty. At this point, ssh to the image > works fine. > > - reboot VM (my paranoid reboot thinking I needed to check that the VM > would still reboot) > > If you check /var/lib/cloud now it will have the instance directory and > some other files/directories > > Now at this point, ssh to the image is broken even before preparing and > uploading it to glance - I get the Permission denied message. > > - shutdown VM > > - run virt-sysprep on image (per recommendation in docs) > > - upload to glance and launch instance using the defined keypair > > - trying to use ssh -i with the same keypair results in Permission > > denied same as above > > > > I'm fairly certain the image I sent to Lars was an image created by using > these steps. So the filesystem I sent to Lars was one that had the > /var/lib/cloud/instance directory because I had rebooted it and it was the > problem image that I needed help with. > > > > Hope that helps, > > Dave > Dave, > > Can you add your reproducer to: > https://bugzilla.redhat.com/show_bug.cgi?id=1025749 Hi, Steve, I would but my id (my email address) doesn't have access to the bugzilla report. Dave > > Even if Lars can't reproduce it, better to have the reproducer in bugzilla then > lost in a mailing list :) > > Regards > -steve > > >> -----Original Message----- > >> From: rhos-list-bounces at redhat.com [mailto:rhos-list- > >> bounces at redhat.com] On Behalf Of Steven Dake > >> Sent: Friday, November 08, 2013 1:36 PM > >> To: Lars Kellogg-Stedman > >> Cc: Perry Myers; rhos-list at redhat.com > >> Subject: Re: [rhos-list] cloud-init configuration for ssh access > >> > >> On 11/08/2013 12:16 PM, Lars Kellogg-Stedman wrote: > >>> On Fri, Nov 08, 2013 at 12:08:15PM -0700, Steven Dake wrote: > >>>> The tl;dr version is the code is working as intended. Recommend > >>>> more > >> docs. > >>> I think the actual situation was something different. > >>> > >>> In the filesystem image Dave sent to me, something had created a > >>> *directory* called /var/lib/cloud/instance. Normally this is a > >>> symlink into instances/, and having this a directory > >>> was preventing cloud-init from creating the symlink and was making > >>> it fall over. > >>> > >>> Removing this directory allowed cloud-init to proceed as intended. > >>> I was not able to reproduce this failure mode in my own testing, so > >>> I'm not sure how that directory go there. > >> I see. If cloud-init spits out some kind of odd error like "couldn't > >> create symlink, giving up", then that sounds like a bug that needs > >> root-cause analysis. IIRC the lock files are stored in that instance > >> directory, but they may actually have been in the symlinked instance-id > dirs. > >> > >> In recommend opening a bug so we can track logs/progress/image > >> contents as opposed to using a mailing list to triage bugs. If we > >> can figure out how the image was created, it is something that can be > reproduced. > >> > >> But without a reproducer, the problem may go unsolved. > >> > >> Regards > >> -steve > >> > >>>> The cloud-init code is designed to only run the cloud config > >>>> modules once by default. > >>> This wasn't an issue with the per-instance semaphores. > >>> > >> _______________________________________________ > >> rhos-list mailing list > >> rhos-list at redhat.com > >> https://www.redhat.com/mailman/listinfo/rhos-list From lars at redhat.com Sat Nov 9 00:41:39 2013 From: lars at redhat.com (Lars Kellogg-Stedman) Date: Fri, 8 Nov 2013 19:41:39 -0500 Subject: [rhos-list] cloud-init configuration for ssh access In-Reply-To: <18CF1869BE7AB04DB1E4CC93FD43702A1B731DD7@P-EXMB2-DC21.corp.sgi.com> References: <18CF1869BE7AB04DB1E4CC93FD43702A1B731106@P-EXMB2-DC21.corp.sgi.com> <52739750.7060207@redhat.com> <527D369F.4010104@redhat.com> <20131108191602.GC16571@redhat.com> <527D3D35.2070200@redhat.com> <18CF1869BE7AB04DB1E4CC93FD43702A1B731D73@P-EXMB2-DC21.corp.sgi.com> <527D5193.7030800@redhat.com> <18CF1869BE7AB04DB1E4CC93FD43702A1B731DD7@P-EXMB2-DC21.corp.sgi.com> Message-ID: <20131109004139.GH16571@redhat.com> On Fri, Nov 08, 2013 at 09:51:15PM +0000, David Raddatz wrote: > Hi, Steve, I would but my id (my email address) doesn't have access > to the bugzilla report. I've added the information to the bug. -- Lars Kellogg-Stedman | larsks @ irc Cloud Engineering / OpenStack | " " @ twitter -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From prmarino1 at gmail.com Sun Nov 10 21:10:08 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Sun, 10 Nov 2013 16:10:08 -0500 Subject: [rhos-list] Openstack GSSAPI (Kerberos 5) and sasl for services question In-Reply-To: <5272e451.c549e00a.08e6.ffff9c66@mx.google.com> References: <1383229169.8612.90.camel@willson.li.ssimo.org> <5272e451.c549e00a.08e6.ffff9c66@mx.google.com> Message-ID: hey every one I did work this out with saslauthd. Sorry but what I've heard from every one so far was a little scary and has lead me to believe that part of the code in OpenStack just isn't ready to properly utilize keytabs yet. So far with saslauthd its working fine but there is one problem I found. Essentially the username nova/ for example wont work because of the / in the resulting URI. I even tried HTML encoding the / and it still wont work. but it does work with just a simple username like nova. If there is any one else interested I would be happy to do a short writeup on how to get it to work with saslauthd. The one warning is unless QPID is using SSL this method sends the password in clear text over the network. That said if you have configured QPID to use SSL it works very well and does make it easier to cluster QPID. Furthermore since QPID works well with keytabs it allows you to utilize Kerberos 5 in an adhock manner now and have an easy transition in the future once that portion of OpenStack's code is ready. by the way if any one is wondering the method I used would work with a MIT Kerberos 5 servers as well so it is compatible with FreeIPA. On Thu, Oct 31, 2013 at 7:14 PM, Paul Robert Marino wrote: > Thanks guys > That's interesting the doc also mentions a couple of environment variable > which I might be able to put in the appropriate /etc/sysconfig file, one of > which I knew about but hadn't thought of. > I'll do some experimenting when I get back into the office on Sunday. > > The other thing that occurred to me is I could use saslauthd as a proxy to > translate an other mechanism into GSSAPI but I've never liked to use that > method if I could avoid it because it weakens the security but at least its > better than a local SASL password file. The other thing is the RHEL init > script expects you only to have one instance running at any time which is > not quite the intended behaviour for saslauthd. You can actually spawn an > instance for each Mechanism you want to proxy into your target mechanism the > standard RHEL script only allows you to proxy one mechanism which means you > have to pick the lowest common denominator. SuSE use to support running an > instance for each mechanism back in the early 2000's and it always worked > well with my LDAP 3 servers providing support for old or broken clients. > > -- Sent from my HP Pre3 > > ________________________________ > On Oct 31, 2013 10:31, Simo Sorce wrote: > > On Thu, 2013-10-31 at 09:41 -0400, Adam Young wrote: >> On 10/30/2013 12:14 PM, Paul Robert Marino wrote: >> > Ive been looking over this doc because I would like to secure the >> > backend component of openstack with Kerberos. >> > http://openstack.redhat.com/Keystone_integration_with_IDM >> > >> > I don't want to do a full IPA server for this just Kerberos which for >> > the most part is fairly simple. >> > I already have preexisting Heimdal Kerberos 5 server cluster from an >> > other project which I can utilize in the environment which works fine >> > with the MIT client libraries and does its own replication without >> > using LDAP as a backend. >> > >> > so far most of it seems fairly strait forward but I found one thing I >> > found in the doc thats messy and was hoping the doc is out of date and >> > maybe there was a cleaner solution. here is what I have an issue with >> > >> > " >> > >> > The problem with this is that the key we just obtained is only good >> > for a specified period of time: 24 hours by default. Once 24 hours >> > passes the Kerberos ticket will no longer be valid and nova and cinder >> > will no longer be able to communicate with qpidd. >> > >> > The fix for now is to create a cron job which will renew these >> > credentials. >> > >> > >> > " >> > I also assume the same would be true for all of the openstack services >> > not just nova and cinder, >> > has the ability to specify and utilize a keytab been added or does any >> > one know if there are any plans to add the feature in the future. If >> > not who should I be nagging :-) . >> > Really it needs to be added to all of the openstack services it it >> > isn't there already >> >> It is a shortcoming addressed at the GSSAPI level, but that code is not >> in the RHEL 6 series yet. In the future, you will be able to put a >> Keytab in the appropriate subdirectory under /var/run and the new TGT >> will be fetched upon demand. >> >> Simo Sorce was involved with the projkect to do that and can provide >> more details. > > This is the MIT project page: > http://k5wiki.kerberos.org/wiki/Projects/Keytab_initiation > > It boils down to putting a keytab > in /var/kerberos/krb5/user//client.keytab and then make gssapi > initiation calls without trying to check for credentials using direct > krb5 calls or anything like that. > > not all the software does the right thing yet, but we will collaborate > with authors and help fix what doesn't work. > > Simo. > > -- > Simo Sorce * Red Hat, Inc * New York > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From dfv at eurotux.com Mon Nov 11 10:59:33 2013 From: dfv at eurotux.com (Diogo Vieira) Date: Mon, 11 Nov 2013 10:59:33 +0000 Subject: [rhos-list] Lockup after trying to detach a volume in Havana Message-ID: <7E1BFF67-0320-48DB-AF56-173537758AB5@eurotux.com> Hello once again, I'm sorry for troubling you again but this has happened to me twice now. After trying to detach a volume from an instance the system completely locks up, which forced a reboot. Can't even ssh into it and there's nothing relevant in the cinder logs. Not sure if relevant or not but I remember the instance was active while I tried to detach the volume. After the reboot, the first time I logged in to horizon it showed a success message for the detachment but the volume status is stuck on detaching. Can someone help me understand why this happens and how can I remove that volume? Thank you very much, Diogo Vieira From prmarino1 at gmail.com Mon Nov 11 16:01:58 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Mon, 11 Nov 2013 11:01:58 -0500 Subject: [rhos-list] migrations with gluster isue Message-ID: hello every one Ive been doing some testing with RDO grizzly and found an issue with the compute nodes doing a migration. Live migration works as expected, but regular migrations don't. I wouldn't be too worried about it except for the migrate instance option in the admin tab in horizon. for some reason they don't seem to detect that they are on shared storage and instead try to ssh to each other and do filesystem modifications. the first one it tries is an mkdir then it fails right now ive left the shell for the nova user as /sbin/nologin so even if i did add keys it wouldnt work and besides the mkdir command would return an error any way. here are the settings ive been tinkering with " instances_path = /nova-cluster/instances live_migration_bandwidth=0 live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE live_migration_retry_count=30 live_migration_uri=qemu+tcp://%s/system image_cache_manager_interval=0 block_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER glusterfs_mount_point_base=/nova-cluster " does any one have any ideas about this or know of any settings I may have missed? From hateya at redhat.com Mon Nov 11 16:16:52 2013 From: hateya at redhat.com (Haim Ateya) Date: Mon, 11 Nov 2013 11:16:52 -0500 (EST) Subject: [rhos-list] Lockup after trying to detach a volume in Havana In-Reply-To: <7E1BFF67-0320-48DB-AF56-173537758AB5@eurotux.com> References: <7E1BFF67-0320-48DB-AF56-173537758AB5@eurotux.com> Message-ID: <1259177536.29466184.1384186612040.JavaMail.root@redhat.com> Hi diogo, few questions: 1) I would like to know if issue reproducible (if so, can I examine the machine\run the flow?). 2) what topology are you running, is it all-in-one or components are located remotely (independently from controller). 3) what storage type are you using? NFS\iSCSI (lvmDriver)? 4) do you have console access to the machine? can you run top and see if some process consume memory\cpu (had it once - some qpid leak). Haim ----- Original Message ----- > From: "Diogo Vieira" > To: rhos-list at redhat.com > Sent: Monday, November 11, 2013 12:59:33 PM > Subject: [rhos-list] Lockup after trying to detach a volume in Havana > > Hello once again, > > I'm sorry for troubling you again but this has happened to me twice now. > After trying to detach a volume from an instance the system completely locks > up, which forced a reboot. Can't even ssh into it and there's nothing > relevant in the cinder logs. Not sure if relevant or not but I remember the > instance was active while I tried to detach the volume. > > After the reboot, the first time I logged in to horizon it showed a success > message for the detachment but the volume status is stuck on detaching. > > Can someone help me understand why this happens and how can I remove that > volume? > > Thank you very much, > Diogo Vieira > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From dfv at eurotux.com Mon Nov 11 17:59:00 2013 From: dfv at eurotux.com (Diogo Vieira) Date: Mon, 11 Nov 2013 17:59:00 +0000 Subject: [rhos-list] Lockup after trying to detach a volume in Havana In-Reply-To: <1259177536.29466184.1384186612040.JavaMail.root@redhat.com> References: <7E1BFF67-0320-48DB-AF56-173537758AB5@eurotux.com> <1259177536.29466184.1384186612040.JavaMail.root@redhat.com> Message-ID: <76A254C4-410A-4E69-91E2-1C958536F060@eurotux.com> Hi 1) It is, I was able to reproduce it (but only when the volume is in-use; when I tried to detach with the instance suspended it worked normally). The necessary steps to reproduce it are: - Create a test volume and a test instance; - Run the instance; - Try to detach the volume. If you need to know something more, please tell me. 2) I have two machines: one running all the openstack services and another one running a compute node. The instance was in the first one and it was that machine that became unresponsive. My other compute node doesn't seem to have any problem. 3) Fibre channel. 4) I do have console access but it becomes unresponsive too. Diogo Vieira On Nov 11, 2013, at 4:16 PM, Haim Ateya wrote: > Hi diogo, > > few questions: > > 1) I would like to know if issue reproducible (if so, can I examine the machine\run the flow?). > 2) what topology are you running, is it all-in-one or components are located remotely (independently from controller). > 3) what storage type are you using? NFS\iSCSI (lvmDriver)? > 4) do you have console access to the machine? can you run top and see if some process consume memory\cpu (had it once - some qpid leak). > > Haim > > ----- Original Message ----- >> From: "Diogo Vieira" >> To: rhos-list at redhat.com >> Sent: Monday, November 11, 2013 12:59:33 PM >> Subject: [rhos-list] Lockup after trying to detach a volume in Havana >> >> Hello once again, >> >> I'm sorry for troubling you again but this has happened to me twice now. >> After trying to detach a volume from an instance the system completely locks >> up, which forced a reboot. Can't even ssh into it and there's nothing >> relevant in the cinder logs. Not sure if relevant or not but I remember the >> instance was active while I tried to detach the volume. >> >> After the reboot, the first time I logged in to horizon it showed a success >> message for the detachment but the volume status is stuck on detaching. >> >> Can someone help me understand why this happens and how can I remove that >> volume? >> >> Thank you very much, >> Diogo Vieira >> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sunny_Anthony at symantec.com Wed Nov 13 13:54:55 2013 From: Sunny_Anthony at symantec.com (Sunny Anthony) Date: Wed, 13 Nov 2013 05:54:55 -0800 Subject: [rhos-list] How to get network namespaces support kernel for RHEL 6.4 Message-ID: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A06@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> Hi Experts, Setting up OpenStack for the first time on RHEL6.4. Please help in getting/download the network namespaces support kernel rpm for RHEL 6.4. Regards, Sunny -------------- next part -------------- An HTML attachment was scrubbed... URL: From sputhenp at redhat.com Wed Nov 13 14:23:48 2013 From: sputhenp at redhat.com (Sadique Puthen) Date: Wed, 13 Nov 2013 19:53:48 +0530 Subject: [rhos-list] How to get network namespaces support kernel for RHEL 6.4 In-Reply-To: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A06@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> References: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A06@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> Message-ID: <52838B74.9040809@redhat.com> On 11/13/2013 07:24 PM, Sunny Anthony wrote: > > Hi Experts, > > Setting up OpenStack for the first time on RHEL6.4. > > Please help in getting/download the network namespaces support kernel > rpm for RHEL 6.4. > > Regards, > Sunny > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list This is automatically installed by packstack from rhos channel. You can also install it manually by running "yum install kernel-*openstack*". rpm -q kernel will show below output. kernel-2.6.32-358.123.2.openstack.el6.x86_64 --Sadique -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sunny_Anthony at symantec.com Wed Nov 13 14:51:45 2013 From: Sunny_Anthony at symantec.com (Sunny Anthony) Date: Wed, 13 Nov 2013 06:51:45 -0800 Subject: [rhos-list] How to get network namespaces support kernel for RHEL 6.4 In-Reply-To: <52838B74.9040809@redhat.com> References: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A06@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> <52838B74.9040809@redhat.com> Message-ID: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A38@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> I think first we need to download/register for the RPM source. My system is not registered any RPM repository. Please help with any RDO repository from where I can download the respective RPM or register my system. Regards, Sunny From: Sadique Puthen [mailto:sputhenp at redhat.com] Sent: Wednesday, November 13, 2013 7:54 PM To: Sunny Anthony Cc: rhos-list at redhat.com Subject: Re: [rhos-list] How to get network namespaces support kernel for RHEL 6.4 On 11/13/2013 07:24 PM, Sunny Anthony wrote: Hi Experts, Setting up OpenStack for the first time on RHEL6.4. Please help in getting/download the network namespaces support kernel rpm for RHEL 6.4. Regards, Sunny _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list This is automatically installed by packstack from rhos channel. You can also install it manually by running "yum install kernel-*openstack*". rpm -q kernel will show below output. kernel-2.6.32-358.123.2.openstack.el6.x86_64 --Sadique -------------- next part -------------- An HTML attachment was scrubbed... URL: From tvvcox at redhat.com Wed Nov 13 15:09:50 2013 From: tvvcox at redhat.com (Tomas Von Veschler) Date: Wed, 13 Nov 2013 16:09:50 +0100 Subject: [rhos-list] How to get network namespaces support kernel for RHEL 6.4 In-Reply-To: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A38@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> References: <46A6A5BAC1E78D489E00C35CED46F73C0517E63A06@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> <52838B74.9040809@redhat.com> <46A6A5BAC1E78D489E00C35CED46F73C0517E63A38@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> Message-ID: <5283963E.2060103@redhat.com> For RDO: http://openstack.redhat.com/Quickstart For RHOS, you'll need a valid RHN subscription including RHEL and Openstack channels. Regards, Tomas On 11/13/2013 03:51 PM, Sunny Anthony wrote: > I think first we need to download/register for the RPM source. > > My system is not registered any RPM repository. > > Please help with any RDO repository from where I can download the > respective RPM or register my system. > > Regards, > Sunny > > *From:*Sadique Puthen [mailto:sputhenp at redhat.com] > *Sent:* Wednesday, November 13, 2013 7:54 PM > *To:* Sunny Anthony > *Cc:* rhos-list at redhat.com > *Subject:* Re: [rhos-list] How to get network namespaces support kernel > for RHEL 6.4 > > On 11/13/2013 07:24 PM, Sunny Anthony wrote: > > Hi Experts, > > Setting up OpenStack for the first time on RHEL6.4. > > Please help in getting/download the network namespaces support > kernel rpm for RHEL 6.4. > > Regards, > Sunny > > > > > _______________________________________________ > > rhos-list mailing list > > rhos-list at redhat.com > > https://www.redhat.com/mailman/listinfo/rhos-list > > > This is automatically installed by packstack from rhos channel. You can > also install it manually by running "yum install kernel-*openstack*". > > rpm -q kernel will show below output. > > kernel-2.6.32-358.123.2.openstack.el6.x86_64 > > --Sadique > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From prashanth.prahal at gmail.com Thu Nov 14 06:06:06 2013 From: prashanth.prahal at gmail.com (Prashanth Prahalad) Date: Thu, 14 Nov 2013 11:36:06 +0530 Subject: [rhos-list] quantum security model Message-ID: Hi Folks, I'm trying to understand the quantum security model behavior in certain cases. I've OVS plugin configured with VLAN isolation. I've a tenant project (alt_demo) (admin) > keystone tenant-list +----------------------------------+----------+---------+ | id | name | enabled | +----------------------------------+----------+---------+ | c19f9a2d16b74c3c971dbfbc1afdc687 | admin | True | | a37209139af44a8a8a2a8e519e3f8478 | alt_demo | True | | 70e910a7296d4a19be4b32d5bcaf3996 | services | True | +----------------------------------+----------+---------+ I've a user (alt_demo) who is a 'member' of project alt_demo. (alt_demo is not an admin) (admin > keystone user-list +----------------------------------+----------+---------+-------------------+ | id | name | enabled | email | +----------------------------------+----------+---------+-------------------+ | 338a1897720a4be48023a6987c76191d | admin | True | test at test.com | | c2dc7ac0e8bf4628bc7d3b2fe285793a | alt_demo | True | alt_demo at demo.com | | 94936f26d48e481dadacda322fc51858 | cinder | True | cinder at localhost | | b7db5ef2f2d849b1a8dfc7f043bf4289 | glance | True | glance at localhost | | a42b0ca85f914cf88dc6361da5e08a0c | nova | True | nova at localhost | | 2f0f85cb85f242c7b9c5f620886b9537 | quantum | True | quantum at localhost | +----------------------------------+----------+---------+-------------------+ As alt_demo, try to create a network (alt_demo) > quantum net-create alt-net Created a new network: +-----------------+--------------------------------------+ | Field | Value | +-----------------+--------------------------------------+ | admin_state_up | True | | id | c1629dac-91dd-424a-bc82-8b97323f5059 | | name | alt-net | | router:external | False | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | +-----------------+--------------------------------------+ List the network details for the network which was just created (alt_demo) > quantum net-show alt-net +-----------------+--------------------------------------+ | Field | Value | +-----------------+--------------------------------------+ | admin_state_up | True | | id | c1629dac-91dd-424a-bc82-8b97323f5059 | | name | alt-net | | router:external | False | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | +-----------------+--------------------------------------+ Here's what an "admin" user sees : (admin) > quantum net-show alt-net +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | c1629dac-91dd-424a-bc82-8b97323f5059 | | name | alt-net | | provider:network_type | vlan | | provider:physical_network | physnet1 | | provider:segmentation_id | 46 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | +---------------------------+--------------------------------------+ Now, the question I've is the user "alt_demo" cannot see the VLAN/provider-network and other details which is very confusing (when the user was able to create the network, he should be able to see details of the network he just created). Thanks ! Prashanth -------------- next part -------------- An HTML attachment was scrubbed... URL: From sputhenp at redhat.com Thu Nov 14 07:25:28 2013 From: sputhenp at redhat.com (Sadique Puthen) Date: Thu, 14 Nov 2013 12:55:28 +0530 Subject: [rhos-list] quantum security model In-Reply-To: References: Message-ID: <52847AE8.9080203@redhat.com> On 11/14/2013 11:36 AM, Prashanth Prahalad wrote: > > Hi Folks, > > I'm trying to understand the quantum security model behavior in > certain cases. I've OVS plugin configured with VLAN isolation. > > I've a tenant project (alt_demo) > > (admin) > keystone tenant-list > +----------------------------------+----------+---------+ > | id | name | enabled | > +----------------------------------+----------+---------+ > | c19f9a2d16b74c3c971dbfbc1afdc687 | admin | True | > | a37209139af44a8a8a2a8e519e3f8478 | alt_demo | True | > | 70e910a7296d4a19be4b32d5bcaf3996 | services | True | > +----------------------------------+----------+---------+ > > I've a user (alt_demo) who is a 'member' of project alt_demo. > (alt_demo is not an admin) > > (admin > keystone user-list > +----------------------------------+----------+---------+-------------------+ > | id | name | enabled | email | > +----------------------------------+----------+---------+-------------------+ > | 338a1897720a4be48023a6987c76191d | admin | True | test at test.com > | > | c2dc7ac0e8bf4628bc7d3b2fe285793a | alt_demo | True | > alt_demo at demo.com | > | 94936f26d48e481dadacda322fc51858 | cinder | True | cinder at localhost | > | b7db5ef2f2d849b1a8dfc7f043bf4289 | glance | True | glance at localhost | > | a42b0ca85f914cf88dc6361da5e08a0c | nova | True | nova at localhost | > | 2f0f85cb85f242c7b9c5f620886b9537 | quantum | True | quantum at localhost | > +----------------------------------+----------+---------+-------------------+ > > As alt_demo, try to create a network > > (alt_demo) > quantum net-create alt-net > Created a new network: > +-----------------+--------------------------------------+ > | Field | Value | > +-----------------+--------------------------------------+ > | admin_state_up | True | > | id | c1629dac-91dd-424a-bc82-8b97323f5059 | > | name | alt-net | > | router:external | False | > | shared | False | > | status | ACTIVE | > | subnets | | > | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | > +-----------------+--------------------------------------+ > > List the network details for the network which was just created > > (alt_demo) > quantum net-show alt-net > +-----------------+--------------------------------------+ > | Field | Value | > +-----------------+--------------------------------------+ > | admin_state_up | True | > | id | c1629dac-91dd-424a-bc82-8b97323f5059 | > | name | alt-net | > | router:external | False | > | shared | False | > | status | ACTIVE | > | subnets | | > | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | > +-----------------+--------------------------------------+ > > Here's what an "admin" user sees : > > (admin) > quantum net-show alt-net > +---------------------------+--------------------------------------+ > | Field | Value | > +---------------------------+--------------------------------------+ > | admin_state_up | True | > | id | c1629dac-91dd-424a-bc82-8b97323f5059 | > | name | alt-net | > | provider:network_type | vlan | > | provider:physical_network | physnet1 | > | provider:segmentation_id | 46 | > | router:external | False | > | shared | False | > | status | ACTIVE | > | subnets | | > | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | > +---------------------------+--------------------------------------+ > > Now, the question I've is the user "alt_demo" cannot see the > VLAN/provider-network and other details which is very confusing (when > the user was able to create the network, he should be able to see > details of the network he just created). > Why does the user need to bother about segmentation id and other details? It just need to work for him and no need to know how it work internally. That may be the reason it's not exposed to him. > Thanks ! > Prashanth > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From rkukura at redhat.com Thu Nov 14 14:51:14 2013 From: rkukura at redhat.com (Robert Kukura) Date: Thu, 14 Nov 2013 09:51:14 -0500 Subject: [rhos-list] quantum security model In-Reply-To: <52847AE8.9080203@redhat.com> References: <52847AE8.9080203@redhat.com> Message-ID: <5284E362.4010405@redhat.com> On 11/14/2013 02:25 AM, Sadique Puthen wrote: > On 11/14/2013 11:36 AM, Prashanth Prahalad wrote: >> >> Hi Folks, >> >> I'm trying to understand the quantum security model behavior in >> certain cases. I've OVS plugin configured with VLAN isolation. >> >> I've a tenant project (alt_demo) >> >> (admin) > keystone tenant-list >> +----------------------------------+----------+---------+ >> | id | name | enabled | >> +----------------------------------+----------+---------+ >> | c19f9a2d16b74c3c971dbfbc1afdc687 | admin | True | >> | a37209139af44a8a8a2a8e519e3f8478 | alt_demo | True | >> | 70e910a7296d4a19be4b32d5bcaf3996 | services | True | >> +----------------------------------+----------+---------+ >> >> I've a user (alt_demo) who is a 'member' of project alt_demo. >> (alt_demo is not an admin) >> >> (admin > keystone user-list >> +----------------------------------+----------+---------+-------------------+ >> | id | name | enabled | email | >> +----------------------------------+----------+---------+-------------------+ >> | 338a1897720a4be48023a6987c76191d | admin | True | test at test.com >> | >> | c2dc7ac0e8bf4628bc7d3b2fe285793a | alt_demo | True | >> alt_demo at demo.com | >> | 94936f26d48e481dadacda322fc51858 | cinder | True | cinder at localhost | >> | b7db5ef2f2d849b1a8dfc7f043bf4289 | glance | True | glance at localhost | >> | a42b0ca85f914cf88dc6361da5e08a0c | nova | True | nova at localhost | >> | 2f0f85cb85f242c7b9c5f620886b9537 | quantum | True | quantum at localhost | >> +----------------------------------+----------+---------+-------------------+ >> >> As alt_demo, try to create a network >> >> (alt_demo) > quantum net-create alt-net >> Created a new network: >> +-----------------+--------------------------------------+ >> | Field | Value | >> +-----------------+--------------------------------------+ >> | admin_state_up | True | >> | id | c1629dac-91dd-424a-bc82-8b97323f5059 | >> | name | alt-net | >> | router:external | False | >> | shared | False | >> | status | ACTIVE | >> | subnets | | >> | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | >> +-----------------+--------------------------------------+ >> >> List the network details for the network which was just created >> >> (alt_demo) > quantum net-show alt-net >> +-----------------+--------------------------------------+ >> | Field | Value | >> +-----------------+--------------------------------------+ >> | admin_state_up | True | >> | id | c1629dac-91dd-424a-bc82-8b97323f5059 | >> | name | alt-net | >> | router:external | False | >> | shared | False | >> | status | ACTIVE | >> | subnets | | >> | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | >> +-----------------+--------------------------------------+ >> >> Here's what an "admin" user sees : >> >> (admin) > quantum net-show alt-net >> +---------------------------+--------------------------------------+ >> | Field | Value | >> +---------------------------+--------------------------------------+ >> | admin_state_up | True | >> | id | c1629dac-91dd-424a-bc82-8b97323f5059 | >> | name | alt-net | >> | provider:network_type | vlan | >> | provider:physical_network | physnet1 | >> | provider:segmentation_id | 46 | >> | router:external | False | >> | shared | False | >> | status | ACTIVE | >> | subnets | | >> | tenant_id | a37209139af44a8a8a2a8e519e3f8478 | >> +---------------------------+--------------------------------------+ >> >> Now, the question I've is the user "alt_demo" cannot see the >> VLAN/provider-network and other details which is very confusing (when >> the user was able to create the network, he should be able to see >> details of the network he just created). >> > > Why does the user need to bother about segmentation id and other > details? It just need to work for him and no need to know how it work > internally. That may be the reason it's not exposed to him. Correct. Similarly, a normal tenant running "nova show " does not see what compute node the instance is running on, but an admin user does. In general, tenants are assumed to not need to be aware of how the cloud is implemented. They just deal with the abstractions OpenStack provides. Note that this is all driven by policy files that can be modified if a deployment has different requirements. -Bob > >> Thanks ! >> Prashanth >> >> >> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From xzhao at bnl.gov Thu Nov 14 17:21:38 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Thu, 14 Nov 2013 12:21:38 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: References: <5282A928.1040701@bnl.gov> Message-ID: <528506A2.9030504@bnl.gov> The regular conf files look fine, from the physical network host, nslookup works fine, I also tried with iptables stopped on the network host, it doesn't help. Below is some output from tcpdump run on the network host: (10.0.1.3 is the dns server for the VM network, 10.0.1.5 is an instance) I can resolve names for other VMs within the same VM subnet: 12:03:13.862982 IP 10.0.1.5.49737 > 10.0.1.3.domain: 42560+ PTR? 2.1.0.10.in-addr.arpa. (39) 12:03:13.863109 IP 10.0.1.3.domain > 10.0.1.5.49737: 42560* 1/0/0 PTR host-10-0-1-2.openstacklocal. (81) But when I try "nslookup www.google.com", it gave the "Refused" message: 12:03:21.991820 IP 10.0.1.5.56262 > 10.0.1.3.domain: 39784+ A? www.google.com. (32) 12:03:21.991931 IP 10.0.1.3.domain > 10.0.1.5.56262: 39784 Refused 0/0/0 (32) 12:03:21.992711 IP 10.0.1.5.47311 > 10.0.1.3.domain: 38835+ A? www.google.com.openstacklocal. (47) 12:03:21.992788 IP 10.0.1.3.domain > 10.0.1.5.47311: 38835 Refused 0/0/0 (47) The error message from the instance is "server can't find www.google.com.openstacklocal: REFUSED" Below is the dnsmasq processes running on the network host (the -conf-file is empty, is that normal?) : nobody 28843 1 0 Nov13 ? 00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap4aafbf9c-49 --except-interface=lo --pid-file=/var/lib/quantum/dhcp/fea3d2fb-2b50-47e8-ba28-e68f094606bc/pid --dhcp-hostsfile=/var/lib/quantum/dhcp/fea3d2fb-2b50-47e8-ba28-e68f094606bc/host --dhcp-optsfile=/var/lib/quantum/dhcp/fea3d2fb-2b50-47e8-ba28-e68f094606bc/opts --dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update --leasefile-ro --dhcp-range=tag0,10.0.1.0,static,120s --conf-file= --domain=openstacklocal root 28844 28843 0 Nov13 ? 00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap4aafbf9c-49 --except-interface=lo --pid-file=/var/lib/quantum/dhcp/fea3d2fb-2b50-47e8-ba28-e68f094606bc/pid --dhcp-hostsfile=/var/lib/quantum/dhcp/fea3d2fb-2b50-47e8-ba28-e68f094606bc/host --dhcp-optsfile=/var/lib/quantum/dhcp/fea3d2fb-2b50-47e8-ba28-e68f094606bc/opts --dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update --leasefile-ro --dhcp-range=tag0,10.0.1.0,static,120s --conf-file= --domain=openstacklocal Any help will be greatly appreciated. Thanks, Xin On 11/12/2013 6:55 PM, Remo Mattei wrote: > RH does have firewall rules you may want to see if DNS is going out. I > know you said that it goes outside but you can also check the order if > in nsswitch.conf etc.. > > Have a good day, > > ciao > -- > Remo Mattei > > > November 12, 2013 at 14:32:52, Xin Zhao (xzhao at bnl.gov > ) ha scritto: > >> Hello, >> >> I have a multi-host grizzly RHEL6 install, using OVS. From the instance, >> I can ping external ips, but DNS resolv doesn't work, it only works for >> other instances on the VM network. >> If I do subnet-update to add public DNS server ips to the vm network, >> DNS resolv works for external hosts, but stops working for other >> instances on the same VM network. >> Do I miss some configuration here? >> >> Thanks, >> Xin >> >> _______________________________________________ >> Mailing list: >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> Post to : openstack at lists.openstack.org >> Unsubscribe : >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> >> !DSPAM:2,5282ac94271465380316102! >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From twilson at redhat.com Thu Nov 14 17:36:52 2013 From: twilson at redhat.com (Terry Wilson) Date: Thu, 14 Nov 2013 12:36:52 -0500 (EST) Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <528506A2.9030504@bnl.gov> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> Message-ID: <816219590.24004983.1384450612377.JavaMail.root@redhat.com> > Hello, > > I have a multi-host grizzly RHEL6 install, using OVS. From the instance, > I can ping external ips, but DNS resolv doesn't work, it only works for > other instances on the VM network. > If I do subnet-update to add public DNS server ips to the vm network, > DNS resolv works for external hosts, but stops working for other > instances on the same VM network. > Do I miss some configuration here? > > Thanks, > Xin Which version of quantum are you using? In version 2013.1.4, a bug was fixed for adding a default route in the namespace in which the dnsmasq agent runs so that external DNS requests could be handled. See: https://bugs.launchpad.net/neutron/+bug/1181378 for more details. Terry From xzhao at bnl.gov Thu Nov 14 19:11:25 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Thu, 14 Nov 2013 14:11:25 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <816219590.24004983.1384450612377.JavaMail.root@redhat.com> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> Message-ID: <5285205D.4020305@bnl.gov> Hi Terry, Thanks for the info. I am using openstack-quantum-2013.1.3 on controller and compute hosts, and openstack-quantum-2013.1.4-3 on the network host. I will try to update the rpm on the compute and controller nodes. Thanks, Xin On 11/14/2013 12:36 PM, Terry Wilson wrote: >> Hello, >> >> I have a multi-host grizzly RHEL6 install, using OVS. From the instance, >> I can ping external ips, but DNS resolv doesn't work, it only works for >> other instances on the VM network. >> If I do subnet-update to add public DNS server ips to the vm network, >> DNS resolv works for external hosts, but stops working for other >> instances on the same VM network. >> Do I miss some configuration here? >> >> Thanks, >> Xin > Which version of quantum are you using? In version 2013.1.4, a bug was fixed for adding a default route in the namespace in which the dnsmasq agent runs so that external DNS requests could be handled. See: https://bugs.launchpad.net/neutron/+bug/1181378 for more details. > > Terry From prmarino1 at gmail.com Thu Nov 14 20:37:09 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 14 Nov 2013 15:37:09 -0500 Subject: [rhos-list] Lbass seems not to be functioning in RDO grizzly Message-ID: hello every one Im tinkering with RDO grizzly and trying to get the lbaas agent working. im using openvswitch with gre tunnels which is working well. in /etc/quantum/lbaas_agent.ini I set the following interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver ovs_use_veth = True device_driver = quantum.plugins.services.agent_loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver use_namespaces = True in /etc/quantum/quantum.conf i set in the [DEFAULT] section service_plugins = quantum.plugins.services.agent_loadbalancer.plugin.LoadBalancerPlugin I restarted quantum server and started the lbass agent. haproxy is installed. when I run quantum lb-pool-list i get this error " 404 Not Found The resource could not be found. " the weird thing is when I look at the logs I done see any errors of any kind not even in debug logging, and there is no indication on the secure log that a sudo involving haproxy was ever attempted. also i dont see it listed if i do a quantum ext-list The only thing that I can see that may indicate the cause is durring the startup of quantum server I see the message " WARNING [quantum.api.extensions] Extension lbaas not supported by any of loaded plugins " this said I havent seen any indication on how to fix it on any of the openstack docs or any of the blog sites where people have configured it in the past. according to every thing Ive read it should be working. does any one have any ideas on what may be the problem? From xzhao at bnl.gov Thu Nov 14 20:39:14 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Thu, 14 Nov 2013 15:39:14 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <816219590.24004983.1384450612377.JavaMail.root@redhat.com> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> Message-ID: <528534F2.7000403@bnl.gov> Hi Terry, I upgrade to version 2013.1.4 on all 3 hosts (controller/network/compute). Unfortunately that doesn't solve the DNS issue for instances. In the dhcp-agent.log, there is a message of: WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR DNSMASQ. DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its version is 2.59 or above! Not sure if it's critical or not ... Thanks, Xin On 11/14/2013 12:36 PM, Terry Wilson wrote: >> Hello, >> >> I have a multi-host grizzly RHEL6 install, using OVS. From the instance, >> I can ping external ips, but DNS resolv doesn't work, it only works for >> other instances on the VM network. >> If I do subnet-update to add public DNS server ips to the vm network, >> DNS resolv works for external hosts, but stops working for other >> instances on the same VM network. >> Do I miss some configuration here? >> >> Thanks, >> Xin > Which version of quantum are you using? In version 2013.1.4, a bug was fixed for adding a default route in the namespace in which the dnsmasq agent runs so that external DNS requests could be handled. See: https://bugs.launchpad.net/neutron/+bug/1181378 for more details. > > Terry From prmarino1 at gmail.com Thu Nov 14 20:54:07 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 14 Nov 2013 15:54:07 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <528534F2.7000403@bnl.gov> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> <528534F2.7000403@bnl.gov> Message-ID: that has been patched but the error message still shows up in RDO The underlying issue was patched along time ago but the latter patch to remove that message was never backported to RDO it will work just fine without it you can safely ignore that message. On Thu, Nov 14, 2013 at 3:39 PM, Xin Zhao wrote: > Hi Terry, > > I upgrade to version 2013.1.4 on all 3 hosts (controller/network/compute). > Unfortunately that doesn't solve the DNS issue for instances. > > In the dhcp-agent.log, there is a message of: > > WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR DNSMASQ. > DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its version is 2.59 or > above! > > Not sure if it's critical or not ... > > > Thanks, > Xin > > On 11/14/2013 12:36 PM, Terry Wilson wrote: >>> >>> Hello, >>> >>> I have a multi-host grizzly RHEL6 install, using OVS. From the instance, >>> I can ping external ips, but DNS resolv doesn't work, it only works for >>> other instances on the VM network. >>> If I do subnet-update to add public DNS server ips to the vm network, >>> DNS resolv works for external hosts, but stops working for other >>> instances on the same VM network. >>> Do I miss some configuration here? >>> >>> Thanks, >>> Xin >> >> Which version of quantum are you using? In version 2013.1.4, a bug was >> fixed for adding a default route in the namespace in which the dnsmasq agent >> runs so that external DNS requests could be handled. See: >> https://bugs.launchpad.net/neutron/+bug/1181378 for more details. >> >> Terry > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From prmarino1 at gmail.com Thu Nov 14 21:07:55 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 14 Nov 2013 16:07:55 -0500 Subject: [rhos-list] Lbass seems not to be functioning in RDO grizzly In-Reply-To: References: Message-ID: correction I have found something in the lbaas log every few seconds im seeing this which indicates an issue with qpid " 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0a710]: ConnectionHeartbeat() 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0a710]: '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0a710]: '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0a710]: ConnectionHeartbeat() 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0d6c8]: ConnectionHeartbeat() 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0d6c8]: '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0d6c8]: '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0d6c8]: ConnectionHeartbeat() 2013-11-14 15:59:45 ERROR [quantum.openstack.common.rpc.amqp] Timed out waiting for RPC response. Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 495, in __iter__ data = self._dataqueue.get(timeout=self._timeout) File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, in get return waiter.wait() File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, in wait return get_hub().switch() File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch return self.greenlet.switch() Empty 2013-11-14 15:59:45 ERROR [quantum.plugins.services.agent_loadbalancer.agent.manager] Unable to retrieve ready devices Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/manager.py", line 168, in sync_state ready_logical_devices = set(self.plugin_rpc.get_ready_devices()) File "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/api.py", line 36, in get_ready_devices topic=self.topic File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", line 630, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 614, in call rv = list(rv) File "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", line 500, in __iter__ raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-11-14 15:59:45 DEBUG [quantum.openstack.common.periodic_task] Skipping LbaasAgentManager.collect_stats, 1 ticks left until next run " whats strange about it is QPID seems to be working fine for all the other quantum agents. here is my qpid configuration in the quantum.conf note any fields encpasulated int <> have been scrubbed " fake_rabbit = False rpc_backend = quantum.openstack.common.rpc.impl_qpid qpid_hostname = qpid_port = 5672 qpid_hosts = :5672 qpid_username = qpid_password = qpid_sasl_mechanisms = PLAIN qpid_reconnect = True qpid_reconnect_timeout = 30 qpid_reconnect_limit = 0 qpid_reconnect_interval_min = 0 qpid_reconnect_interval_max = 0 qpid_reconnect_interval = 20 qpid_heartbeat = 30 qpid_protocol = tcp qpid_tcp_nodelay = True default_publisher_id = " On Thu, Nov 14, 2013 at 3:37 PM, Paul Robert Marino wrote: > hello every one > Im tinkering with RDO grizzly and trying to get the lbaas agent working. > im using openvswitch with gre tunnels which is working well. > > in /etc/quantum/lbaas_agent.ini I set the following > > interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver > ovs_use_veth = True > device_driver = > quantum.plugins.services.agent_loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver > use_namespaces = True > > in /etc/quantum/quantum.conf i set in the [DEFAULT] section > > service_plugins = > quantum.plugins.services.agent_loadbalancer.plugin.LoadBalancerPlugin > > > I restarted quantum server and started the lbass agent. > > haproxy is installed. > > when I run quantum lb-pool-list > > i get this error > " > 404 Not Found > > The resource could not be found. > > " > > the weird thing is when I look at the logs I done see any errors of > any kind not even in debug logging, and there is no indication on the > secure log that a sudo involving haproxy was ever attempted. > > also i dont see it listed if i do a quantum ext-list > > The only thing that I can see that may indicate the cause is durring > the startup of quantum server I see the message > " > WARNING [quantum.api.extensions] Extension lbaas not supported by any > of loaded plugins > " > > this said I havent seen any indication on how to fix it on any of the > openstack docs or any of the blog sites where people have configured > it in the past. according to every thing Ive read it should be > working. > > does any one have any ideas on what may be the problem? From prmarino1 at gmail.com Thu Nov 14 21:17:06 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 14 Nov 2013 16:17:06 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> <528534F2.7000403@bnl.gov> Message-ID: by the way if you are curious that message is an artifact from this bug fix https://bugzilla.redhat.com/show_bug.cgi?id=887369 On Thu, Nov 14, 2013 at 3:54 PM, Paul Robert Marino wrote: > that has been patched but the error message still shows up in RDO > > The underlying issue was patched along time ago but the latter patch > to remove that message was never backported to RDO > it will work just fine without it you can safely ignore that message. > > > On Thu, Nov 14, 2013 at 3:39 PM, Xin Zhao wrote: >> Hi Terry, >> >> I upgrade to version 2013.1.4 on all 3 hosts (controller/network/compute). >> Unfortunately that doesn't solve the DNS issue for instances. >> >> In the dhcp-agent.log, there is a message of: >> >> WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR DNSMASQ. >> DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its version is 2.59 or >> above! >> >> Not sure if it's critical or not ... >> >> >> Thanks, >> Xin >> >> On 11/14/2013 12:36 PM, Terry Wilson wrote: >>>> >>>> Hello, >>>> >>>> I have a multi-host grizzly RHEL6 install, using OVS. From the instance, >>>> I can ping external ips, but DNS resolv doesn't work, it only works for >>>> other instances on the VM network. >>>> If I do subnet-update to add public DNS server ips to the vm network, >>>> DNS resolv works for external hosts, but stops working for other >>>> instances on the same VM network. >>>> Do I miss some configuration here? >>>> >>>> Thanks, >>>> Xin >>> >>> Which version of quantum are you using? In version 2013.1.4, a bug was >>> fixed for adding a default route in the namespace in which the dnsmasq agent >>> runs so that external DNS requests could be handled. See: >>> https://bugs.launchpad.net/neutron/+bug/1181378 for more details. >>> >>> Terry >> >> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list From vkarani1 at in.ibm.com Thu Nov 14 22:37:22 2013 From: vkarani1 at in.ibm.com (Velayutham Karani1) Date: Fri, 15 Nov 2013 04:07:22 +0530 Subject: [rhos-list] AUTO: Velayutham Karani1 is out of the office (returning 19-11-2013) Message-ID: I am out of the office until 19-11-2013. I will be out of office till 18 November 2013 with limited access to email and phone. I will return to office by 19 Nov 2013. I can reply to your mail as and when possible. If urgent response needed, please contact my manager Shanmugam Raman at shanmugam at in.ibm.com. Thanks, Velu. Note: This is an automated response to your message "rhos-list Digest, Vol 16, Issue 15" sent on 15/11/2013 2:24:11. This is the only notification you will receive while this person is away. From pmyers at redhat.com Thu Nov 14 22:49:02 2013 From: pmyers at redhat.com (Perry Myers) Date: Thu, 14 Nov 2013 17:49:02 -0500 Subject: [rhos-list] AUTO: Velayutham Karani1 is out of the office (returning 19-11-2013) In-Reply-To: References: Message-ID: <5285535E.1090605@redhat.com> I'm not being rude here, but when I see emails like this, I'm going to start unsubscribing the sender. Folks need to fix their mail systems to not do this. On 11/14/2013 05:37 PM, Velayutham Karani1 wrote: > > I am out of the office until 19-11-2013. > > I will be out of office till 18 November 2013 with limited access to email > and phone. I will return to office by 19 Nov 2013. I can reply to your > mail as and when possible. If urgent response needed, please contact my > manager Shanmugam Raman at shanmugam at in.ibm.com. > > Thanks, > Velu. > > > Note: This is an automated response to your message "rhos-list Digest, Vol > 16, Issue 15" sent on 15/11/2013 2:24:11. > > This is the only notification you will receive while this person is away. > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From prmarino1 at gmail.com Thu Nov 14 23:24:12 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 14 Nov 2013 18:24:12 -0500 Subject: [rhos-list] AUTO: Velayutham Karani1 is out of the office (returning 19-11-2013) In-Reply-To: <5285535E.1090605@redhat.com> Message-ID: <52855b9d.ea42420a.4f48.0c5b@mx.google.com> An HTML attachment was scrubbed... URL: From juanfra.rodriguez.cardoso at gmail.com Fri Nov 15 09:35:45 2013 From: juanfra.rodriguez.cardoso at gmail.com (JuanFra Rodriguez Cardoso) Date: Fri, 15 Nov 2013 10:35:45 +0100 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <528534F2.7000403@bnl.gov> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> <528534F2.7000403@bnl.gov> Message-ID: You can get a newer dnsmasq version here: http://pkgs.repoforge.org/dnsmasq/dnsmasq-2.65-1.el6.rfx.x86_64.rpm Regards, --- JuanFra 2013/11/14 Xin Zhao > Hi Terry, > > I upgrade to version 2013.1.4 on all 3 hosts (controller/network/compute). > Unfortunately that doesn't solve the DNS issue for instances. > > In the dhcp-agent.log, there is a message of: > > WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR DNSMASQ. > DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its version is 2.59 or > above! > > Not sure if it's critical or not ... > > Thanks, > Xin > > > On 11/14/2013 12:36 PM, Terry Wilson wrote: > >> Hello, >>> >>> I have a multi-host grizzly RHEL6 install, using OVS. From the instance, >>> I can ping external ips, but DNS resolv doesn't work, it only works for >>> other instances on the VM network. >>> If I do subnet-update to add public DNS server ips to the vm network, >>> DNS resolv works for external hosts, but stops working for other >>> instances on the same VM network. >>> Do I miss some configuration here? >>> >>> Thanks, >>> Xin >>> >> Which version of quantum are you using? In version 2013.1.4, a bug was >> fixed for adding a default route in the namespace in which the dnsmasq >> agent runs so that external DNS requests could be handled. See: >> https://bugs.launchpad.net/neutron/+bug/1181378 for more details. >> >> Terry >> > > > _______________________________________________ > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/ > openstack > Post to : openstack at lists.openstack.org > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/ > openstack > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rohara at redhat.com Fri Nov 15 19:50:01 2013 From: rohara at redhat.com (Ryan O'Hara) Date: Fri, 15 Nov 2013 13:50:01 -0600 Subject: [rhos-list] Fwd: Re: Lbass seems not to be functioning in RDO grizzly In-Reply-To: <52854B8B.1090905@redhat.com> References: <52854B8B.1090905@redhat.com> Message-ID: <20131115195001.GB4757@redhat.com> Please let me know what operating system you are using and the version your openstack-quantum package. Thanks. Ryan > -------- Original Message -------- > Subject: Re: [rhos-list] Lbass seems not to be functioning in RDO grizzly > Date: Thu, 14 Nov 2013 16:07:55 -0500 > From: Paul Robert Marino > To: rhos-list at redhat.com > > correction I have found something in the lbaas log > > every few seconds im seeing this which indicates an issue with qpid > " > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0a710]: > ConnectionHeartbeat() > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0a710]: > '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0a710]: > '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0a710]: > ConnectionHeartbeat() > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0d6c8]: > ConnectionHeartbeat() > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0d6c8]: > '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0d6c8]: > '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' > 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0d6c8]: > ConnectionHeartbeat() > 2013-11-14 15:59:45 ERROR [quantum.openstack.common.rpc.amqp] Timed > out waiting for RPC response. > Traceback (most recent call last): > File > "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > line 495, in __iter__ > data = self._dataqueue.get(timeout=self._timeout) > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, > in get > return waiter.wait() > File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, > in wait > return get_hub().switch() > File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line > 177, in switch > return self.greenlet.switch() > Empty > 2013-11-14 15:59:45 ERROR > [quantum.plugins.services.agent_loadbalancer.agent.manager] Unable to > retrieve ready devices > Traceback (most recent call last): > File > "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/manager.py", > line 168, in sync_state > ready_logical_devices = set(self.plugin_rpc.get_ready_devices()) > File > "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/api.py", > line 36, in get_ready_devices > topic=self.topic > File > "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", > line 80, in call > return rpc.call(context, self._get_topic(topic), msg, timeout) > File > "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", > line 140, in call > return _get_impl().call(CONF, context, topic, msg, timeout) > File > "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", > line 630, in call > rpc_amqp.get_connection_pool(conf, Connection)) > File > "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > line 614, in call > rv = list(rv) > File > "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", > line 500, in __iter__ > raise rpc_common.Timeout() > Timeout: Timeout while waiting on RPC response. > 2013-11-14 15:59:45 DEBUG [quantum.openstack.common.periodic_task] > Skipping LbaasAgentManager.collect_stats, 1 ticks left until next run > > " > whats strange about it is QPID seems to be working fine for all the > other quantum agents. > > here is my qpid configuration in the quantum.conf note any fields > encpasulated int <> have been scrubbed > > " > fake_rabbit = False > rpc_backend = quantum.openstack.common.rpc.impl_qpid > qpid_hostname = > qpid_port = 5672 > qpid_hosts = :5672 > qpid_username = > qpid_password = > qpid_sasl_mechanisms = PLAIN > qpid_reconnect = True > qpid_reconnect_timeout = 30 > qpid_reconnect_limit = 0 > qpid_reconnect_interval_min = 0 > qpid_reconnect_interval_max = 0 > qpid_reconnect_interval = 20 > qpid_heartbeat = 30 > qpid_protocol = tcp > qpid_tcp_nodelay = True > default_publisher_id = > " > > > On Thu, Nov 14, 2013 at 3:37 PM, Paul Robert Marino > wrote: > > hello every one > > Im tinkering with RDO grizzly and trying to get the lbaas agent working. > > im using openvswitch with gre tunnels which is working well. > > > > in /etc/quantum/lbaas_agent.ini I set the following > > > > interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver > > ovs_use_veth = True > > device_driver = > > quantum.plugins.services.agent_loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver > > use_namespaces = True > > > > in /etc/quantum/quantum.conf i set in the [DEFAULT] section > > > > service_plugins = > > quantum.plugins.services.agent_loadbalancer.plugin.LoadBalancerPlugin > > > > > > I restarted quantum server and started the lbass agent. > > > > haproxy is installed. > > > > when I run quantum lb-pool-list > > > > i get this error > > " > > 404 Not Found > > x> > The resource could not be found. > > > > " > > > > the weird thing is when I look at the logs I done see any errors of > > any kind not even in debug logging, and there is no indication on the > > secure log that a sudo involving haproxy was ever attempted. > > > > also i dont see it listed if i do a quantum ext-list > > > > The only thing that I can see that may indicate the cause is durring > > the startup of quantum server I see the message > > " > > WARNING [quantum.api.extensions] Extension lbaas not supported by any > > of loaded plugins > > " > > > > this said I havent seen any indication on how to fix it on any of the > > openstack docs or any of the blog sites where people have configured > > it in the past. according to every thing Ive read it should be > > working. > > > > does any one have any ideas on what may be the problem? > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > > From xzhao at bnl.gov Fri Nov 15 22:16:15 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Fri, 15 Nov 2013 17:16:15 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> <528534F2.7000403@bnl.gov> Message-ID: <52869D2F.9010306@bnl.gov> Thanks for all the reply, as Paul said, the dnsmasq version doesn't appear to be the issue here. I also tested dns between 2 different VM subnets, instances can ping each other across subnets, they can also reach dns server of the other subnet. But hostname can't be resolved across subnets. Although I am not sure if this is the same issue as it is for the outgoing nameserver, or a feature by design ... Thanks, Xin On 11/15/2013 4:35 AM, JuanFra Rodriguez Cardoso wrote: > You can get a newer dnsmasq version here: > http://pkgs.repoforge.org/dnsmasq/dnsmasq-2.65-1.el6.rfx.x86_64.rpm > > Regards, > --- > JuanFra > > > 2013/11/14 Xin Zhao > > > Hi Terry, > > I upgrade to version 2013.1.4 on all 3 hosts > (controller/network/compute). Unfortunately that doesn't solve the > DNS issue for instances. > > In the dhcp-agent.log, there is a message of: > > WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR > DNSMASQ. DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its > version is 2.59 or above! > > Not sure if it's critical or not ... > > Thanks, > Xin > > > On 11/14/2013 12:36 PM, Terry Wilson wrote: > > Hello, > > I have a multi-host grizzly RHEL6 install, using OVS. From > the instance, > I can ping external ips, but DNS resolv doesn't work, it > only works for > other instances on the VM network. > If I do subnet-update to add public DNS server ips to the > vm network, > DNS resolv works for external hosts, but stops working for > other > instances on the same VM network. > Do I miss some configuration here? > > Thanks, > Xin > > Which version of quantum are you using? In version 2013.1.4, a > bug was fixed for adding a default route in the namespace in > which the dnsmasq agent runs so that external DNS requests > could be handled. See: > https://bugs.launchpad.net/neutron/+bug/1181378 for more details. > > Terry > > > > _______________________________________________ > Mailing list: > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack at lists.openstack.org > > Unsubscribe : > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From prmarino1 at gmail.com Sun Nov 17 14:37:20 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Sun, 17 Nov 2013 09:37:20 -0500 Subject: [rhos-list] Fwd: Re: Lbass seems not to be functioning in RDO grizzly In-Reply-To: <20131115195001.GB4757@redhat.com> References: <52854B8B.1090905@redhat.com> <20131115195001.GB4757@redhat.com> Message-ID: sorry for the delayed response I have a rather odd work schedule The box I'm currently testing on is Scientific Linux 6.4 the quantum version is 2013.1.4-3 further version numbers for possibly relevant RPM's below " rpm -qa|grep -Pi '(quantum|haproxy|kernel|qpid)' qpid-cpp-server-cluster-0.14-22.el6_3.x86_64 qpid-tools-0.14-6.el6_3.noarch openstack-quantum-openvswitch-2013.1.4-3.el6.noarch kernel-2.6.32-358.118.1.openstack.el6.x86_64 python-quantumclient-2.2.1-2.el6.noarch dracut-kernel-004-303.el6.noarch python-qpid-0.14-11.el6_3.noarch qpid-cpp-client-ssl-0.14-22.el6_3.x86_64 qpid-qmf-0.14-14.el6_3.x86_64 python-quantum-2013.1.4-3.el6.noarch haproxy-1.4.22-5.el6_4.x86_64 kernel-2.6.32-358.114.1.openstack.el6.gre.2.x86_64 qpid-cpp-server-ssl-0.14-22.el6_3.x86_64 openstack-quantum-2013.1.4-3.el6.noarch kernel-firmware-2.6.32-358.123.2.openstack.el6.noarch kernel-2.6.32-358.123.2.openstack.el6.x86_64 qpid-cpp-server-0.14-22.el6_3.x86_64 python-qpid-qmf-0.14-14.el6_3.x86_64 openstack-quantum-linuxbridge-2013.1.4-3.el6.noarch qpid-cpp-client-0.14-22.el6_3.x86_64 " On Fri, Nov 15, 2013 at 2:50 PM, Ryan O'Hara wrote: > > Please let me know what operating system you are using and the version > your openstack-quantum package. Thanks. > > Ryan > >> -------- Original Message -------- >> Subject: Re: [rhos-list] Lbass seems not to be functioning in RDO grizzly >> Date: Thu, 14 Nov 2013 16:07:55 -0500 >> From: Paul Robert Marino >> To: rhos-list at redhat.com >> >> correction I have found something in the lbaas log >> >> every few seconds im seeing this which indicates an issue with qpid >> " >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0a710]: >> ConnectionHeartbeat() >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0a710]: >> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0a710]: >> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0a710]: >> ConnectionHeartbeat() >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0d6c8]: >> ConnectionHeartbeat() >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0d6c8]: >> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0d6c8]: >> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0d6c8]: >> ConnectionHeartbeat() >> 2013-11-14 15:59:45 ERROR [quantum.openstack.common.rpc.amqp] Timed >> out waiting for RPC response. >> Traceback (most recent call last): >> File >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", >> line 495, in __iter__ >> data = self._dataqueue.get(timeout=self._timeout) >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, >> in get >> return waiter.wait() >> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, >> in wait >> return get_hub().switch() >> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line >> 177, in switch >> return self.greenlet.switch() >> Empty >> 2013-11-14 15:59:45 ERROR >> [quantum.plugins.services.agent_loadbalancer.agent.manager] Unable to >> retrieve ready devices >> Traceback (most recent call last): >> File >> "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/manager.py", >> line 168, in sync_state >> ready_logical_devices = set(self.plugin_rpc.get_ready_devices()) >> File >> "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/api.py", >> line 36, in get_ready_devices >> topic=self.topic >> File >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", >> line 80, in call >> return rpc.call(context, self._get_topic(topic), msg, timeout) >> File >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", >> line 140, in call >> return _get_impl().call(CONF, context, topic, msg, timeout) >> File >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", >> line 630, in call >> rpc_amqp.get_connection_pool(conf, Connection)) >> File >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", >> line 614, in call >> rv = list(rv) >> File >> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", >> line 500, in __iter__ >> raise rpc_common.Timeout() >> Timeout: Timeout while waiting on RPC response. >> 2013-11-14 15:59:45 DEBUG [quantum.openstack.common.periodic_task] >> Skipping LbaasAgentManager.collect_stats, 1 ticks left until next run >> >> " >> whats strange about it is QPID seems to be working fine for all the >> other quantum agents. >> >> here is my qpid configuration in the quantum.conf note any fields >> encpasulated int <> have been scrubbed >> >> " >> fake_rabbit = False >> rpc_backend = quantum.openstack.common.rpc.impl_qpid >> qpid_hostname = >> qpid_port = 5672 >> qpid_hosts = :5672 >> qpid_username = >> qpid_password = >> qpid_sasl_mechanisms = PLAIN >> qpid_reconnect = True >> qpid_reconnect_timeout = 30 >> qpid_reconnect_limit = 0 >> qpid_reconnect_interval_min = 0 >> qpid_reconnect_interval_max = 0 >> qpid_reconnect_interval = 20 >> qpid_heartbeat = 30 >> qpid_protocol = tcp >> qpid_tcp_nodelay = True >> default_publisher_id = >> " >> >> >> On Thu, Nov 14, 2013 at 3:37 PM, Paul Robert Marino >> wrote: >> > hello every one >> > Im tinkering with RDO grizzly and trying to get the lbaas agent working. >> > im using openvswitch with gre tunnels which is working well. >> > >> > in /etc/quantum/lbaas_agent.ini I set the following >> > >> > interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver >> > ovs_use_veth = True >> > device_driver = >> > quantum.plugins.services.agent_loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver >> > use_namespaces = True >> > >> > in /etc/quantum/quantum.conf i set in the [DEFAULT] section >> > >> > service_plugins = >> > quantum.plugins.services.agent_loadbalancer.plugin.LoadBalancerPlugin >> > >> > >> > I restarted quantum server and started the lbass agent. >> > >> > haproxy is installed. >> > >> > when I run quantum lb-pool-list >> > >> > i get this error >> > " >> > 404 Not Found >> > > x> > The resource could not be found. >> > >> > " >> > >> > the weird thing is when I look at the logs I done see any errors of >> > any kind not even in debug logging, and there is no indication on the >> > secure log that a sudo involving haproxy was ever attempted. >> > >> > also i dont see it listed if i do a quantum ext-list >> > >> > The only thing that I can see that may indicate the cause is durring >> > the startup of quantum server I see the message >> > " >> > WARNING [quantum.api.extensions] Extension lbaas not supported by any >> > of loaded plugins >> > " >> > >> > this said I havent seen any indication on how to fix it on any of the >> > openstack docs or any of the blog sites where people have configured >> > it in the past. according to every thing Ive read it should be >> > working. >> > >> > does any one have any ideas on what may be the problem? >> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list >> >> > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From prmarino1 at gmail.com Sun Nov 17 22:04:32 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Sun, 17 Nov 2013 17:04:32 -0500 Subject: [rhos-list] Fwd: Re: Lbass seems not to be functioning in RDO grizzly In-Reply-To: References: <52854B8B.1090905@redhat.com> <20131115195001.GB4757@redhat.com> Message-ID: Well the good news is I looked at it with fresh eyes this morning and I found the primary problem. the issue was a little goofy it seams service_plugins was specified twice in the config the first time with quantum.plugins.services.agent_loadbalancer.plugin.LoadBalancerPlugin and the second time latter in the configuration where it was blank. I removed the second blank version of the line then restarted quantum server and the lbaas service and its working now. This seems to be what caused the problem, but it brings to light an other one in my mind. This kind of goes to a deeper issue though, I've noticed that whenever there is a bad config option or just an odd state in the system like a service being restarted and trying to execute commands it doesn't need too OpenStack tends to blame the amqp code in the traces rather than giving a more useful message. these messages arent particularly helpful with diagnosing the real error and in some cases distract from finding them. On Sun, Nov 17, 2013 at 9:37 AM, Paul Robert Marino wrote: > sorry for the delayed response I have a rather odd work schedule > > The box I'm currently testing on is Scientific Linux 6.4 > the quantum version is 2013.1.4-3 > > further version numbers for possibly relevant RPM's below > > " > rpm -qa|grep -Pi '(quantum|haproxy|kernel|qpid)' > qpid-cpp-server-cluster-0.14-22.el6_3.x86_64 > qpid-tools-0.14-6.el6_3.noarch > openstack-quantum-openvswitch-2013.1.4-3.el6.noarch > kernel-2.6.32-358.118.1.openstack.el6.x86_64 > python-quantumclient-2.2.1-2.el6.noarch > dracut-kernel-004-303.el6.noarch > python-qpid-0.14-11.el6_3.noarch > qpid-cpp-client-ssl-0.14-22.el6_3.x86_64 > qpid-qmf-0.14-14.el6_3.x86_64 > python-quantum-2013.1.4-3.el6.noarch > haproxy-1.4.22-5.el6_4.x86_64 > kernel-2.6.32-358.114.1.openstack.el6.gre.2.x86_64 > qpid-cpp-server-ssl-0.14-22.el6_3.x86_64 > openstack-quantum-2013.1.4-3.el6.noarch > kernel-firmware-2.6.32-358.123.2.openstack.el6.noarch > kernel-2.6.32-358.123.2.openstack.el6.x86_64 > qpid-cpp-server-0.14-22.el6_3.x86_64 > python-qpid-qmf-0.14-14.el6_3.x86_64 > openstack-quantum-linuxbridge-2013.1.4-3.el6.noarch > qpid-cpp-client-0.14-22.el6_3.x86_64 > " > > On Fri, Nov 15, 2013 at 2:50 PM, Ryan O'Hara wrote: >> >> Please let me know what operating system you are using and the version >> your openstack-quantum package. Thanks. >> >> Ryan >> >>> -------- Original Message -------- >>> Subject: Re: [rhos-list] Lbass seems not to be functioning in RDO grizzly >>> Date: Thu, 14 Nov 2013 16:07:55 -0500 >>> From: Paul Robert Marino >>> To: rhos-list at redhat.com >>> >>> correction I have found something in the lbaas log >>> >>> every few seconds im seeing this which indicates an issue with qpid >>> " >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0a710]: >>> ConnectionHeartbeat() >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0a710]: >>> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0a710]: >>> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0a710]: >>> ConnectionHeartbeat() >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] SENT[2b0d6c8]: >>> ConnectionHeartbeat() >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] SENT[2b0d6c8]: >>> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.raw] READ[2b0d6c8]: >>> '\x0f\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x01\n\x00\x00' >>> 2013-11-14 15:59:45 DEBUG [qpid.messaging.io.ops] RCVD[2b0d6c8]: >>> ConnectionHeartbeat() >>> 2013-11-14 15:59:45 ERROR [quantum.openstack.common.rpc.amqp] Timed >>> out waiting for RPC response. >>> Traceback (most recent call last): >>> File >>> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", >>> line 495, in __iter__ >>> data = self._dataqueue.get(timeout=self._timeout) >>> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 298, >>> in get >>> return waiter.wait() >>> File "/usr/lib/python2.6/site-packages/eventlet/queue.py", line 129, >>> in wait >>> return get_hub().switch() >>> File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line >>> 177, in switch >>> return self.greenlet.switch() >>> Empty >>> 2013-11-14 15:59:45 ERROR >>> [quantum.plugins.services.agent_loadbalancer.agent.manager] Unable to >>> retrieve ready devices >>> Traceback (most recent call last): >>> File >>> "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/manager.py", >>> line 168, in sync_state >>> ready_logical_devices = set(self.plugin_rpc.get_ready_devices()) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/plugins/services/agent_loadbalancer/agent/api.py", >>> line 36, in get_ready_devices >>> topic=self.topic >>> File >>> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/proxy.py", >>> line 80, in call >>> return rpc.call(context, self._get_topic(topic), msg, timeout) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/__init__.py", >>> line 140, in call >>> return _get_impl().call(CONF, context, topic, msg, timeout) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/impl_qpid.py", >>> line 630, in call >>> rpc_amqp.get_connection_pool(conf, Connection)) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", >>> line 614, in call >>> rv = list(rv) >>> File >>> "/usr/lib/python2.6/site-packages/quantum/openstack/common/rpc/amqp.py", >>> line 500, in __iter__ >>> raise rpc_common.Timeout() >>> Timeout: Timeout while waiting on RPC response. >>> 2013-11-14 15:59:45 DEBUG [quantum.openstack.common.periodic_task] >>> Skipping LbaasAgentManager.collect_stats, 1 ticks left until next run >>> >>> " >>> whats strange about it is QPID seems to be working fine for all the >>> other quantum agents. >>> >>> here is my qpid configuration in the quantum.conf note any fields >>> encpasulated int <> have been scrubbed >>> >>> " >>> fake_rabbit = False >>> rpc_backend = quantum.openstack.common.rpc.impl_qpid >>> qpid_hostname = >>> qpid_port = 5672 >>> qpid_hosts = :5672 >>> qpid_username = >>> qpid_password = >>> qpid_sasl_mechanisms = PLAIN >>> qpid_reconnect = True >>> qpid_reconnect_timeout = 30 >>> qpid_reconnect_limit = 0 >>> qpid_reconnect_interval_min = 0 >>> qpid_reconnect_interval_max = 0 >>> qpid_reconnect_interval = 20 >>> qpid_heartbeat = 30 >>> qpid_protocol = tcp >>> qpid_tcp_nodelay = True >>> default_publisher_id = >>> " >>> >>> >>> On Thu, Nov 14, 2013 at 3:37 PM, Paul Robert Marino >>> wrote: >>> > hello every one >>> > Im tinkering with RDO grizzly and trying to get the lbaas agent working. >>> > im using openvswitch with gre tunnels which is working well. >>> > >>> > in /etc/quantum/lbaas_agent.ini I set the following >>> > >>> > interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver >>> > ovs_use_veth = True >>> > device_driver = >>> > quantum.plugins.services.agent_loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver >>> > use_namespaces = True >>> > >>> > in /etc/quantum/quantum.conf i set in the [DEFAULT] section >>> > >>> > service_plugins = >>> > quantum.plugins.services.agent_loadbalancer.plugin.LoadBalancerPlugin >>> > >>> > >>> > I restarted quantum server and started the lbass agent. >>> > >>> > haproxy is installed. >>> > >>> > when I run quantum lb-pool-list >>> > >>> > i get this error >>> > " >>> > 404 Not Found >>> > >> x> > The resource could not be found. >>> > >>> > " >>> > >>> > the weird thing is when I look at the logs I done see any errors of >>> > any kind not even in debug logging, and there is no indication on the >>> > secure log that a sudo involving haproxy was ever attempted. >>> > >>> > also i dont see it listed if i do a quantum ext-list >>> > >>> > The only thing that I can see that may indicate the cause is durring >>> > the startup of quantum server I see the message >>> > " >>> > WARNING [quantum.api.extensions] Extension lbaas not supported by any >>> > of loaded plugins >>> > " >>> > >>> > this said I havent seen any indication on how to fix it on any of the >>> > openstack docs or any of the blog sites where people have configured >>> > it in the past. according to every thing Ive read it should be >>> > working. >>> > >>> > does any one have any ideas on what may be the problem? >>> >>> _______________________________________________ >>> rhos-list mailing list >>> rhos-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/rhos-list >>> >>> >> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list From tbrunell at redhat.com Mon Nov 18 05:26:52 2013 From: tbrunell at redhat.com (Ted Brunell) Date: Mon, 18 Nov 2013 00:26:52 -0500 (EST) Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <52869D2F.9010306@bnl.gov> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> <528534F2.7000403@bnl.gov> <52869D2F.9010306@bnl.gov> Message-ID: <1212907069.7588482.1384752412080.JavaMail.root@redhat.com> I have a blog post on getting LBaaS working at http://myopensourcelife.com/2013/08/08/openstack-load-balancer-as-a-service-lbaas/ I used RHOS 3 (grizzly) for the post, but should work on RDO Grizzly. R/ Ted ----- Original Message ----- From: "Xin Zhao" To: "JuanFra Rodriguez Cardoso" Cc: "redhat openstack" , "openstack at lists.openstack.org Openstack" Sent: Friday, November 15, 2013 5:16:15 PM Subject: Re: [rhos-list] [Openstack] dns problem for instances Thanks for all the reply, as Paul said, the dnsmasq version doesn't appear to be the issue here. I also tested dns between 2 different VM subnets, instances can ping each other across subnets, they can also reach dns server of the other subnet. But hostname can't be resolved across subnets. Although I am not sure if this is the same issue as it is for the outgoing nameserver, or a feature by design ... Thanks, Xin On 11/15/2013 4:35 AM, JuanFra Rodriguez Cardoso wrote: You can get a newer dnsmasq version here: http://pkgs.repoforge.org/dnsmasq/dnsmasq-2.65-1.el6.rfx.x86_64.rpm Regards, --- JuanFra 2013/11/14 Xin Zhao < xzhao at bnl.gov > Hi Terry, I upgrade to version 2013.1.4 on all 3 hosts (controller/network/compute). Unfortunately that doesn't solve the DNS issue for instances. In the dhcp-agent.log, there is a message of: WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR DNSMASQ. DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its version is 2.59 or above! Not sure if it's critical or not ... Thanks, Xin On 11/14/2013 12:36 PM, Terry Wilson wrote: Hello, I have a multi-host grizzly RHEL6 install, using OVS. From the instance, I can ping external ips, but DNS resolv doesn't work, it only works for other instances on the VM network. If I do subnet-update to add public DNS server ips to the vm network, DNS resolv works for external hosts, but stops working for other instances on the same VM network. Do I miss some configuration here? Thanks, Xin Which version of quantum are you using? In version 2013.1.4, a bug was fixed for adding a default route in the namespace in which the dnsmasq agent runs so that external DNS requests could be handled. See: https://bugs.launchpad.net/neutron/+bug/1181378 for more details. Terry _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack at lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From xuzhong at cisco.com Wed Nov 20 17:36:10 2013 From: xuzhong at cisco.com (Xuan Zhong (xuzhong)) Date: Wed, 20 Nov 2013 17:36:10 +0000 Subject: [rhos-list] How to access OpenStack rpms? Message-ID: I couldn't find the information on how to get the subscription to the Openstack channel and gain access to the OpenStack rpms for RHEL6.4. Can anyone provide some information on this? Thanks Xuan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hateya at redhat.com Wed Nov 20 17:49:02 2013 From: hateya at redhat.com (Haim Ateya) Date: Wed, 20 Nov 2013 12:49:02 -0500 (EST) Subject: [rhos-list] How to access OpenStack rpms? In-Reply-To: References: Message-ID: <1237710559.37911871.1384969742156.JavaMail.root@redhat.com> I would have try: rhel-x86_64-server-6-ost-4 rhel-x86_64-server-6-ost-4-cts rhel-x86_64-server-6-ost-4-cts-debuginfo rhel-x86_64-server-6-ost-4-debuginfo ----- Original Message ----- > From: "Xuan Zhong (xuzhong)" > To: rhos-list at redhat.com > Sent: Wednesday, November 20, 2013 7:36:10 PM > Subject: [rhos-list] How to access OpenStack rpms? > > I couldn't find the information on how to get the subscription to the > Openstack channel and gain access to the OpenStack rpms for RHEL6.4. Can > anyone provide some information on this? > > Thanks > Xuan > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From tvvcox at redhat.com Wed Nov 20 19:09:32 2013 From: tvvcox at redhat.com (Tomas Von Veschler) Date: Wed, 20 Nov 2013 20:09:32 +0100 Subject: [rhos-list] How to access OpenStack rpms? In-Reply-To: References: Message-ID: <528D08EC.3070808@redhat.com> On 11/20/2013 06:36 PM, Xuan Zhong (xuzhong) wrote: > I couldn't find the information on how to get the subscription to the > Openstack channel and gain access to the OpenStack rpms for RHEL6.4. Can > anyone provide some information on this? Hi Xuan, First I'd check if you have a valid subscription: https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/3/html/Getting_Started_Guide/ch02.html#idp17784560 In case you don't, a subscription can be obtained from: 1. NFR (not-for-resale) partner subscription 2. Via Red Hat sales 3. Via free 90-day evaluation: https://www.redhat.com/products/cloud-computing/openstack/ As Cisco is a partner with Red Hat Openstack, I'd suggest to contact your internal Red Hat alliance manager, maybe he can provide access to NFRs. Then you can access the online manuals with all the information at: https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux_OpenStack_Platform/ Regards, Tomas From mwaite at redhat.com Wed Nov 20 19:19:55 2013 From: mwaite at redhat.com (Michael Waite) Date: Wed, 20 Nov 2013 14:19:55 -0500 Subject: [rhos-list] How to access OpenStack rpms? In-Reply-To: <528D08EC.3070808@redhat.com> References: <528D08EC.3070808@redhat.com> Message-ID: <528D0B5B.4060007@redhat.com> Cisco is a partner in our OpenStack partner network and as such has fast track for all our downloads. It is not necessary for Cisco to have to use any of the three paths listed below. Cisco also has support as part of being a partner of ours as well. I have attached the partner access process document. Yes I know there are others on this list as well but I feel this is important information to show what we offer our partners. Xuan, May I suggest connecting with your partner manager that manages the relationship with Red Hat? ------Mike On 11/20/2013 02:09 PM, Tomas Von Veschler wrote: > On 11/20/2013 06:36 PM, Xuan Zhong (xuzhong) wrote: >> I couldn't find the information on how to get the subscription to the >> Openstack channel and gain access to the OpenStack rpms for RHEL6.4. Can >> anyone provide some information on this? > > Hi Xuan, > > First I'd check if you have a valid subscription: > > https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/3/html/Getting_Started_Guide/ch02.html#idp17784560 > > > In case you don't, a subscription can be obtained from: > > 1. NFR (not-for-resale) partner subscription > 2. Via Red Hat sales > 3. Via free 90-day evaluation: > > https://www.redhat.com/products/cloud-computing/openstack/ > > As Cisco is a partner with Red Hat Openstack, I'd suggest to contact > your internal Red Hat alliance manager, maybe he can provide access to > NFRs. > > Then you can access the online manuals with all the information at: > > https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux_OpenStack_Platform/ > > > Regards, > Tomas > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- A non-text attachment was scrubbed... Name: RHOSCIPN_Software_access_guide_V1.7.pdf Type: application/x-octet-stream Size: 226310 bytes Desc: not available URL: From xuzhong at cisco.com Wed Nov 20 19:36:35 2013 From: xuzhong at cisco.com (Xuan Zhong (xuzhong)) Date: Wed, 20 Nov 2013 19:36:35 +0000 Subject: [rhos-list] How to access OpenStack rpms? In-Reply-To: <528D0B5B.4060007@redhat.com> Message-ID: Thanks for the information. May I ask where I can find the contact information of Cisco Redhat partner manager? Xuan On 11/20/13 2:19 PM, "Michael Waite" wrote: >Cisco is a partner in our OpenStack partner network and as such has fast >track for all our downloads. >It is not necessary for Cisco to have to use any of the three paths >listed below. >Cisco also has support as part of being a partner of ours as well. > >I have attached the partner access process document. >Yes I know there are others on this list as well but I feel this is >important information to show what we offer our partners. > >Xuan, >May I suggest connecting with your partner manager that manages the >relationship with Red Hat? > >------Mike > > > >On 11/20/2013 02:09 PM, Tomas Von Veschler wrote: >> On 11/20/2013 06:36 PM, Xuan Zhong (xuzhong) wrote: >>> I couldn't find the information on how to get the subscription to the >>> Openstack channel and gain access to the OpenStack rpms for RHEL6.4. >>>Can >>> anyone provide some information on this? >> >> Hi Xuan, >> >> First I'd check if you have a valid subscription: >> >> >>https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Lin >>ux_OpenStack_Platform/3/html/Getting_Started_Guide/ch02.html#idp17784560 >> >> >> In case you don't, a subscription can be obtained from: >> >> 1. NFR (not-for-resale) partner subscription >> 2. Via Red Hat sales >> 3. Via free 90-day evaluation: >> >> https://www.redhat.com/products/cloud-computing/openstack/ >> >> As Cisco is a partner with Red Hat Openstack, I'd suggest to contact >> your internal Red Hat alliance manager, maybe he can provide access to >> NFRs. >> >> Then you can access the online manuals with all the information at: >> >> >>https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux_Ope >>nStack_Platform/ >> >> >> Regards, >> Tomas >> >> _______________________________________________ >> rhos-list mailing list >> rhos-list at redhat.com >> https://www.redhat.com/mailman/listinfo/rhos-list From xzhao at bnl.gov Wed Nov 20 22:48:08 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Wed, 20 Nov 2013 17:48:08 -0500 Subject: [rhos-list] metadata service not working for VMs Message-ID: <528D3C28.5070508@bnl.gov> Hello, I am installing grizzly with quantum/OVS using kernel-2.6.32-358.123.2.openstack.el6.x86_64 and openstack-XXX-2013.1.4-3. From inside the VM, I can ping 169.254.169.254 (it's available in the routing table), but curl commands fail with the following errors: $>curl http://169.254.169.254 About to connect to 169.254.169.254 port 80 ... Connection refused Does the metadata service run on the controller node or the network node, on which port and which namespace ? The VMs can only talk to the network host via the physical VM network, they don't have access to the management network. Below is the relevant configuration information. Another info is that I still have some DNS issue for the VMs, external DNS and internal DNS can't work at the same time, meaning if I assign public DNS servers to the VM virtual subnets, VM can resolve external hostnames, but doesn't work for other VMs inside the same subnet, and if I use the default internal DNS, VMs can't resolve external hostnames but they can resolve names within the same VM subnet. I am not sure if this is related to the metadata issue or not, I would think not, as the above metadata command uses ip directly... Thanks, Xin on controller node: nova.conf: service_neutron_metadata_proxy=true quantum_metadata_proxy_shared_secret= On network node: dhcp_agent.ini: enable_isolated_metadata = True metadata_agent.ini: [DEFAULT] auth_url = http://localhost:35357/v2.0 auth_region = RegionOne admin_tenant_name = %SERVICE_TENANT_NAME% admin_user = %SERVICE_USER% admin_password = %SERVICE_PASSWORD% auth_strategy = keystone metadata_proxy_shared_secret = [keystone_authtoken] auth_host = admin_tenant_name = services admin_user = quantum admin_password = The VM internal subnet info: +------------------+--------------------------------------------+ | Field | Value | +------------------+--------------------------------------------+ | allocation_pools | {"start": "10.0.1.2", "end": "10.0.1.254"} | | cidr | 10.0.1.0/24 | | dns_nameservers | 8.8.4.4 | | | 8.8.8.8 | | enable_dhcp | True | | gateway_ip | 10.0.1.1 | | host_routes | | | id | 505949ed-30bb-4c5e-8d1b-9ef2745f9455 | | ip_version | 4 | | name | | | network_id | 31f9d39b-012f-4447-92a4-1a3b5514b37d | | tenant_id | 22b1956ec62a49e88fb93b53a4f10337 | +------------------+--------------------------------------------+ From xzhao at bnl.gov Wed Nov 20 23:05:25 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Wed, 20 Nov 2013 18:05:25 -0500 Subject: [rhos-list] metadata service not working for VMs In-Reply-To: <528D3C28.5070508@bnl.gov> References: <528D3C28.5070508@bnl.gov> Message-ID: <528D4035.9050000@bnl.gov> Some more info: from the router namespace, I can see the metadata service is listening on port 9697, and an NAT rule for it: [root at cldnet01 quantum(keystone_admin)]# ip netns exec qrouter-183f4dda-cb26-4822-af6d-941b4b0831b4 netstat -lpnt Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:9697 0.0.0.0:* LISTEN 2703/python [root at cldnet01 quantum(keystone_admin)]# ip netns exec qrouter-183f4dda-cb26-4822-af6d-941b4b0831b4 iptables -L -t nat ...... Chain quantum-l3-agent-PREROUTING (1 references) target prot opt source destination REDIRECT tcp -- anywhere 169.254.169.254 tcp dpt:http redir ports 9697 ...... On 11/20/2013 5:48 PM, Xin Zhao wrote: > Hello, > > I am installing grizzly with quantum/OVS using > kernel-2.6.32-358.123.2.openstack.el6.x86_64 and > openstack-XXX-2013.1.4-3. > From inside the VM, I can ping 169.254.169.254 (it's available in the > routing table), but curl commands fail with the following errors: > > $>curl http://169.254.169.254 > About to connect to 169.254.169.254 port 80 ... > Connection refused > > Does the metadata service run on the controller node or the network > node, on which port and which namespace ? The VMs can only talk to > the network > host via the physical VM network, they don't have access to the > management network. > > Below is the relevant configuration information. Another info is that > I still have some DNS issue for the VMs, external DNS and internal DNS > can't work at the same time, > meaning if I assign public DNS servers to the VM virtual subnets, VM > can resolve external hostnames, but doesn't work for other VMs inside > the same subnet, and if I use > the default internal DNS, VMs can't resolve external hostnames but > they can resolve names within the same VM subnet. I am not sure if > this is related to the metadata issue or not, I > would think not, as the above metadata command uses ip directly... > > Thanks, > Xin > > > on controller node: > nova.conf: > service_neutron_metadata_proxy=true > quantum_metadata_proxy_shared_secret= > > On network node: > dhcp_agent.ini: > enable_isolated_metadata = True > metadata_agent.ini: > [DEFAULT] > auth_url = http://localhost:35357/v2.0 > auth_region = RegionOne > admin_tenant_name = %SERVICE_TENANT_NAME% > admin_user = %SERVICE_USER% > admin_password = %SERVICE_PASSWORD% > auth_strategy = keystone > > metadata_proxy_shared_secret = > [keystone_authtoken] > auth_host = > admin_tenant_name = services > admin_user = quantum > admin_password = > > The VM internal subnet info: > > +------------------+--------------------------------------------+ > | Field | Value | > +------------------+--------------------------------------------+ > | allocation_pools | {"start": "10.0.1.2", "end": "10.0.1.254"} | > | cidr | 10.0.1.0/24 | > | dns_nameservers | 8.8.4.4 | > | | 8.8.8.8 | > | enable_dhcp | True | > | gateway_ip | 10.0.1.1 | > | host_routes | | > | id | 505949ed-30bb-4c5e-8d1b-9ef2745f9455 | > | ip_version | 4 | > | name | | > | network_id | 31f9d39b-012f-4447-92a4-1a3b5514b37d | > | tenant_id | 22b1956ec62a49e88fb93b53a4f10337 | > +------------------+--------------------------------------------+ > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From prmarino1 at gmail.com Thu Nov 21 02:06:17 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Wed, 20 Nov 2013 21:06:17 -0500 Subject: [rhos-list] metadata service not working for VMs In-Reply-To: <528D4035.9050000@bnl.gov> Message-ID: <528d6a99.8a10e00a.5eb7.494d@mx.google.com> An HTML attachment was scrubbed... URL: From dfv at eurotux.com Mon Nov 25 16:22:27 2013 From: dfv at eurotux.com (Diogo Vieira) Date: Mon, 25 Nov 2013 16:22:27 +0000 Subject: [rhos-list] Problems Installing Swift with Packstack Message-ID: Hi, I'm trying to install a Swift cluster (1 Proxy Node and 5 Storage Nodes) with Packstack, but I keep getting this error: > Error: Parameter source failed on Firewall[001 swift storage and rsync incoming 10.10.6.16/vdb]: Munging failed for value "10.10.6.16/vdb" in class source: no address for 10.10.6.16/vdb This is what I think is relevant of the answer file: > # A comma separated list of IP addresses on which to install the > # Swift Storage services, each entry should take the format > # [/dev], for example 127.0.0.1/vdb will install /dev/vdb > # on 127.0.0.1 as a swift storage device(packstack does not create the > # filesystem, you must do this first), if /dev is omitted Packstack > # will create a loopback device for a test setup > CONFIG_SWIFT_STORAGE_HOSTS=10.10.4.21/mapper/mpatha,10.10.6.14/vdb,10.10.6.15/vdb,10.10.6.16/vdb,10.10.6.17/vdb I have, on 10.10.6.14, 10.10.6.15, 10.10.6.16 and 10.10.6.17 a vdb device with a vdb1 partition formatted with xfs. Can someone help me? Thank you very much, Diogo Vieira From gilles at redhat.com Tue Nov 26 02:59:58 2013 From: gilles at redhat.com (Gilles Dubreuil) Date: Tue, 26 Nov 2013 13:59:58 +1100 Subject: [rhos-list] RHEL6: Nested virtualization support for kvm_intel? Message-ID: <1385434798.2367.163.camel@gil.surfgate.org> Hi, Maybe I'm dreaming but I thought soon we'll able to go Inception level 3, well at least that's how I explain to my partner what we're doing. I'm not sure if any previous OpenStack kernels did have the feature. But latest RHEL6.5 kernel seems not to support nested virtualization on kvm_intel kernel module. I'm not 100% sure if CPU support is required besides normal vmx flag. But I can't see any flag advertising such feature or not while comparing Intel specs as well with my laptop where it works with Fedora. If there is no specific CPU support needed then the blocker will be the module. In such case, do there is any road map for the feature? Didn't find much rhel6 support for it (google/BZ) either. Thanks Gilles # uname -a Linux host10.oslab.priv 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64 x86_64 x86_64 GNU/Linux # modinfo kvm_intel | grep -i nested # cat /proc/cpuinfo | grep name | uniq model name : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz # cat /proc/cpuinfo | grep flags | uniq flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid From acathrow at redhat.com Tue Nov 26 03:06:08 2013 From: acathrow at redhat.com (Andrew Cathrow) Date: Mon, 25 Nov 2013 22:06:08 -0500 (EST) Subject: [rhos-list] RHEL6: Nested virtualization support for kvm_intel? In-Reply-To: <1385434798.2367.163.camel@gil.surfgate.org> References: <1385434798.2367.163.camel@gil.surfgate.org> Message-ID: <2146564566.15716147.1385435168932.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Gilles Dubreuil" > To: rhos-list at redhat.com > Sent: Monday, November 25, 2013 9:59:58 PM > Subject: [rhos-list] RHEL6: Nested virtualization support for kvm_intel? > > Hi, > > Maybe I'm dreaming but I thought soon we'll able to go Inception level > 3, well at least that's how I explain to my partner what we're doing. > > I'm not sure if any previous OpenStack kernels did have the feature. > But latest RHEL6.5 kernel seems not to support nested virtualization on > kvm_intel kernel module. > > I'm not 100% sure if CPU support is required besides normal vmx flag. > But I can't see any flag advertising such feature or not while comparing > Intel specs as well with my laptop where it works with Fedora. > > If there is no specific CPU support needed then the blocker will be the > module. In such case, do there is any road map for the feature? > Didn't find much rhel6 support for it (google/BZ) either. Not until RHEL 7, however it does work (but not supported) on AMD hardware. > > Thanks > Gilles > > > # uname -a > Linux host10.oslab.priv 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 > EST 2013 x86_64 x86_64 x86_64 GNU/Linux > # modinfo kvm_intel | grep -i nested > # cat /proc/cpuinfo | grep name | uniq > model name : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz > # cat /proc/cpuinfo | grep flags | uniq > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge > mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe > syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good > xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx > smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt > tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts > dts tpr_shadow vnmi flexpriority ept vpid > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > > > From gilles at redhat.com Tue Nov 26 03:08:30 2013 From: gilles at redhat.com (Gilles Dubreuil) Date: Tue, 26 Nov 2013 14:08:30 +1100 Subject: [rhos-list] RHEL6: Nested virtualization support for kvm_intel? In-Reply-To: <2146564566.15716147.1385435168932.JavaMail.root@redhat.com> References: <1385434798.2367.163.camel@gil.surfgate.org> <2146564566.15716147.1385435168932.JavaMail.root@redhat.com> Message-ID: <1385435310.2367.165.camel@gil.surfgate.org> On Mon, 2013-11-25 at 22:06 -0500, Andrew Cathrow wrote: > > ----- Original Message ----- > > From: "Gilles Dubreuil" > > To: rhos-list at redhat.com > > Sent: Monday, November 25, 2013 9:59:58 PM > > Subject: [rhos-list] RHEL6: Nested virtualization support for kvm_intel? > > > > Hi, > > > > Maybe I'm dreaming but I thought soon we'll able to go Inception level > > 3, well at least that's how I explain to my partner what we're doing. > > > > I'm not sure if any previous OpenStack kernels did have the feature. > > But latest RHEL6.5 kernel seems not to support nested virtualization on > > kvm_intel kernel module. > > > > I'm not 100% sure if CPU support is required besides normal vmx flag. > > But I can't see any flag advertising such feature or not while comparing > > Intel specs as well with my laptop where it works with Fedora. > > > > If there is no specific CPU support needed then the blocker will be the > > module. In such case, do there is any road map for the feature? > > Didn't find much rhel6 support for it (google/BZ) either. > > Not until RHEL 7, however it does work (but not supported) on AMD hardware. > > Thanks. > > > > Thanks > > Gilles > > > > > > # uname -a > > Linux host10.oslab.priv 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 > > EST 2013 x86_64 x86_64 x86_64 GNU/Linux > > # modinfo kvm_intel | grep -i nested > > # cat /proc/cpuinfo | grep name | uniq > > model name : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz > > # cat /proc/cpuinfo | grep flags | uniq > > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge > > mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe > > syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good > > xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx > > smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt > > tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts > > dts tpr_shadow vnmi flexpriority ept vpid > > > > > > _______________________________________________ > > rhos-list mailing list > > rhos-list at redhat.com > > https://www.redhat.com/mailman/listinfo/rhos-list > > > > > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From Sunny_Anthony at symantec.com Tue Nov 26 14:14:36 2013 From: Sunny_Anthony at symantec.com (Sunny Anthony) Date: Tue, 26 Nov 2013 06:14:36 -0800 Subject: [rhos-list] Problems in attaching volumes to the instances Message-ID: <46A6A5BAC1E78D489E00C35CED46F73C05193C45DA@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> Hi Experts, I'm trying to attach a newly created volume to an instance but there is no "Edit Attachments" option available in the Project --> Volumes category. Please guide. Regards, Sunny _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From Sunny_Anthony at symantec.com Tue Nov 26 18:23:54 2013 From: Sunny_Anthony at symantec.com (Sunny Anthony) Date: Tue, 26 Nov 2013 10:23:54 -0800 Subject: [rhos-list] Problems in attaching volumes to the instances In-Reply-To: <46A6A5BAC1E78D489E00C35CED46F73C05193C45DA@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> References: <46A6A5BAC1E78D489E00C35CED46F73C05193C45DA@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> Message-ID: <46A6A5BAC1E78D489E00C35CED46F73C05193C4624@APJ1XCHEVSPIN31.SYMC.SYMANTEC.COM> Hello Experts, Observing weird behavior in RDO , "Edit Attachments" option available for 2nd volume created. :-( "Edit Attachments" option is not available for first created volume. Now issue is none of the volumes are getting attached to the instance even after attaching using "Edit Attachments" option. Any inputs is highly appreciated. Regards, Sunny -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Sunny Anthony Sent: Tuesday, November 26, 2013 7:45 PM To: rhos-list at redhat.com Subject: [rhos-list] Problems in attaching volumes to the instances Hi Experts, I'm trying to attach a newly created volume to an instance but there is no "Edit Attachments" option available in the Project --> Volumes category. Please guide. Regards, Sunny _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From xzhao at bnl.gov Wed Nov 27 19:42:52 2013 From: xzhao at bnl.gov (Xin Zhao) Date: Wed, 27 Nov 2013 14:42:52 -0500 Subject: [rhos-list] [Openstack] dns problem for instances In-Reply-To: <52869D2F.9010306@bnl.gov> References: <5282A928.1040701@bnl.gov> <528506A2.9030504@bnl.gov> <816219590.24004983.1384450612377.JavaMail.root@redhat.com> <528534F2.7000403@bnl.gov> <52869D2F.9010306@bnl.gov> Message-ID: <52964B3C.4020504@bnl.gov> Hello, Just to close the loop : after assigning public DNS servers to dnsmasq_dns_server option in dhcp_agent.ini, both the internal and external hostnames can be resolved. The DNS server in /etc/resolv.conf is on the internal management subnet, which is not reachable from the dhcp network namespace, that's probably why it wasn't working before. Thanks, Xin On 11/15/2013 5:16 PM, Xin Zhao wrote: > Thanks for all the reply, as Paul said, the dnsmasq version doesn't > appear to be the issue here. > > I also tested dns between 2 different VM subnets, instances can ping > each other across subnets, they can also reach dns server of the other > subnet. But hostname can't be resolved across subnets. > Although I am not sure if this is the same issue as it is for the > outgoing nameserver, or a feature by design ... > > Thanks, > Xin > > On 11/15/2013 4:35 AM, JuanFra Rodriguez Cardoso wrote: >> You can get a newer dnsmasq version here: >> http://pkgs.repoforge.org/dnsmasq/dnsmasq-2.65-1.el6.rfx.x86_64.rpm >> >> Regards, >> --- >> JuanFra >> >> >> 2013/11/14 Xin Zhao > >> >> Hi Terry, >> >> I upgrade to version 2013.1.4 on all 3 hosts >> (controller/network/compute). Unfortunately that doesn't solve >> the DNS issue for instances. >> >> In the dhcp-agent.log, there is a message of: >> >> WARNING [quantum.agent.linux.dhcp] FAILED VERSION REQUIREMENT FOR >> DNSMASQ. DHCP AGENT MAY NOT RUN CORRECTLY! Please ensure that its >> version is 2.59 or above! >> >> Not sure if it's critical or not ... >> >> Thanks, >> Xin >> >> >> On 11/14/2013 12:36 PM, Terry Wilson wrote: >> >> Hello, >> >> I have a multi-host grizzly RHEL6 install, using OVS. >> From the instance, >> I can ping external ips, but DNS resolv doesn't work, it >> only works for >> other instances on the VM network. >> If I do subnet-update to add public DNS server ips to the >> vm network, >> DNS resolv works for external hosts, but stops working >> for other >> instances on the same VM network. >> Do I miss some configuration here? >> >> Thanks, >> Xin >> >> Which version of quantum are you using? In version 2013.1.4, >> a bug was fixed for adding a default route in the namespace >> in which the dnsmasq agent runs so that external DNS requests >> could be handled. See: >> https://bugs.launchpad.net/neutron/+bug/1181378 for more details. >> >> Terry >> >> >> >> _______________________________________________ >> Mailing list: >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> Post to : openstack at lists.openstack.org >> >> Unsubscribe : >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> >> > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: