From rich.minton at lmco.com Fri Mar 1 17:06:04 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 1 Mar 2013 17:06:04 +0000 Subject: [rhos-list] Instances showing up on the wrong host. Message-ID: I created an instance that came up on my second compute node, as shown in the Horizon Dashboard when logged in as admin. In the VNC console it is seen as instance-0000008a. But when I look at the list of instances on compute2 using "virsh list --all" I do not see that instance in the list. If I do a "virsh list --all" on the controller it does show up in the list (see below). I also logged into the VM and checked the MAC address. It is in the list when I do a "brctl showmacs" on the controller node. Has anyone seen this before? Also, does anyone know how to get rid of the instances marked as "shut off" in the virsh list? These instances have no associated directory in the instances directory. It's like they are in some database somewhere but are stranded. On Controller: Virsh list --all Id Name State ---------------------------------------------------- 15 instance-00000088 running 16 instance-00000089 running 17 instance-0000008a running 18 instance-0000008b running 19 instance-0000008d running 20 instance-0000008f running 21 instance-00000098 running - instance-00000067 shut off - instance-0000006e shut off - instance-0000006f shut off - instance-00000071 shut off - instance-00000072 shut off - instance-00000073 shut off - instance-00000074 shut off - instance-00000076 shut off - instance-00000077 shut off - instance-00000078 shut off brctl showmacs br976 port no mac addr is local? ageing timer 6 fa:16:3e:0e:45:56 no 35.32 1 fa:16:3e:17:cb:7c yes 0.00 4 fa:16:3e:23:22:32 no 42.46 3 fa:16:3e:30:83:a5 no 231.03 5 fa:16:3e:40:23:ee no 292.82 6 fe:16:3e:0e:45:56 yes 0.00 4 fe:16:3e:23:22:32 yes 0.00 7 fe:16:3e:23:d1:16 yes 0.00 8 fe:16:3e:2a:bd:ef yes 0.00 3 fe:16:3e:30:83:a5 yes 0.00 5 fe:16:3e:40:23:ee yes 0.00 2 fe:16:3e:7b:f9:d9 yes 0.00 Thanks for the help! Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From SAM.W.GABRIEL at saic.com Fri Mar 1 20:21:50 2013 From: SAM.W.GABRIEL at saic.com (SAM GABRIEL) Date: Fri, 01 Mar 2013 12:21:50 -0800 Subject: [rhos-list] Nova-manage command failure Message-ID: I?m following the instructions in https://access.redhat.com/knowledge/docs/en-US/Red_Hat_OpenStack/2/html/Gett ing_Started_Guide/chap-Deploying_Compute_Services.html (Chapter 11 of ?Red Hat OpenStack Getting Started Guide, Getting Started with Red Hat OpenStack 2.0 (Folsom) Preview, Edition 1.0?). I am using Nova networking. My installation has proceeded without error up until the following command: sudo nova-manage network create demonet 10.0.0.0/24 1 256 --bridge=demonetbr0 Which resulted in the following error message: Command failed, please check log for more info 2013-03-01 11:55:55 CRITICAL nova [req-68875b1d-cd10-4e24-a55d-434ea7e6845f None None] 'vlan_start' None of the Nova logs had any further details. I am installing OpenStack on RHEL 6.4 in a VirtualBox VM with my network interfaces configured as follows: 1: lo: mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:b8:98:2e brd ff:ff:ff:ff:ff:ff inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0 inet6 fe80::a00:27ff:feb8:982e/64 scope link valid_lft forever preferred_lft forever 3: eth1: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 08:00:27:18:b0:e6 brd ff:ff:ff:ff:ff:ff inet6 fe80::a00:27ff:fe18:b0e6/64 scope link valid_lft forever preferred_lft forever 4: virbr0: mtu 1500 qdisc noqueue state UNKNOWN link/ether 52:54:00:81:f8:23 brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 5: virbr0-nic: mtu 1500 qdisc noop state DOWN qlen 500 link/ether 52:54:00:81:f8:23 brd ff:ff:ff:ff:ff:ff I configured my flat and public interfaces with the following commands: sudo openstack-config --set /etc/nova/nova.conf DEFAULT flat_interface eth1 sudo openstack-config --set /etc/nova/nova.conf DEFAULT public_interface eth0 Why am I getting an error with the nova-manage network command? What am I doing wrong? Sam -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbryant at redhat.com Sat Mar 2 03:44:08 2013 From: rbryant at redhat.com (Russell Bryant) Date: Fri, 01 Mar 2013 22:44:08 -0500 Subject: [rhos-list] Nova-manage command failure In-Reply-To: References: Message-ID: <51317588.6030600@redhat.com> On 03/01/2013 03:21 PM, SAM GABRIEL wrote: > I?m following the instructions in > https://access.redhat.com/knowledge/docs/en-US/Red_Hat_OpenStack/2/html/Getting_Started_Guide/chap-Deploying_Compute_Services.html > (Chapter 11 of ?Red Hat OpenStack Getting Started Guide, Getting Started > with Red Hat OpenStack 2.0 (Folsom) Preview, Edition 1.0?). I am using > Nova networking. My installation has proceeded without error up until > the following command: > > sudo nova-manage network create demonet 10.0.0.0/24 1 256 > --bridge=demonetbr0 > > Which resulted in the following error message: > > Command failed, please check log for more info > 2013-03-01 11:55:55 CRITICAL nova > [req-68875b1d-cd10-4e24-a55d-434ea7e6845f None None] 'vlan_start' What version of openstack-nova do you have installed? Assuming you haven't changed this option yourself, can you try adding this line to the [DEFAULT] section of /etc/nova/nova.conf? network_manager = nova.network.manager.FlatDHCPManager My guess is that you hit this because of a recent change we made to configuration file handling and the version you have doesn't have the fix. -- Russell Bryant From gilles at redhat.com Mon Mar 4 05:58:53 2013 From: gilles at redhat.com (Gilles Dubreuil) Date: Mon, 04 Mar 2013 16:58:53 +1100 Subject: [rhos-list] RHOS Preview documentations not available Message-ID: <1362376733.4455.67.camel@gil.surfgate.org> Hi All, FYI All the links in the Red Hat OpenStack Preview documentation page are not available. https://access.redhat.com/knowledge/docs/Red_Hat_OpenStack_Preview/ I created support case #00799572 Regards Gilles From sgordon at redhat.com Mon Mar 4 12:43:26 2013 From: sgordon at redhat.com (Steve Gordon) Date: Mon, 4 Mar 2013 07:43:26 -0500 (EST) Subject: [rhos-list] RHOS Preview documentations not available In-Reply-To: <1362376733.4455.67.camel@gil.surfgate.org> Message-ID: <1406905148.12108717.1362401006929.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Gilles Dubreuil" > To: rhos-list at redhat.com > Sent: Monday, March 4, 2013 12:58:53 AM > Subject: [rhos-list] RHOS Preview documentations not available > > Hi All, > > FYI > > All the links in the Red Hat OpenStack Preview documentation page are > not available. > > https://access.redhat.com/knowledge/docs/Red_Hat_OpenStack_Preview/ > > > I created support case #00799572 > > Regards > Gilles Hi Gilles, The documentation is now available here: https://access.redhat.com/knowledge/docs/Red_Hat_OpenStack/ Thanks, Steve From rich.minton at lmco.com Tue Mar 5 20:12:55 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Tue, 5 Mar 2013 20:12:55 +0000 Subject: [rhos-list] Problems with metadata. Message-ID: I'm having some problems with my instances accessing metadata... they don't. The instances are not able to contact dhcp to get their IP and as a result I cannot pull in the metadata for ssh keys, hostname, etc. I can ping the metadata IP, 169.254.169.254 from my host but when I try to test the connection using curl I get a 404 error. I'm also not sure if iptables is being setup properly. I believe I have nova.conf configured properly. I'm using nova-metadata-api on each of my compute nodes and have nova.conf configured with the hosts IP for the metadata server and I do not have "metadata" in the list of enabled apis. I also tried running nova-metadata-api from the command line in debug mode to see if I was getting any errors. There were none, which leads me to believe it's something in iptables and the VMs are never accessing the service. Also if I flush iptables and restart the openstack services, iptables is repopulated but it is very sparse. Seems like there should be a lot more entries in the tables. I just need some hints on where to look to find out what's going on. Thanks in advance. Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From prmarino1 at gmail.com Tue Mar 5 22:12:29 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Tue, 5 Mar 2013 17:12:29 -0500 Subject: [rhos-list] issues with cinder and the nfs driver Message-ID: I seem to be having a weird issue with the cinder NFS driver. for some reason its not calling sudo with rootwrapper to mount the nfs filesystem as a matter of fact it doesn't seem to be calling sudo at all here is a sample of my config " [DEFAULT] verbose = True debug = True #glance_port = 9292 #glance_api_servers = [\'glancevip01.example.domain:9292\'] sql_max_retries = 10 rpc_backend = cinder.openstack.common.rpc.impl_qpid rootwrap_config = /etc/cinder/rootwrap.conf publish_errors = True logdir = /var/log/cinder auth_strategy = keystone sql_connection = mysql://cinder:cinder at mysqlvip01.example.domain/cinder api_paste_config = /etc/cinder/api-paste.ini state_path = /var/lib/cinder lock_path = /var/lib/cinder/tmp #glance_host = glancevip01.example.domain storage_availability_zone = nova volume_driver = cinder.volume.nfs.NfsDriver nfs_shares_config = /etc/cinder/nfs.conf nfs_mount_point_base = /etc/cinder/volumes volumes_dir = /etc/cinder/volumes nfs_disk_util = df nfs_sparsed_volumes = True [keystone_authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory auth_host = keystonevip01.example.domain auth_port = 35357 auth_protocol = http admin_user = cinder admin_password = cinderpasswd admin_tenant_name = service memcache_servers = testopenstack03.example.domain:11211,testopenstack04.example.domain:11211 " here is the error im getting " DEBUG cinder.utils [] Running cmd (subprocess): mount.nfs execute /usr/lib/python2.6/site-packages/cinder/utils.py:163 DEBUG cinder.utils [] Result was 1 execute /usr/lib/python2.6/site-packages/cinder/utils.py:180 " Now the reason I know neither sudo or cinder-rootwrapper are being called first i noticed that the sudo attempts weren't showing up in /var/log/secure, then I backed up cinder-rootwrapper and put a symlink to the logger command in its place to see what would happen and nothing showed up in /var/log/messages. whats worse is I even tried setting root_helper to logger then /usr/bin/logger and still nothing showed up in syslog. Does any one have any ideas as to what might be wrong, From chrisw at redhat.com Wed Mar 6 01:38:48 2013 From: chrisw at redhat.com (Chris Wright) Date: Tue, 5 Mar 2013 17:38:48 -0800 Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: References: Message-ID: <20130306013848.GV30456@x200.localdomain> * Paul Robert Marino (prmarino1 at gmail.com) wrote: > here is the error im getting > > " > DEBUG cinder.utils [] Running cmd (subprocess): mount.nfs execute > /usr/lib/python2.6/site-packages/cinder/utils.py:163 > DEBUG cinder.utils [] Result was 1 execute > /usr/lib/python2.6/site-packages/cinder/utils.py:180 > " That's just testing that mount.nfs exists (and won't run as root, or trigger rootwrap). Your config file looks correct to me. Have you tried "cinder create 1 test" or something to actually require rootwrap. thanks, -chris From prmarino1 at gmail.com Wed Mar 6 01:45:06 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Tue, 05 Mar 2013 20:45:06 -0500 Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: <20130306013848.GV30456@x200.localdomain> Message-ID: <51369fa3.0647310a.2d99.ffffe4df@mx.google.com> An HTML attachment was scrubbed... URL: From chrisw at redhat.com Wed Mar 6 02:19:39 2013 From: chrisw at redhat.com (Chris Wright) Date: Tue, 5 Mar 2013 21:19:39 -0500 (EST) Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: <51369fa3.0647310a.2d99.ffffe4df@mx.google.com> Message-ID: <672430350.10619156.1362536379622.JavaMail.root@redhat.com> ----- Original Message ----- > Yes > It returns error code 400 but there isn't any thing in the logs to > indicate why hmm, you sure volume request is going to cinder, is osapi_volume removed from nova.conf? might be helpful to see the full logs (at least /var/log/cinder/volume.log and perhaps api and shcedule) to see the request go through the system thanks, -chris (btw, harder to reply inline to html email) From gfidente at fedoraproject.org Wed Mar 6 03:46:07 2013 From: gfidente at fedoraproject.org (Giulio Fidente) Date: Wed, 06 Mar 2013 04:46:07 +0100 Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: <672430350.10619156.1362536379622.JavaMail.root@redhat.com> References: <672430350.10619156.1362536379622.JavaMail.root@redhat.com> Message-ID: <5136BBFF.5070103@fedoraproject.org> On 03/06/2013 03:19 AM, Chris Wright wrote: > ----- Original Message ----- >> Yes >> It returns error code 400 but there isn't any thing in the logs to >> indicate why > > hmm, you sure volume request is going to cinder, is osapi_volume removed > from nova.conf? > > might be helpful to see the full logs (at least /var/log/cinder/volume.log > and perhaps api and shcedule) to see the request go through the system I'm not seeing any issue on some test setup either but, as also Chris suggests, mount.nfs is not even rootwrapped so I suggest those DEBUG lines are not the root cause. It seems to be executing 'mount.nfs' with no arguments just to make sure it is available and, by default, mount.nfs exits with 1 when no arguments are given... that shouldn't be blocking execution, those are just DEBUG lines. -- Giulio Fidente IRC: giulivo From chrisw at redhat.com Wed Mar 6 06:44:09 2013 From: chrisw at redhat.com (Chris Wright) Date: Wed, 6 Mar 2013 01:44:09 -0500 (EST) Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: <51369fa3.0647310a.2d99.ffffe4df@mx.google.com> Message-ID: <59329134.10696478.1362552249943.JavaMail.root@redhat.com> ---- Original Message ----- > Yes > It returns error code 400 but there isn't any thing in the logs to > indicate why hmm, are you sure it's going to cinder service or do you have osapi_volume in your nova.conf? i think it'd be helpful to see the full /var/log/cinder/volume.log (perhaps others too, just to see the request going through the whole system). thanks, -chris (btw, harder to quote and reply to html email) From llange at redhat.com Wed Mar 6 08:48:27 2013 From: llange at redhat.com (Lutz Lange) Date: Wed, 06 Mar 2013 09:48:27 +0100 Subject: [rhos-list] RHOS running on VmWare - supported Use Case? Message-ID: <513702DB.9090305@redhat.com> Hi there, i have a customer that is interested in our OpenStack. They want to run on VmWare. I know that we OpenStack can do this, but are there any plans on supporting this with RHOS? Cheers Lutz -- Lutz Lange, RHCA lutz at redhat.com Solution Architect Red Hat GmbH Wankelstrasse 5 Cell : +49 172 75 285 17 D-70563 Stuttgart ____________________________________________________________________ Reg. Adresse: Red Hat GmbH, Werner-von-Siemens-Ring 11-15 D-85630 Grasbrunn Handelsregister: Amtsgericht Muenchen HRB 153243 Geschaeftsfuehrer: Mark Hegarty, Charlie Peters, Michael Cunningham, Charles Cachera -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 263 bytes Desc: OpenPGP digital signature URL: From thomas.oulevey at cern.ch Wed Mar 6 12:58:17 2013 From: thomas.oulevey at cern.ch (Thomas Oulevey) Date: Wed, 6 Mar 2013 13:58:17 +0100 Subject: [rhos-list] Folsom test with Cinder and Netapp iScsi driver. In-Reply-To: <59329134.10696478.1362552249943.JavaMail.root@redhat.com> References: <59329134.10696478.1362552249943.JavaMail.root@redhat.com> Message-ID: <51373D69.8060809@cern.ch> Hi, I was wondering if someone has some experience with Cinder and the iScsi Netapp driver. We managed to use it through the OnCommand API and to create/attach/detach/reattach volume. 2 questions: - The snapshots seem broken. It is not clear if it is available or not. Any idea ? Success ? The API seems to handle the request but we could not make it to work. https://wiki.openstack.org/wiki/CinderSupportMatrix - When you attach a volume it seems the device name choice is not respected: volume end up all the time with /dev/vdb. (If vdb exist it will go to the next available vdc in this case) but the cli and horizon show the user choice (/dev/vdd for example). Even the dumpxml show the user chosen device. I didn't find a bugzilla but there is a discussion here : https://bugs.launchpad.net/nova/+bug/1004328 The solution seems more a workaround e.g using a SN based naming. Let me know if I should open a bugzilla issue where I can attached the details, logs, etc.... cheers, Thomas. From prmarino1 at gmail.com Wed Mar 6 14:37:47 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Wed, 6 Mar 2013 09:37:47 -0500 Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: <59329134.10696478.1362552249943.JavaMail.root@redhat.com> References: <51369fa3.0647310a.2d99.ffffe4df@mx.google.com> <59329134.10696478.1362552249943.JavaMail.root@redhat.com> Message-ID: I found the issue there were a couple of minor mistakes in keystone. 1) I had keystone was pointing to a VIP for the endpoint that hadn't been started yet. 2) I used a script I wrote to automatically create the keystone entries and the part for cinder had a mistake where it was using the username as the password so I brought the VIP up (huped keepalived), corrected the keystone script, deleted all of the entries for cinder, and reran the script it now works perfectly. On Wed, Mar 6, 2013 at 1:44 AM, Chris Wright wrote: > ---- Original Message ----- >> Yes >> It returns error code 400 but there isn't any thing in the logs to >> indicate why > > hmm, are you sure it's going to cinder service or do you have > osapi_volume in your nova.conf? > > i think it'd be helpful to see the full /var/log/cinder/volume.log > (perhaps others too, just to see the request going through the whole > system). > > thanks, > -chris > > (btw, harder to quote and reply to html email) From prmarino1 at gmail.com Wed Mar 6 14:41:16 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Wed, 6 Mar 2013 09:41:16 -0500 Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: References: <51369fa3.0647310a.2d99.ffffe4df@mx.google.com> <59329134.10696478.1362552249943.JavaMail.root@redhat.com> Message-ID: oh and by the way thanks for the help :-) The replies are what made me realize the entries in keystone might be the problem. On Wed, Mar 6, 2013 at 9:37 AM, Paul Robert Marino wrote: > I found the issue > there were a couple of minor mistakes in keystone. > 1) I had keystone was pointing to a VIP for the endpoint that hadn't > been started yet. > 2) I used a script I wrote to automatically create the keystone > entries and the part for cinder had a mistake where it was using the > username as the password > so I brought the VIP up (huped keepalived), corrected the keystone > script, deleted all of the entries for cinder, and reran the script it > now works perfectly. > > > On Wed, Mar 6, 2013 at 1:44 AM, Chris Wright wrote: >> ---- Original Message ----- >>> Yes >>> It returns error code 400 but there isn't any thing in the logs to >>> indicate why >> >> hmm, are you sure it's going to cinder service or do you have >> osapi_volume in your nova.conf? >> >> i think it'd be helpful to see the full /var/log/cinder/volume.log >> (perhaps others too, just to see the request going through the whole >> system). >> >> thanks, >> -chris >> >> (btw, harder to quote and reply to html email) From daniel at dumdan.com Wed Mar 6 16:58:35 2013 From: daniel at dumdan.com (Daniel Dumitriu) Date: Wed, 06 Mar 2013 11:58:35 -0500 Subject: [rhos-list] RHOS running on VmWare - supported Use Case? In-Reply-To: <513702DB.9090305@redhat.com> References: <513702DB.9090305@redhat.com> Message-ID: <1362589115.2068.37.camel@camelot.ddse.ca> Sorry, I am, obviously, not one of the *intended* recipients of such a question. Having seen it, though, I must confess I was puzzled, at first... My familiarity with VMware's contribution to OpenStack is minimal. Especially when it comes down to the Core OpenStack Components. We do know, however, that ESX is a supported hypervisor. This does mean we can have have Compute Nodes running ESX while all the control infrastructure and other Core Components would be RHOS. We can, of course, go as far as to admit all the non-physical nodes to be running as VMware VMs. Can anyone tell me what the business case could be, for such a setup ? Besides the very valid one "that's what the client wants" ! == Never mind, I just figured it out... What if you had a bunch of VMware infrastructure and wanted to try things out before jumping in ? Besides, this could smooth-out the transition to a mixed environment. Chances are the client will be more at-ease with a smooth transition. -- ___________ Daniel Dumitriu daniel at dumdan.com Telephone: +1-416-626-9345 Mobile: +1-416-318-2487 -----Original Message----- From: Lutz Lange To: rhos-list at redhat.com Subject: [rhos-list] RHOS running on VmWare - supported Use Case? Date: Wed, 06 Mar 2013 09:48:27 +0100 Hi there, i have a customer that is interested in our OpenStack. They want to run on VmWare. I know that we OpenStack can do this, but are there any plans on supporting this with RHOS? Cheers Lutz _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From SAM.W.GABRIEL at saic.com Wed Mar 6 18:43:38 2013 From: SAM.W.GABRIEL at saic.com (SAM GABRIEL) Date: Wed, 06 Mar 2013 10:43:38 -0800 Subject: [rhos-list] Nova-manage command failure In-Reply-To: <51317588.6030600@redhat.com> Message-ID: Russell, Thank you for the quick reply - I've been out on vacation for a couple of days. I have Nova version 2.10.0, and the change to /etc/nova/nova.conf did fix the problem! Sam On 3/1/13 7:44 PM, "Russell Bryant" wrote: > On 03/01/2013 03:21 PM, SAM GABRIEL wrote: >> I?m following the instructions in >> https://access.redhat.com/knowledge/docs/en-US/Red_Hat_OpenStack/2/html/Getti >> ng_Started_Guide/chap-Deploying_Compute_Services.html >> (Chapter 11 of ?Red Hat OpenStack Getting Started Guide, Getting Started >> with Red Hat OpenStack 2.0 (Folsom) Preview, Edition 1.0?). I am using >> Nova networking. My installation has proceeded without error up until >> the following command: >> >> sudo nova-manage network create demonet 10.0.0.0/24 1 256 >> --bridge=demonetbr0 >> >> Which resulted in the following error message: >> >> Command failed, please check log for more info >> 2013-03-01 11:55:55 CRITICAL nova >> [req-68875b1d-cd10-4e24-a55d-434ea7e6845f None None] 'vlan_start' > > What version of openstack-nova do you have installed? > > Assuming you haven't changed this option yourself, can you try adding > this line to the [DEFAULT] section of /etc/nova/nova.conf? > > network_manager = nova.network.manager.FlatDHCPManager > > My guess is that you hit this because of a recent change we made to > configuration file handling and the version you have doesn't have the fix. From kbasil at redhat.com Wed Mar 6 20:24:10 2013 From: kbasil at redhat.com (Keith Basil) Date: Wed, 6 Mar 2013 15:24:10 -0500 Subject: [rhos-list] RHOS running on VmWare - supported Use Case? In-Reply-To: References: Message-ID: > Sorry, I am, obviously, not one of the *intended* recipients of such a > question. > > Having seen it, though, I must confess I was puzzled, at first... > > My familiarity with VMware's contribution to OpenStack is minimal. > Especially when it comes down to the Core OpenStack Components. We do > know, however, that ESX is a supported hypervisor. > > This does mean we can have have Compute Nodes running ESX while all the > control infrastructure and other Core Components would be RHOS. We can, > of course, go as far as to admit all the non-physical nodes to be > running as VMware VMs. > > Can anyone tell me what the business case could be, for such a setup ? > Besides the very valid one "that's what the client wants" ! Not sure if there is a business case to be made here. If this is requested, I would like to understand the rationale behind it? Does the customer think that because the hypervisor is VMware based that they will get HA and vMotion-like benefits with OpenStack? Are they erroneously mixing and matching cloud types (enterprise virtualization vs. an elastic cloud)? > Never mind, I just figured it out... What if you had a bunch of VMware > infrastructure and wanted to try things out before jumping in ? Besides, > this could smooth-out the transition to a mixed environment. Chances are > the client will be more at-ease with a smooth transition. As a proof of concept, maybe. As production infrastructure, probably not for many reasons. My guess is that it's a philosophical misunderstanding of the nature of the two cloud types: one designed for virtualization, static/legacy workloads, etc (VMware) and one designed for new elastic applications where the apps *expect* the infrastructure to fail (OpenStack). -k -- keith basil | principal product manager, OpenStack kbasil at redhat.com | redhat.com/openstack 757-769-8674 (m) | 978-813-1323 (o) skype/twitter/github/irc, life: noslzzp From chrisw at redhat.com Wed Mar 6 21:55:43 2013 From: chrisw at redhat.com (Chris Wright) Date: Wed, 6 Mar 2013 13:55:43 -0800 Subject: [rhos-list] issues with cinder and the nfs driver In-Reply-To: References: <51369fa3.0647310a.2d99.ffffe4df@mx.google.com> <59329134.10696478.1362552249943.JavaMail.root@redhat.com> Message-ID: <20130306215543.GP30456@x200.localdomain> * Paul Robert Marino (prmarino1 at gmail.com) wrote: > oh and by the way thanks for the help :-) > The replies are what made me realize the entries in keystone might be > the problem. Glad you got it working, and thanks for reporting back on the details of your fix. thanks, -chris > On Wed, Mar 6, 2013 at 9:37 AM, Paul Robert Marino wrote: > > I found the issue > > there were a couple of minor mistakes in keystone. > > 1) I had keystone was pointing to a VIP for the endpoint that hadn't > > been started yet. > > 2) I used a script I wrote to automatically create the keystone > > entries and the part for cinder had a mistake where it was using the > > username as the password > > so I brought the VIP up (huped keepalived), corrected the keystone > > script, deleted all of the entries for cinder, and reran the script it > > now works perfectly. > > > > > > On Wed, Mar 6, 2013 at 1:44 AM, Chris Wright wrote: > >> ---- Original Message ----- > >>> Yes > >>> It returns error code 400 but there isn't any thing in the logs to > >>> indicate why > >> > >> hmm, are you sure it's going to cinder service or do you have > >> osapi_volume in your nova.conf? > >> > >> i think it'd be helpful to see the full /var/log/cinder/volume.log > >> (perhaps others too, just to see the request going through the whole > >> system). > >> > >> thanks, > >> -chris > >> > >> (btw, harder to quote and reply to html email) From chrisw at redhat.com Wed Mar 6 22:04:52 2013 From: chrisw at redhat.com (Chris Wright) Date: Wed, 6 Mar 2013 14:04:52 -0800 Subject: [rhos-list] Folsom test with Cinder and Netapp iScsi driver. In-Reply-To: <51373D69.8060809@cern.ch> References: <59329134.10696478.1362552249943.JavaMail.root@redhat.com> <51373D69.8060809@cern.ch> Message-ID: <20130306220452.GQ30456@x200.localdomain> * Thomas Oulevey (thomas.oulevey at cern.ch) wrote: > I was wondering if someone has some experience with Cinder and the iScsi > Netapp driver. I haven't tried it. > We managed to use it through the OnCommand API and to > create/attach/detach/reattach volume. > > 2 questions: > - The snapshots seem broken. It is not clear if it is available or not. Any > idea ? Success ? > The API seems to handle the request but we could not make it to work. > https://wiki.openstack.org/wiki/CinderSupportMatrix > - When you attach a volume it seems the device name choice is not respected: > volume end up all the time with /dev/vdb. (If vdb exist it will go to the > next available vdc in this case) but > the cli and horizon show the user choice (/dev/vdd for example). > Even the dumpxml show the user chosen device. Good questions above. I don't have answers, perhaps Eric (Cc'd) does. > I didn't find a bugzilla but there is a discussion here : > https://bugs.launchpad.net/nova/+bug/1004328 > The solution seems more a workaround e.g using a SN based naming. > > Let me know if I should open a bugzilla issue where I can attached the > details, logs, etc.... Yes, open a bug to help track. We can always close it if it turns out to be some simple configuration type error. thanks, -chris From eharney at redhat.com Wed Mar 6 23:02:42 2013 From: eharney at redhat.com (Eric Harney) Date: Wed, 06 Mar 2013 18:02:42 -0500 Subject: [rhos-list] Folsom test with Cinder and Netapp iScsi driver. In-Reply-To: <20130306220452.GQ30456@x200.localdomain> References: <59329134.10696478.1362552249943.JavaMail.root@redhat.com> <51373D69.8060809@cern.ch> <20130306220452.GQ30456@x200.localdomain> Message-ID: <1362610962.14125.32.camel@eharney.localdomain> On Wed, 2013-03-06 at 14:04 -0800, Chris Wright wrote: > * Thomas Oulevey (thomas.oulevey at cern.ch) wrote: > > I was wondering if someone has some experience with Cinder and the iScsi > > Netapp driver. > > I haven't tried it. I also haven't tried the Netapp driver. > > > We managed to use it through the OnCommand API and to > > create/attach/detach/reattach volume. > > > > 2 questions: > > - The snapshots seem broken. It is not clear if it is available or not. Any > > idea ? Success ? > > The API seems to handle the request but we could not make it to work. > > https://wiki.openstack.org/wiki/CinderSupportMatrix > > - When you attach a volume it seems the device name choice is not respected: > > volume end up all the time with /dev/vdb. (If vdb exist it will go to the > > next available vdc in this case) but > > the cli and horizon show the user choice (/dev/vdd for example). > > Even the dumpxml show the user chosen device. > > Good questions above. I don't have answers, perhaps Eric (Cc'd) does. > > > I didn't find a bugzilla but there is a discussion here : > > https://bugs.launchpad.net/nova/+bug/1004328 > > The solution seems more a workaround e.g using a SN based naming. > > > > Let me know if I should open a bugzilla issue where I can attached the > > details, logs, etc.... > > Yes, open a bug to help track. We can always close it if it turns out to > be some simple configuration type error. > I went ahead and opened a doc bug for the device node issue to start, because my impression is also that it doesn't behave as expected. I don't know all of the specifics about how this works since it's more in the Nova/virt area, but I do think there may be some unexpected limitation here that we need to look at. https://bugzilla.redhat.com/show_bug.cgi?id=918809 Thanks for pointing this out. > thanks, > -chris From prmarino1 at gmail.com Wed Mar 6 23:03:14 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Wed, 06 Mar 2013 18:03:14 -0500 Subject: [rhos-list] RHOS running on VmWare - supported Use Case? In-Reply-To: Message-ID: <5137cb32.02adec0a.524e.36b6@mx.google.com> An HTML attachment was scrubbed... URL: From rich.minton at lmco.com Thu Mar 7 17:21:03 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 7 Mar 2013 17:21:03 +0000 Subject: [rhos-list] Problems with metadata. Message-ID: I know you guys are busy trying to get the new release out the door but if someone could take a quick peek at this and provide a suggestion on where to look, I would be very grateful. I could really use a suggestion on compute node configuration for metadata and vnc proxy. I'm having some problems with my instances accessing metadata from my compute nodes... they don't. The instances running on compute nodes are not able to contact dhcp to get their IP and as a result I cannot pull in the metadata for ssh keys, hostname, etc. I can ping the metadata IP, 169.254.169.254 from my host but when I try to test the connection using curl I get a 404 error. I'm also not sure if iptables is being setup properly. I believe I have nova.conf configured properly. I'm using nova-metadata-api on each of my compute nodes and have nova.conf configured with the hosts IP for the metadata server and I do not have "metadata" in the list of enabled apis. I also tried running nova-metadata-api from the command line in debug mode to see if I was getting any errors. There were none, which leads me to believe it's something in iptables and the VMs are never accessing the service. Also if I flush iptables and restart the openstack services, iptables is repopulated but it is very sparse. Seems like there should be a lot more entries in the tables. Also, when I try to access the vnc console on instances running on compute nodes (not the controller) I get a connection failure. I have the vnc service running on each compute node and set the variables in nova.conf to point to the compute node instead of the controller. I actually had this working at one point and then something went wrong... again maybe iptables. I just need some hints on where to look to find out what's going on. Thanks in advance. Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jthomas at redhat.com Thu Mar 7 18:11:05 2013 From: jthomas at redhat.com (Jon Thomas) Date: Thu, 07 Mar 2013 13:11:05 -0500 Subject: [rhos-list] Problems with metadata. In-Reply-To: References: Message-ID: <1362679865.2911.14.camel@basin.redhat.com> On Thu, 2013-03-07 at 17:21 +0000, Minton, Rich wrote: > I know you guys are busy trying to get the new release out the door > but if someone could take a quick peek at this and provide a > suggestion on where to look, I would be very grateful. I could really > use a suggestion on compute node configuration for metadata and vnc > proxy. > > > > I?m having some problems with my instances accessing metadata from my > compute nodes? they don?t. The instances running on compute nodes are > not able to contact dhcp to get their IP and as a result I cannot pull > in the metadata for ssh keys, hostname, etc. I can ping the metadata > IP, 169.254.169.254 from my host but when I try to test the connection > using curl I get a 404 error. I?m also not sure if iptables is being What curl command? curl http://169.254.169.254 should give ...

The document has moved here.

... You can open up dhcp with -A INPUT -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A FORWARD -p udp -m udp --sport 68 --dport 67 -j ACCEPT > setup properly. I believe I have nova.conf configured properly. I?m > using nova-metadata-api on each of my compute nodes and have nova.conf > configured with the hosts IP for the metadata server and I do not have > ?metadata? in the list of enabled apis. I also tried running > nova-metadata-api from the command line in debug mode to see if I was > getting any errors. There were none, which leads me to believe it?s > something in iptables and the VMs are never accessing the service. > Also if I flush iptables and restart the openstack services, iptables > is repopulated but it is very sparse. Seems like there should be a lot > more entries in the tables. > Sounds like something with iptables. You can compare to thsi set from a packstack based install. 192.168.2.113 is public/admin interface. $ cat /etc/sys*/iptables # Generated by iptables-save v1.4.7 on Thu Mar 7 12:29:40 2013 *mangle :PREROUTING ACCEPT [63439:31483572] :INPUT ACCEPT [63439:31483572] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [62191:32543722] :POSTROUTING ACCEPT [62191:32543722] :nova-api-POSTROUTING - [0:0] :nova-network-POSTROUTING - [0:0] -A POSTROUTING -j nova-network-POSTROUTING -A POSTROUTING -j nova-api-POSTROUTING -A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill COMMIT # Completed on Thu Mar 7 12:29:40 2013 # Generated by iptables-save v1.4.7 on Thu Mar 7 12:29:40 2013 *filter :INPUT ACCEPT [970:292442] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [1291:434252] :nova-api-FORWARD - [0:0] :nova-api-INPUT - [0:0] :nova-api-OUTPUT - [0:0] :nova-api-local - [0:0] :nova-filter-top - [0:0] :nova-network-FORWARD - [0:0] :nova-network-INPUT - [0:0] :nova-network-OUTPUT - [0:0] :nova-network-local - [0:0] -A INPUT -j nova-network-INPUT -A INPUT -j nova-api-INPUT -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT -A INPUT -p tcp -m multiport --dports 5000,35357 -m comment --comment "001 keystone incoming" -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT -A INPUT -p tcp -m multiport --dports 3260,8776 -m comment --comment "001 cinder incoming" -j ACCEPT -A INPUT -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9292 -m comment --comment "001 glance incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 80 -m comment --comment "001 horizon incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 80 -m comment --comment "001 horizon incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3306 -m comment --comment "001 mysql incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9292 -m comment --comment "001 glance incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5000,35357 -m comment --comment "001 keystone incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5900:5999 -m comment --comment "001 nove compute incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 6080 -m comment --comment "001 novncproxy incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5672 -m comment --comment "001 qpid incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3306 -m comment --comment "001 mysql incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774 -m comment --comment "001 novaapi incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5672 -m comment --comment "001 qpid incomming" -j ACCEPT -A FORWARD -j nova-filter-top -A FORWARD -j nova-network-FORWARD -A FORWARD -j nova-api-FORWARD -A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT -A FORWARD -i virbr0 -o virbr0 -j ACCEPT -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable -A FORWARD -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A OUTPUT -j nova-filter-top -A OUTPUT -j nova-network-OUTPUT -A OUTPUT -j nova-api-OUTPUT -A nova-api-INPUT -d 192.168.2.113/32 -p tcp -m tcp --dport 8775 -j ACCEPT -A nova-filter-top -j nova-network-local -A nova-filter-top -j nova-api-local -A nova-network-FORWARD -i br100 -j ACCEPT -A nova-network-FORWARD -o br100 -j ACCEPT -A nova-network-INPUT -i br100 -p udp -m udp --dport 67 -j ACCEPT -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 67 -j ACCEPT -A nova-network-INPUT -i br100 -p udp -m udp --dport 53 -j ACCEPT -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 53 -j ACCEPT COMMIT # Completed on Thu Mar 7 12:29:40 2013 # Generated by iptables-save v1.4.7 on Thu Mar 7 12:29:40 2013 *nat :PREROUTING ACCEPT [13:752] :POSTROUTING ACCEPT [589:137435] :OUTPUT ACCEPT [589:137435] :nova-api-OUTPUT - [0:0] :nova-api-POSTROUTING - [0:0] :nova-api-PREROUTING - [0:0] :nova-api-float-snat - [0:0] :nova-api-snat - [0:0] :nova-network-OUTPUT - [0:0] :nova-network-POSTROUTING - [0:0] :nova-network-PREROUTING - [0:0] :nova-network-float-snat - [0:0] :nova-network-snat - [0:0] :nova-postrouting-bottom - [0:0] -A PREROUTING -j nova-network-PREROUTING -A PREROUTING -j nova-api-PREROUTING -A POSTROUTING -j nova-network-POSTROUTING -A POSTROUTING -j nova-api-POSTROUTING -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE -A POSTROUTING -j nova-postrouting-bottom -A OUTPUT -j nova-network-OUTPUT -A OUTPUT -j nova-api-OUTPUT -A nova-api-snat -j nova-api-float-snat -A nova-network-POSTROUTING -s 192.168.32.0/22 -d 192.168.2.113/32 -j ACCEPT -A nova-network-POSTROUTING -s 192.168.32.0/22 -d 192.168.32.0/22 -m conntrack ! --ctstate DNAT -j ACCEPT -A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.2.113:8775 -A nova-network-snat -j nova-network-float-snat -A nova-network-snat -s 192.168.32.0/22 -o eth0 -j SNAT --to-source 192.168.2.113 -A nova-postrouting-bottom -j nova-network-snat -A nova-postrouting-bottom -j nova-api-snat COMMIT # Completed on Thu Mar 7 12:29:40 2013 > > > Also, when I try to access the vnc console on instances running on > compute nodes (not the controller) I get a connection failure. I have > the vnc service running on each compute node and set the variables in > nova.conf to point to the compute node instead of the controller. I > actually had this working at one point and then something went wrong? > again maybe iptables. > > > > I just need some hints on where to look to find out what?s going on. > > > > Thanks in advance. > > Rick > > > > > > > > Richard Minton > > LMICC Systems Administrator > > 4000 Geerdes Blvd, 13D31 > > King of Prussia, PA 19406 > > Phone: 610-354-5482 > > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list From prmarino1 at gmail.com Thu Mar 7 19:00:36 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Thu, 07 Mar 2013 14:00:36 -0500 Subject: [rhos-list] Problems with metadata. In-Reply-To: <1362679865.2911.14.camel@basin.redhat.com> Message-ID: <5138e3d7.0373ec0a.68cc.1682@mx.google.com> An HTML attachment was scrubbed... URL: From rich.minton at lmco.com Thu Mar 7 19:08:24 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 7 Mar 2013 19:08:24 +0000 Subject: [rhos-list] EXTERNAL: Re: Problems with metadata. In-Reply-To: <5138e3d7.0373ec0a.68cc.1682@mx.google.com> References: <1362679865.2911.14.camel@basin.redhat.com> <5138e3d7.0373ec0a.68cc.1682@mx.google.com> Message-ID: Yes, good question. Using Nova-Network in VlanManager mode. Two NICs bonded together. One bridge and vlan for my hosts and another vlan and bridge for my single tenant. Eth0--\ /--bond0.972--->br972----10.10.12.xx host network >----bond0---< Eth1--/ \--bond0.976--->br976----10.10.16.xx instance private IPs \--10.10.12.xx instance floating IPs From: Paul Robert Marino [mailto:prmarino1 at gmail.com] Sent: Thursday, March 07, 2013 2:01 PM To: Jon Thomas; Minton, Rich Cc: rhos-list at redhat.com; Pothapragada, Kiran Subject: EXTERNAL: Re: [rhos-list] Problems with metadata. Well my first question is are you using quantum or nova network? -- Sent from my HP Pre3 ________________________________ On Mar 7, 2013 1:12 PM, Jon Thomas > wrote: On Thu, 2013-03-07 at 17:21 +0000, Minton, Rich wrote: > I know you guys are busy trying to get the new release out the door > but if someone could take a quick peek at this and provide a > suggestion on where to look, I would be very grateful. I could really > use a suggestion on compute node configuration for metadata and vnc > proxy. > > > > I?m having some problems with my instances accessing metadata from my > compute nodes? they don?t. The instances running on compute nodes are > not able to contact dhcp to get their IP and as a result I cannot pull > in the metadata for ssh keys, hostname, etc. I can ping the metadata > IP, 169.254.169.254 from my host but when I try to test the connection > using curl I get a 404 error. I?m also not sure if iptables is being What curl command? curl http://169.254.169.254 should give ...

The document has moved here.

... You can open up dhcp with -A INPUT -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A FORWARD -p udp -m udp --sport 68 --dport 67 -j ACCEPT > setup properly. I believe I have nova.conf configured properly. I?m > using nova-metadata-api on each of my compute nodes and have nova.conf > configured with the hosts IP for the metadata server and I do not have > ?metadata? in the list of enabled apis. I also tried running > nova-metadata-api from the command line in debug mode to see if I was > getting any errors. There were none, which leads me to believe it?s > something in iptables and the VMs are never accessing the service. > Also if I flush iptables and restart the openstack services, iptables > is repopulated but it is very sparse. Seems like there should be a lot > more entries in the tables. > Sounds like something with iptables. You can compare to thsi set from a packstack based install. 192.168.2.113 is public/admin interface. $ cat /etc/sys*/iptables # Generated by iptables-save v1.4.7 on Thu Mar 7 12:29:40 2013 *mangle :PREROUTING ACCEPT [63439:31483572] :INPUT ACCEPT [63439:31483572] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [62191:32543722] :POSTROUTING ACCEPT [62191:32543722] :nova-api-POSTROUTING - [0:0] :nova-network-POSTROUTING - [0:0] -A POSTROUTING -j nova-network-POSTROUTING -A POSTROUTING -j nova-api-POSTROUTING -A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill COMMIT # Completed on Thu Mar 7 12:29:40 2013 # Generated by iptables-save v1.4.7 on Thu Mar 7 12:29:40 2013 *filter :INPUT ACCEPT [970:292442] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [1291:434252] :nova-api-FORWARD - [0:0] :nova-api-INPUT - [0:0] :nova-api-OUTPUT - [0:0] :nova-api-local - [0:0] :nova-filter-top - [0:0] :nova-network-FORWARD - [0:0] :nova-network-INPUT - [0:0] :nova-network-OUTPUT - [0:0] :nova-network-local - [0:0] -A INPUT -j nova-network-INPUT -A INPUT -j nova-api-INPUT -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT -A INPUT -p tcp -m multiport --dports 5000,35357 -m comment --comment "001 keystone incoming" -j ACCEPT -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT -A INPUT -p tcp -m multiport --dports 3260,8776 -m comment --comment "001 cinder incoming" -j ACCEPT -A INPUT -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9292 -m comment --comment "001 glance incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 80 -m comment --comment "001 horizon incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774,8775 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 80 -m comment --comment "001 horizon incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3306 -m comment --comment "001 mysql incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9292 -m comment --comment "001 glance incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5000,35357 -m comment --comment "001 keystone incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5900:5999 -m comment --comment "001 nove compute incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 6080 -m comment --comment "001 novncproxy incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5672 -m comment --comment "001 qpid incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3306 -m comment --comment "001 mysql incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8773,8774 -m comment --comment "001 novaapi incomming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5672 -m comment --comment "001 qpid incomming" -j ACCEPT -A FORWARD -j nova-filter-top -A FORWARD -j nova-network-FORWARD -A FORWARD -j nova-api-FORWARD -A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT -A FORWARD -i virbr0 -o virbr0 -j ACCEPT -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable -A FORWARD -p udp -m udp --sport 68 --dport 67 -j ACCEPT -A OUTPUT -j nova-filter-top -A OUTPUT -j nova-network-OUTPUT -A OUTPUT -j nova-api-OUTPUT -A nova-api-INPUT -d 192.168.2.113/32 -p tcp -m tcp --dport 8775 -j ACCEPT -A nova-filter-top -j nova-network-local -A nova-filter-top -j nova-api-local -A nova-network-FORWARD -i br100 -j ACCEPT -A nova-network-FORWARD -o br100 -j ACCEPT -A nova-network-INPUT -i br100 -p udp -m udp --dport 67 -j ACCEPT -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 67 -j ACCEPT -A nova-network-INPUT -i br100 -p udp -m udp --dport 53 -j ACCEPT -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 53 -j ACCEPT COMMIT # Completed on Thu Mar 7 12:29:40 2013 # Generated by iptables-save v1.4.7 on Thu Mar 7 12:29:40 2013 *nat :PREROUTING ACCEPT [13:752] :POSTROUTING ACCEPT [589:137435] :OUTPUT ACCEPT [589:137435] :nova-api-OUTPUT - [0:0] :nova-api-POSTROUTING - [0:0] :nova-api-PREROUTING - [0:0] :nova-api-float-snat - [0:0] :nova-api-snat - [0:0] :nova-network-OUTPUT - [0:0] :nova-network-POSTROUTING - [0:0] :nova-network-PREROUTING - [0:0] :nova-network-float-snat - [0:0] :nova-network-snat - [0:0] :nova-postrouting-bottom - [0:0] -A PREROUTING -j nova-network-PREROUTING -A PREROUTING -j nova-api-PREROUTING -A POSTROUTING -j nova-network-POSTROUTING -A POSTROUTING -j nova-api-POSTROUTING -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE -A POSTROUTING -j nova-postrouting-bottom -A OUTPUT -j nova-network-OUTPUT -A OUTPUT -j nova-api-OUTPUT -A nova-api-snat -j nova-api-float-snat -A nova-network-POSTROUTING -s 192.168.32.0/22 -d 192.168.2.113/32 -j ACCEPT -A nova-network-POSTROUTING -s 192.168.32.0/22 -d 192.168.32.0/22 -m conntrack ! --ctstate DNAT -j ACCEPT -A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.2.113:8775 -A nova-network-snat -j nova-network-float-snat -A nova-network-snat -s 192.168.32.0/22 -o eth0 -j SNAT --to-source 192.168.2.113 -A nova-postrouting-bottom -j nova-network-snat -A nova-postrouting-bottom -j nova-api-snat COMMIT # Completed on Thu Mar 7 12:29:40 2013 > > > Also, when I try to access the vnc console on instances running on > compute nodes (not the controller) I get a connection failure. I have > the vnc service running on each compute node and set the variables in > nova.conf to point to the compute node instead of the controller. I > actually had this working at one point and then something went wrong? > again maybe iptables. > > > > I just need some hints on where to look to find out what?s going on. > > > > Thanks in advance. > > Rick > > > > > > > > Richard Minton > > LMICC Systems Administrator > > 4000 Geerdes Blvd, 13D31 > > King of Prussia, PA 19406 > > Phone: 610-354-5482 > > > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmyers at redhat.com Fri Mar 8 21:56:16 2013 From: pmyers at redhat.com (Perry Myers) Date: Fri, 08 Mar 2013 16:56:16 -0500 Subject: [rhos-list] two node configuration with packstack In-Reply-To: <8B91F67B8509F34A8588E7EFD4005BECEDC5C3@ANKPHEXC.destekas.local> References: <8B91F67B8509F34A8588E7EFD4005BECEDC20B@ANKPHEXC.destekas.local>, <512E3A07.4080104@redhat.com> <8B91F67B8509F34A8588E7EFD4005BECEDC5C3@ANKPHEXC.destekas.local> Message-ID: <513A5E80.8080802@redhat.com> On 03/05/2013 05:41 PM, Aydin PAYKOC wrote: > Hi, > > Perry, many thanks for your answer. I have managed to install > openstack with packstack. > After the installation when I do "source keystonerc_admin" from the > root user and > keystone token-get for testing it says "Invalid user / password (HTTP 401)" > Also when I do openstack-status it says all the services are active > but also says; > == Keystone users == > Invalid user / password (HTTP 401) > == Glance images == > Invalid user / password (HTTP 401) > == Nova instances == > ERROR: Invalid OpenStack Nova credentials > > I have done a fresh installation and does not changed any passwd. > > I have than looked to the log files, in all services it there is a > line similar to; > 1045, "Access denied for user 'nova'@'192.168.100.252' (using > password: YES)") None None > > So I checked mysql login; > when I try "mysql -unova -psecret" it logs in without any problem but when I do > "mysql -unova -psecret at 192.168.100.252" it gives the message, > 1045, "Access denied for user 'nova'@'localhost' (using password YES)" > > Can anyone help me out ? Not sure what the problem could be offhand, but one of the other engineers on the list should be able to assist you. Cheers, Perry From obasan at redhat.com Sun Mar 10 16:16:57 2013 From: obasan at redhat.com (Ohad Basan) Date: Sun, 10 Mar 2013 12:16:57 -0400 (EDT) Subject: [rhos-list] Openstack live usb In-Reply-To: <1657472711.17689726.1362932119514.JavaMail.root@redhat.com> Message-ID: <135703391.17689797.1362932217131.JavaMail.root@redhat.com> Hey everyone I have prepared a rhel 6.4 based openstack live usb. (packstack installation) it's an initial release so comments are welcome. you can get it here by tomorrow I hope to commit the scripts+kickstart files. http://ci-web.eng.lab.tlv.redhat.com/repos/livecd/ thank you, Ohad From nux at li.nux.ro Sun Mar 10 19:30:21 2013 From: nux at li.nux.ro (Nux!) Date: Sun, 10 Mar 2013 19:30:21 +0000 Subject: [rhos-list] Openstack live usb In-Reply-To: <135703391.17689797.1362932217131.JavaMail.root@redhat.com> References: <135703391.17689797.1362932217131.JavaMail.root@redhat.com> Message-ID: On 10.03.2013 16:16, Ohad Basan wrote: > Hey everyone > > I have prepared a rhel 6.4 based openstack live usb. (packstack > installation) > it's an initial release so comments are welcome. > you can get it here > by tomorrow I hope to commit the scripts+kickstart files. > > http://ci-web.eng.lab.tlv.redhat.com/repos/livecd/ > Awesome, I've been meaning to build something similar. Sadly that url does not resolve. Looking forward to kickstarts. Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro From Srinivas_G_Gowda at Dell.com Mon Mar 11 04:50:54 2013 From: Srinivas_G_Gowda at Dell.com (Srinivas_G_Gowda at Dell.com) Date: Sun, 10 Mar 2013 21:50:54 -0700 Subject: [rhos-list] Openstack live usb In-Reply-To: References: <135703391.17689797.1362932217131.JavaMail.root@redhat.com>, Message-ID: <75F7F7632819D94BA80703D8B1F10B6D25301AD7F2@BLRX7MCDC201.AMER.DELL.COM> Link doesn't work for me either... Thanks, G ________________________________________ From: rhos-list-bounces at redhat.com [rhos-list-bounces at redhat.com] On Behalf Of Nux! [nux at li.nux.ro] Sent: Monday, March 11, 2013 1:00 AM To: rhos-list at redhat.com Subject: Re: [rhos-list] Openstack live usb On 10.03.2013 16:16, Ohad Basan wrote: > Hey everyone > > I have prepared a rhel 6.4 based openstack live usb. (packstack > installation) > it's an initial release so comments are welcome. > you can get it here > by tomorrow I hope to commit the scripts+kickstart files. > > http://ci-web.eng.lab.tlv.redhat.com/repos/livecd/ > Awesome, I've been meaning to build something similar. Sadly that url does not resolve. Looking forward to kickstarts. Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From derekh at redhat.com Mon Mar 11 10:10:52 2013 From: derekh at redhat.com (Derek Higgins) Date: Mon, 11 Mar 2013 10:10:52 +0000 Subject: [rhos-list] two node configuration with packstack In-Reply-To: <513A5E80.8080802@redhat.com> References: <8B91F67B8509F34A8588E7EFD4005BECEDC20B@ANKPHEXC.destekas.local>, <512E3A07.4080104@redhat.com> <8B91F67B8509F34A8588E7EFD4005BECEDC5C3@ANKPHEXC.destekas.local> <513A5E80.8080802@redhat.com> Message-ID: <513DADAC.8010302@redhat.com> On 03/08/2013 09:56 PM, Perry Myers wrote: > On 03/05/2013 05:41 PM, Aydin PAYKOC wrote: >> Hi, >> >> Perry, many thanks for your answer. I have managed to install >> openstack with packstack. >> After the installation when I do "source keystonerc_admin" from the >> root user and >> keystone token-get for testing it says "Invalid user / password (HTTP 401)" >> Also when I do openstack-status it says all the services are active >> but also says; >> == Keystone users == >> Invalid user / password (HTTP 401) >> == Glance images == >> Invalid user / password (HTTP 401) >> == Nova instances == >> ERROR: Invalid OpenStack Nova credentials >> >> I have done a fresh installation and does not changed any passwd. >> >> I have than looked to the log files, in all services it there is a >> line similar to; >> 1045, "Access denied for user 'nova'@'192.168.100.252' (using >> password: YES)") None None >> >> So I checked mysql login; >> when I try "mysql -unova -psecret" it logs in without any problem but when I do >> "mysql -unova -psecret at 192.168.100.252" it gives the message, this command doesn't look correct to me, try $ mysql -u nova -psecret -h 192.168.100.252 nova >> 1045, "Access denied for user 'nova'@'localhost' (using password YES)" >> >> Can anyone help me out ? when running packstack in interactive mode all of the passwords for the various services are randomly generated. If you run it in interactive mode a second time then new ones get generated. I think two things are going on here 1. the various mysql passwords have changed but the services were running before packstack was run so didn't pick up the changes in the config files, restarting the services using the wrong password should solve this 2. the admin password in keystonerc_admin has now changed but because the keystone user had previously been created with a different password the new password in keystonerc_admin is now incorrect, if you have the original admin password you can use this. If you don't have it you can set a new one using the admin token (admin_token in /etc/keystone/keystone.conf) # get the the uuid for the admin user $ keystone --os-token --os-endpoint http://127.0.0.1:35357/v2.0/ user-list # change the password for this user $ keystone --os-token --os-endpoint http://127.0.0.1:35357/v2.0/ user-password-update --pass newpassword another option is to start again using an answerfile (after removing packages, dropping data bases etc...), the benefit of doing this is running packstack multiple times with the answerfile doesn't have any undesired side effects. In general if running packstack multiple times and answer file should be used so that the randomly generated passwords from the first run don't change, the newest versions of packstack generates a answerfile when ran in interactive mode to facilitate this. hope this helps, thanks, Derek. > > Not sure what the problem could be offhand, but one of the other > engineers on the list should be able to assist you. > > Cheers, > > Perry > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From rich.minton at lmco.com Mon Mar 11 13:27:58 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 11 Mar 2013 13:27:58 +0000 Subject: [rhos-list] IPTables question. Message-ID: I'm looking at the iptables for both my controller node and compute node and I see something that doesn't seem right. In the "nova-network-POSTROUTING" NAT chain I have the same hostname as the destination on both nodes. I have Nova Network running on both nodes so I would think that the destination hostname would equal the hostname that nova-network is running on, i.e., the destination hostname on the compute node should be uvslp-diu05os and not be the same as on the Controller node. Can anyone tell me if this is the case? (Controller Node, hostname uvslp-diu04os - "iptables -t nat -L) Chain nova-network-POSTROUTING (1 references) target prot opt source destination ACCEPT all -- 10.10.18.0/24 uvslp-diu04os.lmdit.us.lmco.com ACCEPT all -- 10.10.18.0/24 10.10.18.0/24 ! ctstate DNAT (Compute Node, hostname uvslp-diu05os - "iptables -t nat -L) Chain nova-network-POSTROUTING (1 references) target prot opt source destination ACCEPT all -- 10.10.18.0/24 uvslp-diu04os.lmdit.us.lmco.com ACCEPT all -- 10.10.18.0/24 10.10.18.0/24 ! ctstate DNAT Thank you, Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbryant at redhat.com Mon Mar 11 14:17:31 2013 From: rbryant at redhat.com (Russell Bryant) Date: Mon, 11 Mar 2013 10:17:31 -0400 Subject: [rhos-list] IPTables question. In-Reply-To: References: Message-ID: <513DE77B.1090506@redhat.com> On 03/11/2013 09:27 AM, Minton, Rich wrote: > I?m looking at the iptables for both my controller node and compute node > and I see something that doesn?t seem right. In the > ?nova-network-POSTROUTING? NAT chain I have the same hostname as the > destination on both nodes. I have Nova Network running on both nodes so > I would think that the destination hostname would equal the hostname > that nova-network is running on, i.e., the destination hostname on the > compute node should be uvslp-diu05os and not be the same as on the > Controller node. Can anyone tell me if this is the case? Have you configured nova-network to run in multi-host mode? multi_host=true in nova.conf. -- Russell Bryant From rich.minton at lmco.com Mon Mar 11 14:27:20 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 11 Mar 2013 14:27:20 +0000 Subject: [rhos-list] EXTERNAL: Re: IPTables question. In-Reply-To: <513DE77B.1090506@redhat.com> References: <513DE77B.1090506@redhat.com> Message-ID: Yes, I have that set. -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Russell Bryant Sent: Monday, March 11, 2013 10:18 AM To: rhos-list at redhat.com Subject: EXTERNAL: Re: [rhos-list] IPTables question. On 03/11/2013 09:27 AM, Minton, Rich wrote: > I'm looking at the iptables for both my controller node and compute > node and I see something that doesn't seem right. In the > "nova-network-POSTROUTING" NAT chain I have the same hostname as the > destination on both nodes. I have Nova Network running on both nodes > so I would think that the destination hostname would equal the > hostname that nova-network is running on, i.e., the destination > hostname on the compute node should be uvslp-diu05os and not be the > same as on the Controller node. Can anyone tell me if this is the case? Have you configured nova-network to run in multi-host mode? multi_host=true in nova.conf. -- Russell Bryant _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From obasan at redhat.com Thu Mar 14 10:20:38 2013 From: obasan at redhat.com (Ohad Basan) Date: Thu, 14 Mar 2013 06:20:38 -0400 (EDT) Subject: [rhos-list] centos openstack livecd In-Reply-To: <567518993.19988599.1363256051211.JavaMail.root@redhat.com> Message-ID: <420775160.19990304.1363256438281.JavaMail.root@redhat.com> Hey everyone I have opened a github project that creates an iso with a live env of openstack on centos feel free to clone it from https://github.com/Ohadbasan/centos-openstack generate the iso on your centos instance and send patches thanks Ohad From mail-lists at karan.org Thu Mar 14 11:57:07 2013 From: mail-lists at karan.org (Karanbir Singh) Date: Thu, 14 Mar 2013 11:57:07 +0000 Subject: [rhos-list] centos openstack livecd In-Reply-To: <420775160.19990304.1363256438281.JavaMail.root@redhat.com> References: <420775160.19990304.1363256438281.JavaMail.root@redhat.com> Message-ID: <5141BB13.1060301@karan.org> Hi, On 03/14/2013 10:20 AM, Ohad Basan wrote: > I have opened a github project that creates an iso with a live env of openstack on centos > feel free to clone it from https://github.com/Ohadbasan/centos-openstack > generate the iso on your centos instance and send patches thanks for this script and kickstart Ohad. I've done a quick clone + build and pushed iso to : http://mirrors.karan.org/CentOS64-EPEL-openstack.iso The buildlog is at : http://mirrors.karan.org/CentOS64-EPEL-openstack.iso.log.gz Regards, - KB -- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc From rich.minton at lmco.com Fri Mar 15 16:00:23 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 15 Mar 2013 16:00:23 +0000 Subject: [rhos-list] Nova-network stops with iptables-restore error. Message-ID: Does anyone have an idea why I get this when I start nova-network on a compute node? 2013-03-15 11:06:58 2770 AUDIT nova.service [-] Starting network node (version 2012.2.2-9.el6ost) 2013-03-15 11:07:07 2770 CRITICAL nova [-] Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c Exit code: 2 Stdout: '' Stderr: "iptables-restore v1.4.7: host/network `None' not found\nError occurred at line: 25\nTry `iptables-restore -h' or 'iptables-restore --help' for more information.\n" 2013-03-15 11:57:43 4021 AUDIT nova.service [-] Starting network node (version 2012.2.2-9.el6ost) 2013-03-15 11:57:52 4021 CRITICAL nova [-] Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c Exit code: 2 Stdout: '' Stderr: "iptables-restore v1.4.7: host/network `None' not found\nError occurred at line: 29\nTry `iptables-restore -h' or 'iptables-restore --help' for more information.\n" Thanks for any help! Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmyers at redhat.com Fri Mar 15 23:57:50 2013 From: pmyers at redhat.com (Perry Myers) Date: Fri, 15 Mar 2013 19:57:50 -0400 Subject: [rhos-list] Nova-network stops with iptables-restore error. In-Reply-To: References: Message-ID: <5143B57E.1050208@redhat.com> On 03/15/2013 12:00 PM, Minton, Rich wrote: > Does anyone have an idea why I get this when I start nova-network on a > compute node? > > > > 2013-03-15 11:06:58 2770 AUDIT nova.service [-] Starting network node > (version 2012.2.2-9.el6ost) > > 2013-03-15 11:07:07 2770 CRITICAL nova [-] Unexpected error while > running command. > > Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c > > Exit code: 2 > > Stdout: '' > > Stderr: "iptables-restore v1.4.7: host/network `None' not found\nError > occurred at line: 25\nTry `iptables-restore -h' or 'iptables-restore > --help' for more information.\n" > > 2013-03-15 11:57:43 4021 AUDIT nova.service [-] Starting network node > (version 2012.2.2-9.el6ost) > > 2013-03-15 11:57:52 4021 CRITICAL nova [-] Unexpected error while > running command. > > Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c > > Exit code: 2 > > Stdout: '' > > Stderr: "iptables-restore v1.4.7: host/network `None' not found\nError > occurred at line: 29\nTry `iptables-restore -h' or 'iptables-restore > --help' for more information.\n" Hi Rich, What are the contents of /etc/nova/nova.conf and what does your host network configuration look like (devices, logical networks, etc) Cheers, Perry From rich.minton at lmco.com Mon Mar 18 16:39:48 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 18 Mar 2013 16:39:48 +0000 Subject: [rhos-list] EXTERNAL: Re: Nova-network stops with iptables-restore error. In-Reply-To: <5143B57E.1050208@redhat.com> References: <5143B57E.1050208@redhat.com> Message-ID: Perry, Thanks for the response. I found my problem. I had converted from VlanManager for FlatDHCPManager but I still had my third node set to Vlan (my bad). Thanks, Rick -----Original Message----- From: Perry Myers [mailto:pmyers at redhat.com] Sent: Friday, March 15, 2013 7:58 PM To: Minton, Rich Cc: rhos-list at redhat.com; Brent Eagles; Russell Bryant; Dave Allan Subject: EXTERNAL: Re: [rhos-list] Nova-network stops with iptables-restore error. On 03/15/2013 12:00 PM, Minton, Rich wrote: > Does anyone have an idea why I get this when I start nova-network on a > compute node? > > > > 2013-03-15 11:06:58 2770 AUDIT nova.service [-] Starting network node > (version 2012.2.2-9.el6ost) > > 2013-03-15 11:07:07 2770 CRITICAL nova [-] Unexpected error while > running command. > > Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore > -c > > Exit code: 2 > > Stdout: '' > > Stderr: "iptables-restore v1.4.7: host/network `None' not found\nError > occurred at line: 25\nTry `iptables-restore -h' or 'iptables-restore > --help' for more information.\n" > > 2013-03-15 11:57:43 4021 AUDIT nova.service [-] Starting network node > (version 2012.2.2-9.el6ost) > > 2013-03-15 11:57:52 4021 CRITICAL nova [-] Unexpected error while > running command. > > Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore > -c > > Exit code: 2 > > Stdout: '' > > Stderr: "iptables-restore v1.4.7: host/network `None' not found\nError > occurred at line: 29\nTry `iptables-restore -h' or 'iptables-restore > --help' for more information.\n" Hi Rich, What are the contents of /etc/nova/nova.conf and what does your host network configuration look like (devices, logical networks, etc) Cheers, Perry From berrange at redhat.com Wed Mar 20 11:47:19 2013 From: berrange at redhat.com (Daniel P. Berrange) Date: Wed, 20 Mar 2013 11:47:19 +0000 Subject: [rhos-list] Instances showing up on the wrong host. In-Reply-To: References: Message-ID: <20130320114719.GA18112@redhat.com> On Fri, Mar 01, 2013 at 05:06:04PM +0000, Minton, Rich wrote: > I created an instance that came up on my second compute node, > as shown in the Horizon Dashboard when logged in as admin. > In the VNC console it is seen as instance-0000008a. But when > I look at the list of instances on compute2 using "virsh list --all" > I do not see that instance in the list. If I do a "virsh list --all" > on the controller it does show up in the list (see below). I also > logged into the VM and checked the MAC address. It is in the list > when I do a "brctl showmacs" on the controller node. Has anyone > seen this before? Not really enough info here to give a good answer. If you are still seeing the problem, can you show output from various 'nova' commands to clearly illustrate the problem. As a starting point something like these commands - nova list - nova show - nova host-list - nova host-show - nova hypervisor-list - nova hypervisor-show - virsh -c qemu+ssh://root@/system list For "" use the hostname visibee in the 'nova show' output for the instance in question. > Also, does anyone know how to get rid of the instances marked > as "shut off" in the virsh list? These instances have no associated > directory in the instances directory. It's like they are in some > database somewhere but are stranded. Nova ought to clean these up itself, but it must have had some problem in doing so. You remove them from libvirt's view by doing 'virsh undefine VMNAME' Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| From rich.minton at lmco.com Wed Mar 20 13:01:47 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Wed, 20 Mar 2013 13:01:47 +0000 Subject: [rhos-list] EXTERNAL: Re: Instances showing up on the wrong host. In-Reply-To: <20130320114719.GA18112@redhat.com> References: <20130320114719.GA18112@redhat.com> Message-ID: This has been resolved. I was able to us "virsh undefined" to get rid of the stale shutdown entries. Thanks, Rick -----Original Message----- From: Daniel P. Berrange [mailto:berrange at redhat.com] Sent: Wednesday, March 20, 2013 7:47 AM To: Minton, Rich Cc: rhos-list at redhat.com Subject: EXTERNAL: Re: [rhos-list] Instances showing up on the wrong host. On Fri, Mar 01, 2013 at 05:06:04PM +0000, Minton, Rich wrote: > I created an instance that came up on my second compute node, as shown > in the Horizon Dashboard when logged in as admin. > In the VNC console it is seen as instance-0000008a. But when I look at > the list of instances on compute2 using "virsh list --all" > I do not see that instance in the list. If I do a "virsh list --all" > on the controller it does show up in the list (see below). I also > logged into the VM and checked the MAC address. It is in the list when > I do a "brctl showmacs" on the controller node. Has anyone seen this > before? Not really enough info here to give a good answer. If you are still seeing the problem, can you show output from various 'nova' commands to clearly illustrate the problem. As a starting point something like these commands - nova list - nova show - nova host-list - nova host-show - nova hypervisor-list - nova hypervisor-show - virsh -c qemu+ssh://root@/system list For "" use the hostname visibee in the 'nova show' output for the instance in question. > Also, does anyone know how to get rid of the instances marked as "shut > off" in the virsh list? These instances have no associated directory > in the instances directory. It's like they are in some database > somewhere but are stranded. Nova ought to clean these up itself, but it must have had some problem in doing so. You remove them from libvirt's view by doing 'virsh undefine VMNAME' Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| From marc.jadoul at gmail.com Wed Mar 20 14:40:33 2013 From: marc.jadoul at gmail.com (Marc Jadoul) Date: Wed, 20 Mar 2013 15:40:33 +0100 Subject: [rhos-list] Glance config Message-ID: Hello, I have 2 small questions related to Glance config in the "preview getting started guide". First, in some document on internet, a specific user is created for glance service and configured in glance configuration files. For instance: admin_tenant_name = service admin_user = glance admin_password = glance But the documentation seems to indicate using the admin user of keystone..... Second, I supposed glance would use swift. But Glance is configured and tested first and Swift after.... May be I messed something, but is seems Glance use file instead? Figure 1.1 indicate it use swift. Where is that configured? Regards, Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgordon at redhat.com Wed Mar 20 14:52:35 2013 From: sgordon at redhat.com (Steve Gordon) Date: Wed, 20 Mar 2013 10:52:35 -0400 (EDT) Subject: [rhos-list] Glance config In-Reply-To: Message-ID: <1186878786.12002557.1363791155493.JavaMail.root@redhat.com> ----- Original Message ----- > From: "Marc Jadoul" > To: rhos-list at redhat.com > Sent: Wednesday, March 20, 2013 10:40:33 AM > Subject: [rhos-list] Glance config > > Hello, > > I have 2 small questions related to Glance config in the "preview > getting > started guide". > > First, in some document on internet, a specific user is created for > glance > service and configured in glance configuration files. > For instance: > admin_tenant_name = service > admin_user = glance > admin_password = glance > > But the documentation seems to indicate using the admin user of > keystone..... Hi Marc, Both are valid strategies as I understand it, you will find that currently we configure most of the services in this manner with a view to getting the user up and running as quickly as possible (I think the notable exception is OpenStack Networking). > Second, > I supposed glance would use swift. But Glance is configured and > tested > first and Swift after.... May be I messed something, but is seems > Glance > use file instead? > Figure 1.1 indicate it use swift. Where is that configured? Currently, it's not, I will raise a bug to address this and make readers aware of both options. The diagram should likely be updated to indicate that this is optional as well. Thanks, Steve From sgordon at redhat.com Wed Mar 20 15:07:34 2013 From: sgordon at redhat.com (Steve Gordon) Date: Wed, 20 Mar 2013 11:07:34 -0400 (EDT) Subject: [rhos-list] Glance config In-Reply-To: <1186878786.12002557.1363791155493.JavaMail.root@redhat.com> Message-ID: <1996623623.12013695.1363792054236.JavaMail.root@redhat.com> ---- Original Message ----- > From: "Steve Gordon" > To: "Marc Jadoul" > Cc: rhos-list at redhat.com > Sent: Wednesday, March 20, 2013 10:52:35 AM > Subject: Re: [rhos-list] Glance config > > > ----- Original Message ----- > > From: "Marc Jadoul" > > To: rhos-list at redhat.com > > Sent: Wednesday, March 20, 2013 10:40:33 AM > > Subject: [rhos-list] Glance config > > > > Hello, > > > > I have 2 small questions related to Glance config in the "preview > > getting > > started guide". > > > > First, in some document on internet, a specific user is created for > > glance > > service and configured in glance configuration files. > > For instance: > > admin_tenant_name = service > > admin_user = glance > > admin_password = glance > > > > But the documentation seems to indicate using the admin user of > > keystone..... > > Hi Marc, > > Both are valid strategies as I understand it, you will find that > currently we configure most of the services in this manner with a > view to getting the user up and running as quickly as possible (I > think the notable exception is OpenStack Networking). > > > Second, > > I supposed glance would use swift. But Glance is configured and > > tested > > first and Swift after.... May be I messed something, but is seems > > Glance > > use file instead? > > Figure 1.1 indicate it use swift. Where is that configured? > > Currently, it's not, I will raise a bug to address this and make > readers aware of both options. The diagram should likely be updated > to indicate that this is optional as well. > > Thanks, > > Steve I have raised bugs to track these issues here: https://bugzilla.redhat.com/show_bug.cgi?id=923839 https://bugzilla.redhat.com/show_bug.cgi?id=923845 They should be publicly viewable but let me know if you have any issues accessing them. Thanks, Steve From k.mohammad1 at physics.ox.ac.uk Wed Mar 20 17:23:16 2013 From: k.mohammad1 at physics.ox.ac.uk (Kashif Mohammad) Date: Wed, 20 Mar 2013 17:23:16 +0000 Subject: [rhos-list] Nova-network woes Message-ID: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> Hi Guys I am using rhel6.4 and RedHat Openstack Folsom rpms. I have one controller node which is running glance, cinder, scheduler, api server, consoleauth and other node is running compute, network and api. Both nodes have one public and one private IP and running in multi_host mode. I am not running nova-network and nova-compute on controller node. Network bit is like this on compute node public_interface = em1 flat_interface = em2 fixed_range = 10.0.1.0/24 flat_network_dhcp_start = 10.0.1.5 connection_type = libvirt flat_injected = False multi_host = True flat_network_bridge = br100 created network like this nova-manage network create private 10.0.1.0/24 1 256 --bridge=br100 --multi_host=True I can create VM through dashboard and initially it couldn't get ip address from dhcp server running on compute node but after adding iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill it is getting ip address and I can log in into vm from compute node. It can reach metadata server and grab public keys. My problem is that VM's can not access outside word. I think that issue is in this line of iptable 47 3247 nova-network-float-snat all -- any any anywhere anywhere 2 168 SNAT all -- any em1 10.0.1.0/24 anywhere to:192.168.9.3 It is changing source IP address to local ip rather than external ip. If insert a rule iptables -t nat -I nova-network-snat 2 -o em1 -j SNAT --to-source 163.1.5.40 Then it vm can access interner but the moment I create a new VM this line disappear from iptable. My n/w setting is like this 2: em1: mtu 1500 qdisc mq state UP qlen 1000 link/ether 00:22:19:6a:bc:09 brd ff:ff:ff:ff:ff:ff inet 163.1.5.40/24 brd 163.1.5.255 scope global em1 inet6 fe80::222:19ff:fe6a:bc09/64 scope link valid_lft forever preferred_lft forever 3: em2: mtu 1500 qdisc mq state UP qlen 1000 link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff inet6 fe80::222:19ff:fe6a:bc0b/64 scope link valid_lft forever preferred_lft forever 5: br100: mtu 1500 qdisc noqueue state UNKNOWN link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff inet 10.0.1.1/24 brd 10.0.1.255 scope global br100 inet 192.168.9.3/24 brd 192.168.9.255 scope global br100 inet6 fe80::80cd:6fff:fef7:d955/64 scope link valid_lft forever preferred_lft forever I can not reach vnc as well but I first fix this n/w issue and then look into vnc. Thanks Kashif From shake.chen at gmail.com Thu Mar 21 01:28:50 2013 From: shake.chen at gmail.com (Shake Chen) Date: Thu, 21 Mar 2013 09:28:50 +0800 Subject: [rhos-list] Nova-network woes In-Reply-To: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> Message-ID: On Thu, Mar 21, 2013 at 1:23 AM, Kashif Mohammad < k.mohammad1 at physics.ox.ac.uk> wrote: > > Hi Guys > > I am using rhel6.4 and RedHat Openstack Folsom rpms. I have one > controller node which is running glance, cinder, scheduler, api server, > consoleauth and other node is running compute, network and api. Both nodes > have one public and one private IP and running in multi_host mode. > > I am not running nova-network and nova-compute on controller node. > Network bit is like this on compute node > > public_interface = em1 > flat_interface = em2 > fixed_range = 10.0.1.0/24 > flat_network_dhcp_start = 10.0.1.5 > connection_type = libvirt > flat_injected = False > multi_host = True > flat_network_bridge = br100 > > created network like this > nova-manage network create private 10.0.1.0/24 1 256 --bridge=br100 > --multi_host=True > try it nova-manage network create private --fixed_range_v4=10.9.1.0/24 \ --num_networks=1 --bridge=br100 --bridge_interface=eth1 \ --network_size=256 --multi_host=T > > > I can create VM through dashboard and initially it couldn't get ip address > from dhcp server running on compute node but after adding > > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM > --checksum-fill > > it is getting ip address and I can log in into vm from compute node. It > can reach metadata server and grab public keys. > My problem is that VM's can not access outside word. I think that issue is > in this line of iptable > > 47 3247 nova-network-float-snat all -- any any anywhere > anywhere > 2 168 SNAT all -- any em1 10.0.1.0/24 > anywhere to:192.168.9.3 > > It is changing source IP address to local ip rather than external ip. If > insert a rule > > iptables -t nat -I nova-network-snat 2 -o em1 -j SNAT --to-source > 163.1.5.40 > > Then it vm can access interner but the moment I create a new VM this line > disappear from iptable. > > My n/w setting is like this > > 2: em1: mtu 1500 qdisc mq state UP qlen > 1000 > link/ether 00:22:19:6a:bc:09 brd ff:ff:ff:ff:ff:ff > inet 163.1.5.40/24 brd 163.1.5.255 scope global em1 > inet6 fe80::222:19ff:fe6a:bc09/64 scope link > valid_lft forever preferred_lft forever > 3: em2: mtu 1500 qdisc mq state UP qlen > 1000 > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet6 fe80::222:19ff:fe6a:bc0b/64 scope link > valid_lft forever preferred_lft forever > 5: br100: mtu 1500 qdisc noqueue state > UNKNOWN > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet 10.0.1.1/24 brd 10.0.1.255 scope global br100 > inet 192.168.9.3/24 brd 192.168.9.255 scope global br100 > inet6 fe80::80cd:6fff:fef7:d955/64 scope link > valid_lft forever preferred_lft forever > > I can not reach vnc as well but I first fix this n/w issue and then look > into vnc. > > Thanks > Kashif > > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > -- Shake Chen -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicolas.vogel at heig-vd.ch Thu Mar 21 07:38:15 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Thu, 21 Mar 2013 07:38:15 +0000 Subject: [rhos-list] Nova-Network problem Message-ID: Hello, I?ve installed Openstack Folsom with Keystone, Nova, Cinder and Glance on a single node with CentOS 6.3. I?m using Nova-network and not Quantum. My server has 2 NICs, and I used one of them for management (em1: 10.192.75.190/24) and the second for my VM (em2 + br100: 10.192.73.193/26). All the VM I start get a right IP address but I can?t reach them. Here is my config: em1: DEVICE="em1" BOOTPROTO="none" HWADDR="84:2B:2B:xx:xx:xx" NM_CONTROLLED="yes" ONBOOT="yes" TYPE="Ethernet" IPADDR=10.192.75.190 PREFIX=24 GATEWAY=10.192.75.1 DNS1=10.192.48.100 DNS2=10.192.48.101 UUID="f1d642aa-c64c-4696-b849-xxxxxxxxxxxx" em2: DEVICE="em2" #BOOTPROTO="none" HWADDR="84:2B:2B:6C:FD:10" BRIDGE=br100 NM_CONTROLLED="yes" ONBOOT="yes" TYPE="Ethernet" UUID="014a008d-02cb-45c2-98b4-xxxxxxxxxxxx" br100: DEVICE=br100 TYPE=Bridge ONBOOT=yes DELAY=0 BOOTPROTO=static IPADDR=10.192.73.193 NETMASK=255.255.255.192 I created my network with following commands: nova-manage network create private --multi_host=T --fixed_range_v4=10.192.73.192/26 --bridge=br100 --bridge_interface=em2 --network_size=64 nova.conf: [DEFAULT] logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatDHCPManager iscsi_helper = tgtadm sql_connection = mysql://nova:nova at localhost/nova compute_driver = libvirt.LibvirtDriver firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver rpc_backend = nova.openstack.common.rpc.impl_qpid rootwrap_config = /etc/nova/rootwrap.conf auth_strategy = keystone flat_interface = em2 public_interface = em2 volume_api_class = nova.volume.cinder.API enabled_apis = ec2,osapi_compute,metadata [keystone_authtoken] admin_tenant_name = %SERVICE_TENANT_NAME% admin_user = %SERVICE_USER% admin_password = %SERVICE_PASSWORD% auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http signing_dirname = /tmp/keystone-signing-nova and the routing table: Destination Passerelle Genmask Indic Metric Ref Use Iface 10.192.73.192 * 255.255.255.192 U 0 0 0 br100 10.192.75.0 * 255.255.255.0 U 0 0 0 em1 192.168.122.0 * 255.255.255.0 U 0 0 0 virbr0 link-local * 255.255.0.0 U 1002 0 0 em1 link-local * 255.255.0.0 U 1006 0 0 br100 default 10.192.75.1 0.0.0.0 UG 0 0 0 em1 What did I make wrong in my config? If I want to assign my VM a floating IP (public IP), I have to create a pool. But do I have to use a third NIC which is available on my server and create a br101 interface for that? I?m also trying to configure a second node. Is the network configuration identical as for the controller (with a br100 = IP 10.192.73.194 in this case) ? Thanks for all your answers and help, Cheers, Nicolas. -------------- next part -------------- An HTML attachment was scrubbed... URL: From anniec at yahoo-inc.com Thu Mar 21 13:19:13 2013 From: anniec at yahoo-inc.com (Annie Cheng) Date: Thu, 21 Mar 2013 13:19:13 +0000 Subject: [rhos-list] Oslo errors? In-Reply-To: Message-ID: + rhos-list at redhat.com Mark, let us know whether this should go to rhos-list or community upstream list, cuz it's RHEL specific question and upstream doesn't officially support RHEL .. Whatever works for you we will try to follow the protocol to get help. Josh, can you please attach the output file again? Thanks! Annie From: Joshua Harlow > Reply-To: "openstack-dev at yahoo-inc.com" >, Joshua Harlow > Date: Wednesday, March 20, 2013 6:51 PM To: "openstack-dev at yahoo-inc.com" >, Joshua Harlow >, Mark McLoughlin > Subject: Re: Oslo errors? And system version? $ lsb_release -a LSB Version: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 6.3 (Santiago) Release: 6.3 Codename: Santiago And dependency list? $ pip-python freeze | grep -v github.com Babel==0.9.6 Cheetah==2.4.4 IPy==0.75 Jinja2==2.6 M2Crypto==0.20.2 Magic-file-extensions==0.1 Mako==0.7.3 Markdown==2.0.1 MarkupSafe==0.15 MySQL-python==1.2.3c1 Paste==1.7.4 PasteDeploy==1.5.0 PyYAML==3.10 Pygments==1.6 Routes==1.12.3 SQLAlchemy==0.7.9 Sphinx==1.1.3 Tempita==0.5.1 WebOb==1.2.3 WebTest==1.4.3 alembic==0.4.2 amqplib==0.6.1 anyjson==0.2.4 argparse==1.2.1 boto==2.1.1 bouncer==0.1.1 cas==0.15 cliff==1.3 cliff-tablib==1.0 cloud-init==0.7.2 cmd2==0.6.4 colorama==0.2.4 configobj==4.6.0 coverage==3.6 decorator==3.4.0 discover==0.4.0 distribute==0.6.35 docutils==0.10 egenix-mx-base==3.1.1 euca2ools==2.1.1 eventlet==0.12.1 extras==0.0.3 feedparser==5.0.1 fixtures==0.3.12 greenlet==0.4.0 hgtools==2.0.2 httplib2==0.7.7 httpretty==0.5.6 iniparse==0.4 iotop==0.3.2 iso8601==0.1.4 jsonschema==0.2 keyring==1.2 keystone==2012.2.4 kombu==1.0.4 lockfile==0.8 logilab-astng==0.20.1 logilab-common==0.50.3 lxml==2.3.5 mock==0.8.0 monkeypatch==0.1rc3 mox==0.5.3 netaddr==0.7.5 netifaces==0.5 netsnmp-python==1.0a1 nose==1.2.1 nose-exclude==0.1.9 nosehtmloutput==0.0.4 nosexcover==1.0.7 oauth==1.0.1 openstack.nose-plugin==0.11 ordereddict==1.1 oslo.config==1.1.0 pam==0.1.4 paramiko==1.7.5 passlib==1.6.1 pep8==1.3.3 perf==0.1 pretty==0.1 prettytable==0.6.1 progressbar==2.2 psutil==0.6.1 pyOpenSSL==0.10 pycrypto==2.6 pycurl==7.19.0 pyflakes==0.6.1 pygpgme==0.1 pylint==0.21.1 pyparsing==1.5.6 pysendfile==2.0.0 python-daemon==1.5.5 python-dateutil==1.4.1 python-distutils-extra==2.32 python-dmidecode==3.10.13 python-subunit==0.0.9 pyudev==0.16.1 pyzmq==2.2.0.1 qpid-python==0.20 raidctl==0.0.7 redis==2.7.2 requests==0.14.2 simplejson==2.0.9 sqlalchemy-migrate==0.7.2 stevedore==0.8 suds==0.4 tablib==0.9.11 termcolor==1.1.0 testrepository==0.0.13 testtools==0.9.29 unittest2==0.5.1 urlgrabber==3.9.1 warlock==0.1.0 wsgiref==0.1.2 xattr==0.6.4 yum-metadata-parser==1.1.2 yum-presto==0.4.4 From: Joshua Harlow > Reply-To: openstack-dev >, Joshua Harlow > Date: Wednesday, March 20, 2013 6:44 PM To: Mark McLoughlin > Cc: openstack-dev > Subject: Re: Oslo errors? +openstack-dev (at y!) From: Joshua Harlow > Date: Wednesday, March 20, 2013 6:37 PM To: Mark McLoughlin > Subject: Oslo errors? Hi mark, I am just trying to run nosetests in the oslo-incubator code on RHEL6.3. How are u running these tests, any attempt in RHEL6.X? Shouldn't CI be catching these? Attached is a big output with some of these? -Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: From beagles at redhat.com Thu Mar 21 15:20:07 2013 From: beagles at redhat.com (Brent Eagles) Date: Thu, 21 Mar 2013 12:50:07 -0230 Subject: [rhos-list] Nova-network woes In-Reply-To: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> Message-ID: <20130321152007.GA18983@beagles.redhat.com> Hi Kashif, On Wed, Mar 20, 2013 at 05:23:16PM +0000, Kashif Mohammad wrote: > > Hi Guys > > I am using rhel6.4 and RedHat Openstack Folsom rpms. I have one > controller node which is running glance, cinder, scheduler, api > server, consoleauth and other node is running compute, network and > api. Both nodes have one public and one private IP and running in > multi_host mode. > > I am not running nova-network and nova-compute on controller node. > Network bit is like this on compute node > > public_interface = em1 > flat_interface = em2 > fixed_range = 10.0.1.0/24 > flat_network_dhcp_start = 10.0.1.5 > connection_type = libvirt > flat_injected = False > multi_host = True > flat_network_bridge = br100 > > created network like this > nova-manage network create private 10.0.1.0/24 1 256 --bridge=br100 --multi_host=True I would try running nova-manage using the keyword=value approach instead of relying on positional arguments (e.g. --fixed_range_ipv4=10.0.1.0/24). It would avoid issues with getting the order wrong which can lead to surprising results. > I can create VM through dashboard and initially it couldn't get ip > address from dhcp server running on compute node but after adding > > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill Mmmm.. if everything is configured properly, you shouldn't have to modify the firewall rules for this to work. Was there something you came across while debugging your setup that gave you the idea to add this? > it is getting ip address and I can log in into vm from compute node. > It can reach metadata server and grab public keys. My problem is that > VM's can not access outside word. I think that issue is in this line > of iptable > > 47 3247 nova-network-float-snat all -- any any anywhere anywhere > 2 168 SNAT all -- any em1 10.0.1.0/24 anywhere to:192.168.9.3 > > It is changing source IP address to local ip rather than external ip. If insert a rule > > iptables -t nat -I nova-network-snat 2 -o em1 -j SNAT --to-source 163.1.5.40 That's weird. Can you describe how you configured and allocated the floating IPs? This is another thing that generally just works. > Then it vm can access interner but the moment I create a new VM this line disappear from iptable. > > My n/w setting is like this > > 2: em1: mtu 1500 qdisc mq state UP qlen 1000 > link/ether 00:22:19:6a:bc:09 brd ff:ff:ff:ff:ff:ff > inet 163.1.5.40/24 brd 163.1.5.255 scope global em1 > inet6 fe80::222:19ff:fe6a:bc09/64 scope link > valid_lft forever preferred_lft forever > 3: em2: mtu 1500 qdisc mq state UP qlen 1000 > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet6 fe80::222:19ff:fe6a:bc0b/64 scope link > valid_lft forever preferred_lft forever > 5: br100: mtu 1500 qdisc noqueue state UNKNOWN > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet 10.0.1.1/24 brd 10.0.1.255 scope global br100 > inet 192.168.9.3/24 brd 192.168.9.255 scope global br100 > inet6 fe80::80cd:6fff:fef7:d955/64 scope link > valid_lft forever preferred_lft forever > > I can not reach vnc as well but I first fix this n/w issue and then look into vnc. > > Thanks > Kashif Heh. No VNC? It is the one thing you should *not* live without! I would say that something got crossed during configuration. You might want to "back up" to the beginning and try each thing in turn. If the "cookbook" type instructions didn't work as described, it might be best to figure out why instead of tweaking until things just kind of work. As one bit of configuration tends to depend on another, if the most fundamental thing does not work properly (e.g. allocating a private IP to a VM) it is very possible that things are going to go awry with other configuration items (e.g. allocating floating IPs). However, instead of just leaving with you a note to "try again", and it is just a thought/guess, but if iptables is giving you grief and you have a pre-existing firewall configuration, you might want to look for rules that might interfere with what openstack is trying to do. What I have done in the past when trying a new configuration is take a snapshot of the current configuration with iptables-save (e.g. iptables-save > iptables.before) and again when everything is "done" (iptables.after?) and visually compare the two. It is useful pedagogically because I see what Openstack is doing and it helps identify configuration (and sometimes bugs!) before I go too far. Cheers, Brent From rbryant at redhat.com Thu Mar 21 15:27:44 2013 From: rbryant at redhat.com (Russell Bryant) Date: Thu, 21 Mar 2013 11:27:44 -0400 Subject: [rhos-list] Nova-network woes In-Reply-To: <20130321152007.GA18983@beagles.redhat.com> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> Message-ID: <514B26F0.60503@redhat.com> On 03/21/2013 11:20 AM, Brent Eagles wrote: >> I can create VM through dashboard and initially it couldn't get ip >> address from dhcp server running on compute node but after adding >> >> iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill > > Mmmm.. if everything is configured properly, you shouldn't have to > modify the firewall rules for this to work. Was there something you came > across while debugging your setup that gave you the idea to add this? This is actually a known issue. Mark describes it pretty well in this comment: https://bugzilla.redhat.com/show_bug.cgi?id=910619#c6 It should be fixed such that you don't need this rule as of openstack-nova-2012.2.3-1.el6ost. -- Russell Bryant From rich.minton at lmco.com Thu Mar 21 15:45:22 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 21 Mar 2013 15:45:22 +0000 Subject: [rhos-list] Problems after Update. Message-ID: Where to begin... Basically, I'm hosed. I performed an update to the latest versions of the Openstack distro and rebooted my hosts. Then problems started... * Can't launch new instances-the in progress star comes up and stays there almost forever. * Several of my instances went to an error state. When I tried to reset their state to "active" they went into shutdown/shutoff state. These are the instances on my controller/compute node. I typed "virsh list" and the list was empty. * I tried to bring up a VNC Console on one of the running instances and after several minutes received this error in the browser: ------------------------------------------------------------------------------------------------------------------------------------------- CommunicationError at /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/ Error communicating with http://10.10.12.245:9292 timed out Request Method: GET Request URL: http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/?tab=instance_details__vnc Django Version: 1.4.2 Exception Type: CommunicationError Exception Value: Error communicating with http://10.10.12.245:9292 timed out Exception Location: /usr/lib/python2.6/site-packages/glanceclient/common/http.py in _http_request, line 145 Python Executable: /usr/bin/python Python Version: 2.6.6 Python Path: ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', '/usr/lib64/python26.zip', '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old', '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages', '/usr/lib64/python2.6/site-packages/gtk-2.0', '/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg', '/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg', '/usr/lib/python2.6/site-packages', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/usr/share/openstack-dashboard/openstack_dashboard'] Server time: Thu, 21 Mar 2013 15:26:38 +0000 ------------------------------------------------------------------------------------------------------------------------------------ * I tried deleting iptables and rebooting but did not solve anything. * I'm not seeing anything out of the ordinary in the nova logs. Any ideas on where I should start looking? Thanks for any help. Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.mohammad1 at physics.ox.ac.uk Thu Mar 21 15:47:27 2013 From: k.mohammad1 at physics.ox.ac.uk (Kashif Mohammad) Date: Thu, 21 Mar 2013 15:47:27 +0000 Subject: [rhos-list] Nova-network woes In-Reply-To: <20130321152007.GA18983@beagles.redhat.com> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> Message-ID: <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> Hi Brent Thanks for looking into it. Fixed one of the main issue so my vm can get access to internet. I have one public and one private ip on my compute node and have defined private ip as my_ip in nova.conf at compute node. Looking more deeply I found that routing_source_ip defaults to $my_ip. Explicitly defined routing_source_ip to public ip address and it started working. Nova network is now creating correct iptables rule so vm can access internet. >> iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill Found the solution here https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/issues/14 But still my vnc not working. Running openstack-nova-consoleauth, openstack-nova-novncproxy openstack-nova-xvpvncproxy on controller node. I have this configuration in nova.conf at controller node novnc_enabled = true vnc_enabled = true novncproxy_base_url = http://163.1.5.243:6080/vnc_auto.html xvpvncproxy_base_url = http://163.1.5.243:6081/console On Compute node vnc_enabled = true novnc_enabled = true novncproxy_base_url = http://163.1.5.243:6080/vnc_auto.html xvpvncproxy_base_url = http://163.1.5.243:6081/console vncserver_proxyclient_address = 192.168.9.3 vncserver_listen = 0.0.0.0 I can get vnc url using get-vnc-console so probably authentication bit is working but when I try to access vnc console through browser it just give this error Failed to connect to server (code: 1006) Get same error if try to connect through horizon dashboard. I will very much appreciate your help. Cheers Kashif -----Original Message----- From: Brent Eagles [mailto:beagles at redhat.com] Sent: 21 March 2013 15:20 To: Kashif Mohammad Cc: rhos-list at redhat.com Subject: Re: [rhos-list] Nova-network woes Hi Kashif, On Wed, Mar 20, 2013 at 05:23:16PM +0000, Kashif Mohammad wrote: > > Hi Guys > > I am using rhel6.4 and RedHat Openstack Folsom rpms. I have one > controller node which is running glance, cinder, scheduler, api > server, consoleauth and other node is running compute, network and > api. Both nodes have one public and one private IP and running in > multi_host mode. > > I am not running nova-network and nova-compute on controller node. > Network bit is like this on compute node > > public_interface = em1 > flat_interface = em2 > fixed_range = 10.0.1.0/24 > flat_network_dhcp_start = 10.0.1.5 > connection_type = libvirt > flat_injected = False > multi_host = True > flat_network_bridge = br100 > > created network like this > nova-manage network create private 10.0.1.0/24 1 256 --bridge=br100 --multi_host=True I would try running nova-manage using the keyword=value approach instead of relying on positional arguments (e.g. --fixed_range_ipv4=10.0.1.0/24). It would avoid issues with getting the order wrong which can lead to surprising results. > I can create VM through dashboard and initially it couldn't get ip > address from dhcp server running on compute node but after adding > > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill Mmmm.. if everything is configured properly, you shouldn't have to modify the firewall rules for this to work. Was there something you came across while debugging your setup that gave you the idea to add this? > it is getting ip address and I can log in into vm from compute node. > It can reach metadata server and grab public keys. My problem is that > VM's can not access outside word. I think that issue is in this line > of iptable > > 47 3247 nova-network-float-snat all -- any any anywhere anywhere > 2 168 SNAT all -- any em1 10.0.1.0/24 anywhere to:192.168.9.3 > > It is changing source IP address to local ip rather than external ip. If insert a rule > > iptables -t nat -I nova-network-snat 2 -o em1 -j SNAT --to-source 163.1.5.40 That's weird. Can you describe how you configured and allocated the floating IPs? This is another thing that generally just works. > Then it vm can access interner but the moment I create a new VM this line disappear from iptable. > > My n/w setting is like this > > 2: em1: mtu 1500 qdisc mq state UP qlen 1000 > link/ether 00:22:19:6a:bc:09 brd ff:ff:ff:ff:ff:ff > inet 163.1.5.40/24 brd 163.1.5.255 scope global em1 > inet6 fe80::222:19ff:fe6a:bc09/64 scope link > valid_lft forever preferred_lft forever > 3: em2: mtu 1500 qdisc mq state UP qlen 1000 > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet6 fe80::222:19ff:fe6a:bc0b/64 scope link > valid_lft forever preferred_lft forever > 5: br100: mtu 1500 qdisc noqueue state UNKNOWN > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet 10.0.1.1/24 brd 10.0.1.255 scope global br100 > inet 192.168.9.3/24 brd 192.168.9.255 scope global br100 > inet6 fe80::80cd:6fff:fef7:d955/64 scope link > valid_lft forever preferred_lft forever > > I can not reach vnc as well but I first fix this n/w issue and then look into vnc. > > Thanks > Kashif Heh. No VNC? It is the one thing you should *not* live without! I would say that something got crossed during configuration. You might want to "back up" to the beginning and try each thing in turn. If the "cookbook" type instructions didn't work as described, it might be best to figure out why instead of tweaking until things just kind of work. As one bit of configuration tends to depend on another, if the most fundamental thing does not work properly (e.g. allocating a private IP to a VM) it is very possible that things are going to go awry with other configuration items (e.g. allocating floating IPs). However, instead of just leaving with you a note to "try again", and it is just a thought/guess, but if iptables is giving you grief and you have a pre-existing firewall configuration, you might want to look for rules that might interfere with what openstack is trying to do. What I have done in the past when trying a new configuration is take a snapshot of the current configuration with iptables-save (e.g. iptables-save > iptables.before) and again when everything is "done" (iptables.after?) and visually compare the two. It is useful pedagogically because I see what Openstack is doing and it helps identify configuration (and sometimes bugs!) before I go too far. Cheers, Brent From beagles at redhat.com Thu Mar 21 15:50:32 2013 From: beagles at redhat.com (Brent Eagles) Date: Thu, 21 Mar 2013 13:20:32 -0230 Subject: [rhos-list] Nova-Network problem In-Reply-To: References: Message-ID: <20130321155032.GB18983@beagles.redhat.com> On Thu, Mar 21, 2013 at 07:38:15AM +0000, Vogel Nicolas wrote: > Hello, > > I?ve installed Openstack Folsom with Keystone, Nova, Cinder and Glance > on a single node with CentOS 6.3. I?m using Nova-network and not > Quantum. My server has 2 NICs, and I used one of them for management > (em1: 10.192.75.190/24) and the second for my VM (em2 + br100: > 10.192.73.193/26). All the VM I start get a right IP address but I > can?t reach them. Here is my config: What do you mean by "can't reach them"? Do you mean SSH or ping? Did you define the security group rules (see http://docs.openstack.org/trunk/openstack-compute/admin/content/enabling-ping-and-ssh-on-vms.html Without them, iptables is likely blocking access. > em2: > DEVICE="em2" > #BOOTPROTO="none" > HWADDR="84:2B:2B:6C:FD:10" > BRIDGE=br100 > NM_CONTROLLED="yes" > ONBOOT="yes" > TYPE="Ethernet" > UUID="014a008d-02cb-45c2-98b4-xxxxxxxxxxxx" Is this interface really configured with NetworkManager, or did you do this with a configuation file? If the latter, you might want to set NM_CONTROLLED=no. It may not make a difference, but... > > br100: > DEVICE=br100 > TYPE=Bridge > ONBOOT=yes > DELAY=0 > BOOTPROTO=static > IPADDR=10.192.73.193 > NETMASK=255.255.255.192 > > What did I make wrong in my config? Might be best to verify the security group thing before looking at the rest. > > If I want to assign my VM a floating IP (public IP), I have to create > a pool. But do I have to use a third NIC which is available on my > server and create a br101 interface for that? No, you do not need another interface. What happens when you allocate and assign a floating IP is that Openstack adds rules to your iptables that perform source and destination address translation at appropriate points according to how you have things configured. For example, as a packet from a VM goes out your public interface, there is a SNAT to the assigned public IP address ensuring that return communications come back a.) to the proper destination and b.) addressed to the proper assigned address. There is a companion rule that DNAT's it back to the private address for packets coming back in. > I?m also trying to configure a second node. Is the network > configuration identical as for the controller (with a br100 = IP > 10.192.73.194 in this case) ? The question is a little ambiguous. Do you mean a second standalone compute node or a compute/network node? I would recommend giving the following a read if it is the latter: http://docs.openstack.org/trunk/openstack-compute/admin/content/existing-ha-networking-options.html Just a heads up when it comes to configuing your second node, be wary of instances in your configuration where "localhost" is used (e.g. the database access configuration). It can be a bit frustrating (debug=True/verbose=True is your friend!) when you miss them. You also need to make sure that the firewall rules aren't filtering out the required traffic. Another thing that tripped me up when I first started were the mysql accounts (if you are using mysql) as they can be restrictive with respect to the source host/ip for the connection. Try running mysql -e "select host, user from mysql.user" . If the host values are all "%" (percent signs) then I think you are golden. If not, you might need to do some tweaking to get things to work. Cheers, Brent From rbryant at redhat.com Thu Mar 21 15:55:47 2013 From: rbryant at redhat.com (Russell Bryant) Date: Thu, 21 Mar 2013 11:55:47 -0400 Subject: [rhos-list] Problems after Update. In-Reply-To: References: Message-ID: <514B2D83.40908@redhat.com> On 03/21/2013 11:45 AM, Minton, Rich wrote: > Where to begin? Basically, I?m hosed. > > > > I performed an update to the latest versions of the Openstack distro and > rebooted my hosts. Then problems started? > > ? Can?t launch new instances?the in progress star comes up and > stays there almost forever. Can you try starting an instance from the command line? What response do you get? Can you post the output of 'nova show ' after it goes into an ERROR state? -- Russell Bryant From nicolas.vogel at heig-vd.ch Thu Mar 21 16:00:20 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Thu, 21 Mar 2013 16:00:20 +0000 Subject: [rhos-list] unable to delete an instance Message-ID: Hi, I had an instance who couldn?t terminate properly so I used the command ?nova reset-state ?active ? to modify her status to Active. It works but after that, using nova delete , my instance is still standing in the ?Deleting? Task. Can I delete this instance direct in the database? What is the way to manually terminate an instance? This instance had a volume attached to her so I think I will have to manually detach and delete this volume too. But command nova volume-list is giving me ?ERROR: ConnectionRefused: '[Errno 111] Connection refused'?. Thanks, Nicolas. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.minton at lmco.com Thu Mar 21 16:23:00 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 21 Mar 2013 16:23:00 +0000 Subject: [rhos-list] Problems after Update. In-Reply-To: References: Message-ID: I found the source of a couple of my problems... The update wiped out my glance-api-paste.ini and glance-registry-paste.ini. Thank fully I had backups. I still have instances that are in shutoff/shutdown mode. I was able to get them to back to "active" but then they went to "shutoff" soon after. Here is the "nova show" for one of the instances. +-------------------------------------+------------------------------------------------------------------------------------------+ | Property | Value | +-------------------------------------+------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-SRV-ATTR:host | uvslp-diu01os.lmdit.us.lmco.com | | OS-EXT-SRV-ATTR:hypervisor_hostname | uvslp-diu01os.lmdit.us.lmco.com | | OS-EXT-SRV-ATTR:instance_name | instance-000000ea | | OS-EXT-STS:power_state | 4 | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | error | | accessIPv4 | | | accessIPv6 | | | config_drive | | | created | 2013-03-18T15:39:29Z | | fault | {u'message': u'IOError', u'code': 500, u'created': u'2013-03-21T16:17:12Z'} | | flavor | m1.medium (3) | | hostId | e55588f13411a788271d5273f0dabfa2eb755e0323dab51039dab866 | | id | a39787b8-2c66-4aaa-b154-42ec05c0e3c8 | | image | Red Hat Enterprise Linux 6.4, x86_64, Base Server (71c092f5-55de-4b3b-a9d9-4fcb25e15a89) | | key_name | TestKeys | | lmicc-access-nic network | 10.10.16.11 | | metadata | {} | | name | demo | | security_groups | [{u'name': u'default'}] | | status | ERROR | | tenant_id | e4ba09fbf76c41a29218dc68cacf78d5 | | updated | 2013-03-21T16:17:12Z | | user_id | f015ea92bf4848a891b73ae2fbf6c75b | +-------------------------------------+------------------------------------------------------------------------------------------+ From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Minton, Rich Sent: Thursday, March 21, 2013 11:45 AM To: rhos-list at redhat.com Subject: EXTERNAL: [rhos-list] Problems after Update. Where to begin... Basically, I'm hosed. I performed an update to the latest versions of the Openstack distro and rebooted my hosts. Then problems started... * Can't launch new instances-the in progress star comes up and stays there almost forever. * Several of my instances went to an error state. When I tried to reset their state to "active" they went into shutdown/shutoff state. These are the instances on my controller/compute node. I typed "virsh list" and the list was empty. * I tried to bring up a VNC Console on one of the running instances and after several minutes received this error in the browser: ------------------------------------------------------------------------------------------------------------------------------------------- CommunicationError at /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/ Error communicating with http://10.10.12.245:9292 timed out Request Method: GET Request URL: http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/?tab=instance_details__vnc Django Version: 1.4.2 Exception Type: CommunicationError Exception Value: Error communicating with http://10.10.12.245:9292 timed out Exception Location: /usr/lib/python2.6/site-packages/glanceclient/common/http.py in _http_request, line 145 Python Executable: /usr/bin/python Python Version: 2.6.6 Python Path: ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', '/usr/lib64/python26.zip', '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old', '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages', '/usr/lib64/python2.6/site-packages/gtk-2.0', '/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg', '/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg', '/usr/lib/python2.6/site-packages', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/usr/share/openstack-dashboard/openstack_dashboard'] Server time: Thu, 21 Mar 2013 15:26:38 +0000 ------------------------------------------------------------------------------------------------------------------------------------ * I tried deleting iptables and rebooting but did not solve anything. * I'm not seeing anything out of the ordinary in the nova logs. Any ideas on where I should start looking? Thanks for any help. Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbryant at redhat.com Thu Mar 21 16:41:54 2013 From: rbryant at redhat.com (Russell Bryant) Date: Thu, 21 Mar 2013 12:41:54 -0400 Subject: [rhos-list] Problems after Update. In-Reply-To: References: Message-ID: <514B3852.1080807@redhat.com> Based on the instance data, there should be a Traceback in the log somewhere. Check nova-compute.log. -- Russell Bryant On 03/21/2013 12:23 PM, Minton, Rich wrote: > I found the source of a couple of my problems? > > > > The update wiped out my glance-api-paste.ini and > glance-registry-paste.ini. Thank fully I had backups. > > > > I still have instances that are in shutoff/shutdown mode. I was able to > get them to back to ?active? but then they went to ?shutoff? soon > after. Here is the ?nova show? for one of the instances. > > > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | Property | > Value > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | OS-DCF:diskConfig | > MANUAL > | > > | OS-EXT-SRV-ATTR:host | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:hypervisor_hostname | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:instance_name | > instance-000000ea > | > > | OS-EXT-STS:power_state | > 4 > | > > | OS-EXT-STS:task_state | > None > | > > | OS-EXT-STS:vm_state | > error > | > > | accessIPv4 > | > | > > | accessIPv6 > | > | > > | config_drive > | > | > > | created | > 2013-03-18T15:39:29Z > | > > | fault | {u'message': u'IOError', > u'code': 500, u'created': u'2013-03-21T16:17:12Z'} | > > | flavor | m1.medium > (3) > | > > | hostId | > e55588f13411a788271d5273f0dabfa2eb755e0323dab51039dab866 > | > > | id | > a39787b8-2c66-4aaa-b154-42ec05c0e3c8 > | > > | image | Red Hat Enterprise Linux 6.4, > x86_64, Base Server (71c092f5-55de-4b3b-a9d9-4fcb25e15a89) | > > | key_name | > TestKeys > | > > | lmicc-access-nic network | > 10.10.16.11 > | > > | metadata | > {} > | > > | name | > demo > | > > | security_groups | [{u'name': > u'default'}] > | > > | status | > ERROR > | > > | tenant_id | > e4ba09fbf76c41a29218dc68cacf78d5 > | > > | updated | > 2013-03-21T16:17:12Z > | > > | user_id | > f015ea92bf4848a891b73ae2fbf6c75b > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > > > *From:*rhos-list-bounces at redhat.com > [mailto:rhos-list-bounces at redhat.com] *On Behalf Of *Minton, Rich > *Sent:* Thursday, March 21, 2013 11:45 AM > *To:* rhos-list at redhat.com > *Subject:* EXTERNAL: [rhos-list] Problems after Update. > > > > Where to begin? Basically, I?m hosed. > > > > I performed an update to the latest versions of the Openstack distro and > rebooted my hosts. Then problems started? > > ? Can?t launch new instances?the in progress star comes up and > stays there almost forever. > > ? Several of my instances went to an error state. When I tried > to reset their state to ?active? they went into shutdown/shutoff state. > These are the instances on my controller/compute node. I typed ?virsh > list? and the list was empty. > > ? I tried to bring up a VNC Console on one of the running > instances and after several minutes received this error in the browser: > > ------------------------------------------------------------------------------------------------------------------------------------------- > > CommunicationError at /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/ > > > > _Error communicating with http://10.10.12.245:9292 timed out_ > > > > Request Method: GET > > Request URL: > http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/?tab=instance_details__vnc > > Django Version: 1.4.2 > > Exception Type: CommunicationError > > Exception Value: > > > > Error communicating with http://10.10.12.245:9292 timed out > > > > Exception Location: > /usr/lib/python2.6/site-packages/glanceclient/common/http.py in > _http_request, line 145 > > Python Executable: /usr/bin/python > > Python Version: 2.6.6 > > Python Path: > > > > ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', > > '/usr/lib64/python26.zip', > > '/usr/lib64/python2.6', > > '/usr/lib64/python2.6/plat-linux2', > > '/usr/lib64/python2.6/lib-tk', > > '/usr/lib64/python2.6/lib-old', > > '/usr/lib64/python2.6/lib-dynload', > > '/usr/lib64/python2.6/site-packages', > > '/usr/lib64/python2.6/site-packages/gtk-2.0', > > '/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg', > > '/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg', > > '/usr/lib/python2.6/site-packages', > > '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', > > '/usr/share/openstack-dashboard/openstack_dashboard'] > > > > Server time: Thu, 21 Mar 2013 15:26:38 +0000 > > ------------------------------------------------------------------------------------------------------------------------------------ > > ? I tried deleting iptables and rebooting but did not solve > anything. > > ? I?m not seeing anything out of the ordinary in the nova logs. > > > > Any ideas on where I should start looking? > > > > Thanks for any help. > > Rick > > > > _Richard Minton_ > > LMICC Systems Administrator > > 4000 Geerdes Blvd, 13D31 > > King of Prussia, PA 19406 > > Phone: 610-354-5482 > > > From rbryant at redhat.com Thu Mar 21 16:45:09 2013 From: rbryant at redhat.com (Russell Bryant) Date: Thu, 21 Mar 2013 12:45:09 -0400 Subject: [rhos-list] unable to delete an instance In-Reply-To: References: Message-ID: <514B3915.7050609@redhat.com> On 03/21/2013 12:00 PM, Vogel Nicolas wrote: > Hi, > > > > I had an instance who couldn?t terminate properly so I used the command > ?nova reset-state ?active ? to modify her status to Active. It works > but after that, using nova delete , my instance is still standing in > the ?Deleting? Task. > > Can I delete this instance direct in the database? What is the way to > manually terminate an instance? We need to go back to figuring out why the instance couldn't be deleted originally. Do you know how you got it into that state? If all else fails, the brute force is manually removing the instance from the database and the hypervisor, but you should never have to do that. > This instance had a volume attached to her so I think I will have to > manually detach and delete this volume too. But command nova volume-list > is giving me ?ERROR: ConnectionRefused: '[Errno 111] Connection refused'?. Do other nova commands give you that error? Can you provide the full command and output? -- Russell Bryant From rich.minton at lmco.com Thu Mar 21 17:03:27 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 21 Mar 2013 17:03:27 +0000 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: <514B3852.1080807@redhat.com> References: <514B3852.1080807@redhat.com> Message-ID: Here is some of what I found in "compute.log" and it doesn't look good. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] Found 5 in the database and 0 on the hypervisor. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: 2d87a89a-fcbb-4581-a7c3-0e2b2c30b568] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: b7e64392-1f28-446f-8fb3-2b75dc895951] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 INFO nova.virt.libvirt.driver [-] [instance: 2d87a89a-fcbb-4581-a7c3-0e2b2c30b568] Instance destroyed successfully. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: 40153f6e-e963-4a30-bb0a-3f7e612d5638] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 INFO nova.virt.libvirt.driver [-] [instance: b7e64392-1f28-446f-8fb3-2b75dc895951] Instance destroyed successfully. 2013-03-21 12:15:22 42682 WARNING nova.compute.manager [-] [instance: a39787b8-2c66-4aaa-b154-42ec05c0e3c8] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:22 42682 INFO nova.virt.libvirt.driver [-] [instance: 40153f6e-e963-4a30-bb0a-3f7e612d5638] Instance destroyed successfully. 2013-03-21 12:15:22 42682 WARNING nova.compute.manager [-] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:22 42682 INFO nova.virt.libvirt.driver [-] [instance: a39787b8-2c66-4aaa-b154-42ec05c0e3c8] Instance destroyed successfully. 2013-03-21 12:15:23 42682 INFO nova.virt.libvirt.driver [-] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Instance destroyed successfully. 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free ram (MB): 237303 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free disk (GB): -14 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 38 2013-03-21 12:15:23 42682 INFO nova.compute.resource_tracker [-] Compute_service record updated for uvslp-diu01os.lmdit.us.lmco.com 2013-03-21 12:16:02 AUDIT nova.compute.manager [req-a00e05db-32f4-4be0-8885-e789e8474b4f f015ea92bf4848a891b73ae2fbf6c75b e4ba09fbf76c41a29218dc68cacf78d5] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Get console output 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp temp_level, payload) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 196, in decorated_function 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp kwargs['instance']['uuid'], e, sys.exc_info()) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 190, in decorated_function 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1931, in get_console_output 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp output = self.driver.get_console_output(instance) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp temp_level, payload) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1150, in get_console_output 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp libvirt_utils.chown(path, os.getuid()) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 332, in chown 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp execute('chown', owner, path, run_as_root=True) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 53, in execute 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return utils.execute(*args, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd)) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command. 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Exit code: 1 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Stdout: '' 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Stderr: "/bin/chown: cannot access `/var/lib/nova/instances/instance-000000f2/console.log': No such file or directory\n" 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.common [-] Returning exception Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log Exit code: 1 Stdout: '' Stderr: "/bin/chown: cannot access `/var/lib/nova/instances/instance-000000f2/console.log': No such file or directory\n" to caller 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', ' F ile "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python 2.6/site-packages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 196, in decorated_function\n kwargs[\'instance\'][\'uuid\'], e, sys.exc_info())\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 190, in decorated_function\n retu rn function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1931, in get_console_output\n output = self.driver.get_console_output(instance)\n', ' File "/usr/lib /python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-pack ages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1150, in get_console_output\n libvirt_utils.chown(path, os.getuid())\ n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 332, in chown\n execute(\'chown\', owner, path, run_as_root=True)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", l ine 53, in execute\n return utils.execute(*args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute\n cmd=\' \'.join(cmd))\n', 'ProcessExecutionError: Unexpected error while run ning command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log\nExit code: 1\nStdout: \'\'\nStderr: "/bin/chown: cannot access `/var/lib/nova/instances/insta nce-000000f2/console.log\': No such file or directory\\n"\n'] -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Thursday, March 21, 2013 12:42 PM To: Minton, Rich Cc: rhos-list at redhat.com Subject: EXTERNAL: Re: Problems after Update. Based on the instance data, there should be a Traceback in the log somewhere. Check nova-compute.log. -- Russell Bryant On 03/21/2013 12:23 PM, Minton, Rich wrote: > I found the source of a couple of my problems... > > > > The update wiped out my glance-api-paste.ini and > glance-registry-paste.ini. Thank fully I had backups. > > > > I still have instances that are in shutoff/shutdown mode. I was able > to get them to back to "active" but then they went to "shutoff" soon > after. Here is the "nova show" for one of the instances. > > > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | Property | > Value > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | OS-DCF:diskConfig | > MANUAL > | > > | OS-EXT-SRV-ATTR:host | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:hypervisor_hostname | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:instance_name | > instance-000000ea > | > > | OS-EXT-STS:power_state | > 4 > | > > | OS-EXT-STS:task_state | > None > | > > | OS-EXT-STS:vm_state | > error > | > > | accessIPv4 > | > | > > | accessIPv6 > | > | > > | config_drive > | > | > > | created | > 2013-03-18T15:39:29Z > | > > | fault | {u'message': u'IOError', > u'code': 500, u'created': u'2013-03-21T16:17:12Z'} | > > | flavor | m1.medium > (3) > | > > | hostId | > e55588f13411a788271d5273f0dabfa2eb755e0323dab51039dab866 > | > > | id | > a39787b8-2c66-4aaa-b154-42ec05c0e3c8 > | > > | image | Red Hat Enterprise Linux 6.4, > x86_64, Base Server (71c092f5-55de-4b3b-a9d9-4fcb25e15a89) | > > | key_name | > TestKeys > | > > | lmicc-access-nic network | > 10.10.16.11 > | > > | metadata | > {} > | > > | name | > demo > | > > | security_groups | [{u'name': > u'default'}] > | > > | status | > ERROR > | > > | tenant_id | > e4ba09fbf76c41a29218dc68cacf78d5 > | > > | updated | > 2013-03-21T16:17:12Z > | > > | user_id | > f015ea92bf4848a891b73ae2fbf6c75b > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > > > *From:*rhos-list-bounces at redhat.com > [mailto:rhos-list-bounces at redhat.com] *On Behalf Of *Minton, Rich > *Sent:* Thursday, March 21, 2013 11:45 AM > *To:* rhos-list at redhat.com > *Subject:* EXTERNAL: [rhos-list] Problems after Update. > > > > Where to begin... Basically, I'm hosed. > > > > I performed an update to the latest versions of the Openstack distro > and rebooted my hosts. Then problems started... > > * Can't launch new instances-the in progress star comes up and > stays there almost forever. > > * Several of my instances went to an error state. When I tried > to reset their state to "active" they went into shutdown/shutoff state. > These are the instances on my controller/compute node. I typed "virsh > list" and the list was empty. > > * I tried to bring up a VNC Console on one of the running > instances and after several minutes received this error in the browser: > > ---------------------------------------------------------------------- > --------------------------------------------------------------------- > > CommunicationError at > /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/ > > > > _Error communicating with http://10.10.12.245:9292 timed out_ > > > > Request Method: GET > > Request URL: > http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-b > f2770a90c82/?tab=instance_details__vnc > > Django Version: 1.4.2 > > Exception Type: CommunicationError > > Exception Value: > > > > Error communicating with http://10.10.12.245:9292 timed out > > > > Exception Location: > /usr/lib/python2.6/site-packages/glanceclient/common/http.py in > _http_request, line 145 > > Python Executable: /usr/bin/python > > Python Version: 2.6.6 > > Python Path: > > > > ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', > > '/usr/lib64/python26.zip', > > '/usr/lib64/python2.6', > > '/usr/lib64/python2.6/plat-linux2', > > '/usr/lib64/python2.6/lib-tk', > > '/usr/lib64/python2.6/lib-old', > > '/usr/lib64/python2.6/lib-dynload', > > '/usr/lib64/python2.6/site-packages', > > '/usr/lib64/python2.6/site-packages/gtk-2.0', > > '/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg', > > '/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg', > > '/usr/lib/python2.6/site-packages', > > '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', > > '/usr/share/openstack-dashboard/openstack_dashboard'] > > > > Server time: Thu, 21 Mar 2013 15:26:38 +0000 > > ---------------------------------------------------------------------- > -------------------------------------------------------------- > > * I tried deleting iptables and rebooting but did not solve > anything. > > * I'm not seeing anything out of the ordinary in the nova logs. > > > > Any ideas on where I should start looking? > > > > Thanks for any help. > > Rick > > > > _Richard Minton_ > > LMICC Systems Administrator > > 4000 Geerdes Blvd, 13D31 > > King of Prussia, PA 19406 > > Phone: 610-354-5482 > > > From rich.minton at lmco.com Thu Mar 21 17:16:52 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 21 Mar 2013 17:16:52 +0000 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: References: <514B3852.1080807@redhat.com> Message-ID: Found it! The NFS mount to my instances directory was missing. I remounted the directory and I can see all of my instances now. I have one other issue with metadata and ssh keys but I'll create a new thread for that. Thanks for the help. -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Minton, Rich Sent: Thursday, March 21, 2013 1:03 PM To: Russell Bryant Cc: rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: Problems after Update. Here is some of what I found in "compute.log" and it doesn't look good. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] Found 5 in the database and 0 on the hypervisor. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: 2d87a89a-fcbb-4581-a7c3-0e2b2c30b568] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: b7e64392-1f28-446f-8fb3-2b75dc895951] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 INFO nova.virt.libvirt.driver [-] [instance: 2d87a89a-fcbb-4581-a7c3-0e2b2c30b568] Instance destroyed successfully. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: 40153f6e-e963-4a30-bb0a-3f7e612d5638] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 INFO nova.virt.libvirt.driver [-] [instance: b7e64392-1f28-446f-8fb3-2b75dc895951] Instance destroyed successfully. 2013-03-21 12:15:22 42682 WARNING nova.compute.manager [-] [instance: a39787b8-2c66-4aaa-b154-42ec05c0e3c8] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:22 42682 INFO nova.virt.libvirt.driver [-] [instance: 40153f6e-e963-4a30-bb0a-3f7e612d5638] Instance destroyed successfully. 2013-03-21 12:15:22 42682 WARNING nova.compute.manager [-] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:22 42682 INFO nova.virt.libvirt.driver [-] [instance: a39787b8-2c66-4aaa-b154-42ec05c0e3c8] Instance destroyed successfully. 2013-03-21 12:15:23 42682 INFO nova.virt.libvirt.driver [-] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Instance destroyed successfully. 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free ram (MB): 237303 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free disk (GB): -14 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 38 2013-03-21 12:15:23 42682 INFO nova.compute.resource_tracker [-] Compute_service record updated for uvslp-diu01os.lmdit.us.lmco.com 2013-03-21 12:16:02 AUDIT nova.compute.manager [req-a00e05db-32f4-4be0-8885-e789e8474b4f f015ea92bf4848a891b73ae2fbf6c75b e4ba09fbf76c41a29218dc68cacf78d5] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Get console output 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp temp_level, payload) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 196, in decorated_function 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp kwargs['instance']['uuid'], e, sys.exc_info()) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 190, in decorated_function 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1931, in get_console_output 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp output = self.driver.get_console_output(instance) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp temp_level, payload) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1150, in get_console_output 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp libvirt_utils.chown(path, os.getuid()) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 332, in chown 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp execute('chown', owner, path, run_as_root=True) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 53, in execute 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return utils.execute(*args, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd)) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command. 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Exit code: 1 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Stdout: '' 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Stderr: "/bin/chown: cannot access `/var/lib/nova/instances/instance-000000f2/console.log': No such file or directory\n" 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.common [-] Returning exception Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log Exit code: 1 Stdout: '' Stderr: "/bin/chown: cannot access `/var/lib/nova/instances/instance-000000f2/console.log': No such file or directory\n" to caller 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', ' F ile "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python 2.6/site-packages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 196, in decorated_function\n kwargs[\'instance\'][\'uuid\'], e, sys.exc_info())\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 190, in decorated_function\n retu rn function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1931, in get_console_output\n output = self.driver.get_console_output(instance)\n', ' File "/usr/lib /python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-pack ages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1150, in get_console_output\n libvirt_utils.chown(path, os.getuid())\ n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 332, in chown\n execute(\'chown\', owner, path, run_as_root=True)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", l ine 53, in execute\n return utils.execute(*args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute\n cmd=\' \'.join(cmd))\n', 'ProcessExecutionError: Unexpected error while run ning command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log\nExit code: 1\nStdout: \'\'\nStderr: "/bin/chown: cannot access `/var/lib/nova/instances/insta nce-000000f2/console.log\': No such file or directory\\n"\n'] -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Thursday, March 21, 2013 12:42 PM To: Minton, Rich Cc: rhos-list at redhat.com Subject: EXTERNAL: Re: Problems after Update. Based on the instance data, there should be a Traceback in the log somewhere. Check nova-compute.log. -- Russell Bryant On 03/21/2013 12:23 PM, Minton, Rich wrote: > I found the source of a couple of my problems... > > > > The update wiped out my glance-api-paste.ini and > glance-registry-paste.ini. Thank fully I had backups. > > > > I still have instances that are in shutoff/shutdown mode. I was able > to get them to back to "active" but then they went to "shutoff" soon > after. Here is the "nova show" for one of the instances. > > > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | Property | > Value > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | OS-DCF:diskConfig | > MANUAL > | > > | OS-EXT-SRV-ATTR:host | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:hypervisor_hostname | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:instance_name | > instance-000000ea > | > > | OS-EXT-STS:power_state | > 4 > | > > | OS-EXT-STS:task_state | > None > | > > | OS-EXT-STS:vm_state | > error > | > > | accessIPv4 > | > | > > | accessIPv6 > | > | > > | config_drive > | > | > > | created | > 2013-03-18T15:39:29Z > | > > | fault | {u'message': u'IOError', > u'code': 500, u'created': u'2013-03-21T16:17:12Z'} | > > | flavor | m1.medium > (3) > | > > | hostId | > e55588f13411a788271d5273f0dabfa2eb755e0323dab51039dab866 > | > > | id | > a39787b8-2c66-4aaa-b154-42ec05c0e3c8 > | > > | image | Red Hat Enterprise Linux 6.4, > x86_64, Base Server (71c092f5-55de-4b3b-a9d9-4fcb25e15a89) | > > | key_name | > TestKeys > | > > | lmicc-access-nic network | > 10.10.16.11 > | > > | metadata | > {} > | > > | name | > demo > | > > | security_groups | [{u'name': > u'default'}] > | > > | status | > ERROR > | > > | tenant_id | > e4ba09fbf76c41a29218dc68cacf78d5 > | > > | updated | > 2013-03-21T16:17:12Z > | > > | user_id | > f015ea92bf4848a891b73ae2fbf6c75b > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > > > *From:*rhos-list-bounces at redhat.com > [mailto:rhos-list-bounces at redhat.com] *On Behalf Of *Minton, Rich > *Sent:* Thursday, March 21, 2013 11:45 AM > *To:* rhos-list at redhat.com > *Subject:* EXTERNAL: [rhos-list] Problems after Update. > > > > Where to begin... Basically, I'm hosed. > > > > I performed an update to the latest versions of the Openstack distro > and rebooted my hosts. Then problems started... > > * Can't launch new instances-the in progress star comes up and > stays there almost forever. > > * Several of my instances went to an error state. When I tried > to reset their state to "active" they went into shutdown/shutoff state. > These are the instances on my controller/compute node. I typed "virsh > list" and the list was empty. > > * I tried to bring up a VNC Console on one of the running > instances and after several minutes received this error in the browser: > > ---------------------------------------------------------------------- > --------------------------------------------------------------------- > > CommunicationError at > /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/ > > > > _Error communicating with http://10.10.12.245:9292 timed out_ > > > > Request Method: GET > > Request URL: > http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-b > f2770a90c82/?tab=instance_details__vnc > > Django Version: 1.4.2 > > Exception Type: CommunicationError > > Exception Value: > > > > Error communicating with http://10.10.12.245:9292 timed out > > > > Exception Location: > /usr/lib/python2.6/site-packages/glanceclient/common/http.py in > _http_request, line 145 > > Python Executable: /usr/bin/python > > Python Version: 2.6.6 > > Python Path: > > > > ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', > > '/usr/lib64/python26.zip', > > '/usr/lib64/python2.6', > > '/usr/lib64/python2.6/plat-linux2', > > '/usr/lib64/python2.6/lib-tk', > > '/usr/lib64/python2.6/lib-old', > > '/usr/lib64/python2.6/lib-dynload', > > '/usr/lib64/python2.6/site-packages', > > '/usr/lib64/python2.6/site-packages/gtk-2.0', > > '/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg', > > '/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg', > > '/usr/lib/python2.6/site-packages', > > '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', > > '/usr/share/openstack-dashboard/openstack_dashboard'] > > > > Server time: Thu, 21 Mar 2013 15:26:38 +0000 > > ---------------------------------------------------------------------- > -------------------------------------------------------------- > > * I tried deleting iptables and rebooting but did not solve > anything. > > * I'm not seeing anything out of the ordinary in the nova logs. > > > > Any ideas on where I should start looking? > > > > Thanks for any help. > > Rick > > > > _Richard Minton_ > > LMICC Systems Administrator > > 4000 Geerdes Blvd, 13D31 > > King of Prussia, PA 19406 > > Phone: 610-354-5482 > > > _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From nsavin at yahoo-inc.com Thu Mar 21 17:15:54 2013 From: nsavin at yahoo-inc.com (Nikita Savin) Date: Thu, 21 Mar 2013 17:15:54 +0000 Subject: [rhos-list] Oslo errors? In-Reply-To: References: , Message-ID: Hi, works in my test env: # cat /etc/redhat-release CentOS release 6.3 (Final) # pip freeze | grep -v github.com Babel==0.9.6 Cheetah==2.4.4 Jinja2==2.6 Mako==0.7.3 Markdown==2.2.1 MarkupSafe==0.15 MySQL-python==1.2.4 Paste==1.7.5.1 PasteDeploy==1.5.0 Pygments==1.6 Routes==1.12.3 SQLAlchemy==0.7.9 Sphinx==1.1.3 Tempita==0.5.1 WebOb==1.2.3 WebTest==1.3.3 alembic==0.4.2 amqp==1.0.9 amqplib==0.6.1 anyjson==0.2.4 argparse==1.2.1 beautifulsoup4==4.1.3 boto==2.8.0 cliff==1.3.1 cliff-tablib==1.0 cmd2==0.6.4 colorama==0.2.5 configobj==4.7.2 coverage==3.6 decorator==3.4.0 discover==0.4.0 distribute==0.6.35 docutils==0.10 eventlet==0.12.1 extras==0.0.3 feedparser==5.1.3 fixtures==0.3.12 glance==2013.1.a105.g65fbf67 greenlet==0.4.0 httplib2==0.8 importlib==1.0.2 iso8601==0.1.4 jsonpatch==0.12 jsonpointer==0.7 jsonschema==0.8.0 keyring==1.2.2 kombu==1.0.4 logilab-astng==0.24.2 logilab-common==0.59.0 lxml==3.1.0 mock==1.0.1 mox==0.5.3 netaddr==0.7.10 netifaces==0.8 nose==1.1.2 nose-exclude==0.1.9 nosehtmloutput==0.0.4 nosexcover==1.0.8 numpy==1.7.0 openstack.common==2013.1.a5.g8ede926 ordereddict==1.1 oslo.config==1.1.0 pam==0.1.4 paramiko==1.10.0 passlib==1.6.1 pep8==1.3.3 prettytable==0.6.1 psycopg2==2.4.6 pyOpenSSL==0.13 pyasn1==0.1.6 pycrypto==2.6 pyflakes==0.6.1 pylint==0.25.2 pyparsing==1.5.7 pysendfile==2.0.0 pysqlite==2.6.3 python-cinderclient==1.0.2 python-glanceclient==0.8.0 python-keystoneclient==0.2.2 python-ldap==2.3.13 python-memcached==1.48 python-novaclient==2.11.1 python-quantumclient==2.2.0 python-subunit==0.0.10 python-swiftclient==1.3.0 pyudev==0.16.1 pyzmq==2.2.0.1 qpid-python==0.20 quantum==2013.2.a234.g915599d redis==2.7.2 repoze.lru==0.6 requests==0.14.2 setuptools-git==1.0b1 simplejson==3.1.0 six==1.2.0 sqlalchemy-migrate==0.7.2 stevedore==0.8 suds==0.4 swift==1.7.7 tablib==0.9.11 termcolor==1.1.0 testrepository==0.0.14 testtools==0.9.29 unittest2==0.5.1 waitress==0.8.2 warlock==0.8.1 websockify==0.3.0 wsgiref==0.1.2 xattr==0.6.4 ________________________________________ From: Annie Cheng [anniec at yahoo-inc.com] Sent: Thursday, March 21, 2013 6:19 AM To: Joshua Harlow; Mark McLoughlin; rhos-list at redhat.com Cc: openstack-dev at yahoo-inc.com; Perry Myers; Debra Gordon; Vartika Agarwal Subject: Re: Oslo errors? + rhos-list at redhat.com Mark, let us know whether this should go to rhos-list or community upstream list, cuz it's RHEL specific question and upstream doesn't officially support RHEL .. Whatever works for you we will try to follow the protocol to get help. Josh, can you please attach the output file again? Thanks! Annie From: Joshua Harlow > Reply-To: "openstack-dev at yahoo-inc.com" >, Joshua Harlow > Date: Wednesday, March 20, 2013 6:51 PM To: "openstack-dev at yahoo-inc.com" >, Joshua Harlow >, Mark McLoughlin > Subject: Re: Oslo errors? And system version? $ lsb_release -a LSB Version: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 6.3 (Santiago) Release: 6.3 Codename: Santiago And dependency list? $ pip-python freeze | grep -v github.com Babel==0.9.6 Cheetah==2.4.4 IPy==0.75 Jinja2==2.6 M2Crypto==0.20.2 Magic-file-extensions==0.1 Mako==0.7.3 Markdown==2.0.1 MarkupSafe==0.15 MySQL-python==1.2.3c1 Paste==1.7.4 PasteDeploy==1.5.0 PyYAML==3.10 Pygments==1.6 Routes==1.12.3 SQLAlchemy==0.7.9 Sphinx==1.1.3 Tempita==0.5.1 WebOb==1.2.3 WebTest==1.4.3 alembic==0.4.2 amqplib==0.6.1 anyjson==0.2.4 argparse==1.2.1 boto==2.1.1 bouncer==0.1.1 cas==0.15 cliff==1.3 cliff-tablib==1.0 cloud-init==0.7.2 cmd2==0.6.4 colorama==0.2.4 configobj==4.6.0 coverage==3.6 decorator==3.4.0 discover==0.4.0 distribute==0.6.35 docutils==0.10 egenix-mx-base==3.1.1 euca2ools==2.1.1 eventlet==0.12.1 extras==0.0.3 feedparser==5.0.1 fixtures==0.3.12 greenlet==0.4.0 hgtools==2.0.2 httplib2==0.7.7 httpretty==0.5.6 iniparse==0.4 iotop==0.3.2 iso8601==0.1.4 jsonschema==0.2 keyring==1.2 keystone==2012.2.4 kombu==1.0.4 lockfile==0.8 logilab-astng==0.20.1 logilab-common==0.50.3 lxml==2.3.5 mock==0.8.0 monkeypatch==0.1rc3 mox==0.5.3 netaddr==0.7.5 netifaces==0.5 netsnmp-python==1.0a1 nose==1.2.1 nose-exclude==0.1.9 nosehtmloutput==0.0.4 nosexcover==1.0.7 oauth==1.0.1 openstack.nose-plugin==0.11 ordereddict==1.1 oslo.config==1.1.0 pam==0.1.4 paramiko==1.7.5 passlib==1.6.1 pep8==1.3.3 perf==0.1 pretty==0.1 prettytable==0.6.1 progressbar==2.2 psutil==0.6.1 pyOpenSSL==0.10 pycrypto==2.6 pycurl==7.19.0 pyflakes==0.6.1 pygpgme==0.1 pylint==0.21.1 pyparsing==1.5.6 pysendfile==2.0.0 python-daemon==1.5.5 python-dateutil==1.4.1 python-distutils-extra==2.32 python-dmidecode==3.10.13 python-subunit==0.0.9 pyudev==0.16.1 pyzmq==2.2.0.1 qpid-python==0.20 raidctl==0.0.7 redis==2.7.2 requests==0.14.2 simplejson==2.0.9 sqlalchemy-migrate==0.7.2 stevedore==0.8 suds==0.4 tablib==0.9.11 termcolor==1.1.0 testrepository==0.0.13 testtools==0.9.29 unittest2==0.5.1 urlgrabber==3.9.1 warlock==0.1.0 wsgiref==0.1.2 xattr==0.6.4 yum-metadata-parser==1.1.2 yum-presto==0.4.4 From: Joshua Harlow > Reply-To: openstack-dev >, Joshua Harlow > Date: Wednesday, March 20, 2013 6:44 PM To: Mark McLoughlin > Cc: openstack-dev > Subject: Re: Oslo errors? +openstack-dev (at y!) From: Joshua Harlow > Date: Wednesday, March 20, 2013 6:37 PM To: Mark McLoughlin > Subject: Oslo errors? Hi mark, I am just trying to run nosetests in the oslo-incubator code on RHEL6.3. How are u running these tests, any attempt in RHEL6.X? Shouldn't CI be catching these? Attached is a big output with some of these? -Josh From rbryant at redhat.com Thu Mar 21 18:06:59 2013 From: rbryant at redhat.com (Russell Bryant) Date: Thu, 21 Mar 2013 14:06:59 -0400 Subject: [rhos-list] Oslo errors? In-Reply-To: References: , Message-ID: <514B4C43.6080606@redhat.com> On 03/21/2013 01:15 PM, Nikita Savin wrote: > Hi, > > works in my test env: Can you guys clarify how you're trying to run the unit tests? There are different ways to do it and they don't all work the same way. -- Russell Bryant From rbryant at redhat.com Thu Mar 21 18:07:41 2013 From: rbryant at redhat.com (Russell Bryant) Date: Thu, 21 Mar 2013 14:07:41 -0400 Subject: [rhos-list] Oslo errors? In-Reply-To: References: , Message-ID: <514B4C6D.1050704@redhat.com> On 03/21/2013 01:15 PM, Nikita Savin wrote: > Hi, > > works in my test env: Can you guys clarify how you're trying to run the unit tests? There are different ways to do it and they don't all work the same way. -- Russell Bryant From rich.minton at lmco.com Thu Mar 21 22:13:53 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Thu, 21 Mar 2013 22:13:53 +0000 Subject: [rhos-list] SSH Key Injection not working. Message-ID: I'm having a problem with injecting keys through the metadata service (actually nova-api). The keys are not being written to the authorized_keys file. Actually it doesn't look like they are available from the metadata api. When the instance is booting I get this error: "cc_ssh.py[WARNING]: applying credentials failed!" This is the from api.log. Notice "404" when trying get the keys. 2013-03-21 17:54:58 16357 INFO nova.api.ec2 [-] 0.388398s 10.10.16.17 GET /latest/metadata/public-keys/0/openssh-key None:None 404 [curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2] text/plain text/html 2013-03-21 17:54:58 16357 INFO nova.metadata.wsgi.server [-] 10.10.16.17 - - [21/Mar/2013 17:54:58] "GET /latest/metadata/public-keys/0/openssh-key HTTP/1.1" 404 278 0.389426 I'm not running the metadata-api, only nova-api on each of three nodes. Nova.conf related to meta-data config is as follows: Controller/Compute 1 enabled_apis = ec2,osapi_compute,metadata metadata_host = 10.10.12.245 metadata_port = 8775 Compute 2 enabled_apis = ec2,osapi_compute,metadata metadata_host = 10.10.12.246 metadata_port = 8775 Compute 3 enabled_apis = ec2,osapi_compute,metadata metadata_host = 10.10.12.247 metadata_port = 8775 While logged into the VM, I can run http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and the ssh key is returned. So I guess the metadata service is working properly. Do I need to have the .ssh directory and the authorized_keys file already in place and with the correct permissions? Thanks, Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicolas.vogel at heig-vd.ch Fri Mar 22 09:29:14 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Fri, 22 Mar 2013 09:29:14 +0000 Subject: [rhos-list] unable to delete an instance In-Reply-To: <514B3915.7050609@redhat.com> References: <514B3915.7050609@redhat.com> Message-ID: Here are some logs which could help: Compute.log : 2013-03-22 09:32:08 2343 WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor. 2013-03-22 09:32:08 2343 INFO nova.compute.manager [-] [instance: 273c7a2b-a27c-42ef-9cdb-dbe5d592b194] During sync_power_state the instance has a pending task. Skip. ... 2013-03-22 09:34:19 2343 INFO nova.compute.manager [-] Updating host status 2013-03-22 09:34:55 AUDIT nova.compute.manager [req-8c181208-125d-487b-8933-edf44c8088bf fdfcf45f412a46e4880e6d8bd8143b61 bc308f566c2b46aea7812a033de1f21c] [instance: 273c7a2b-a27c-42ef-9cdb-dbe5d592b194] Rebooting instance 2013-03-22 09:35:00 2343 ERROR nova.virt.libvirt.driver [-] [instance: 273c7a2b-a27c-42ef-9cdb-dbe5d592b194] During wait destroy, instance disappeared. 2013-03-22 09:35:00 INFO nova.virt.libvirt.firewall [req-8c181208-125d-487b-8933-edf44c8088bf fdfcf45f412a46e4880e6d8bd8143b61 bc308f566c2b46aea7812a033de1f21c] [instance: 273c7a2b-a27c-42ef-9cdb-dbe5d592b194] Called setup_basic_filtering in nwfilter 2013-03-22 09:35:00 INFO nova.virt.libvirt.firewall [req-8c181208-125d-487b-8933-edf44c8088bf fdfcf45f412a46e4880e6d8bd8143b61 bc308f566c2b46aea7812a033de1f21c] [instance: 273c7a2b-a27c-42ef-9cdb-dbe5d592b194] Ensuring static filters 2013-03-22 09:35:04 ERROR nova.compute.manager [req-8c181208-125d-487b-8933-edf44c8088bf fdfcf45f412a46e4880e6d8bd8143b61 bc308f566c2b46aea7812a033de1f21c] [instance: 273c7a2b-a27c-42ef-9cdb-dbe5d592b194] Cannot reboot instance: Unable to pre-create chardev file '/var/lib/nova/instances/instance-0000001f/console.log': Aucun fichier ou dossier de ce type I also have the ERROR "nova [-] need more than 0 values to unpack" or "cinder [-] need more than 0 values to unpack". And the cinder service is unavailable: [root at IICT-SV1259 ~(keystone_admin)]$ cinder list ERROR: n/a (HTTP 400) Is that a problem from keystone? -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: jeudi 21 mars 2013 17:45 To: Vogel Nicolas Cc: 'rhos-list at redhat.com' Subject: Re: [rhos-list] unable to delete an instance On 03/21/2013 12:00 PM, Vogel Nicolas wrote: > Hi, > > > > I had an instance who couldn?t terminate properly so I used the > command ?nova reset-state ?active ? to modify her status to > Active. It works but after that, using nova delete , my instance > is still standing in the ?Deleting? Task. > > Can I delete this instance direct in the database? What is the way to > manually terminate an instance? We need to go back to figuring out why the instance couldn't be deleted originally. Do you know how you got it into that state? If all else fails, the brute force is manually removing the instance from the database and the hypervisor, but you should never have to do that. > This instance had a volume attached to her so I think I will have to > manually detach and delete this volume too. But command nova > volume-list is giving me ?ERROR: ConnectionRefused: '[Errno 111] Connection refused'?. Do other nova commands give you that error? Can you provide the full command and output? -- Russell Bryant From k.mohammad1 at physics.ox.ac.uk Fri Mar 22 10:19:59 2013 From: k.mohammad1 at physics.ox.ac.uk (Kashif Mohammad) Date: Fri, 22 Mar 2013 10:19:59 +0000 Subject: [rhos-list] Nova-network woes (Vnc issue now) In-Reply-To: <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> Message-ID: <88B17E26E0A9F94381C67535AEF2BB5E7990FA29@EXCHNG13.physics.ox.ac.uk> Hi As I mentioned earlier that now everything is working except vnc access. I tried many options but nothing seems to be working. One suspicious thing is that nova-vncproxy is running without nova.conf file nova 2170 0.0 0.1 236268 17148 ? S Mar21 0:01 python /usr/bin/nova-novncproxy --web /usr/share/novnc/ is that expected behaviour ? Looking at /etc/init.d/openstack-nova-novncproxy it seems to be expected behaviour daemon --user nova --pidfile $pidfile "$exec --web /usr/share/novnc/ &>/dev/null & echo \$! > $pidfile" Can any one confirm this please. I am using openstack-nova-novncproxy-0.4-2.el6.noarch from folsom redhat repo. Cheers Kashif -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Kashif Mohammad Sent: 21 March 2013 15:47 To: Brent Eagles Cc: rhos-list at redhat.com Subject: Re: [rhos-list] Nova-network woes Hi Brent Thanks for looking into it. Fixed one of the main issue so my vm can get access to internet. I have one public and one private ip on my compute node and have defined private ip as my_ip in nova.conf at compute node. Looking more deeply I found that routing_source_ip defaults to $my_ip. Explicitly defined routing_source_ip to public ip address and it started working. Nova network is now creating correct iptables rule so vm can access internet. >> iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill Found the solution here https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/issues/14 But still my vnc not working. Running openstack-nova-consoleauth, openstack-nova-novncproxy openstack-nova-xvpvncproxy on controller node. I have this configuration in nova.conf at controller node novnc_enabled = true vnc_enabled = true novncproxy_base_url = http://163.1.5.243:6080/vnc_auto.html xvpvncproxy_base_url = http://163.1.5.243:6081/console On Compute node vnc_enabled = true novnc_enabled = true novncproxy_base_url = http://163.1.5.243:6080/vnc_auto.html xvpvncproxy_base_url = http://163.1.5.243:6081/console vncserver_proxyclient_address = 192.168.9.3 vncserver_listen = 0.0.0.0 I can get vnc url using get-vnc-console so probably authentication bit is working but when I try to access vnc console through browser it just give this error Failed to connect to server (code: 1006) Get same error if try to connect through horizon dashboard. I will very much appreciate your help. Cheers Kashif -----Original Message----- From: Brent Eagles [mailto:beagles at redhat.com] Sent: 21 March 2013 15:20 To: Kashif Mohammad Cc: rhos-list at redhat.com Subject: Re: [rhos-list] Nova-network woes Hi Kashif, On Wed, Mar 20, 2013 at 05:23:16PM +0000, Kashif Mohammad wrote: > > Hi Guys > > I am using rhel6.4 and RedHat Openstack Folsom rpms. I have one > controller node which is running glance, cinder, scheduler, api > server, consoleauth and other node is running compute, network and > api. Both nodes have one public and one private IP and running in > multi_host mode. > > I am not running nova-network and nova-compute on controller node. > Network bit is like this on compute node > > public_interface = em1 > flat_interface = em2 > fixed_range = 10.0.1.0/24 > flat_network_dhcp_start = 10.0.1.5 > connection_type = libvirt > flat_injected = False > multi_host = True > flat_network_bridge = br100 > > created network like this > nova-manage network create private 10.0.1.0/24 1 256 --bridge=br100 --multi_host=True I would try running nova-manage using the keyword=value approach instead of relying on positional arguments (e.g. --fixed_range_ipv4=10.0.1.0/24). It would avoid issues with getting the order wrong which can lead to surprising results. > I can create VM through dashboard and initially it couldn't get ip > address from dhcp server running on compute node but after adding > > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill Mmmm.. if everything is configured properly, you shouldn't have to modify the firewall rules for this to work. Was there something you came across while debugging your setup that gave you the idea to add this? > it is getting ip address and I can log in into vm from compute node. > It can reach metadata server and grab public keys. My problem is that > VM's can not access outside word. I think that issue is in this line > of iptable > > 47 3247 nova-network-float-snat all -- any any anywhere anywhere > 2 168 SNAT all -- any em1 10.0.1.0/24 anywhere to:192.168.9.3 > > It is changing source IP address to local ip rather than external ip. If insert a rule > > iptables -t nat -I nova-network-snat 2 -o em1 -j SNAT --to-source 163.1.5.40 That's weird. Can you describe how you configured and allocated the floating IPs? This is another thing that generally just works. > Then it vm can access interner but the moment I create a new VM this line disappear from iptable. > > My n/w setting is like this > > 2: em1: mtu 1500 qdisc mq state UP qlen 1000 > link/ether 00:22:19:6a:bc:09 brd ff:ff:ff:ff:ff:ff > inet 163.1.5.40/24 brd 163.1.5.255 scope global em1 > inet6 fe80::222:19ff:fe6a:bc09/64 scope link > valid_lft forever preferred_lft forever > 3: em2: mtu 1500 qdisc mq state UP qlen 1000 > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet6 fe80::222:19ff:fe6a:bc0b/64 scope link > valid_lft forever preferred_lft forever > 5: br100: mtu 1500 qdisc noqueue state UNKNOWN > link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff > inet 10.0.1.1/24 brd 10.0.1.255 scope global br100 > inet 192.168.9.3/24 brd 192.168.9.255 scope global br100 > inet6 fe80::80cd:6fff:fef7:d955/64 scope link > valid_lft forever preferred_lft forever > > I can not reach vnc as well but I first fix this n/w issue and then look into vnc. > > Thanks > Kashif Heh. No VNC? It is the one thing you should *not* live without! I would say that something got crossed during configuration. You might want to "back up" to the beginning and try each thing in turn. If the "cookbook" type instructions didn't work as described, it might be best to figure out why instead of tweaking until things just kind of work. As one bit of configuration tends to depend on another, if the most fundamental thing does not work properly (e.g. allocating a private IP to a VM) it is very possible that things are going to go awry with other configuration items (e.g. allocating floating IPs). However, instead of just leaving with you a note to "try again", and it is just a thought/guess, but if iptables is giving you grief and you have a pre-existing firewall configuration, you might want to look for rules that might interfere with what openstack is trying to do. What I have done in the past when trying a new configuration is take a snapshot of the current configuration with iptables-save (e.g. iptables-save > iptables.before) and again when everything is "done" (iptables.after?) and visually compare the two. It is useful pedagogically because I see what Openstack is doing and it helps identify configuration (and sometimes bugs!) before I go too far. Cheers, Brent _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From rbryant at redhat.com Fri Mar 22 10:31:55 2013 From: rbryant at redhat.com (Russell Bryant) Date: Fri, 22 Mar 2013 06:31:55 -0400 Subject: [rhos-list] SSH Key Injection not working. In-Reply-To: References: Message-ID: <514C331B.2010406@redhat.com> On 03/21/2013 06:13 PM, Minton, Rich wrote: > While logged into the VM, I can run > http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and > the ssh key is returned. So I guess the metadata service is working > properly. Do I need to have the .ssh directory and the authorized_keys > file already in place and with the correct permissions? If you were able to verify that the metadata server returns the SSH key, then the problem seems to be with however the instance is trying to get it (as in, an issue with the image, not OpenStack). -- Russell Bryant From nicolas.vogel at heig-vd.ch Fri Mar 22 10:33:32 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Fri, 22 Mar 2013 10:33:32 +0000 Subject: [rhos-list] unable to delete an instance In-Reply-To: <514B3915.7050609@redhat.com> References: <514B3915.7050609@redhat.com> Message-ID: It's OK I could fix it! It was because I added an endpoint in the keystone endpoint-list. I think I didn't really understand how to configure networking services and a compute node. I will continue my researches. Thanks for helping! Cheers, Nicolas -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: jeudi 21 mars 2013 17:45 To: Vogel Nicolas Cc: 'rhos-list at redhat.com' Subject: Re: [rhos-list] unable to delete an instance On 03/21/2013 12:00 PM, Vogel Nicolas wrote: > Hi, > > > > I had an instance who couldn?t terminate properly so I used the > command ?nova reset-state ?active ? to modify her status to > Active. It works but after that, using nova delete , my instance > is still standing in the ?Deleting? Task. > > Can I delete this instance direct in the database? What is the way to > manually terminate an instance? We need to go back to figuring out why the instance couldn't be deleted originally. Do you know how you got it into that state? If all else fails, the brute force is manually removing the instance from the database and the hypervisor, but you should never have to do that. > This instance had a volume attached to her so I think I will have to > manually detach and delete this volume too. But command nova > volume-list is giving me ?ERROR: ConnectionRefused: '[Errno 111] Connection refused'?. Do other nova commands give you that error? Can you provide the full command and output? -- Russell Bryant From rbryant at redhat.com Fri Mar 22 12:55:15 2013 From: rbryant at redhat.com (Russell Bryant) Date: Fri, 22 Mar 2013 08:55:15 -0400 Subject: [rhos-list] unable to delete an instance In-Reply-To: References: <514B3915.7050609@redhat.com> Message-ID: <514C54B3.1020006@redhat.com> On 03/22/2013 06:33 AM, Vogel Nicolas wrote: > It's OK I could fix it! > > It was because I added an endpoint in the keystone endpoint-list. > I think I didn't really understand how to configure networking services and a compute node. I will continue my researches. > Thanks for helping! Great, I'm glad you got it figured out! -- Russell Bryant From tbrunell at redhat.com Fri Mar 22 13:48:07 2013 From: tbrunell at redhat.com (Ted Brunell) Date: Fri, 22 Mar 2013 09:48:07 -0400 (EDT) Subject: [rhos-list] Quantum or Nova in RHOS In-Reply-To: <311039439.12999388.1363959857622.JavaMail.root@redhat.com> Message-ID: <253945448.13000734.1363960087974.JavaMail.root@redhat.com> Hello, I just have a couple of quick questions about the network stack that will be included in RHOS when it comes out of preview. When the RHOS is released, will it include Quantum + Open vSwitch or will nova-network be the preferred network stack? The current preview docs seem to indicate that Quantum is the route that Red Hat is going. If nova is used, will there be an upgrade path to Quantum or will it require a large amount of re-engineering of deployed RHOS systems to get it working? Are there any limitations with Quantum right now that make it not on par with nova-network? Thanks, Ted From rbryant at redhat.com Fri Mar 22 14:54:07 2013 From: rbryant at redhat.com (Russell Bryant) Date: Fri, 22 Mar 2013 10:54:07 -0400 Subject: [rhos-list] Nova-network woes (Vnc issue now) In-Reply-To: <88B17E26E0A9F94381C67535AEF2BB5E7990FA29@EXCHNG13.physics.ox.ac.uk> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> <88B17E26E0A9F94381C67535AEF2BB5E7990FA29@EXCHNG13.physics.ox.ac.uk> Message-ID: <514C708F.4070406@redhat.com> On 03/22/2013 06:19 AM, Kashif Mohammad wrote: > > Hi > > As I mentioned earlier that now everything is working except vnc access. I tried many options but nothing seems to be working. One suspicious thing is that nova-vncproxy is running without nova.conf file > > nova 2170 0.0 0.1 236268 17148 ? S Mar21 0:01 python /usr/bin/nova-novncproxy --web /usr/share/novnc/ > > is that expected behaviour ? Looking at /etc/init.d/openstack-nova-novncproxy it seems to be expected behaviour > > daemon --user nova --pidfile $pidfile "$exec --web /usr/share/novnc/ &>/dev/null & echo \$! > $pidfile" Yes, that is fine. Your configuration looks sane. Can you describe exactly what happens when it fails? Does the VNC client load in the browser, and then you get a VNC connection error? Or does the browser fail to connect to anything at all? -- Russell Bryant From k.mohammad1 at physics.ox.ac.uk Fri Mar 22 15:48:04 2013 From: k.mohammad1 at physics.ox.ac.uk (Kashif Mohammad) Date: Fri, 22 Mar 2013 15:48:04 +0000 Subject: [rhos-list] Nova-network woes (Vnc issue now) In-Reply-To: <514C708F.4070406@redhat.com> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> <88B17E26E0A9F94381C67535AEF2BB5E7990FA29@EXCHNG13.physics.ox.ac.uk> <514C708F.4070406@redhat.com> Message-ID: <88B17E26E0A9F94381C67535AEF2BB5E79910FBA@EXCHNG13.physics.ox.ac.uk> Hi Russell >> does the browser fail to connect to anything at all? Yes, it fails to connect to anything at all. It fails with this error Failed to connect to server (code: 1006) I can get a token through get-vnc-console. What I understand that request is coming on public address on controller node and vnc is running on private address on a different compute node. Does nova suppose to add a route for this? Looking at route table route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 163.1.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.9.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1 0.0.0.0 163.1.5.254 0.0.0.0 UG 0 0 0 eth0 VM is running on 192.168.9.3 with this parameters > Hi > > As I mentioned earlier that now everything is working except vnc access. I tried many options but nothing seems to be working. One suspicious thing is that nova-vncproxy is running without nova.conf file > > nova 2170 0.0 0.1 236268 17148 ? S Mar21 0:01 python /usr/bin/nova-novncproxy --web /usr/share/novnc/ > > is that expected behaviour ? Looking at /etc/init.d/openstack-nova-novncproxy it seems to be expected behaviour > > daemon --user nova --pidfile $pidfile "$exec --web /usr/share/novnc/ &>/dev/null & echo \$! > $pidfile" Yes, that is fine. Your configuration looks sane. Can you describe exactly what happens when it fails? Does the VNC client load in the browser, and then you get a VNC connection error? Or does the browser fail to connect to anything at all? -- Russell Bryant _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From rich.minton at lmco.com Fri Mar 22 16:33:59 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 22 Mar 2013 16:33:59 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: <514C331B.2010406@redhat.com> References: <514C331B.2010406@redhat.com> Message-ID: It looks like my problem was with the older cloud-init package. I updated to the latest Red Hat version of cloud-init and now everything works great. Thanks, Rick -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Russell Bryant Sent: Friday, March 22, 2013 6:32 AM To: rhos-list at redhat.com Subject: EXTERNAL: Re: [rhos-list] SSH Key Injection not working. On 03/21/2013 06:13 PM, Minton, Rich wrote: > While logged into the VM, I can run > http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and > the ssh key is returned. So I guess the metadata service is working > properly. Do I need to have the .ssh directory and the authorized_keys > file already in place and with the correct permissions? If you were able to verify that the metadata server returns the SSH key, then the problem seems to be with however the instance is trying to get it (as in, an issue with the image, not OpenStack). -- Russell Bryant _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From rbryant at redhat.com Fri Mar 22 17:06:17 2013 From: rbryant at redhat.com (Russell Bryant) Date: Fri, 22 Mar 2013 13:06:17 -0400 Subject: [rhos-list] Nova-network woes (Vnc issue now) In-Reply-To: <88B17E26E0A9F94381C67535AEF2BB5E79910FBA@EXCHNG13.physics.ox.ac.uk> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> <88B17E26E0A9F94381C67535AEF2BB5E7990FA29@EXCHNG13.physics.ox.ac.uk> <514C708F.4070406@redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E79910FBA@EXCHNG13.physics.ox.ac.uk> Message-ID: <514C8F89.5050404@redhat.com> On 03/22/2013 11:48 AM, Kashif Mohammad wrote: > Hi Russell > >>> does the browser fail to connect to anything at all? > > Yes, it fails to connect to anything at all. It fails with this error > > Failed to connect to server (code: 1006) Actually, it is connecting to the VNC proxy. That's what I was trying to get at. That message is inside of the novnc HTML5 vnc viewer. So the VNC proxy is failing to make a VNC connection to your instance. > I can get a token through get-vnc-console. What I understand that request is coming on public address on controller node and vnc is running on private address on a different compute node. Does nova suppose to add a route for this? Looking at route table Your browser connects to the VNC proxy successfully via the public IP address. The proxy should then connect to the VNC server for the instance. The address used to connect to the VNC server internally to nova is from the vncserver_proxyclient_address option, which you have set to 192.168.9.3 on your compute node. Check to see if the VNC connection is getting blocked by iptables on your compute node. -- Russell Bryant From harlowja at yahoo-inc.com Fri Mar 22 17:13:13 2013 From: harlowja at yahoo-inc.com (Joshua Harlow) Date: Fri, 22 Mar 2013 10:13:13 -0700 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: Message-ID: If u need more help. I am the second most active dev (and added all the RH stuffs) in cloud-init. Here to serve :-P -Josh On 3/22/13 9:33 AM, "Minton, Rich" wrote: >It looks like my problem was with the older cloud-init package. I updated >to the latest Red Hat version of cloud-init and now everything works >great. > >Thanks, >Rick > >-----Original Message----- >From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] >On Behalf Of Russell Bryant >Sent: Friday, March 22, 2013 6:32 AM >To: rhos-list at redhat.com >Subject: EXTERNAL: Re: [rhos-list] SSH Key Injection not working. > >On 03/21/2013 06:13 PM, Minton, Rich wrote: >> While logged into the VM, I can run >> http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and >> the ssh key is returned. So I guess the metadata service is working >> properly. Do I need to have the .ssh directory and the authorized_keys >> file already in place and with the correct permissions? > >If you were able to verify that the metadata server returns the SSH key, >then the problem seems to be with however the instance is trying to get >it (as in, an issue with the image, not OpenStack). > >-- >Russell Bryant > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list From rich.minton at lmco.com Fri Mar 22 17:40:29 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 22 Mar 2013 17:40:29 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: Message-ID: Super. It looks like I'm still having an issue with ssh keys. Cloud-init doesn't seem to insert the key into "authorized_keys". Here is the output of the boot.log. Starting cloud-init: Cloud-init v. 0.7.1 running 'init-local' at Fri, 22 Mar 2013 17:21:51 +0000. Up 66.13 seconds. Starting cloud-init: Cloud-init v. 0.7.1 running 'init' at Fri, 22 Mar 2013 17:21:53 +0000. Up 67.51 seconds. ci-info: ++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++ ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: | Device | Up | Address | Mask | Hw-Address | ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | ci-info: | eth0 | True | 10.10.16.13 | 255.255.255.0 | fa:16:3e:43:6d:4d | ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: +++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++ ci-info: +-------+-------------+------------+---------------+-----------+-------+ ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags | ci-info: +-------+-------------+------------+---------------+-----------+-------+ ci-info: | 0 | 10.10.16.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U | ci-info: | 1 | 169.254.0.0 | 0.0.0.0 | 255.255.0.0 | eth0 | U | ci-info: | 2 | 0.0.0.0 | 10.10.16.7 | 0.0.0.0 | eth0 | UG | ci-info: +-------+-------------+------------+---------------+-----------+-------+ 2013-03-22 13:21:56,519 - util.py[WARNING]: Applying ssh credentials failed! Starting cloud-init: Cloud-init v. 0.7.1 running 'modules:config' at Fri, 22 Mar 2013 17:21:57 +0000. Up 71.30 seconds. Starting cloud-init: Cloud-init v. 0.7.1 running 'modules:final' at Fri, 22 Mar 2013 17:21:58 +0000. Up 72.85 seconds. 2013-03-22 13:21:59,147 - util.py[WARNING]: Running ssh-authkey-fingerprints () failed ec2: ec2: ############################################################# ec2: -----BEGIN SSH HOST KEY FINGERPRINTS----- ec2: 1024 de:0d:95:89:10:b3:f8:a5:12:09:62:87:58:f5:52:35 /etc/ssh/ssh_host_dsa_key.pub (DSA) ec2: 2048 48:d3:2c:ac:98:4e:57:dc:07:8a:4e:c8:0c:68:56:35 /etc/ssh/ssh_host_key.pub (RSA1) ec2: 2048 97:62:2b:dd:09:3c:5d:c6:30:7e:8a:16:89:27:82:6a /etc/ssh/ssh_host_rsa_key.pub (RSA) ec2: -----END SSH HOST KEY FINGERPRINTS----- ec2: ############################################################# -----BEGIN SSH HOST KEY KEYS----- 2048 35 21889849139367763563389351601377820796073793617449436599221977659445135002476223129833833651351484232076837788805006750680052997128685757303892593965137923198447956864014079923478870309110547046664268287380298838332284048302681336615302514348725862981947986908242642891955318438217141018134624513598829945193978261667926153861849588173917651884170461212835461964580843245838698025157376233972830294897970640494259212138531890829666635458094297349958505817038457555402332013792018319388616117427759769435072425962956358797493569252499044371829936847980644362073198927847473883205821775406782358666256423235107872534147 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAs/IUMJ56v2OLNGNKiooyV0cY4gITD56Km64WL3RtHc9TrHGOXOcxNJHRsHS0JFkxPY9PXNlIuoqF6/FxTmPwW1KeqDigT4/a9Ho5WSaVEmRSdID2fPtBawZeUlZQH3KHMIcCGYkixQpb8o5JNNn0LNUEI1N7YkbnliK2zuewWwMoXYUqKvqi6LRlov6HFRP9EbyrZYVFWCpbRy6ZlP781VF3QR7PdAMl/IZWZmG7uMuRGCqfhUEJVYMFxLgShL/1QE6/eW7jc0tcHBaPo5CZReMQy3kE1TO9XltCTLsWYbhc1Sn7O1BaqIC3W8mOfEz39QKQ/tBMpgOcRK88Xtbz2Q== -----END SSH HOST KEY KEYS----- Cloud-init v. 0.7.1 finished at Fri, 22 Mar 2013 17:21:59 +0000. Datasource DataSourceEc2. Up 73.49 seconds Starting ntpd: [ OK ] NetBackup Authentication daemon not started. NetBackup network daemon started. NetBackup client daemon started. NetBackup SAN Client Fibre Transport daemon started. NetBackup Bare Metal Restore Boot Server daemon started. Starting postfix: [ OK ] Starting abrt daemon: [ OK ] Starting crond: [ OK ] Starting McAfee Agent services... [ OK ] Starting SMB services: [ OK ] Starting atd: [ OK ] Starting Red Hat Network Daemon: [ OK ] Starting rhsmcertd...[ OK ] Starting certmonger: [ OK ] AUTHORIZED_KEYS: ******************************** ******************************** [root at uvslv-diu04dv log]# -----Original Message----- From: Joshua Harlow [mailto:harlowja at yahoo-inc.com] Sent: Friday, March 22, 2013 1:13 PM To: Minton, Rich; Russell Bryant; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. If u need more help. I am the second most active dev (and added all the RH stuffs) in cloud-init. Here to serve :-P -Josh On 3/22/13 9:33 AM, "Minton, Rich" > wrote: >It looks like my problem was with the older cloud-init package. I >updated to the latest Red Hat version of cloud-init and now everything >works great. > >Thanks, >Rick > >-----Original Message----- >From: rhos-list-bounces at redhat.com >[mailto:rhos-list-bounces at redhat.com] >On Behalf Of Russell Bryant >Sent: Friday, March 22, 2013 6:32 AM >To: rhos-list at redhat.com >Subject: EXTERNAL: Re: [rhos-list] SSH Key Injection not working. > >On 03/21/2013 06:13 PM, Minton, Rich wrote: >> While logged into the VM, I can run >> http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and >> the ssh key is returned. So I guess the metadata service is working >> properly. Do I need to have the .ssh directory and the >> authorized_keys file already in place and with the correct permissions? > >If you were able to verify that the metadata server returns the SSH >key, then the problem seems to be with however the instance is trying >to get it (as in, an issue with the image, not OpenStack). > >-- >Russell Bryant > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.minton at lmco.com Fri Mar 22 18:00:18 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 22 Mar 2013 18:00:18 +0000 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: References: <514B3852.1080807@redhat.com> Message-ID: I just updated another cluster with the new openstack releases and after the update my glance-api-paste.ini nad glance-registry-paste.ini files are 0 bytes. I thought it would have created a save file. This might be a bug. -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-api-paste.ini -rw-r-----. 1 glance glance 5408 Mar 5 17:52 glance-cache.conf -rw-r-----. 1 glance glance 3117 Mar 5 17:52 glance-registry.conf -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-registry-paste.ini -rw-r----- 1 root glance 1091 Feb 27 13:16 glance-scrubber.conf -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Minton, Rich Sent: Thursday, March 21, 2013 1:17 PM To: Russell Bryant Cc: rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: Problems after Update. Found it! The NFS mount to my instances directory was missing. I remounted the directory and I can see all of my instances now. I have one other issue with metadata and ssh keys but I'll create a new thread for that. Thanks for the help. -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Minton, Rich Sent: Thursday, March 21, 2013 1:03 PM To: Russell Bryant Cc: rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: Problems after Update. Here is some of what I found in "compute.log" and it doesn't look good. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] Found 5 in the database and 0 on the hypervisor. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: 2d87a89a-fcbb-4581-a7c3-0e2b2c30b568] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: b7e64392-1f28-446f-8fb3-2b75dc895951] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 INFO nova.virt.libvirt.driver [-] [instance: 2d87a89a-fcbb-4581-a7c3-0e2b2c30b568] Instance destroyed successfully. 2013-03-21 12:15:21 42682 WARNING nova.compute.manager [-] [instance: 40153f6e-e963-4a30-bb0a-3f7e612d5638] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:21 42682 INFO nova.virt.libvirt.driver [-] [instance: b7e64392-1f28-446f-8fb3-2b75dc895951] Instance destroyed successfully. 2013-03-21 12:15:22 42682 WARNING nova.compute.manager [-] [instance: a39787b8-2c66-4aaa-b154-42ec05c0e3c8] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:22 42682 INFO nova.virt.libvirt.driver [-] [instance: 40153f6e-e963-4a30-bb0a-3f7e612d5638] Instance destroyed successfully. 2013-03-21 12:15:22 42682 WARNING nova.compute.manager [-] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Instance shutdown by itself. Calling the stop API. 2013-03-21 12:15:22 42682 INFO nova.virt.libvirt.driver [-] [instance: a39787b8-2c66-4aaa-b154-42ec05c0e3c8] Instance destroyed successfully. 2013-03-21 12:15:23 42682 INFO nova.virt.libvirt.driver [-] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Instance destroyed successfully. 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free ram (MB): 237303 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free disk (GB): -14 2013-03-21 12:15:23 42682 AUDIT nova.compute.resource_tracker [-] Free VCPUS: 38 2013-03-21 12:15:23 42682 INFO nova.compute.resource_tracker [-] Compute_service record updated for uvslp-diu01os.lmdit.us.lmco.com 2013-03-21 12:16:02 AUDIT nova.compute.manager [req-a00e05db-32f4-4be0-8885-e789e8474b4f f015ea92bf4848a891b73ae2fbf6c75b e4ba09fbf76c41a29218dc68cacf78d5] [instance: be67a3ef-036b-4df8-90dd-04cbd0ffafdb] Get console output 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp temp_level, payload) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 196, in decorated_function 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp kwargs['instance']['uuid'], e, sys.exc_info()) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 190, in decorated_function 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1931, in get_console_output 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp output = self.driver.get_console_output(instance) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp temp_level, payload) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp self.gen.next() 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1150, in get_console_output 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp libvirt_utils.chown(path, os.getuid()) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 332, in chown 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp execute('chown', owner, path, run_as_root=True) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 53, in execute 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp return utils.execute(*args, **kwargs) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd)) 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command. 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Exit code: 1 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Stdout: '' 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp Stderr: "/bin/chown: cannot access `/var/lib/nova/instances/instance-000000f2/console.log': No such file or directory\n" 2013-03-21 12:16:02 42682 TRACE nova.openstack.common.rpc.amqp 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.common [-] Returning exception Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log Exit code: 1 Stdout: '' Stderr: "/bin/chown: cannot access `/var/lib/nova/instances/instance-000000f2/console.log': No such file or directory\n" to caller 2013-03-21 12:16:02 42682 ERROR nova.openstack.common.rpc.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', ' F ile "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python 2.6/site-packages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 196, in decorated_function\n kwargs[\'instance\'][\'uuid\'], e, sys.exc_info())\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 190, in decorated_function\n retu rn function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1931, in get_console_output\n output = self.driver.get_console_output(instance)\n', ' File "/usr/lib /python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-pack ages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1150, in get_console_output\n libvirt_utils.chown(path, os.getuid())\ n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 332, in chown\n execute(\'chown\', owner, path, run_as_root=True)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", l ine 53, in execute\n return utils.execute(*args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/utils.py", line 206, in execute\n cmd=\' \'.join(cmd))\n', 'ProcessExecutionError: Unexpected error while run ning command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 162 /var/lib/nova/instances/instance-000000f2/console.log\nExit code: 1\nStdout: \'\'\nStderr: "/bin/chown: cannot access `/var/lib/nova/instances/insta nce-000000f2/console.log\': No such file or directory\\n"\n'] -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Thursday, March 21, 2013 12:42 PM To: Minton, Rich Cc: rhos-list at redhat.com Subject: EXTERNAL: Re: Problems after Update. Based on the instance data, there should be a Traceback in the log somewhere. Check nova-compute.log. -- Russell Bryant On 03/21/2013 12:23 PM, Minton, Rich wrote: > I found the source of a couple of my problems... > > > > The update wiped out my glance-api-paste.ini and > glance-registry-paste.ini. Thank fully I had backups. > > > > I still have instances that are in shutoff/shutdown mode. I was able > to get them to back to "active" but then they went to "shutoff" soon > after. Here is the "nova show" for one of the instances. > > > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | Property | > Value > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > | OS-DCF:diskConfig | > MANUAL > | > > | OS-EXT-SRV-ATTR:host | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:hypervisor_hostname | > uvslp-diu01os.lmdit.us.lmco.com > | > > | OS-EXT-SRV-ATTR:instance_name | > instance-000000ea > | > > | OS-EXT-STS:power_state | > 4 > | > > | OS-EXT-STS:task_state | > None > | > > | OS-EXT-STS:vm_state | > error > | > > | accessIPv4 > | > | > > | accessIPv6 > | > | > > | config_drive > | > | > > | created | > 2013-03-18T15:39:29Z > | > > | fault | {u'message': u'IOError', > u'code': 500, u'created': u'2013-03-21T16:17:12Z'} | > > | flavor | m1.medium > (3) > | > > | hostId | > e55588f13411a788271d5273f0dabfa2eb755e0323dab51039dab866 > | > > | id | > a39787b8-2c66-4aaa-b154-42ec05c0e3c8 > | > > | image | Red Hat Enterprise Linux 6.4, > x86_64, Base Server (71c092f5-55de-4b3b-a9d9-4fcb25e15a89) | > > | key_name | > TestKeys > | > > | lmicc-access-nic network | > 10.10.16.11 > | > > | metadata | > {} > | > > | name | > demo > | > > | security_groups | [{u'name': > u'default'}] > | > > | status | > ERROR > | > > | tenant_id | > e4ba09fbf76c41a29218dc68cacf78d5 > | > > | updated | > 2013-03-21T16:17:12Z > | > > | user_id | > f015ea92bf4848a891b73ae2fbf6c75b > | > > +-------------------------------------+------------------------------------------------------------------------------------------+ > > > > *From:*rhos-list-bounces at redhat.com > [mailto:rhos-list-bounces at redhat.com] *On Behalf Of *Minton, Rich > *Sent:* Thursday, March 21, 2013 11:45 AM > *To:* rhos-list at redhat.com > *Subject:* EXTERNAL: [rhos-list] Problems after Update. > > > > Where to begin... Basically, I'm hosed. > > > > I performed an update to the latest versions of the Openstack distro > and rebooted my hosts. Then problems started... > > * Can't launch new instances-the in progress star comes up and > stays there almost forever. > > * Several of my instances went to an error state. When I tried > to reset their state to "active" they went into shutdown/shutoff state. > These are the instances on my controller/compute node. I typed "virsh > list" and the list was empty. > > * I tried to bring up a VNC Console on one of the running > instances and after several minutes received this error in the browser: > > ---------------------------------------------------------------------- > --------------------------------------------------------------------- > > CommunicationError at > /nova/instances/1f6cb3f6-c4d4-45ac-890b-bf2770a90c82/ > > > > _Error communicating with http://10.10.12.245:9292 timed out_ > > > > Request Method: GET > > Request URL: > http://10.10.12.245/dashboard/nova/instances/1f6cb3f6-c4d4-45ac-890b-b > f2770a90c82/?tab=instance_details__vnc > > Django Version: 1.4.2 > > Exception Type: CommunicationError > > Exception Value: > > > > Error communicating with http://10.10.12.245:9292 timed out > > > > Exception Location: > /usr/lib/python2.6/site-packages/glanceclient/common/http.py in > _http_request, line 145 > > Python Executable: /usr/bin/python > > Python Version: 2.6.6 > > Python Path: > > > > ['/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../..', > > '/usr/lib64/python26.zip', > > '/usr/lib64/python2.6', > > '/usr/lib64/python2.6/plat-linux2', > > '/usr/lib64/python2.6/lib-tk', > > '/usr/lib64/python2.6/lib-old', > > '/usr/lib64/python2.6/lib-dynload', > > '/usr/lib64/python2.6/site-packages', > > '/usr/lib64/python2.6/site-packages/gtk-2.0', > > '/usr/lib/python2.6/site-packages/WebOb-1.0.8-py2.6.egg', > > '/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg', > > '/usr/lib/python2.6/site-packages', > > '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', > > '/usr/share/openstack-dashboard/openstack_dashboard'] > > > > Server time: Thu, 21 Mar 2013 15:26:38 +0000 > > ---------------------------------------------------------------------- > -------------------------------------------------------------- > > * I tried deleting iptables and rebooting but did not solve > anything. > > * I'm not seeing anything out of the ordinary in the nova logs. > > > > Any ideas on where I should start looking? > > > > Thanks for any help. > > Rick > > > > _Richard Minton_ > > LMICC Systems Administrator > > 4000 Geerdes Blvd, 13D31 > > King of Prussia, PA 19406 > > Phone: 610-354-5482 > > > _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbryant at redhat.com Fri Mar 22 18:14:50 2013 From: rbryant at redhat.com (Russell Bryant) Date: Fri, 22 Mar 2013 14:14:50 -0400 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: References: <514B3852.1080807@redhat.com> Message-ID: <514C9F9A.2070901@redhat.com> On 03/22/2013 02:00 PM, Minton, Rich wrote: > I just updated another cluster with the new openstack releases and after > the update my glance-api-paste.ini nad glance-registry-paste.ini files > are 0 bytes. I thought it would have created a save file. This might be > a bug. > > > > -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-api-paste.ini > > -rw-r-----. 1 glance glance 5408 Mar 5 17:52 glance-cache.conf > > -rw-r-----. 1 glance glance 3117 Mar 5 17:52 glance-registry.conf > > -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-registry-paste.ini > > -rw-r----- 1 root glance 1091 Feb 27 13:16 glance-scrubber.conf > Well that's odd. Can you describe the exact steps you used to upgrade? -- Russell Bryant From rich.minton at lmco.com Fri Mar 22 18:21:13 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 22 Mar 2013 18:21:13 +0000 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: <514C9F9A.2070901@redhat.com> References: <514B3852.1080807@redhat.com> <514C9F9A.2070901@redhat.com> Message-ID: Yum -y update -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Friday, March 22, 2013 2:15 PM To: Minton, Rich Cc: rhos-list at redhat.com; Perry Myers (pmyers at redhat.com) Subject: Re: EXTERNAL: Re: Problems after Update. On 03/22/2013 02:00 PM, Minton, Rich wrote: > I just updated another cluster with the new openstack releases and > after the update my glance-api-paste.ini nad glance-registry-paste.ini > files are 0 bytes. I thought it would have created a save file. This > might be a bug. > > > > -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-api-paste.ini > > -rw-r-----. 1 glance glance 5408 Mar 5 17:52 glance-cache.conf > > -rw-r-----. 1 glance glance 3117 Mar 5 17:52 glance-registry.conf > > -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-registry-paste.ini > > -rw-r----- 1 root glance 1091 Feb 27 13:16 glance-scrubber.conf > Well that's odd. Can you describe the exact steps you used to upgrade? -- Russell Bryant From rich.minton at lmco.com Fri Mar 22 20:44:23 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Fri, 22 Mar 2013 20:44:23 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: Message-ID: I think I have my key issue figured out. It works fine on my Basic server image but on my Desktop server image I think our proxy is blocking access to the metadata IP. My problem is definitely with my image. So I am going to say that cloud-init works as it should. I sure like the new version a lot better. The old version had some residual Ubuntu specific code putting things in the wrong place. Thanks, Rick From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Minton, Rich Sent: Friday, March 22, 2013 1:40 PM To: Joshua Harlow; Russell Bryant; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. Super. It looks like I'm still having an issue with ssh keys. Cloud-init doesn't seem to insert the key into "authorized_keys". Here is the output of the boot.log. Starting cloud-init: Cloud-init v. 0.7.1 running 'init-local' at Fri, 22 Mar 2013 17:21:51 +0000. Up 66.13 seconds. Starting cloud-init: Cloud-init v. 0.7.1 running 'init' at Fri, 22 Mar 2013 17:21:53 +0000. Up 67.51 seconds. ci-info: ++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++ ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: | Device | Up | Address | Mask | Hw-Address | ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | ci-info: | eth0 | True | 10.10.16.13 | 255.255.255.0 | fa:16:3e:43:6d:4d | ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: +++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++ ci-info: +-------+-------------+------------+---------------+-----------+-------+ ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags | ci-info: +-------+-------------+------------+---------------+-----------+-------+ ci-info: | 0 | 10.10.16.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U | ci-info: | 1 | 169.254.0.0 | 0.0.0.0 | 255.255.0.0 | eth0 | U | ci-info: | 2 | 0.0.0.0 | 10.10.16.7 | 0.0.0.0 | eth0 | UG | ci-info: +-------+-------------+------------+---------------+-----------+-------+ 2013-03-22 13:21:56,519 - util.py[WARNING]: Applying ssh credentials failed! Starting cloud-init: Cloud-init v. 0.7.1 running 'modules:config' at Fri, 22 Mar 2013 17:21:57 +0000. Up 71.30 seconds. Starting cloud-init: Cloud-init v. 0.7.1 running 'modules:final' at Fri, 22 Mar 2013 17:21:58 +0000. Up 72.85 seconds. 2013-03-22 13:21:59,147 - util.py[WARNING]: Running ssh-authkey-fingerprints () failed ec2: ec2: ############################################################# ec2: -----BEGIN SSH HOST KEY FINGERPRINTS----- ec2: 1024 de:0d:95:89:10:b3:f8:a5:12:09:62:87:58:f5:52:35 /etc/ssh/ssh_host_dsa_key.pub (DSA) ec2: 2048 48:d3:2c:ac:98:4e:57:dc:07:8a:4e:c8:0c:68:56:35 /etc/ssh/ssh_host_key.pub (RSA1) ec2: 2048 97:62:2b:dd:09:3c:5d:c6:30:7e:8a:16:89:27:82:6a /etc/ssh/ssh_host_rsa_key.pub (RSA) ec2: -----END SSH HOST KEY FINGERPRINTS----- ec2: ############################################################# -----BEGIN SSH HOST KEY KEYS----- 2048 35 21889849139367763563389351601377820796073793617449436599221977659445135002476223129833833651351484232076837788805006750680052997128685757303892593965137923198447956864014079923478870309110547046664268287380298838332284048302681336615302514348725862981947986908242642891955318438217141018134624513598829945193978261667926153861849588173917651884170461212835461964580843245838698025157376233972830294897970640494259212138531890829666635458094297349958505817038457555402332013792018319388616117427759769435072425962956358797493569252499044371829936847980644362073198927847473883205821775406782358666256423235107872534147 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAs/IUMJ56v2OLNGNKiooyV0cY4gITD56Km64WL3RtHc9TrHGOXOcxNJHRsHS0JFkxPY9PXNlIuoqF6/FxTmPwW1KeqDigT4/a9Ho5WSaVEmRSdID2fPtBawZeUlZQH3KHMIcCGYkixQpb8o5JNNn0LNUEI1N7YkbnliK2zuewWwMoXYUqKvqi6LRlov6HFRP9EbyrZYVFWCpbRy6ZlP781VF3QR7PdAMl/IZWZmG7uMuRGCqfhUEJVYMFxLgShL/1QE6/eW7jc0tcHBaPo5CZReMQy3kE1TO9XltCTLsWYbhc1Sn7O1BaqIC3W8mOfEz39QKQ/tBMpgOcRK88Xtbz2Q== -----END SSH HOST KEY KEYS----- Cloud-init v. 0.7.1 finished at Fri, 22 Mar 2013 17:21:59 +0000. Datasource DataSourceEc2. Up 73.49 seconds Starting ntpd: [ OK ] NetBackup Authentication daemon not started. NetBackup network daemon started. NetBackup client daemon started. NetBackup SAN Client Fibre Transport daemon started. NetBackup Bare Metal Restore Boot Server daemon started. Starting postfix: [ OK ] Starting abrt daemon: [ OK ] Starting crond: [ OK ] Starting McAfee Agent services... [ OK ] Starting SMB services: [ OK ] Starting atd: [ OK ] Starting Red Hat Network Daemon: [ OK ] Starting rhsmcertd...[ OK ] Starting certmonger: [ OK ] AUTHORIZED_KEYS: ******************************** ******************************** [root at uvslv-diu04dv log]# -----Original Message----- From: Joshua Harlow [mailto:harlowja at yahoo-inc.com] Sent: Friday, March 22, 2013 1:13 PM To: Minton, Rich; Russell Bryant; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. If u need more help. I am the second most active dev (and added all the RH stuffs) in cloud-init. Here to serve :-P -Josh On 3/22/13 9:33 AM, "Minton, Rich" > wrote: >It looks like my problem was with the older cloud-init package. I >updated to the latest Red Hat version of cloud-init and now everything >works great. > >Thanks, >Rick > >-----Original Message----- >From: rhos-list-bounces at redhat.com >[mailto:rhos-list-bounces at redhat.com] >On Behalf Of Russell Bryant >Sent: Friday, March 22, 2013 6:32 AM >To: rhos-list at redhat.com >Subject: EXTERNAL: Re: [rhos-list] SSH Key Injection not working. > >On 03/21/2013 06:13 PM, Minton, Rich wrote: >> While logged into the VM, I can run >> http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and >> the ssh key is returned. So I guess the metadata service is working >> properly. Do I need to have the .ssh directory and the >> authorized_keys file already in place and with the correct permissions? > >If you were able to verify that the metadata server returns the SSH >key, then the problem seems to be with however the instance is trying >to get it (as in, an issue with the image, not OpenStack). > >-- >Russell Bryant > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From prmarino1 at gmail.com Fri Mar 22 23:33:34 2013 From: prmarino1 at gmail.com (Paul Robert Marino) Date: Fri, 22 Mar 2013 19:33:34 -0400 Subject: [rhos-list] Quantum or Nova in RHOS In-Reply-To: <253945448.13000734.1363960087974.JavaMail.root@redhat.com> Message-ID: <514cea4f.eb8eec0a.412d.ffffb430@mx.google.com> An HTML attachment was scrubbed... URL: From k.mohammad1 at physics.ox.ac.uk Fri Mar 22 23:51:15 2013 From: k.mohammad1 at physics.ox.ac.uk (Kashif Mohammad) Date: Fri, 22 Mar 2013 23:51:15 +0000 Subject: [rhos-list] Nova-network woes (Vnc issue now) In-Reply-To: <514C8F89.5050404@redhat.com> References: <88B17E26E0A9F94381C67535AEF2BB5E798F776B@EXCHNG13.physics.ox.ac.uk> <20130321152007.GA18983@beagles.redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E7990C261@EXCHNG13.physics.ox.ac.uk> <88B17E26E0A9F94381C67535AEF2BB5E7990FA29@EXCHNG13.physics.ox.ac.uk> <514C708F.4070406@redhat.com> <88B17E26E0A9F94381C67535AEF2BB5E79910FBA@EXCHNG13.physics.ox.ac.uk>, <514C8F89.5050404@redhat.com> Message-ID: <88B17E26E0A9F94381C67535AEF2BB5E799130E6@EXCHNG13.physics.ox.ac.uk> Hi Russell >>Check to see if the VNC connection is getting blocked by iptables on your compute node Thanks for pointing to the right direction. I opened 5900 ports and it started working iptables -I INPUT 11 --proto tcp --dport 5900:5950 -j ACCEPT It may be that I am missing something in my configuration so it didn't work by default. Although one problem remain that I can not log in through dashboard as it is not accepting input from keyboard. It works through browser by getting novnc proxy. Thanks for you help. Kashif ________________________________________ From: Russell Bryant [rbryant at redhat.com] Sent: Friday, March 22, 2013 5:06 PM To: Kashif Mohammad Cc: rhos-list at redhat.com Subject: Re: [rhos-list] Nova-network woes (Vnc issue now) On 03/22/2013 11:48 AM, Kashif Mohammad wrote: > Hi Russell > >>> does the browser fail to connect to anything at all? > > Yes, it fails to connect to anything at all. It fails with this error > > Failed to connect to server (code: 1006) Actually, it is connecting to the VNC proxy. That's what I was trying to get at. That message is inside of the novnc HTML5 vnc viewer. So the VNC proxy is failing to make a VNC connection to your instance. > I can get a token through get-vnc-console. What I understand that request is coming on public address on controller node and vnc is running on private address on a different compute node. Does nova suppose to add a route for this? Looking at route table Your browser connects to the VNC proxy successfully via the public IP address. The proxy should then connect to the VNC server for the instance. The address used to connect to the VNC server internally to nova is from the vncserver_proxyclient_address option, which you have set to 192.168.9.3 on your compute node. Check to see if the VNC connection is getting blocked by iptables on your compute node. -- Russell Bryant From vaibhav.k.agarwal at in.com Mon Mar 25 11:44:54 2013 From: vaibhav.k.agarwal at in.com (Kumar Vaibhav) Date: Mon, 25 Mar 2013 17:14:54 +0530 Subject: [rhos-list] Control Access to instance termination Message-ID: <1364211894.52c5189391854c93e8a0e1326e56c14f@mail.in.com> Hi,I want to restrict the user to see other user's instances. Default behaviour is that a user can see all the servers in his tenantid.He can suspend, delete, pause etc to the instances created by the other users.Can this be restricted? I want the full previlige to the owner(user who created the instance) of the instance. And no privilige to the other users (except admin) on other users' instances.I don't want to create a tenant for each user to solve this problem.Regards,VaibhavDear rhoslist ! Get Yourself a cool, short @in.com Email ID now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From eglynn at redhat.com Mon Mar 25 12:47:21 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Mon, 25 Mar 2013 08:47:21 -0400 (EDT) Subject: [rhos-list] Control Access to instance termination In-Reply-To: <1364211894.52c5189391854c93e8a0e1326e56c14f@mail.in.com> Message-ID: <1219238145.50808120.1364215641217.JavaMail.root@redhat.com> > Hi, > > I want to restrict the user to see other user's instances. > > Default behaviour is that a user can see all the servers in his > tenant-id. > > He can suspend, delete, pause etc to the instances created by the > other users. > > Can this be restricted? I want the full previlige to the owner(user > who created the instance) of the instance. And no privilige to the > other users (except admin) on other users' instances. > > I don't want to create a tenant for each user to solve this problem. > > Regards, > Vaibhav This is a requirement that would ideally be handled via the rules-based access control (RBAC) logic. Currently in nova, a rule such as: "admin_or_owner": "is_admin:True or project_id:%(project_id)s" can be used to specify that an action is restricted to the admin role or the tenant owning the resource. What you would need I believe is a rule like: "admin_or_user": "is_admin:True or user_id:%(user_id)s", then restrict the instance delete etc. actions via: "compute:delete": "rule:admin_or_user", # ... etc. All changes applied to /etc/nova/policy.json followed by: # service openstack-nova-api restart I've tested briefly against master upstream, but I'd be interested hearing your experience with the RHOS version you've got deployed. Cheers, Eoghan From vaibhav.k.agarwal at in.com Mon Mar 25 14:37:31 2013 From: vaibhav.k.agarwal at in.com (Kumar Vaibhav) Date: Mon, 25 Mar 2013 20:07:31 +0530 Subject: [rhos-list] Control Access to instance termination Message-ID: <1364222251.5fef3eff51dc719c4a9f565a742d78f2@mail.in.com> Hi,Where I can get a complete list of policy.json options?I also noticed the policy.json file but couldn't find a documentation for it.I also wanted to restrict the viewership of the servers to the owneruser, projectadmin and admin only.So he can access/change the other users' instances.Regards,Vaibhav Original message From:"Eoghan Glynn"< eglynn at redhat.com >Date: 25 Mar 13 18:21:48Subject: Re: [rhoslist] Control Access to instance terminationTo: Kumar Vaibhav Cc: rhoslist > Hi,> > I want to restrict the user to see other user's instances.> > Default behaviour is that a user can see all the servers in his> tenantid.> > He can suspend, delete, pause etc to the instances created by the> other users.> > Can this be restricted? I want the full previlige to the owner(user> who created the instance) of the instance. And no privilige to the> other users (except admin) on other users' instances.> > I don't want to create a tenant for each user to solve this problem.> > Regards,> Vaib havThis is a requirement that would ideally be handled via the rulesbasedaccess control (RBAC) logic.Currently in nova, a rule such as:"adminorowner":"isadmin:True or projectid:%(projectid)s"can be used to specify that an action is restricted to the adminrole or the tenant owning the resource.What you would need I believe is a rule like:"adminoruser": "isadmin:True or userid:%(userid)s",then restrict the instance delete etc. actions via:"compute:delete": "rule:adminoruser",# ... etc.All changes applied to /etc/nova/policy.json followed by:# service openstacknovaapi restartI've tested briefly against master upstream, but I'd be interestedhearing your experience with the RHOS version you've got deployed.Cheers,EoghanGet Yourself a cool, short @in.com Email ID now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From eglynn at redhat.com Mon Mar 25 16:01:42 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Mon, 25 Mar 2013 12:01:42 -0400 (EDT) Subject: [rhos-list] Control Access to instance termination In-Reply-To: <1364222251.5fef3eff51dc719c4a9f565a742d78f2@mail.in.com> Message-ID: <624323715.51775634.1364227302450.JavaMail.root@redhat.com> > Hi, > > Where I can get a complete list of policy.json options? Good question, I can't find a complete list in the upstream docco. This section "If we wished to restrict all Compute service requests to require this role, the resulting file would look like:" ... http://docs.openstack.org/trunk/openstack-compute/install/yum/content/keystone-concepts.html implies a full list, but the file quoted does not include every fine-grained policy (perhaps relying on the default policy to fill in the gaps). But looking directly at the source, here's a list of policies associated with the compute API: compute:create compute:create:forced_host compute:create:attach_network compute:create:attach_volume compute:get compute:get_all compute:get_all_tenants compute:detach_volume compute:get_instance_faults compute:update compute:soft_delete compute:delete compute:restore compute:force_delete compute:start compute:backup compute:snapshot compute:rebuild compute:revert_resize compute:confirm_resize compute:resize compute:add_fixed_ip compute:remove_fixed_ip compute:pause compute:unpause compute:get_diagnostics compute:suspend compute:resume compute:rescue compute:unrescue compute:set_admin_password compute:inject_file compute:get_vnc_console compute:get_spice_console compute:get_console_output compute:lock compute:unlock compute:get_lock compute:reset_network compute:inject_network_info compute:attach_interface compute:detach_interface compute:get_instance_metadata compute:delete_instance_metadata compute:update_instance_metadata compute:security_groups:add_to_instance compute:security_groups:remove_from_instance > I also noticed the policy.json file but couldn't find a documentation > for it. > > I also wanted to restrict the viewership of the servers to the > owner_user, project_admin and admin only. > So he can access/change the other users' instances. "He" in this context being the admin role? Presumably also project_admin is a custom role you've created yourself? IIUC the rule you'd need would go something like: role:admin or (role:project_admin and project_id:%(project_id)s) or user_id:%(user_id)s or using the older syntax: [["role:admin"], ["role:project_admin", "project_id:%(project_id)s"]], ["user_id:%(user_id)s"]] Cheers, Eoghan > > Regards, > Vaibhav > > > > ---------- Original message ---------- > From:"Eoghan Glynn"< eglynn at redhat.com > > Date: 25 Mar 13 18:21:48 > Subject: Re: [rhos-list] Control Access to instance termination > To: Kumar Vaibhav > Cc: rhos-list > > > > > Hi, > > > > I want to restrict the user to see other user's instances. > > > > Default behaviour is that a user can see all the servers in his > > tenant-id. > > > > He can suspend, delete, pause etc to the instances created by the > > other users. > > > > Can this be restrict ed? I want the full previlige to the > > owner(user > > who created the instance) of the instance. And no privilige to the > > other users (except admin) on other users' instances. > > > > I don't want to create a tenant for each user to solve this > > problem. > > > > Regards, > > Vaibhav > > > This is a requirement that would ideally be handled via the > rules-based > access control (RBAC) logic. > > Currently in nova, a rule such as: > > "admin_or_owner": "is_admin:True or project_id:%(project_id)s" > > can be used to specify that an action is restricted to the admin > role or the tenant owning the resource. > > What you would need I believe is a rule like: > > "admin_or_user": "is_admin:True or user_id:%(user_id)s", > > then restrict the instance delete etc. actions via: > > "compute:delete": "rule:admin_or_user", > # ... etc. > > All changes applied to /etc/nova/policy.json followed by: > > # serv ice openstack-nova-api restart > > I've tested briefly against master upstream, but I'd be interested > hearing your experience with the RHOS version you've got deployed. > > Cheers, > Eoghan > > > > Get Yourself a cool, short @in.com Email ID now! From eglynn at redhat.com Mon Mar 25 16:38:16 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Mon, 25 Mar 2013 12:38:16 -0400 (EDT) Subject: [rhos-list] Control Access to instance termination In-Reply-To: <624323715.51775634.1364227302450.JavaMail.root@redhat.com> Message-ID: <1843910132.51984434.1364229496937.JavaMail.root@redhat.com> > or using the older syntax: > > [["role:admin"], ["role:project_admin", > "project_id:%(project_id)s"]], ["user_id:%(user_id)s"]] Typo: [["role:admin"], ["role:project_admin", "project_id:%(project_id)s"], ["user_id:%(user_id)s"]] From rich.minton at lmco.com Mon Mar 25 16:45:58 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 25 Mar 2013 16:45:58 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: Message-ID: Josh, I did a lot of reading over the weekend and have a better understanding of how cloud-init works. I thought that the ssk key injection was a onetime thing at first boot and by booting my image to make changes was affecting how cloud-init was working when I launched an image. So I went back and created a master image without cloud-init so that I could make changes to the image as needed and then install cloud-init as the last step before powering down the image. Then I ran "glance image-create" with that image so that when an instance was launched cloud-init would be run for the first time. That didn't seem to matter. Still no keys in "authorized_keys" file for root or "cloud-user". I used Wire shark to watch the traffic to the meta-data server and I can even see the ssh keys coming back to the VM. I just don't see them in the authorized_keys file anywhere. No today my VMs on a particular host cannot reach the meta-data server. I get "no route to host" errors but it still seems to be getting the hostname and host keys. It fails on "Applying ssh credentials". I included the boot.log and cloud-init.log. Any help you can provide would be greatly appreciated. I already have grey hair so the next phase would be... I don't even want to mention it. Thanks, Rick Richard Minton LMICC Systems Administrator 4000 Geerdes Blvd, 13D31 King of Prussia, PA 19406 Phone: 610-354-5482 -----Original Message----- From: Joshua Harlow [mailto:harlowja at yahoo-inc.com] Sent: Friday, March 22, 2013 1:13 PM To: Minton, Rich; Russell Bryant; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. If u need more help. I am the second most active dev (and added all the RH stuffs) in cloud-init. Here to serve :-P -Josh On 3/22/13 9:33 AM, "Minton, Rich" wrote: >It looks like my problem was with the older cloud-init package. I >updated to the latest Red Hat version of cloud-init and now everything >works great. > >Thanks, >Rick > >-----Original Message----- >From: rhos-list-bounces at redhat.com >[mailto:rhos-list-bounces at redhat.com] >On Behalf Of Russell Bryant >Sent: Friday, March 22, 2013 6:32 AM >To: rhos-list at redhat.com >Subject: EXTERNAL: Re: [rhos-list] SSH Key Injection not working. > >On 03/21/2013 06:13 PM, Minton, Rich wrote: >> While logged into the VM, I can run >> http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and >> the ssh key is returned. So I guess the metadata service is working >> properly. Do I need to have the .ssh directory and the >> authorized_keys file already in place and with the correct permissions? > >If you were able to verify that the metadata server returns the SSH >key, then the problem seems to be with however the instance is trying >to get it (as in, an issue with the image, not OpenStack). > >-- >Russell Bryant > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list > >_______________________________________________ >rhos-list mailing list >rhos-list at redhat.com >https://www.redhat.com/mailman/listinfo/rhos-list -------------- next part -------------- A non-text attachment was scrubbed... Name: boot.log Type: application/octet-stream Size: 6360 bytes Desc: boot.log URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cloud-init.log Type: application/octet-stream Size: 177264 bytes Desc: cloud-init.log URL: From rbryant at redhat.com Mon Mar 25 17:20:39 2013 From: rbryant at redhat.com (Russell Bryant) Date: Mon, 25 Mar 2013 13:20:39 -0400 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: Message-ID: <51508767.8000202@redhat.com> On 03/25/2013 12:45 PM, Minton, Rich wrote: > Josh, > > I did a lot of reading over the weekend and have a better understanding of how cloud-init works. I thought that the ssk key injection was a onetime thing at first boot and by booting my image to make changes was affecting how cloud-init was working when I launched an image. So I went back and created a master image without cloud-init so that I could make changes to the image as needed and then install cloud-init as the last step before powering down the image. Then I ran "glance image-create" with that image so that when an instance was launched cloud-init would be run for the first time. That didn't seem to matter. Still no keys in "authorized_keys" file for root or "cloud-user". I used Wire shark to watch the traffic to the meta-data server and I can even see the ssh keys coming back to the VM. I just don't see them in the authorized_keys file anywhere. > > No today my VMs on a particular host cannot reach the meta-data server. I get "no route to host" errors but it still seems to be getting the hostname and host keys. It fails on "Applying ssh credentials". I included the boot.log and cloud-init.log. > > Any help you can provide would be greatly appreciated. I already have grey hair so the next phase would be... I don't even want to mention it. Do you just want an image with cloud-init working, or do you need your own image right now? You can use some pre-built Fedora testing images that are lying around. Here's a Fedora 18 image with cloud-init: http://goodsquishy.com/2013/02/fedora-18-qcow2-image/ -- Russell Bryant From rich.minton at lmco.com Mon Mar 25 17:55:53 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 25 Mar 2013 17:55:53 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: <51508767.8000202@redhat.com> References: <51508767.8000202@redhat.com> Message-ID: Thank you Russell. Yes a known good image would be helpful for troubleshooting purposes. I'll give it a shot. Rick -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Monday, March 25, 2013 1:21 PM To: Minton, Rich Cc: Joshua Harlow; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. On 03/25/2013 12:45 PM, Minton, Rich wrote: > Josh, > > I did a lot of reading over the weekend and have a better understanding of how cloud-init works. I thought that the ssk key injection was a onetime thing at first boot and by booting my image to make changes was affecting how cloud-init was working when I launched an image. So I went back and created a master image without cloud-init so that I could make changes to the image as needed and then install cloud-init as the last step before powering down the image. Then I ran "glance image-create" with that image so that when an instance was launched cloud-init would be run for the first time. That didn't seem to matter. Still no keys in "authorized_keys" file for root or "cloud-user". I used Wire shark to watch the traffic to the meta-data server and I can even see the ssh keys coming back to the VM. I just don't see them in the authorized_keys file anywhere. > > No today my VMs on a particular host cannot reach the meta-data server. I get "no route to host" errors but it still seems to be getting the hostname and host keys. It fails on "Applying ssh credentials". I included the boot.log and cloud-init.log. > > Any help you can provide would be greatly appreciated. I already have grey hair so the next phase would be... I don't even want to mention it. Do you just want an image with cloud-init working, or do you need your own image right now? You can use some pre-built Fedora testing images that are lying around. Here's a Fedora 18 image with cloud-init: http://goodsquishy.com/2013/02/fedora-18-qcow2-image/ -- Russell Bryant From rich.minton at lmco.com Mon Mar 25 18:03:38 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 25 Mar 2013 18:03:38 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: <51508767.8000202@redhat.com> References: <51508767.8000202@redhat.com> Message-ID: I do need to have my own (or a RHEL 6.4 Base Server image) to continue with our development work. I'll still give the Fedora image a try. Thanks again. Rick -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Monday, March 25, 2013 1:21 PM To: Minton, Rich Cc: Joshua Harlow; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. On 03/25/2013 12:45 PM, Minton, Rich wrote: > Josh, > > I did a lot of reading over the weekend and have a better understanding of how cloud-init works. I thought that the ssk key injection was a onetime thing at first boot and by booting my image to make changes was affecting how cloud-init was working when I launched an image. So I went back and created a master image without cloud-init so that I could make changes to the image as needed and then install cloud-init as the last step before powering down the image. Then I ran "glance image-create" with that image so that when an instance was launched cloud-init would be run for the first time. That didn't seem to matter. Still no keys in "authorized_keys" file for root or "cloud-user". I used Wire shark to watch the traffic to the meta-data server and I can even see the ssh keys coming back to the VM. I just don't see them in the authorized_keys file anywhere. > > No today my VMs on a particular host cannot reach the meta-data server. I get "no route to host" errors but it still seems to be getting the hostname and host keys. It fails on "Applying ssh credentials". I included the boot.log and cloud-init.log. > > Any help you can provide would be greatly appreciated. I already have grey hair so the next phase would be... I don't even want to mention it. Do you just want an image with cloud-init working, or do you need your own image right now? You can use some pre-built Fedora testing images that are lying around. Here's a Fedora 18 image with cloud-init: http://goodsquishy.com/2013/02/fedora-18-qcow2-image/ -- Russell Bryant From rbryant at redhat.com Mon Mar 25 18:26:25 2013 From: rbryant at redhat.com (Russell Bryant) Date: Mon, 25 Mar 2013 14:26:25 -0400 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: <51508767.8000202@redhat.com> Message-ID: <515096D1.9010409@redhat.com> On 03/25/2013 01:55 PM, Minton, Rich wrote: > Thank you Russell. Yes a known good image would be helpful for troubleshooting purposes. I'll give it a shot. Ok, cool. That's one of the images I use in my development environment, so I know it works. -- Russell Bryant From rbryant at redhat.com Mon Mar 25 19:46:09 2013 From: rbryant at redhat.com (Russell Bryant) Date: Mon, 25 Mar 2013 15:46:09 -0400 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: <51508767.8000202@redhat.com> <515096D1.9010409@redhat.com> Message-ID: <5150A981.4070001@redhat.com> On 03/25/2013 03:27 PM, Minton, Rich wrote: > I just tried it twice and both times it erred on me. I can't get a console or see the log. What does "erred on me" mean in this context? Can you provide more detail on how you tested this, and how you observed the failure? > -----Original Message----- > From: Russell Bryant [mailto:rbryant at redhat.com] > Sent: Monday, March 25, 2013 2:26 PM > To: Minton, Rich > Cc: Joshua Harlow; rhos-list at redhat.com > Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. > > On 03/25/2013 01:55 PM, Minton, Rich wrote: >> Thank you Russell. Yes a known good image would be helpful for troubleshooting purposes. I'll give it a shot. > > Ok, cool. That's one of the images I use in my development environment, so I know it works. > > -- > Russell Bryant > -- Russell Bryant From rich.minton at lmco.com Mon Mar 25 20:35:45 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Mon, 25 Mar 2013 20:35:45 +0000 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: <5150A981.4070001@redhat.com> References: <51508767.8000202@redhat.com> <515096D1.9010409@redhat.com> <5150A981.4070001@redhat.com> Message-ID: I figured out my problem... I had "libvirt_inject_partition = -2" in my nova.conf. I changed the value to -1 and ssh key injection started working. I think that's the reason why the Fedora images were failing. Another good lesson learned. Rick -----Original Message----- From: Russell Bryant [mailto:rbryant at redhat.com] Sent: Monday, March 25, 2013 3:46 PM To: Minton, Rich; rhos-list at redhat.com Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. On 03/25/2013 03:27 PM, Minton, Rich wrote: > I just tried it twice and both times it erred on me. I can't get a console or see the log. What does "erred on me" mean in this context? Can you provide more detail on how you tested this, and how you observed the failure? > -----Original Message----- > From: Russell Bryant [mailto:rbryant at redhat.com] > Sent: Monday, March 25, 2013 2:26 PM > To: Minton, Rich > Cc: Joshua Harlow; rhos-list at redhat.com > Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. > > On 03/25/2013 01:55 PM, Minton, Rich wrote: >> Thank you Russell. Yes a known good image would be helpful for troubleshooting purposes. I'll give it a shot. > > Ok, cool. That's one of the images I use in my development environment, so I know it works. > > -- > Russell Bryant > -- Russell Bryant From harlowja at yahoo-inc.com Mon Mar 25 20:44:10 2013 From: harlowja at yahoo-inc.com (Joshua Harlow) Date: Mon, 25 Mar 2013 13:44:10 -0700 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: Message-ID: More gray hair! J/k, Let me get u to run this module also. Its a new one that shows debug stuff, if u feel comfortable adjusting the cloud-init config to use it. http://paste.openstack.org/show/34557/ Then adding 'verbose: True' in your yaml userdata file will show exactly the userdata/metadata that is available. I should upstream that sometime soon. Btw, if u have any questions, feel free to jump on #cloud-init on freenode irc. Also these errors would explain a lot: - Failed reading from http://169.254.169.254/2009-04-04/ Then after that it defaulted back to the 'none' datasource, which makes me wonder if the ec2 metadata server is alive (and forwarded correctly). I personally use the config drive datasource (no special network stuff required). U can try that also (its a setting in nova.conf). On 3/25/13 9:45 AM, "Minton, Rich" wrote: >Josh, > >I did a lot of reading over the weekend and have a better understanding >of how cloud-init works. I thought that the ssk key injection was a >onetime thing at first boot and by booting my image to make changes was >affecting how cloud-init was working when I launched an image. So I went >back and created a master image without cloud-init so that I could make >changes to the image as needed and then install cloud-init as the last >step before powering down the image. Then I ran "glance image-create" >with that image so that when an instance was launched cloud-init would be >run for the first time. That didn't seem to matter. Still no keys in >"authorized_keys" file for root or "cloud-user". I used Wire shark to >watch the traffic to the meta-data server and I can even see the ssh keys >coming back to the VM. I just don't see them in the authorized_keys file >anywhere. > >No today my VMs on a particular host cannot reach the meta-data server. I >get "no route to host" errors but it still seems to be getting the >hostname and host keys. It fails on "Applying ssh credentials". I >included the boot.log and cloud-init.log. > >Any help you can provide would be greatly appreciated. I already have >grey hair so the next phase would be... I don't even want to mention it. > >Thanks, >Rick > > >Richard Minton >LMICC Systems Administrator >4000 Geerdes Blvd, 13D31 >King of Prussia, PA 19406 >Phone: 610-354-5482 > > > >-----Original Message----- >From: Joshua Harlow [mailto:harlowja at yahoo-inc.com] >Sent: Friday, March 22, 2013 1:13 PM >To: Minton, Rich; Russell Bryant; rhos-list at redhat.com >Subject: Re: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. > >If u need more help. I am the second most active dev (and added all the RH >stuffs) in cloud-init. > >Here to serve :-P > >-Josh > >On 3/22/13 9:33 AM, "Minton, Rich" wrote: > >>It looks like my problem was with the older cloud-init package. I >>updated to the latest Red Hat version of cloud-init and now everything >>works great. >> >>Thanks, >>Rick >> >>-----Original Message----- >>From: rhos-list-bounces at redhat.com >>[mailto:rhos-list-bounces at redhat.com] >>On Behalf Of Russell Bryant >>Sent: Friday, March 22, 2013 6:32 AM >>To: rhos-list at redhat.com >>Subject: EXTERNAL: Re: [rhos-list] SSH Key Injection not working. >> >>On 03/21/2013 06:13 PM, Minton, Rich wrote: >>> While logged into the VM, I can run >>> http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key and >>> the ssh key is returned. So I guess the metadata service is working >>> properly. Do I need to have the .ssh directory and the >>> authorized_keys file already in place and with the correct permissions? >> >>If you were able to verify that the metadata server returns the SSH >>key, then the problem seems to be with however the instance is trying >>to get it (as in, an issue with the image, not OpenStack). >> >>-- >>Russell Bryant >> >>_______________________________________________ >>rhos-list mailing list >>rhos-list at redhat.com >>https://www.redhat.com/mailman/listinfo/rhos-list >> >>_______________________________________________ >>rhos-list mailing list >>rhos-list at redhat.com >>https://www.redhat.com/mailman/listinfo/rhos-list > From rbryant at redhat.com Mon Mar 25 22:13:39 2013 From: rbryant at redhat.com (Russell Bryant) Date: Mon, 25 Mar 2013 18:13:39 -0400 Subject: [rhos-list] EXTERNAL: Re: SSH Key Injection not working. In-Reply-To: References: <51508767.8000202@redhat.com> <515096D1.9010409@redhat.com> <5150A981.4070001@redhat.com> Message-ID: <5150CC13.4050808@redhat.com> On 03/25/2013 04:35 PM, Minton, Rich wrote: > I figured out my problem... > > I had "libvirt_inject_partition = -2" in my nova.conf. I changed the value to -1 and ssh key injection started working. I think that's the reason why the Fedora images were failing. > > Another good lesson learned. Typically you would use either key injection or something like cloud-init, not both. If you expect your images to grab the ssh-key from the metadata server using cloud-init, disable key injection by setting libvirt_inject_key=False in nova.conf. -- Russell Bryant From nicolas.vogel at heig-vd.ch Tue Mar 26 09:23:04 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Tue, 26 Mar 2013 09:23:04 +0000 Subject: [rhos-list] Nova-Network problem In-Reply-To: <20130321155032.GB18983@beagles.redhat.com> References: <20130321155032.GB18983@beagles.redhat.com> Message-ID: Hello, Sorry for this late answer. I modified my configuration but I have some problems yet. My goal is to have a configuration like this "10.3. Flat Network, multiple interfaces, multiple servers" http://docs.openstack.org/trunk/openstack-compute/admin/content/libvirt-flat-networking.html I have my first NIC (em1) configured on a 10.192.75.0/24 network for management. The second network that is available for me is 10.192.72.0/23. This network as already a DHCP server with can give IPs from 72.106 to 72.169. So I think I just have to work with FlatNetworking and not FlatDHCPNetworking because I already have a DHCP server is that correct? I created the rules for the default group like described: http://docs.openstack.org/trunk/openstack-compute/admin/content/enabling-ping-and-ssh-on-vms.html Because I'm in a test phase and totally disconnected from outside world, I set the system in Permissive Mode and disable IPtables for the moment. So for Flat-networking, my em2+br100 interfaces don't need IP+subnet+gateway right? Here my config files for em2: DEVICE="em2" #BOOTPROTO="none" HWADDR="84:2B:2B:6C:FD:10" BRIDGE=br100 NM_CONTROLLED="no" ONBOOT="yes" TYPE="Ethernet" UUID="014a008d-02cb-45c2-98b4-9973fb70420b" And for br100: DEVICE=br100 TYPE=Bridge ONBOOT=yes DELAY=0 BOOTPROTO=none I also modified my nova.conf like this [DEFAULT] logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes #dhcpbridge = /usr/bin/nova-dhcpbridge #dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_inject_partition = -1 network_manager = nova.network.manager.FlatManager iscsi_helper = tgtadm sql_connection = mysql://nova:nova at 10.192.75.190/nova compute_driver = libvirt.LibvirtDriver firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver rpc_backend = nova.openstack.common.rpc.impl_qpid rootwrap_config = /etc/nova/rootwrap.conf auth_strategy = keystone flat_interface = em2 public_interface = em2 volume_api_class = nova.volume.cinder.API enabled_apis = ec2,osapi_compute,metadata flat_network_bridge = br100 # I added this line manually fixed_range = 10.192.72.128/32 # I added this line manually [keystone_authtoken] admin_tenant_name = %SERVICE_TENANT_NAME% admin_user = %SERVICE_USER% admin_password = %SERVICE_PASSWORD% auth_host = 10.192.75.190 auth_port = 35357 auth_protocol = http signing_dirname = /tmp/keystone-signing-nova #VNC Proxy novncproxy_host = 10.192.75.190 novncproxy_port = 6080 novncproxy_base_url = http://10.192.75.190:6080/vnc_auto.html vnc_enabled = true vncserver_listen = 10.192.75.190 vncserver_proxyclient_address = 10.192.75.190 I modified the loopback address with the right IP like you told me and added the two lines "flat_network_bridge" and "fixed_range" manually. Is that correct ? Then I wan't to created a pool for my VMs, which must match a part of my DHCP range: nova-manage network create private --multi_host=T --fixed_range_v4=10.192.72.128/27 --bridge=br100 --bridge_interface=em2 --network_size=32 But when I'm wan't to see the network with "nova network-list" command, I have following error : ERROR: string indices must be integers, not str I'm not connected to the outside world for the moment but I want to test the allocation of floating IP: nova-manage floating create 100.100.100.100 nova floating-ip-create ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-53668eb6-92f6-47f5-98dc-3f6aa8ebf9e7) Why did I have this Error? I have the same error when I'm creating more than one IP (100.100.100.100/29) and then try the floating-ip-create command. And for me second node, I mean a compute/network node, which must be connected to the controller node and receive keystone and glance services from him. On my second Node, I configured for the moment juste Nova and Cinder. I don't know what to do else. Should I complete the Keystone endpoint-list on the controller with this new endpoints (on server 10.192.75.191) ? How can I tell my second node to communicate with the controller? Thanks a lot, Cheers, Nicolas Vogel. -----Original Message----- From: Brent Eagles [mailto:beagles at redhat.com] Sent: jeudi 21 mars 2013 16:51 To: Vogel Nicolas Cc: 'rhos-list at redhat.com' Subject: Re: [rhos-list] Nova-Network problem On Thu, Mar 21, 2013 at 07:38:15AM +0000, Vogel Nicolas wrote: > Hello, > > I?ve installed Openstack Folsom with Keystone, Nova, Cinder and Glance > on a single node with CentOS 6.3. I?m using Nova-network and not > Quantum. My server has 2 NICs, and I used one of them for management > (em1: 10.192.75.190/24) and the second for my VM (em2 + br100: > 10.192.73.193/26). All the VM I start get a right IP address but I > can?t reach them. Here is my config: What do you mean by "can't reach them"? Do you mean SSH or ping? Did you define the security group rules (see http://docs.openstack.org/trunk/openstack-compute/admin/content/enabling-ping-and-ssh-on-vms.html Without them, iptables is likely blocking access. > em2: > DEVICE="em2" > #BOOTPROTO="none" > HWADDR="84:2B:2B:6C:FD:10" > BRIDGE=br100 > NM_CONTROLLED="yes" > ONBOOT="yes" > TYPE="Ethernet" > UUID="014a008d-02cb-45c2-98b4-xxxxxxxxxxxx" Is this interface really configured with NetworkManager, or did you do this with a configuation file? If the latter, you might want to set NM_CONTROLLED=no. It may not make a difference, but... > > br100: > DEVICE=br100 > TYPE=Bridge > ONBOOT=yes > DELAY=0 > BOOTPROTO=static > IPADDR=10.192.73.193 > NETMASK=255.255.255.192 > > What did I make wrong in my config? Might be best to verify the security group thing before looking at the rest. > > If I want to assign my VM a floating IP (public IP), I have to create > a pool. But do I have to use a third NIC which is available on my > server and create a br101 interface for that? No, you do not need another interface. What happens when you allocate and assign a floating IP is that Openstack adds rules to your iptables that perform source and destination address translation at appropriate points according to how you have things configured. For example, as a packet from a VM goes out your public interface, there is a SNAT to the assigned public IP address ensuring that return communications come back a.) to the proper destination and b.) addressed to the proper assigned address. There is a companion rule that DNAT's it back to the private address for packets coming back in. > I?m also trying to configure a second node. Is the network > configuration identical as for the controller (with a br100 = IP > 10.192.73.194 in this case) ? The question is a little ambiguous. Do you mean a second standalone compute node or a compute/network node? I would recommend giving the following a read if it is the latter: http://docs.openstack.org/trunk/openstack-compute/admin/content/existing-ha-networking-options.html Just a heads up when it comes to configuing your second node, be wary of instances in your configuration where "localhost" is used (e.g. the database access configuration). It can be a bit frustrating (debug=True/verbose=True is your friend!) when you miss them. You also need to make sure that the firewall rules aren't filtering out the required traffic. Another thing that tripped me up when I first started were the mysql accounts (if you are using mysql) as they can be restrictive with respect to the source host/ip for the connection. Try running mysql -e "select host, user from mysql.user" . If the host values are all "%" (percent signs) then I think you are golden. If not, you might need to do some tweaking to get things to work. Cheers, Brent From vaibhav.k.agarwal at in.com Tue Mar 26 14:17:51 2013 From: vaibhav.k.agarwal at in.com (Kumar Vaibhav) Date: Tue, 26 Mar 2013 19:47:51 +0530 Subject: [rhos-list] Control Access to instance termination Message-ID: <1364307471.23b023b22d0bf47626029d5961328028@mail.in.com> Hi,Thanks for the help.This seems to solve one part of my problem of changing the state of the instance. A user cannot delete the other users' instance.However the listing problem still continues to exist. I checked the logs and found that getall access control is possible by using the policy.json. But getall function itself uses the filter of 'projectid' from the context. So other part seems to be difficult. Regards,Vaibhav Original message From:"Eoghan Glynn"< eglynn at redhat.com >Date: 25 Mar 13 22:08:26Subject: Re: [rhoslist] Control Access to instance terminationTo: Kumar Vaibhav Cc: rhoslist > or using the older syntax:> > [["role:admin"], ["role:projectadmin",> "projectid:%(projectid)s"]], ["userid:%(userid)s"]]Typo:[["role:admin"], ["role:projectadmin", "projectid:%(projectid)s"], ["userid:%(userid)s"]]Get Yourself a cool, short @in.com Email ID now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From vaibhav.k.agarwal at in.com Tue Mar 26 14:22:18 2013 From: vaibhav.k.agarwal at in.com (Kumar Vaibhav) Date: Tue, 26 Mar 2013 19:52:18 +0530 Subject: [rhos-list] Add user field in the instance panel of dashboard Message-ID: <1364307738.fd9e2ae32b53addc06c63208be3aaa43@mail.in.com> Hi,As a admin I find it difficult to know instanceuser mapping.I want to add the userid information to be added for the Dashboard on the instance page. Is it possible to do this? How I can augment the dashboard to add this feature?Regards,VaibhavDear rhoslist ! Get Yourself a cool, short @in.com Email ID now! -------------- next part -------------- An HTML attachment was scrubbed... URL: From eglynn at redhat.com Tue Mar 26 14:50:56 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Tue, 26 Mar 2013 10:50:56 -0400 (EDT) Subject: [rhos-list] Control Access to instance termination In-Reply-To: <1364307471.23b023b22d0bf47626029d5961328028@mail.in.com> Message-ID: <428238410.57128699.1364309456975.JavaMail.root@redhat.com> > Hi, > > Thanks for the help. > This seems to solve one part of my problem of changing the state of > the instance. > A user cannot delete the other users' instance. Great. > However the listing problem still continues to exist. I checked the > logs and found that get_all access control is possible by using the > policy.json. But get_all function itself uses the filter of > 'project_id' from the context. So other part seems to be difficult. I'm sure I see the problem here, as nova.compute.api.API.get_all bases its policy enforcement check on a target that includes both the project_id *and* user_id: https://github.com/openstack/nova/blob/stable/folsom/nova/compute/api.py#L1116 So it seems to me that a rule based on user_id would be applicable in this case also. Again I've just done a quick test against master, please let me know if the behavior you're seeing with your version of RHOS is different. Cheers, Eoghan > Regards, > Vaibhav > > ---------- Original message ---------- > > > From:"Eoghan Glynn"< eglynn at redhat.com > > Date: 25 Mar 13 22:08:26 > Subject: Re: [rhos-list] Control Access to instance termination > To: Kumar Vaibhav > Cc: rhos-list > > > > > or using the older syntax: > > > > [["role:admin"], ["role:project_admin", > > "project_id:%(project_id)s"]], ["user_id:%(user_id)s"]] > > Typo: > > [["role:admin"], ["role :project_admin", > "project_id:%(project_id)s"], ["user_id:%(user_id)s"]] > > > > > Get Yourself a cool, short @in.com Email ID now! From eglynn at redhat.com Tue Mar 26 14:51:42 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Tue, 26 Mar 2013 10:51:42 -0400 (EDT) Subject: [rhos-list] Control Access to instance termination In-Reply-To: <428238410.57128699.1364309456975.JavaMail.root@redhat.com> Message-ID: <1480937938.57130641.1364309502531.JavaMail.root@redhat.com> > > Hi, > > > > Thanks for the help. > > This seems to solve one part of my problem of changing the state of > > the instance. > > A user cannot delete the other users' instance. > > Great. > > > However the listing problem still continues to exist. I checked the > > logs and found that get_all access control is possible by using the > > policy.json. But get_all function itself uses the filter of > > 'project_id' from the context. So other part seems to be difficult. > > I'm sure I see the problem here, as nova.compute.api.API.get_all s/I'm sure/I'm not sure/ > bases its policy enforcement check on a target that includes both > the project_id *and* user_id: > > https://github.com/openstack/nova/blob/stable/folsom/nova/compute/api.py#L1116 > > So it seems to me that a rule based on user_id would be applicable > in this case also. Again I've just done a quick test against master, > please let me know if the behavior you're seeing with your version > of RHOS is different. > > Cheers, > Eoghan > > > > Regards, > > Vaibhav > > > > ---------- Original message ---------- > > > > > > From:"Eoghan Glynn"< eglynn at redhat.com > > > Date: 25 Mar 13 22:08:26 > > Subject: Re: [rhos-list] Control Access to instance termination > > To: Kumar Vaibhav > > Cc: rhos-list > > > > > > > > > or using the older syntax: > > > > > > [["role:admin"], ["role:project_admin", > > > "project_id:%(project_id)s"]], ["user_id:%(user_id)s"]] > > > > Typo: > > > > [["role:admin"], ["role :project_admin", > > "project_id:%(project_id)s"], ["user_id:%(user_id)s"]] > > > > > > > > > > Get Yourself a cool, short @in.com Email ID now! > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From pmyers at redhat.com Wed Mar 27 13:00:33 2013 From: pmyers at redhat.com (Perry Myers) Date: Wed, 27 Mar 2013 09:00:33 -0400 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: References: <514B3852.1080807@redhat.com> <514C9F9A.2070901@redhat.com> Message-ID: <5152ED71.9070009@redhat.com> On 03/22/2013 02:21 PM, Minton, Rich wrote: > Yum -y update Eoghan/Flavio/John, did we ever get to the bottom of the issues on this thread? i.e. is there something broken with the latest openstack-glance in RHN that is causing the behavior that Rich is seeing? Perry > -----Original Message----- > From: Russell Bryant [mailto:rbryant at redhat.com] > Sent: Friday, March 22, 2013 2:15 PM > To: Minton, Rich > Cc: rhos-list at redhat.com; Perry Myers (pmyers at redhat.com) > Subject: Re: EXTERNAL: Re: Problems after Update. > > On 03/22/2013 02:00 PM, Minton, Rich wrote: >> I just updated another cluster with the new openstack releases and >> after the update my glance-api-paste.ini nad glance-registry-paste.ini >> files are 0 bytes. I thought it would have created a save file. This >> might be a bug. >> >> >> >> -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-api-paste.ini >> >> -rw-r-----. 1 glance glance 5408 Mar 5 17:52 glance-cache.conf >> >> -rw-r-----. 1 glance glance 3117 Mar 5 17:52 glance-registry.conf >> >> -rw-r-----. 1 glance glance 0 Mar 5 17:52 glance-registry-paste.ini >> >> -rw-r----- 1 root glance 1091 Feb 27 13:16 glance-scrubber.conf >> > > Well that's odd. Can you describe the exact steps you used to upgrade? > > -- > Russell Bryant > > _______________________________________________ > rhos-list mailing list > rhos-list at redhat.com > https://www.redhat.com/mailman/listinfo/rhos-list > From eglynn at redhat.com Wed Mar 27 13:07:12 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Wed, 27 Mar 2013 09:07:12 -0400 (EDT) Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: <5152ED71.9070009@redhat.com> Message-ID: <915501635.60939268.1364389632206.JavaMail.root@redhat.com> > On 03/22/2013 02:21 PM, Minton, Rich wrote: > > Yum -y update > > Eoghan/Flavio/John, did we ever get to the bottom of the issues on > this > thread? i.e. is there something broken with the latest > openstack-glance > in RHN that is causing the behavior that Rich is seeing? I'll look into it now. Cheers, Eoghan > Perry > > > -----Original Message----- > > From: Russell Bryant [mailto:rbryant at redhat.com] > > Sent: Friday, March 22, 2013 2:15 PM > > To: Minton, Rich > > Cc: rhos-list at redhat.com; Perry Myers (pmyers at redhat.com) > > Subject: Re: EXTERNAL: Re: Problems after Update. > > > > On 03/22/2013 02:00 PM, Minton, Rich wrote: > >> I just updated another cluster with the new openstack releases and > >> after the update my glance-api-paste.ini nad > >> glance-registry-paste.ini > >> files are 0 bytes. I thought it would have created a save file. > >> This > >> might be a bug. > >> > >> > >> > >> -rw-r-----. 1 glance glance 0 Mar 5 17:52 > >> glance-api-paste.ini > >> > >> -rw-r-----. 1 glance glance 5408 Mar 5 17:52 glance-cache.conf > >> > >> -rw-r-----. 1 glance glance 3117 Mar 5 17:52 > >> glance-registry.conf > >> > >> -rw-r-----. 1 glance glance 0 Mar 5 17:52 > >> glance-registry-paste.ini > >> > >> -rw-r----- 1 root glance 1091 Feb 27 13:16 > >> glance-scrubber.conf > >> > > > > Well that's odd. Can you describe the exact steps you used to > > upgrade? > > > > -- > > Russell Bryant > > > > _______________________________________________ > > rhos-list mailing list > > rhos-list at redhat.com > > https://www.redhat.com/mailman/listinfo/rhos-list > > > > From pmyers at redhat.com Wed Mar 27 13:47:17 2013 From: pmyers at redhat.com (Perry Myers) Date: Wed, 27 Mar 2013 09:47:17 -0400 Subject: [rhos-list] Quantum or Nova in RHOS In-Reply-To: <514cea4f.eb8eec0a.412d.ffffb430@mx.google.com> References: <514cea4f.eb8eec0a.412d.ffffb430@mx.google.com> Message-ID: <5152F865.1040102@redhat.com> Adding the Quantum developers to provide more detail here On 03/22/2013 07:33 PM, Paul Robert Marino wrote: > There are a few features in nova network that haven't been split off yet > that are slated the be released in a different component in the next > release of OpenStack (grizzly) such as the custom DNS functions. > As far as Open vSwitch originally there was something missing (interface > naming) from RHEL's version of the kernel which has been added to the > latest 6.x release and it should work now if you are running the latest > version of RHEL 6. > > > > > -- Sent from my HP Pre3 > > ------------------------------------------------------------------------ > On Mar 22, 2013 9:49 AM, Ted Brunell wrote: > > Hello, > > I just have a couple of quick questions about the network stack that > will be included in RHOS when it comes out of preview. > > When the RHOS is released, will it include Quantum + Open vSwitch or > will nova-network be the preferred network stack? The current preview > docs seem to indicate that Quantum is the route that Red Hat is going. RHOS 2.1 will contain both Quantum and Nova Network, just as upstream Folsom and Grizzly contain both of these. We won't remove Nova Network from RHOS releases until upstream has completely converted to Quantum and they have deprecated Nova Network usage, so it may be a few more releases until we get to that point. Nova Networking is definitely the more mature of the two components, and in the absence of a specific need for a feature that Quantum provides, I think at this point we would steer users to Nova Networking. > If nova is used, will there be an upgrade path to Quantum or will it > require a large amount of re-engineering of deployed RHOS systems to get > it working? There is a manual process for converting from Nova Networking to Quantum, but it is definitely a bit labor intensive and would likely require taking down the cloud to be offline for the conversion. We have some of this on the Fedora wiki: https://fedoraproject.org/wiki/OpenStack https://fedoraproject.org/wiki/Quantum https://fedoraproject.org/wiki/Quantum_Converting_Plugins https://fedoraproject.org/wiki/Packstack_to_Quantum The Packstack to Quantum stuff is covered in part by the official docs as well (on docs.redhat.com) > Are there any limitations with Quantum right now that make it not on par > with nova-network? Yes. I'll let the Quantum developers provide a checklist here Cheers, Perry From flavio at redhat.com Wed Mar 27 13:39:52 2013 From: flavio at redhat.com (Flavio Percoco) Date: Wed, 27 Mar 2013 14:39:52 +0100 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: <915501635.60939268.1364389632206.JavaMail.root@redhat.com> References: <5152ED71.9070009@redhat.com> <915501635.60939268.1364389632206.JavaMail.root@redhat.com> Message-ID: <20130327133952.GB11786@redhat.com> On 27/03/13 09:07 -0400, Eoghan Glynn wrote: > > >> On 03/22/2013 02:21 PM, Minton, Rich wrote: >> > Yum -y update >> >> Eoghan/Flavio/John, did we ever get to the bottom of the issues on >> this >> thread? i.e. is there something broken with the latest >> openstack-glance >> in RHN that is causing the behavior that Rich is seeing? > >I'll look into it now. > There's a bug filed similar to this issue: https://bugzilla.redhat.com/show_bug.cgi?id=924904 Assuming that those files are in /etc/glance this most likely happened because those files where moved to /usr/share/glance/ (as Alan mentioned in that bug). From what version to what version did you update? Bests, FF -- { name: "Flavio Percoco", gpg: "87112EC1", internal: "8261386", phone: "+390687502386", irc: ["fpercoco", "flaper87"]} From eglynn at redhat.com Wed Mar 27 14:05:50 2013 From: eglynn at redhat.com (Eoghan Glynn) Date: Wed, 27 Mar 2013 10:05:50 -0400 (EDT) Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: <20130327133952.GB11786@redhat.com> Message-ID: <1992286997.61172735.1364393150727.JavaMail.root@redhat.com> > On 27/03/13 09:07 -0400, Eoghan Glynn wrote: > > > > > >> On 03/22/2013 02:21 PM, Minton, Rich wrote: > >> > Yum -y update > >> > >> Eoghan/Flavio/John, did we ever get to the bottom of the issues on > >> this > >> thread? i.e. is there something broken with the latest > >> openstack-glance > >> in RHN that is causing the behavior that Rich is seeing? > > > >I'll look into it now. > > > > There's a bug filed similar to this issue: > https://bugzilla.redhat.com/show_bug.cgi?id=924904 > > Assuming that those files are in /etc/glance this most likely > happened > because those files where moved to /usr/share/glance/ (as Alan > mentioned in that bug). Yes, I moved the *-paste.ini files to /usr/share/glance in line with their interpretation as more "code" than user-editable config. Having empty *-paste.ini files under /etc/glance shouldn't be a problem unless the user previously edited those files (which is unlikely, given that the auth config no longer needs to be embedded in the paste config). Also it doesn't seem to have been problematic in this case (where the actual issue was cause by a missing NFS mount of the instances directory). Indeed, restoring the old /etc/glance/*-paste.ini files would have no effect by default, as the new /usr/share/*-dist-paste.ini files take precedence *unless* the user has explicitly set: [paste_deploy] config_file = /etc/glance/glance-X-paste.ini in /usr/share/glance-X-dist-paste.ini. Cheers, Eoghan From gkotton at redhat.com Wed Mar 27 14:07:20 2013 From: gkotton at redhat.com (Gary Kotton) Date: Wed, 27 Mar 2013 16:07:20 +0200 Subject: [rhos-list] Quantum or Nova in RHOS In-Reply-To: <5152F865.1040102@redhat.com> References: <514cea4f.eb8eec0a.412d.ffffb430@mx.google.com> <5152F865.1040102@redhat.com> Message-ID: <5152FD18.7010902@redhat.com> On 03/27/2013 03:47 PM, Perry Myers wrote: > Adding the Quantum developers to provide more detail here > > On 03/22/2013 07:33 PM, Paul Robert Marino wrote: >> There are a few features in nova network that haven't been split off yet >> that are slated the be released in a different component in the next >> release of OpenStack (grizzly) such as the custom DNS functions. The DNS support is not contained in Grizzly release of Quantum. There is a project called Monica but at the moment there is no move to include this into Quantum - I am hoping that it will be done at the up and coming summit. If so it might be available in the Havana release. >> As far as Open vSwitch originally there was something missing (interface >> naming) from RHEL's version of the kernel which has been added to the >> latest 6.x release and it should work now if you are running the latest >> version of RHEL 6. The RHOS 2.1 will not support tunneling. My understanding is that this is in the works. >> >> >> >> >> -- Sent from my HP Pre3 >> >> ------------------------------------------------------------------------ >> On Mar 22, 2013 9:49 AM, Ted Brunell wrote: >> >> Hello, >> >> I just have a couple of quick questions about the network stack that >> will be included in RHOS when it comes out of preview. >> >> When the RHOS is released, will it include Quantum + Open vSwitch or >> will nova-network be the preferred network stack? The current preview >> docs seem to indicate that Quantum is the route that Red Hat is going. I think that this is the way that upstream is going. At the up and coming summit there will be a number of talks about the move. I guess that we can provide some more upstream details after the summit. > RHOS 2.1 will contain both Quantum and Nova Network, just as upstream > Folsom and Grizzly contain both of these. > > We won't remove Nova Network from RHOS releases until upstream has > completely converted to Quantum and they have deprecated Nova Network > usage, so it may be a few more releases until we get to that point. > > Nova Networking is definitely the more mature of the two components, and > in the absence of a specific need for a feature that Quantum provides, I > think at this point we would steer users to Nova Networking. > >> If nova is used, will there be an upgrade path to Quantum or will it >> require a large amount of re-engineering of deployed RHOS systems to get >> it working? > There is a manual process for converting from Nova Networking to > Quantum, but it is definitely a bit labor intensive and would likely > require taking down the cloud to be offline for the conversion. > > We have some of this on the Fedora wiki: > > https://fedoraproject.org/wiki/OpenStack > https://fedoraproject.org/wiki/Quantum > https://fedoraproject.org/wiki/Quantum_Converting_Plugins > https://fedoraproject.org/wiki/Packstack_to_Quantum > > The Packstack to Quantum stuff is covered in part by the official docs > as well (on docs.redhat.com) > >> Are there any limitations with Quantum right now that make it not on par >> with nova-network? > Yes. I'll let the Quantum developers provide a checklist here This is one major limitation regarding the parity - there is no multihost support - that is, in nova you can have floating ip support from all compute nodes. In Quantum there is s single point of failure for the floating IP support - the l3 agent. The HA and scalability of this are only going to be dealt with in Havana. The foundations were laid in Grizzly. Other than that Quantum has a richer feature set that Nova. > > Cheers, > > Perry From rich.minton at lmco.com Wed Mar 27 14:19:25 2013 From: rich.minton at lmco.com (Minton, Rich) Date: Wed, 27 Mar 2013 14:19:25 +0000 Subject: [rhos-list] EXTERNAL: Re: Problems after Update. In-Reply-To: <20130327133952.GB11786@redhat.com> References: <5152ED71.9070009@redhat.com> <915501635.60939268.1364389632206.JavaMail.root@redhat.com> <20130327133952.GB11786@redhat.com> Message-ID: Moving to "openstack-glance-2012.2.3-1.el6ost.noarch" from "openstack-glance-2012.2.1-5.el6ost.noarch". -----Original Message----- From: Flavio Percoco [mailto:flavio at redhat.com] Sent: Wednesday, March 27, 2013 9:40 AM To: Eoghan Glynn Cc: Perry Myers; Russell Bryant; rhos-list at redhat.com; John Bresnahan; Flavio Percoco; Ayal Baron; Minton, Rich Subject: Re: [rhos-list] EXTERNAL: Re: Problems after Update. On 27/03/13 09:07 -0400, Eoghan Glynn wrote: > > >> On 03/22/2013 02:21 PM, Minton, Rich wrote: >> > Yum -y update >> >> Eoghan/Flavio/John, did we ever get to the bottom of the issues on >> this thread? i.e. is there something broken with the latest >> openstack-glance in RHN that is causing the behavior that Rich is >> seeing? > >I'll look into it now. > There's a bug filed similar to this issue: https://bugzilla.redhat.com/show_bug.cgi?id=924904 Assuming that those files are in /etc/glance this most likely happened because those files where moved to /usr/share/glance/ (as Alan mentioned in that bug). From what version to what version did you update? Bests, FF -- { name: "Flavio Percoco", gpg: "87112EC1", internal: "8261386", phone: "+390687502386", irc: ["fpercoco", "flaper87"]} From nicolas.vogel at heig-vd.ch Wed Mar 27 16:03:34 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Wed, 27 Mar 2013 16:03:34 +0000 Subject: [rhos-list] floating IP config Message-ID: Hello, Can I have floating IP to assign to my VM when I?m working with a FlatNetworking mode? I have already a dhcp server in my subnet so I don?t need one. I?m just using the br100 bridged to em2 in order to extend my subnet for my VMs. I can create a pool of floating IP but the then I?m unable to create one with the ?nova floating-ip-create? command. I got following output: ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-46ee5602-e356-47e1-a245-f4f1f0c532e9) Here is the debug if someone could help would be really nice: REQ: curl -i http://10.192.75.190:5000/v2.0/tokens -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-novaclient" -d '{"auth": {"tenantName": "rhsummit", "passwordCredentials": {"username": "username", "password": "secret"}}}' connect: (10.192.75.190, 5000) ************ send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: 10.192.75.190:5000\r\nContent-Length: 107\r\ncontent-type: application/json\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n' send: '{"auth": {"tenantName": "rhsummit", "passwordCredentials": {"username": "username", "password": "secret"}}}' reply: 'HTTP/1.1 200 OK\r\n' header: Vary: X-Auth-Token header: Content-Type: application/json header: Content-Length: 2179 header: Date: Wed, 27 Mar 2013 16:01:49 GMT RESP:{'date': 'Wed, 27 Mar 2013 16:01:49 GMT', 'vary': 'X-Auth-Token', 'content-length': '2179', 'status': '200', 'content-type': 'application/json'} {"access": {"token": {"expires": "2013-03-28T16:01:49Z", "id": "de3c7f42f82945ae98ae623f12fb2332", "tenant": {"enabled": true, "description": null, "name": "rhsummit", "id": "bc308f566c2b46aea7812a033de1f21c"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.192.75.190:8776/v1/bc308f566c2b46aea7812a033de1f21c", "region": "regionOne", "internalURL": "http://10.192.75.190:8776/v1/bc308f566c2b46aea7812a033de1f21c", "id": "115d528d2b804f67bf1f5cd789583b52", "publicURL": "http://10.192.75.190:8776/v1/bc308f566c2b46aea7812a033de1f21c"}], "endpoints_links": [], "type": "volume", "name": "cinder"}, {"endpoints": [{"adminURL": "http://10.192.75.190:8080/v1/AUTH_bc308f566c2b46aea7812a033de1f21c", "region": "regionOne", "internalURL": "http://10.192.75.190:8080/v1/AUTH_bc308f566c2b46aea7812a033de1f21c", "id": "b11d4db9596e4eee90c72962a372d097", "publicURL": "http://10.192.75.190:8080/v1/AUTH_bc308f566c2b46aea7812a033de1f21c"}], "endpoints_links": [], "type": "object-store", "name": "swift"}, {"endpoints": [{"adminURL": "http://10.192.75.190:9292/v1", "region": "regionOne", "internalURL": "http://10.192.75.190:9292/v1", "id": "39cfe232d3ca43e2996f362c4d67e8fe", "publicURL": "http://10.192.75.190:9292/v1"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.192.75.190:8774/v1.1/bc308f566c2b46aea7812a033de1f21c", "region": "regionOne", "internalURL": "http://10.192.75.190:8774/v1.1/bc308f566c2b46aea7812a033de1f21c", "id": "62101f9435204e798be3eac5e961f38f", "publicURL": "http://10.192.75.190:8774/v1.1/bc308f566c2b46aea7812a033de1f21c"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.192.75.190:35357/v2.0", "region": "RegionOne", "internalURL": "http://10.192.75.190:5000/v2.0", "id": "820740662d4b4b9c8a33cbcfc586fb37", "publicURL": "http://10.192.75.190:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "username", "roles_links": [], "id": "fdfcf45f412a46e4880e6d8bd8143b61", "roles": [{"name": "user"}], "name": "username"}, "metadata": {"is_admin": 0, "roles": ["205bacf627554e7e8a66bf97946111bd"]}}} REQ: curl -i http://10.192.75.190:8774/v1.1/bc308f566c2b46aea7812a033de1f21c/os-floating-ips -X POST -H "X-Auth-Project-Id: rhsummit" -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: de3c7f42f82945ae98ae623f12fb2332" -d '{"pool": null}' connect: (10.192.75.190, 8774) ************ send: u'POST /v1.1/bc308f566c2b46aea7812a033de1f21c/os-floating-ips HTTP/1.1\r\nHost: 10.192.75.190:8774\r\nContent-Length: 14\r\nx-auth-project-id: rhsummit\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nx-auth-token: de3c7f42f82945ae98ae623f12fb2332\r\nuser-agent: python-novaclient\r\ncontent-type: application/json\r\n\r\n' send: '{"pool": null}' reply: 'HTTP/1.1 500 Internal Server Error\r\n' header: Content-Length: 128 header: Content-Type: application/json; charset=UTF-8 header: X-Compute-Request-Id: req-a24ce04a-9ccb-41f5-aa68-9d7727ae95f4 header: Date: Wed, 27 Mar 2013 16:01:49 GMT RESP:{'date': 'Wed, 27 Mar 2013 16:01:49 GMT', 'status': '500', 'content-length': '128', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-a24ce04a-9ccb-41f5-aa68-9d7727ae95f4'} {"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}} DEBUG (shell:543) The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-a24ce04a-9ccb-41f5-aa68-9d7727ae95f4) Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/novaclient/shell.py", line 540, in main OpenStackComputeShell().main(sys.argv[1:]) File "/usr/lib/python2.6/site-packages/novaclient/shell.py", line 476, in main args.func(self.cs, args) File "/usr/lib/python2.6/site-packages/novaclient/v1_1/shell.py", line 1405, in do_floating_ip_create _print_floating_ip_list([cs.floating_ips.create(pool=args.pool)]) File "/usr/lib/python2.6/site-packages/novaclient/v1_1/floating_ips.py", line 41, in create return self._create("/os-floating-ips", {'pool': pool}, "floating_ip") File "/usr/lib/python2.6/site-packages/novaclient/base.py", line 148, in _create _resp, body = self.api.client.post(url, body=body) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 244, in post return self._cs_request(url, 'POST', **kwargs) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 228, in _cs_request **kwargs) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 210, in _time_request resp, body = self.request(url, method, **kwargs) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 204, in request raise exceptions.from_response(resp, body) ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-a24ce04a-9ccb-41f5-aa68-9d7727ae95f4) Thanks a lot, Cheers, Nicolas. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rbryant at redhat.com Wed Mar 27 16:34:37 2013 From: rbryant at redhat.com (Russell Bryant) Date: Wed, 27 Mar 2013 12:34:37 -0400 Subject: [rhos-list] floating IP config In-Reply-To: References: Message-ID: <51531F9D.6060808@redhat.com> On 03/27/2013 12:03 PM, Vogel Nicolas wrote: > Hello, > > > > Can I have floating IP to assign to my VM when I?m working with a > FlatNetworking mode? > > I have already a dhcp server in my subnet so I don?t need one. I?m just > using the br100 bridged to em2 in order to extend my subnet for my VMs. > > I can create a pool of floating IP but the then I?m unable to create one > with the ?nova floating-ip-create? command. > > I got following output: > > ERROR: The server has either erred or is incapable of performing the > requested operation. (HTTP 500) (Request-ID: > req-46ee5602-e356-47e1-a245-f4f1f0c532e9) Can you check the server-side logs after doing this? Look for a Traceback in nova-api.log to start with. -- Russell Bryant From nicolas.vogel at heig-vd.ch Wed Mar 27 16:47:32 2013 From: nicolas.vogel at heig-vd.ch (Vogel Nicolas) Date: Wed, 27 Mar 2013 16:47:32 +0000 Subject: [rhos-list] floating IP config In-Reply-To: <51531F9D.6060808@redhat.com> References: <51531F9D.6060808@redhat.com> Message-ID: Thanks for the tip, I checked all the nova logs but didn't saw that the first time. It's a problem with an RPC function which couldn't be find. I read about similar errors on the web but didn't find how to solve it. Here's the nova-api log: 2013-03-27 17:36:01 3890 TRACE nova.api.openstack RemoteError: Remote error: AttributeError No such RPC function 'allocate_floating_ip' 2013-03-27 17:36:01 3890 TRACE nova.api.openstack [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', u' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 148, in dispatch\n raise AttributeError("No such RPC function \'%s\'" % method)\n', u"AttributeError: No such RPC function 'allocate_floating_ip'\n"]. 2013-03-27 17:36:01 3890 TRACE nova.api.openstack 2013-03-27 17:36:01 INFO nova.api.openstack [req-2cc5bf1b-32fa-4d13-a630-a837a4a72b41 c3f5a9dfdffe49838c6f1b332c3ffeca a32f87f42fdd4577b62261b08e4e597c] http://10.192.75.190:8774/v1.1/a32f87f42fdd4577b62261b08e4e597c/os-floating-ips returned with HTTP 500 2013-03-27 17:36:01 INFO nova.osapi_compute.wsgi.server [req-2cc5bf1b-32fa-4d13-a630-a837a4a72b41 c3f5a9dfdffe49838c6f1b332c3ffeca a32f87f42fdd4577b62261b08e4e597c] 10.192.75.190 - - [27/Mar/2013 17:36:01] "POST /v1.1/a32f87f42fdd4577b62261b08e4e597c/os-floating-ips HTTP/1.1" 500 335 0.121185 !!!...No such RPC function 'allocate_floating_ip...!!! I think this is linked with the error I got when entering the "nova service-list" command: ERROR: n/a (HTTP 404) When I'm restarting all the nova services they all look fine but I can't display their status. Here's the debug: REQ: curl -i http://10.192.75.190:35357/v2.0/tokens -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-novaclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "secret"}}}' connect: (10.192.75.190, 35357) ************ send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: 10.192.75.190:35357\r\nContent-Length: 101\r\ncontent-type: application/json\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n' send: '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "secret"}}}' reply: 'HTTP/1.1 200 OK\r\n' header: Vary: X-Auth-Token header: Content-Type: application/json header: Content-Length: 2171 header: Date: Wed, 27 Mar 2013 16:39:24 GMT RESP:{'date': 'Wed, 27 Mar 2013 16:39:24 GMT', 'vary': 'X-Auth-Token', 'content-length': '2171', 'status': '200', 'content-type': 'application/json'} {"access": {"token": {"expires": "2013-03-28T16:39:24Z", "id": "7458ef3f7e464cc7855524a214a04486", "tenant": {"enabled": true, "description": null, "name": "admin", "id": "a32f87f42fdd4577b62261b08e4e597c"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.192.75.190:8776/v1/a32f87f42fdd4577b62261b08e4e597c", "region": "regionOne", "internalURL": "http://10.192.75.190:8776/v1/a32f87f42fdd4577b62261b08e4e597c", "id": "115d528d2b804f67bf1f5cd789583b52", "publicURL": "http://10.192.75.190:8776/v1/a32f87f42fdd4577b62261b08e4e597c"}], "endpoints_links": [], "type": "volume", "name": "cinder"}, {"endpoints": [{"adminURL": "http://10.192.75.190:8080/v1/AUTH_a32f87f42fdd4577b62261b08e4e597c", "region": "regionOne", "internalURL": "http://10.192.75.190:8080/v1/AUTH_a32f87f42fdd4577b62261b08e4e597c", "id": "b11d4db9596e4eee90c72962a372d097", "publicURL": "http://10.192.75.190:8080/v1/AUTH_a32f87f42fdd4577b62261b08e4e597c"}], "endpoints_links": [], "type": "object-store", "name": "swift"}, {"endpoints": [{"adminURL": "http://10.192.75.190:9292/v1", "region": "regionOne", "internalURL": "http://10.192.75.190:9292/v1", "id": "39cfe232d3ca43e2996f362c4d67e8fe", "publicURL": "http://10.192.75.190:9292/v1"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.192.75.190:8774/v1.1/a32f87f42fdd4577b62261b08e4e597c", "region": "regionOne", "internalURL": "http://10.192.75.190:8774/v1.1/a32f87f42fdd4577b62261b08e4e597c", "id": "62101f9435204e798be3eac5e961f38f", "publicURL": "http://10.192.75.190:8774/v1.1/a32f87f42fdd4577b62261b08e4e597c"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.192.75.190:35357/v2.0", "region": "RegionOne", "internalURL": "http://10.192.75.190:5000/v2.0", "id": "820740662d4b4b9c8a33cbcfc586fb37", "publicURL": "http://10.192.75.190:5000/v2.0"}], "endpoi nts_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "admin", "roles_links": [], "id": "c3f5a9dfdffe49838c6f1b332c3ffeca", "roles": [{"name": "admin"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles": ["1104d70fc1014bf8b725cf264731c88e"]}}} REQ: curl -i http://10.192.75.190:8774/v1.1/a32f87f42fdd4577b62261b08e4e597c/os-services -X GET -H "X-Auth-Project-Id: admin" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 7458ef3f7e464cc7855524a214a04486" connect: (10.192.75.190, 8774) ************ send: u'GET /v1.1/a32f87f42fdd4577b62261b08e4e597c/os-services HTTP/1.1\r\nHost: 10.192.75.190:8774\r\nx-auth-project-id: admin\r\nx-auth-token: 7458ef3f7e464cc7855524a214a04486\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n' reply: 'HTTP/1.1 404 Not Found\r\n' header: Content-Length: 52 header: Content-Type: text/plain; charset=UTF-8 header: Date: Wed, 27 Mar 2013 16:39:24 GMT RESP:{'date': 'Wed, 27 Mar 2013 16:39:24 GMT', 'status': '404', 'content-length': '52', 'content-type': 'text/plain; charset=UTF-8'} 404 Not Found The resource could not be found. DEBUG (shell:543) n/a (HTTP 404) Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/novaclient/shell.py", line 540, in main OpenStackComputeShell().main(sys.argv[1:]) File "/usr/lib/python2.6/site-packages/novaclient/shell.py", line 476, in main args.func(self.cs, args) File "/usr/lib/python2.6/site-packages/novaclient/v1_1/shell.py", line 1976, in do_service_list result = cs.services.list(args.host, args.servicename) File "/usr/lib/python2.6/site-packages/novaclient/v1_1/services.py", line 50, in list return self._list(url, "services") File "/usr/lib/python2.6/site-packages/novaclient/base.py", line 62, in _list _resp, body = self.api.client.get(url) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 241, in get return self._cs_request(url, 'GET', **kwargs) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 228, in _cs_request **kwargs) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 210, in _time_request resp, body = self.request(url, method, **kwargs) File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 204, in request raise exceptions.from_response(resp, body) NotFound: n/a (HTTP 404) ERROR: n/a (HTTP 404) The nova-api.log just says the same: 2013-03-27 17:39:24 INFO nova.osapi_compute.wsgi.server [req-6c9ca40f-5ede-4934-9578-5fdeb28118d9 c3f5a9dfdffe49838c6f1b332c3ffeca a32f87f42fdd4577b62261b08e4e597c] 10.192.75.190 - - [27/Mar/2013 17:39:24] "GET /v1.1/a32f87f42fdd4577b62261b08e4e597c/os-services HTTP/1.1" 404 176 0.043507 -----Original Message----- From: rhos-list-bounces at redhat.com [mailto:rhos-list-bounces at redhat.com] On Behalf Of Russell Bryant Sent: mercredi 27 mars 2013 17:35 To: rhos-list at redhat.com Subject: Re: [rhos-list] floating IP config On 03/27/2013 12:03 PM, Vogel Nicolas wrote: > Hello, > > > > Can I have floating IP to assign to my VM when I?m working with a > FlatNetworking mode? > > I have already a dhcp server in my subnet so I don?t need one. I?m > just using the br100 bridged to em2 in order to extend my subnet for my VMs. > > I can create a pool of floating IP but the then I?m unable to create > one with the ?nova floating-ip-create? command. > > I got following output: > > ERROR: The server has either erred or is incapable of performing the > requested operation. (HTTP 500) (Request-ID: > req-46ee5602-e356-47e1-a245-f4f1f0c532e9) Can you check the server-side logs after doing this? Look for a Traceback in nova-api.log to start with. -- Russell Bryant _______________________________________________ rhos-list mailing list rhos-list at redhat.com https://www.redhat.com/mailman/listinfo/rhos-list From rbryant at redhat.com Wed Mar 27 22:57:31 2013 From: rbryant at redhat.com (Russell Bryant) Date: Wed, 27 Mar 2013 18:57:31 -0400 Subject: [rhos-list] floating IP config In-Reply-To: References: <51531F9D.6060808@redhat.com> Message-ID: <5153795B.6050909@redhat.com> On 03/27/2013 12:47 PM, Vogel Nicolas wrote: > Thanks for the tip, I checked all the nova logs but didn't saw that the first time. > It's a problem with an RPC function which couldn't be find. I read about similar errors on the web but didn't find how to solve it. > > Here's the nova-api log: > 2013-03-27 17:36:01 3890 TRACE nova.api.openstack RemoteError: Remote error: AttributeError No such RPC function 'allocate_floating_ip' > 2013-03-27 17:36:01 3890 TRACE nova.api.openstack [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', u' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 148, in dispatch\n raise AttributeError("No such RPC function \'%s\'" % method)\n', u"AttributeError: No such RPC function 'allocate_floating_ip'\n"]. > 2013-03-27 17:36:01 3890 TRACE nova.api.openstack > 2013-03-27 17:36:01 INFO nova.api.openstack [req-2cc5bf1b-32fa-4d13-a630-a837a4a72b41 c3f5a9dfdffe49838c6f1b332c3ffeca a32f87f42fdd4577b62261b08e4e597c] http://10.192.75.190:8774/v1.1/a32f87f42fdd4577b62261b08e4e597c/os-floating-ips returned with HTTP 500 > 2013-03-27 17:36:01 INFO nova.osapi_compute.wsgi.server [req-2cc5bf1b-32fa-4d13-a630-a837a4a72b41 c3f5a9dfdffe49838c6f1b332c3ffeca a32f87f42fdd4577b62261b08e4e597c] 10.192.75.190 - - [27/Mar/2013 17:36:01] "POST /v1.1/a32f87f42fdd4577b62261b08e4e597c/os-floating-ips HTTP/1.1" 500 335 0.121185 > > !!!...No such RPC function 'allocate_floating_ip...!!! Thanks for this. That clarifies exactly what's going on ... back to your original question, floating IPs are *not* supported with the FlatManager. Nova seems to have done a pretty bad job at letting you know that was the problem, though. Feel free to file a bug on bugzilla.redhat.com so that we can make this cleaner for the future. -- Russell Bryant From jpichon at redhat.com Thu Mar 28 10:44:51 2013 From: jpichon at redhat.com (Julie Pichon) Date: Thu, 28 Mar 2013 06:44:51 -0400 (EDT) Subject: [rhos-list] Add user field in the instance panel of dashboard In-Reply-To: <1364307738.fd9e2ae32b53addc06c63208be3aaa43@mail.in.com> Message-ID: <360426092.7708073.1364467491605.JavaMail.root@redhat.com> Hi, "Kumar Vaibhav" wrote: > Hi,As a admin I find it difficult to know instanceuser mapping.I want > to add the userid information to be added for the Dashboard on the > instance page. Is it possible to do this? How I can augment the > dashboard to add this feature?Regards,Vaibhav It should be possible to do this. There is some documentation on how to modify an existing panel: http://docs.openstack.org/developer/horizon/topics/customizing.html#modifying-existing-dashboards-and-panels I haven't tried this but I think your best bet would be to create a new customised panel that extends the existing one, in which you modify the Table to include the additional field. Then you would unregister the existing Instances panel using the previous docs, and register yours to replace it. You can see the existing Table at https://github.com/openstack/horizon/blob/stable/folsom/horizon/dashboards/syspanel/instances/tables.py#L44 . Looking at that code, it looks like there already used to be a user_id (line 64) but it's commented out due to scalability concerns when expanding it into a name. If you only care about the user_id, this is the line you should add to your customised Table. Hope this helps, Julie