From bmr at redhat.com Mon Dec 1 15:02:28 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Mon, 1 Dec 2014 15:02:28 +0000 Subject: [sos-devel] [PATCH] [plugins/openstack_neutron] Add netns dumps for DVR namespaces In-Reply-To: <36AF4B62444F4B4FB04EDD1FDAE1CF1D576725A9@G9W0344.americas.hpqcorp.net> References: <1416639978-3173-1-git-send-email-stephen.ma@hp.com> <20141125115712.GB31857@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D57671BE6@G9W0344.americas.hpqcorp.net> <20141128110352.GA3703@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D576725A9@G9W0344.americas.hpqcorp.net> Message-ID: <20141201150227.GA29725@localhost.localdomain> On Fri, Nov 28, 2014 at 06:09:44PM +0000, Ma, Stephen B. wrote: > Hi Bryn, > > The fip and snat namespaces are created by the neutron-l3-agent when it is running in the dvr or dvr_snat mode. They are not used outside of neutron. That's still not what I mean ;) There is nothing to stop *something else* creating namespaces with those prefixes; therefore we cannot assume that because we find those prefixes in use that we're looking at a Neutron DVR configuration. We should only assume that in the case that we already know that the system is running OpenStack Neutron. > One thing to note is that when Neutron is running on a machine, it does not necessary implies that qdhcp, qrouter, fip, or the snat namespaces are present on the machine. For example, neutron-l3-agent manages the qrouter, fip, and snat namespaces. When the agent crashed, these namespaces are left behind. So if you run sosreport at the time the agent is not running, I would still like to see the information on these 3 namespaces. The information would provide clues as to what is the problem. That's not a problem; we can collect data for them regardless of whether the agent is running or not. Regards, Bryn. From bmr at redhat.com Tue Dec 2 11:44:15 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Tue, 2 Dec 2014 11:44:15 +0000 Subject: [sos-devel] [PATCH] [plugins/openstack_neutron] Add netns dumps for DVR namespaces In-Reply-To: <36AF4B62444F4B4FB04EDD1FDAE1CF1D57672CB2@G9W0344.americas.hpqcorp.net> References: <1416639978-3173-1-git-send-email-stephen.ma@hp.com> <20141125115712.GB31857@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D57671BE6@G9W0344.americas.hpqcorp.net> <20141128110352.GA3703@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D576725A9@G9W0344.americas.hpqcorp.net> <20141201150227.GA29725@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D57672CB2@G9W0344.americas.hpqcorp.net> Message-ID: <20141202114414.GA19827@localhost.localdomain> On Mon, Dec 01, 2014 at 03:52:44PM +0000, Ma, Stephen B. wrote: > Of course *something else* or *some administrator* can create a namespace with starting with "fip", "snat", "qrouter", or "qdhcp". But that is highly unlikely. Only neutron creates and destroy these namespaces. > > Do you need any help in looking into how add checks to see whether neutron is running? Do I need to resubmit my patch again? No need - I'll apply it after I've added a package check to the module (they're unlikely to conflict & it's a trivial change anyway). Regards, Bryn. From sma102874 at gmail.com Thu Dec 4 03:32:46 2014 From: sma102874 at gmail.com (Stephen Ma) Date: Wed, 3 Dec 2014 19:32:46 -0800 Subject: [sos-devel] [PATCH] [plugins/openvswitch] Add flow information from OVS bridges Message-ID: <1417663966-2804-1-git-send-email-stephen.ma@hp.com> Add ovs-ofctl dump-flows output to openvswitch plugin report. Signed-off-by: Stephen Ma --- sos/plugins/openvswitch.py | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/sos/plugins/openvswitch.py b/sos/plugins/openvswitch.py index 3611671..e92d9f4 100644 --- a/sos/plugins/openvswitch.py +++ b/sos/plugins/openvswitch.py @@ -30,7 +30,30 @@ class OpenVSwitch(Plugin): # The '-t 5' adds an upper bound on how long to wait to connect # to the Open vSwitch server, avoiding hangs when running sosreport. - self.add_cmd_output("ovs-vsctl -t 5 show") + vsctl_file = self.get_cmd_output_now("ovs-vsctl -t 5 show") + self.get_bridge_flows(vsctl_file) + + def get_bridge_flows(self, vsctl_file): + bridges = self.get_bridge_names(vsctl_file) + for bridge in bridges: + self.add_cmd_output("ovs-ofctl dump-flows " + bridge) + + def get_bridge_names(self, vsctl_file): + """Return a list for which items are bridge name according to the + output of ovs-vsctl show stored in vsctl_file. + """ + out = [] + try: + vsctl_out = open(vsctl_file).read() + except Exception: + return out + + for line in vsctl_out.splitlines(): + tokens = line.split() + if len(tokens) == 2 and tokens[0] == "Bridge": + out.append(tokens[1]) + + return out class RedHatOpenVSwitch(OpenVSwitch, RedHatPlugin): -- 1.9.1 From christy at linux.vnet.ibm.com Fri Dec 5 21:02:31 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Fri, 5 Dec 2014 15:02:31 -0600 Subject: [sos-devel] [PATCH] Add an sos plugin for the kimchi platform Message-ID: <1417813351-17552-1-git-send-email-christy@linux.vnet.ibm.com> This plugin gathers information about libvirt storage pools defined on the system. Kimchi is an HTML5-based virtualization-management platform For more information, please visit: https://github.com/kimchi-project/kimchi/wiki Signed-off-by: Christy Perez --- sos/plugins/kimchi.py | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 sos/plugins/kimchi.py diff --git a/sos/plugins/kimchi.py b/sos/plugins/kimchi.py new file mode 100644 index 0000000..479cfd1 --- /dev/null +++ b/sos/plugins/kimchi.py @@ -0,0 +1,39 @@ +### This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. + +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. + +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin + +class Kimchi(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): + """kimchi-related information + """ + + plugin_name = 'kimchi' + packages = ('kimchi',) + + def setup(self): + self.add_copy_specs([ + "/etc/kimchi/", + "/var/log/kimchi*" + ]) + + file_name = self.get_cmd_output_now("virsh -r pool-list --details") + if file_name is not None: + with open(file_name, 'r') as pools: + for pool in list(pools)[2:]: + try: + pool_name = pool.lstrip().split()[0] + self.get_cmd_output_now("virsh -r vol-list --pool \ + %s --details" % pool_name) + except: + pass -- 1.9.3 From christy at linux.vnet.ibm.com Sat Dec 6 00:49:51 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Fri, 5 Dec 2014 18:49:51 -0600 Subject: [sos-devel] [PATCH v2] Add an sos plugin for the kimchi platform Message-ID: <1417826991-5998-1-git-send-email-christy@linux.vnet.ibm.com> This plugin gathers kimchi logs as well as information about libvirt storage pools defined on the system. Kimchi is an HTML5-based virtualization-management platform For more information, please visit: https://github.com/kimchi-project/kimchi/wiki Signed-off-by: Christy Perez --- sos/plugins/kimchi.py | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 sos/plugins/kimchi.py diff --git a/sos/plugins/kimchi.py b/sos/plugins/kimchi.py new file mode 100644 index 0000000..69f138e --- /dev/null +++ b/sos/plugins/kimchi.py @@ -0,0 +1,39 @@ +### This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. + +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. + +## You should have received a copy of the GNU General Public License +## along with this program; if not, write to the Free Software +## Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin + +class Kimchi(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): + """kimchi-related information + """ + + plugin_name = 'kimchi' + packages = ('kimchi',) + + def setup(self): + self.add_copy_spec([ + "/etc/kimchi/", + "/var/log/kimchi*" + ]) + + file_name = self.get_cmd_output_now("virsh -r pool-list --details") + if file_name is not None: + with open(file_name, 'r') as pools: + for pool in list(pools)[2:]: + try: + pool_name = pool.lstrip().split()[0] + self.get_cmd_output_now("virsh -r vol-list --pool \ + %s --details" % pool_name) + except: + pass -- 1.9.3 From bmr at redhat.com Tue Dec 9 16:41:58 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Tue, 9 Dec 2014 16:41:58 +0000 Subject: [sos-devel] [PATCH v2] Add an sos plugin for the kimchi platform In-Reply-To: <1417826991-5998-1-git-send-email-christy@linux.vnet.ibm.com> References: <1417826991-5998-1-git-send-email-christy@linux.vnet.ibm.com> Message-ID: <20141209164157.GA12929@localhost.localdomain> On Fri, Dec 05, 2014 at 06:49:51PM -0600, Christy Perez wrote: > This plugin gathers kimchi logs as well as information > about libvirt storage pools defined on the system. > > Kimchi is an HTML5-based virtualization-management platform > For more information, please visit: > https://github.com/kimchi-project/kimchi/wiki > > Signed-off-by: Christy Perez Hi Christy, Thanks for submitting the plugin. > --- > sos/plugins/kimchi.py | 39 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 39 insertions(+) > create mode 100644 sos/plugins/kimchi.py You're welcome to add a copyright header here if you wish (we don't have a strict policy either way - plugin authors are free to add one or not). > +### This program is free software; you can redistribute it and/or modify > +## it under the terms of the GNU General Public License as published by > +## the Free Software Foundation; either version 2 of the License, or > +## (at your option) any later version. > + def setup(self): > + self.add_copy_spec([ > + "/etc/kimchi/", > + "/var/log/kimchi*" > + ]) Because of the possible size of logs (esp. on systems that have been running for some time and that do not have log rotation enabled) we try to limit the size of files collected from /var/log/ by default. The add_copy_spec_limit() method allows you to do this for single files, or lists of files (with optional glob expansion). The only limitation is that the size restrictions do not work for directories. There's also the global 'all_logs' option that overrides this so something like this is common: if not self.get_option("all_logs"): self.add_copy_spec_limit("/var/log/kimchi*.log", sizelimit=limit) else: self.add_copy_spec("/var/log/kimchi*.log") > + file_name = self.get_cmd_output_now("virsh -r pool-list --details") > + if file_name is not None: > + with open(file_name, 'r') as pools: > + for pool in list(pools)[2:]: > + try: > + pool_name = pool.lstrip().split()[0] > + self.get_cmd_output_now("virsh -r vol-list --pool \ > + %s --details" % pool_name) You only need to use get_cmd_output_now() when you both want to store the result in the report and have it immediately available (e.g. to drive further collection). Since that's not the case here you can just use add_cmd_output(). More importantly though I don't really think that these virsh operations belong in the kimchi plugin. There's nothing kimchi-specific that I can see here and we already have a libvirt plugin - it seems to make more sense to add this collections there. Regards, Bryn. From streeter at redhat.com Wed Dec 10 16:21:10 2014 From: streeter at redhat.com (Guy Streeter) Date: Wed, 10 Dec 2014 10:21:10 -0600 Subject: [sos-devel] How to avoid reading /proc/net//rpc/use-gss-proxy ? Message-ID: <548872F6.7040405@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The currently-released MRG RT kernel has a bug that causes a read of /proc/net/rpc/use-gss-proxy to hang in most cases. Is there a way to bypass this and get a successful sosreport from the system? thanks, - --Guy -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEUEARECAAYFAlSIcvYACgkQ0Bme0QyNhPQ0HQCWIZR28jrOYOwGRtbW/p+A6bBH PwCghyMbhrxlazqTbjo6FPe5Tq0ynk8= =6yOC -----END PGP SIGNATURE----- From bmr at redhat.com Wed Dec 10 16:54:10 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Wed, 10 Dec 2014 16:54:10 +0000 Subject: [sos-devel] How to avoid reading /proc/net//rpc/use-gss-proxy ? In-Reply-To: <548872F6.7040405@redhat.com> References: <548872F6.7040405@redhat.com> Message-ID: <20141210165409.GA7321@localhost.localdomain> On Wed, Dec 10, 2014 at 10:21:10AM -0600, Guy Streeter wrote: > The currently-released MRG RT kernel has a bug that causes a read of > /proc/net/rpc/use-gss-proxy > to hang in most cases. Is there a way to bypass this and get a successful > sosreport from the system? You add a line like this to the plugin: self.add_forbidden_path("/proc/net/rpc/use-gss-proxy") But this has been upstream since: commit 44e01afbd86f60760af85ad64810e56cf5e77140 Author: Bryn M. Reeves Date: Wed Aug 14 15:58:13 2013 +0100 Do not attempt to read use-gss-proxy file in procfs The networking plug-in scoops up /proc/net. There are some pseudo- files in here that we should avoid touching. These either have side-effects or hang the reading process. Add a forbidden path for the /proc/net/rpc/use-gss-proxy file. Signed-off-by: Bryn M. Reeves It's been in sos-3.0 and sos-3.2 packages since the start and was backported to sos-2.2 (RHEL6) earlier this year: * Mon Jun 23 2014 Bryn M. Reeves = 2.2-57.el6 - [networking] do not attempt to read use-gss-proxy Resolves: bz1079954 The EL6 version of the change is slightly different: self.addForbiddenPath("/proc/net//rpc/use-gss-proxy") (the API names are different in 2.2 and the '//' is required to work around the handling of /proc/net as a namespace symlink). This was released along with RHEL-6.6. We've not had any requests to backport it to any earlier branches so far. If you're looking at making changes I'd also suggest taking the related channel and flush fixes: >= 3.0: self.add_forbidden_path("/proc/net/rpc/*/channel") self.add_forbidden_path("/proc/net/rpc/*/flush") <= 2.2: self.addForbiddenPath("/proc/net//rpc/*/channel") self.addForbiddenPath("/proc/net//rpc/*/flush") Regards, Bryn. From bmr at redhat.com Wed Dec 10 16:55:54 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Wed, 10 Dec 2014 16:55:54 +0000 Subject: [sos-devel] How to avoid reading /proc/net//rpc/use-gss-proxy ? In-Reply-To: <548872F6.7040405@redhat.com> References: <548872F6.7040405@redhat.com> Message-ID: <20141210165553.GB7321@localhost.localdomain> On Wed, Dec 10, 2014 at 10:21:10AM -0600, Guy Streeter wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > The currently-released MRG RT kernel has a bug that causes a read of > /proc/net/rpc/use-gss-proxy > to hang in most cases. Is there a way to bypass this and get a successful > sosreport from the system? Oh, and in case modifying plugin files or updating the package is not acceptable for whatever reason, just disable the 'networking ' plugin: # sosreport -n networking Although that obviously costs some useful diagnostic info. Regards, Bryn. From bmr at redhat.com Wed Dec 10 17:32:19 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Wed, 10 Dec 2014 17:32:19 +0000 Subject: [sos-devel] [PATCH v3] sosreport: Check for rpm database corruption during initialization In-Reply-To: <20141015104053.32616.94797.stgit@aruna-ThinkPad-T420> References: <20141015104053.32616.94797.stgit@aruna-ThinkPad-T420> Message-ID: <20141210173218.GC7321@localhost.localdomain> Hi Aruna, Sorry for taking so long to get back to this. The sos/utilities.py changes all look fine now but I think there are still a few improvements I'd like to see in the PackageManager changes: On Wed, Oct 15, 2014 at 04:12:38PM +0530, Aruna Balakrishnaiah wrote: > - def __init__(self, query_command=None): > + def __init__(self, query_command=None, timeout=180): Is there a need to make the timeout configurable here (and does it make sense to do so)? We're discussing adding a --timeout option to allow the user to control the timeouts used to collect data but I'm not sure it should be passed through to the PackageManager initialiser (as you then could set it so low that the tool always fails). > diff --git a/sos/policies/redhat.py b/sos/policies/redhat.py > index 2219246..e5a3f3f 100644 > --- a/sos/policies/redhat.py > +++ b/sos/policies/redhat.py > @@ -21,6 +21,7 @@ import sys > from sos.plugins import RedHatPlugin > from sos.policies import LinuxPolicy, PackageManager > from sos import _sos as _ > +from distutils.log import error Why use distutils.log here? It's not used anywhere else in sos; it would be cleaner to just print to stderr: print(error_message, file=sys.stderr) There is also the Policy._print() method (which also obeys the setting of --quiet). > + # If rpm query timed out after timeout duration exit > + if not pkgs: > + error("Could not obtain installed package list") > + sys.exit(1) > + Regards, Bryn. From aruna at linux.vnet.ibm.com Thu Dec 11 06:45:30 2014 From: aruna at linux.vnet.ibm.com (Aruna Balakrishnaiah) Date: Thu, 11 Dec 2014 12:15:30 +0530 Subject: [sos-devel] [PATCH v3] sosreport: Check for rpm database corruption during initialization In-Reply-To: <20141210173218.GC7321@localhost.localdomain> References: <20141015104053.32616.94797.stgit@aruna-ThinkPad-T420> <20141210173218.GC7321@localhost.localdomain> Message-ID: <54893D8A.6040007@linux.vnet.ibm.com> On Wednesday 10 December 2014 11:02 PM, Bryn M. Reeves wrote: > Hi Aruna, > > Sorry for taking so long to get back to this. The sos/utilities.py > changes all look fine now but I think there are still a few > improvements I'd like to see in the PackageManager changes: > > On Wed, Oct 15, 2014 at 04:12:38PM +0530, Aruna Balakrishnaiah wrote: >> - def __init__(self, query_command=None): >> + def __init__(self, query_command=None, timeout=180): > Is there a need to make the timeout configurable here (and does it > make sense to do so)? > > We're discussing adding a --timeout option to allow the user to > control the timeouts used to collect data but I'm not sure it should > be passed through to the PackageManager initialiser (as you then > could set it so low that the tool always fails). > >> diff --git a/sos/policies/redhat.py b/sos/policies/redhat.py >> index 2219246..e5a3f3f 100644 >> --- a/sos/policies/redhat.py >> +++ b/sos/policies/redhat.py >> @@ -21,6 +21,7 @@ import sys >> from sos.plugins import RedHatPlugin >> from sos.policies import LinuxPolicy, PackageManager >> from sos import _sos as _ >> +from distutils.log import error > Why use distutils.log here? It's not used anywhere else in sos; it > would be cleaner to just print to stderr: > > print(error_message, file=sys.stderr) Bryn, This is available only since Python 3.0, lower versions are not supporting this. > > There is also the Policy._print() method (which also obeys the > setting of --quiet). Is there a need to create a new Policy instance just to print?Can I just use print_ ? And should we not print this error message even if its in quiet mode. Regards, Aruna > >> + # If rpm query timed out after timeout duration exit >> + if not pkgs: >> + error("Could not obtain installed package list") >> + sys.exit(1) >> + > Regards, > Bryn. > From bmr at redhat.com Thu Dec 11 11:14:05 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Thu, 11 Dec 2014 11:14:05 +0000 Subject: [sos-devel] [PATCH v3] sosreport: Check for rpm database corruption during initialization In-Reply-To: <54893D8A.6040007@linux.vnet.ibm.com> References: <20141015104053.32616.94797.stgit@aruna-ThinkPad-T420> <20141210173218.GC7321@localhost.localdomain> <54893D8A.6040007@linux.vnet.ibm.com> Message-ID: <20141211111405.GA21604@localhost.localdomain> On Thu, Dec 11, 2014 at 12:15:30PM +0530, Aruna Balakrishnaiah wrote: > On Wednesday 10 December 2014 11:02 PM, Bryn M. Reeves wrote: > >Why use distutils.log here? It's not used anywhere else in sos; it > >would be cleaner to just print to stderr: > > > > print(error_message, file=sys.stderr) > > Bryn, > > This is available only since Python 3.0, lower versions are not supporting this. No that's not correct - 3.0 just makes the print function the default (and remove the print statement). The print() function is available in python 2.6 (our oldest supported runtime) and onwards via the __future__ module: # python Python 2.6.6 (r266:84292, Nov 21 2013, 10:50:32) [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from __future__ import print_statement >>> print("error", file=sys.stderr) error We use __future__ features (with_statement) in several places already so it'd be fine to use for this purpose. > > > > >There is also the Policy._print() method (which also obeys the > >setting of --quiet). > > Is there a need to create a new Policy instance just to print?Can I just use > print_ ? > And should we not print this error message even if its in quiet mode. As it's an error it should also appear on stderr rather than stdout; that's the sort of API change that may be needed to implement a change like this. Regards, Bryn. From christy at linux.vnet.ibm.com Thu Dec 11 15:47:52 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Thu, 11 Dec 2014 09:47:52 -0600 Subject: [sos-devel] [PATCH v2] Add an sos plugin for the kimchi platform In-Reply-To: <20141209164157.GA12929@localhost.localdomain> References: <1417826991-5998-1-git-send-email-christy@linux.vnet.ibm.com> <20141209164157.GA12929@localhost.localdomain> Message-ID: <5489BCA8.4010701@linux.vnet.ibm.com> On 12/09/2014 10:41 AM, Bryn M. Reeves wrote: > On Fri, Dec 05, 2014 at 06:49:51PM -0600, Christy Perez wrote: >> This plugin gathers kimchi logs as well as information >> about libvirt storage pools defined on the system. >> >> Kimchi is an HTML5-based virtualization-management platform >> For more information, please visit: >> https://github.com/kimchi-project/kimchi/wiki >> >> Signed-off-by: Christy Perez > > Hi Christy, > > Thanks for submitting the plugin. Thanks for the suggestions! I'm sending a v3 implementing them all. > >> --- >> sos/plugins/kimchi.py | 39 +++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 39 insertions(+) >> create mode 100644 sos/plugins/kimchi.py > > You're welcome to add a copyright header here if you wish (we don't have > a strict policy either way - plugin authors are free to add one or not). > >> +### This program is free software; you can redistribute it and/or modify >> +## it under the terms of the GNU General Public License as published by >> +## the Free Software Foundation; either version 2 of the License, or >> +## (at your option) any later version. > >> + def setup(self): >> + self.add_copy_spec([ >> + "/etc/kimchi/", >> + "/var/log/kimchi*" >> + ]) > > Because of the possible size of logs (esp. on systems that have been > running for some time and that do not have log rotation enabled) we try > to limit the size of files collected from /var/log/ by default. > > The add_copy_spec_limit() method allows you to do this for single files, > or lists of files (with optional glob expansion). The only limitation is > that the size restrictions do not work for directories. > > There's also the global 'all_logs' option that overrides this so > something like this is common: > > if not self.get_option("all_logs"): > self.add_copy_spec_limit("/var/log/kimchi*.log", sizelimit=limit) > else: > self.add_copy_spec("/var/log/kimchi*.log") > >> + file_name = self.get_cmd_output_now("virsh -r pool-list --details") >> + if file_name is not None: >> + with open(file_name, 'r') as pools: >> + for pool in list(pools)[2:]: >> + try: >> + pool_name = pool.lstrip().split()[0] >> + self.get_cmd_output_now("virsh -r vol-list --pool \ >> + %s --details" % pool_name) > > You only need to use get_cmd_output_now() when you both want to store > the result in the report and have it immediately available (e.g. to > drive further collection). Since that's not the case here you can just > use add_cmd_output(). > > More importantly though I don't really think that these virsh operations > belong in the kimchi plugin. There's nothing kimchi-specific that I can see > here and we already have a libvirt plugin - it seems to make more sense to > add this collections there. > > Regards, > Bryn. > From christy at linux.vnet.ibm.com Thu Dec 11 15:48:22 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Thu, 11 Dec 2014 09:48:22 -0600 Subject: [sos-devel] [PATCH v3] Patch sos plugin for rhev Message-ID: <1418312902-16996-1-git-send-email-christy@linux.vnet.ibm.com> Until there is a community sos plugin for kimchi, this gathers kimichi logs and config information. The original internal plugin had a section to capture virsh pool info, but that is duplicating the libvirt logs and xml files gathered by the libvirt plugin. Removing this also takes care of the issue that when running on RHEV, virsh commands should use the --read-only flag. Signed-off-by: Christy Perez --- src/kimchi/sos.py | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/src/kimchi/sos.py b/src/kimchi/sos.py index 8a0e34a..2b2b7cb 100644 --- a/src/kimchi/sos.py +++ b/src/kimchi/sos.py @@ -12,26 +12,25 @@ ## along with this program; if not, write to the Free Software ## Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin -from sos.utilities import sos_get_command_output class Kimchi(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): """kimchi-related information """ plugin_name = 'kimchi' + packages = ('kimchi',) + + option_list = [('log_limit', 'maximum size (MiB) of each log file', + '', 15)] def setup(self): - self.add_copy_specs([ - "/etc/kimchi/", - "/var/log/kimchi*" - ]) - self.add_cmd_output("virsh pool-list --details") - rc, out, _ = sos_get_command_output('virsh pool-list') - if rc == 0: - for pool in out.splitlines()[2:]: - if pool: - pool_name = pool.split()[0] - self.add_cmd_output("virsh vol-list --pool %s --details" - % pool_name) + log_limit = self.get_option('log_limit') + if not self.get_option('all_logs'): + self.add_copy_spec_limit('/var/log/kimchi/*.log', sizelimit=log_limit) + self.add_copy_spec_limit('/etc/kimchi/kimchi*', sizelimit=log_limit) + self.add_copy_spec_limit('/etc/kimchi/distros.d/*.json', sizelimit=log_limit) + else: + self.add_copy_spec('/var/log/kimchi/') + self.add_copy_spec('/etc/kimchi/') + -- 1.9.3 From christy at linux.vnet.ibm.com Thu Dec 11 15:53:13 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Thu, 11 Dec 2014 09:53:13 -0600 Subject: [sos-devel] [PATCH v3] Patch sos plugin for rhev In-Reply-To: <1418312902-16996-1-git-send-email-christy@linux.vnet.ibm.com> References: <1418312902-16996-1-git-send-email-christy@linux.vnet.ibm.com> Message-ID: <5489BDE9.7050205@linux.vnet.ibm.com> Sorry. Sent this patch from the wrong branch. Please disregard. On 12/11/2014 09:48 AM, Christy Perez wrote: > Until there is a community sos plugin for kimchi, this gathers > kimichi logs and config information. > > The original internal plugin had a section to capture virsh > pool info, but that is duplicating the libvirt logs and xml files > gathered by the libvirt plugin. Removing this also takes care > of the issue that when running on RHEV, virsh commands should > use the --read-only flag. > > Signed-off-by: Christy Perez > --- > src/kimchi/sos.py | 27 +++++++++++++-------------- > 1 file changed, 13 insertions(+), 14 deletions(-) > > diff --git a/src/kimchi/sos.py b/src/kimchi/sos.py > index 8a0e34a..2b2b7cb 100644 > --- a/src/kimchi/sos.py > +++ b/src/kimchi/sos.py > @@ -12,26 +12,25 @@ > ## along with this program; if not, write to the Free Software > ## Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. > > - > from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin > -from sos.utilities import sos_get_command_output > > class Kimchi(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): > """kimchi-related information > """ > > plugin_name = 'kimchi' > + packages = ('kimchi',) > + > + option_list = [('log_limit', 'maximum size (MiB) of each log file', > + '', 15)] > > def setup(self): > - self.add_copy_specs([ > - "/etc/kimchi/", > - "/var/log/kimchi*" > - ]) > - self.add_cmd_output("virsh pool-list --details") > - rc, out, _ = sos_get_command_output('virsh pool-list') > - if rc == 0: > - for pool in out.splitlines()[2:]: > - if pool: > - pool_name = pool.split()[0] > - self.add_cmd_output("virsh vol-list --pool %s --details" > - % pool_name) > + log_limit = self.get_option('log_limit') > + if not self.get_option('all_logs'): > + self.add_copy_spec_limit('/var/log/kimchi/*.log', sizelimit=log_limit) > + self.add_copy_spec_limit('/etc/kimchi/kimchi*', sizelimit=log_limit) > + self.add_copy_spec_limit('/etc/kimchi/distros.d/*.json', sizelimit=log_limit) > + else: > + self.add_copy_spec('/var/log/kimchi/') > + self.add_copy_spec('/etc/kimchi/') > + > From christy at linux.vnet.ibm.com Thu Dec 11 22:12:20 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Thu, 11 Dec 2014 16:12:20 -0600 Subject: [sos-devel] [PATCH v3] Add plugin for Kimchi Message-ID: <1418335940-25000-1-git-send-email-christy@linux.vnet.ibm.com> This plugin gathers kimchi logs and configuration files. Kimchi is an HTML5-based virtualization-management platform For more information, please visit: https://github.com/kimchi-project/kimchi/wiki Signed-off-by: Christy Perez --- sos/plugins/kimchi.py | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 sos/plugins/kimchi.py diff --git a/sos/plugins/kimchi.py b/sos/plugins/kimchi.py new file mode 100644 index 0000000..1d346d1 --- /dev/null +++ b/sos/plugins/kimchi.py @@ -0,0 +1,43 @@ +# Copyright IBM, Corp. 2014, Christy Perez + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin + + +class Kimchi(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): + """kimchi-related information + """ + + plugin_name = 'kimchi' + packages = ('kimchi',) + + option_list = [('logsize', 'maximum size (MiB) of each log file', '', 15), + ('all_logs', 'collect all logs, regardless of size', '', + False) + ] + + def setup(self): + log_limit = self.get_option('logsize') + if not self.get_option('all_logs'): + self.add_copy_spec_limit('/var/log/kimchi/*.log', + sizelimit=log_limit) + self.add_copy_spec_limit('/etc/kimchi/kimchi*', + sizelimit=log_limit) + self.add_copy_spec_limit('/etc/kimchi/distros.d/*.json', + sizelimit=log_limit) + else: + self.add_copy_spec('/var/log/kimchi/') + self.add_copy_spec('/etc/kimchi/') -- 1.9.3 From aruna at linux.vnet.ibm.com Fri Dec 12 05:24:32 2014 From: aruna at linux.vnet.ibm.com (Aruna Balakrishnaiah) Date: Fri, 12 Dec 2014 10:54:32 +0530 Subject: [sos-devel] [PATCH v3] sosreport: Check for rpm database corruption during initialization In-Reply-To: <20141211111405.GA21604@localhost.localdomain> References: <20141015104053.32616.94797.stgit@aruna-ThinkPad-T420> <20141210173218.GC7321@localhost.localdomain> <54893D8A.6040007@linux.vnet.ibm.com> <20141211111405.GA21604@localhost.localdomain> Message-ID: <548A7C10.3080702@linux.vnet.ibm.com> On Thursday 11 December 2014 04:44 PM, Bryn M. Reeves wrote: > On Thu, Dec 11, 2014 at 12:15:30PM +0530, Aruna Balakrishnaiah wrote: >> On Wednesday 10 December 2014 11:02 PM, Bryn M. Reeves wrote: >>> Why use distutils.log here? It's not used anywhere else in sos; it >>> would be cleaner to just print to stderr: >>> >>> print(error_message, file=sys.stderr) >> Bryn, >> >> This is available only since Python 3.0, lower versions are not supporting this. > No that's not correct - 3.0 just makes the print function the default > (and remove the print statement). The print() function is available in > python 2.6 (our oldest supported runtime) and onwards via the __future__ > module: > > # python > Python 2.6.6 (r266:84292, Nov 21 2013, 10:50:32) > [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> from __future__ import print_statement In python 2.7 it is print_function and not print_statement. Will send the patch with this change. Regards, Aruna From aruna at linux.vnet.ibm.com Fri Dec 12 05:57:07 2014 From: aruna at linux.vnet.ibm.com (Aruna Balakrishnaiah) Date: Fri, 12 Dec 2014 11:27:07 +0530 Subject: [sos-devel] [PATCH v4] sosreport: Check for rpm database corruption during initialization Message-ID: <20141212053729.27655.50568.stgit@aruna-ThinkPad-T420> sosreport runs an rpm query to get the package list. If rpmdb is corrupted sosreport hangs for ever hence check for rpmdb consistency before running the rpm query Signed-off-by: Aruna Balakrishnaiah --- Changes from v3: Add default timeout instead of making it configurable Add print function instead of error() Changes from v2: Introduce timeout in shell_out wrapper Changes from v1: Addressed issues of the maintainer Introduce timeout instead of relying on yum. sos/policies/__init__.py | 3 ++- sos/policies/redhat.py | 10 +++++++++- sos/utilities.py | 4 ++-- 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/sos/policies/__init__.py b/sos/policies/__init__.py index 9fcbd55..7ec8d61 100644 --- a/sos/policies/__init__.py +++ b/sos/policies/__init__.py @@ -59,6 +59,7 @@ class PackageManager(object): def __init__(self, query_command=None): self.packages = {} + self.timeout = 180 if query_command: self.query_command = query_command @@ -92,7 +93,7 @@ class PackageManager(object): version': 'major.minor.version'}} """ if self.query_command: - pkg_list = shell_out(self.query_command).splitlines() + pkg_list = shell_out(self.query_command, self.timeout).splitlines() for pkg in pkg_list: if '|' not in pkg: continue diff --git a/sos/policies/redhat.py b/sos/policies/redhat.py index 2219246..38510d9 100644 --- a/sos/policies/redhat.py +++ b/sos/policies/redhat.py @@ -15,6 +15,7 @@ # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. # This enables the use of with syntax in python 2.5 (e.g. jython) +from __future__ import print_function import os import sys @@ -46,8 +47,15 @@ class RedHatPolicy(LinuxPolicy): 'rpm -qa --queryformat "%{NAME}|%{VERSION}\\n"') self.valid_subclasses = [RedHatPlugin] + pkgs = self.package_manager.all_pkgs() + + # If rpm query timed out after timeout duration exit + if not pkgs: + print("Could not obtain installed package list", file=sys.stderr) + sys.exit(1) + # handle PATH for UsrMove - if self.package_manager.all_pkgs()['filesystem']['version'][0] == '3': + if pkgs['filesystem']['version'][0] == '3': self.PATH = "/usr/sbin:/usr/bin:/root/bin" else: self.PATH = "/sbin:/bin:/usr/sbin:/usr/bin:/root/bin" diff --git a/sos/utilities.py b/sos/utilities.py index 8cb4ed6..b2df648 100644 --- a/sos/utilities.py +++ b/sos/utilities.py @@ -181,11 +181,11 @@ def import_module(module_fqname, superclasses=None): return modules -def shell_out(cmd, runat=None): +def shell_out(cmd, timeout=180, runat=None): """Shell out to an external command and return the output or the empty string in case of error. """ - return sos_get_command_output(cmd, runat=runat)['output'] + return sos_get_command_output(cmd, timeout=timeout, runat=runat)['output'] class ImporterHelper(object): From bmr at redhat.com Fri Dec 12 12:13:39 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Fri, 12 Dec 2014 12:13:39 +0000 Subject: [sos-devel] [PATCH v3] sosreport: Check for rpm database corruption during initialization In-Reply-To: <548A7C10.3080702@linux.vnet.ibm.com> References: <20141015104053.32616.94797.stgit@aruna-ThinkPad-T420> <20141210173218.GC7321@localhost.localdomain> <54893D8A.6040007@linux.vnet.ibm.com> <20141211111405.GA21604@localhost.localdomain> <548A7C10.3080702@linux.vnet.ibm.com> Message-ID: <20141212121338.GC2470@localhost.localdomain> On Fri, Dec 12, 2014 at 10:54:32AM +0530, Aruna Balakrishnaiah wrote: > On Thursday 11 December 2014 04:44 PM, Bryn M. Reeves wrote: > >On Thu, Dec 11, 2014 at 12:15:30PM +0530, Aruna Balakrishnaiah wrote: > >>On Wednesday 10 December 2014 11:02 PM, Bryn M. Reeves wrote: > >>>Why use distutils.log here? It's not used anywhere else in sos; it > >>>would be cleaner to just print to stderr: > >>> > >>> print(error_message, file=sys.stderr) > >>Bryn, > >> > >>This is available only since Python 3.0, lower versions are not supporting this. > >No that's not correct - 3.0 just makes the print function the default > >(and remove the print statement). The print() function is available in > >python 2.6 (our oldest supported runtime) and onwards via the __future__ > >module: > > > ># python > >Python 2.6.6 (r266:84292, Nov 21 2013, 10:50:32) > >[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2 > >Type "help", "copyright", "credits" or "license" for more information. > >>>>from __future__ import print_statement > > In python 2.7 it is print_function and not print_statement. You're right (it always was) - that wa a typo in my example, good catch! :) Cheers, Bryn. From bmr at redhat.com Mon Dec 15 12:45:27 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Mon, 15 Dec 2014 12:45:27 +0000 Subject: [sos-devel] [PATCH v4] sosreport: Check for rpm database corruption during initialization In-Reply-To: <20141212053729.27655.50568.stgit@aruna-ThinkPad-T420> References: <20141212053729.27655.50568.stgit@aruna-ThinkPad-T420> Message-ID: <20141215124525.GB20130@localhost.localdomain> On Fri, Dec 12, 2014 at 11:27:07AM +0530, Aruna Balakrishnaiah wrote: > sosreport runs an rpm query to get the package list. If rpmdb is corrupted > sosreport hangs for ever hence check for rpmdb consistency before running > the rpm query Thanks Aruna, this looks good. The only remaining question I think is how long the timeout should be; I'll merge it with the current setting but in most of my tests this operation completes in <30s (actually the vast majority it's <5s - typically 1-2s). The tricky bit here is that sos is designed to run on systems that are in trouble; if we set it too low we'll get false failures and a failure to obtain diagnostics so let's leave it at 180s for now and see if there are any complaints of sosreport taking a long time to fail package manager set up. Regards, Bryn. From bmr at redhat.com Mon Dec 15 17:33:36 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Mon, 15 Dec 2014 17:33:36 +0000 Subject: [sos-devel] [RESEND PATCH v3] sosreport: Capture IBM Power RAID storage adapter configuration information In-Reply-To: <545AFF77.70607@linux.vnet.ibm.com> References: <20141016110015.11503.22082.stgit@aruna-ThinkPad-T420> <545AFF77.70607@linux.vnet.ibm.com> Message-ID: <20141215173335.GA31314@localhost.localdomain> On Thu, Nov 06, 2014 at 10:26:23AM +0530, Aruna Balakrishnaiah wrote: > Bryn, > > Is the patch ok? Can we expect it to be merged in the next release? Hi Aruna, Sorry for the delay; I've been busy with some other projects lately and have a bit of a backlog of sos patches to work through. There are a number of minor issues with the patch; mostly pep8 problems: $ .git/hooks/pre-commit checking pep8 conformance sos/plugins/iprconfig.py:22:1: E302 expected 2 blank lines, found 1 sos/plugins/iprconfig.py:43:16: E111 indentation is not a multiple of four [...] It's a good idea to run the pep8 conformance tool (and also the sos testsuite) for each patch - there were a number of lines with 3-char indents (python requires a multiple of four). I'll post a revised patch in a moment if you'd be able to test it we can get it merged. Regards, Bryn. From bmr at redhat.com Mon Dec 15 17:34:23 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Mon, 15 Dec 2014 17:34:23 +0000 Subject: [sos-devel] [PATCH] [iprconfig] add plugin for IBM Power RAID adapters In-Reply-To: <545AFF77.70607@linux.vnet.ibm.com> References: <20141016110015.11503.22082.stgit@aruna-ThinkPad-T420> <545AFF77.70607@linux.vnet.ibm.com> Message-ID: <20141215173423.GB31314@localhost.localdomain> Capture information which helps in better understanding of IBM Power RAID storage adapter configuration. Since iprconfig is specific to power adding the relevant commands in powerpc plugin. Signed-off-by: Aruna Balakrishnaiah Signed-off-by: Bryn M. Reeves --- sos/plugins/iprconfig.py | 110 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 sos/plugins/iprconfig.py diff --git a/sos/plugins/iprconfig.py b/sos/plugins/iprconfig.py new file mode 100644 index 0000000..b77c192 --- /dev/null +++ b/sos/plugins/iprconfig.py @@ -0,0 +1,110 @@ +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +# This plugin enables collection of logs for Power systems + +import os +import re +from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin +from sos.utilities import is_executable + + +class IprConfig(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): + """IBM Power RAID storage adapter configuration information + """ + + plugin_name = 'iprconfig' + + def check_enabled(self): + arch = self.policy().get_arch() + return arch == "ppc64" and is_executable("iprconfig") + + def setup(self): + self.add_cmd_output([ + "iprconfig -c show-config", + "iprconfig -c show-alt-config", + "iprconfig -c show-arrays", + "iprconfig -c show-jbod-disks", + "iprconfig -c show-ioas", + ]) + + show_ioas = self.call_ext_prog("iprconfig -c show-ioas") + if not show_ioas['status'] == 0: + return + + devices = [] + if show_ioas['output']: + p = re.compile('sg') + for line in show_ioas['output'].splitlines(): + temp = line.split(' ') + # temp[0] holds the device name + if p.search(temp[0]): + devices.append(temp[0]) + + for device in devices: + self.add_cmd_output("iprconfig -c show-details %s" % (device,)) + + # Look for IBM Power RAID enclosures (iprconfig lists them) + show_config = self.call_ext_prog("iprconfig -c show-config") + if not show_config['status'] == 0: + return + + if not show_config['output']: + return + +# iprconfig -c show-config +# Name PCI/SCSI Location Description Status +# ------ ------------------------- ------------------------- ----------------- +# 0005:60:00.0/0: PCI-E SAS RAID Adapter Operational +# sda 0005:60:00.0/0:0:0:0 Physical Disk Active +# sdb 0005:60:00.0/0:1:0:0 Physical Disk Active +# sdc 0005:60:00.0/0:2:0:0 Physical Disk Active +# sdd 0005:60:00.0/0:3:0:0 Physical Disk Active +# sde 0005:60:00.0/0:4:0:0 Physical Disk Active +# sdf 0005:60:00.0/0:5:0:0 Physical Disk Active +# 0005:60:00.0/0:8:0:0 Enclosure Active +# 0005:60:00.0/0:8:1:0 Enclosure Active + + show_alt_config = "iprconfig -c show-alt-config" + altconfig = self.call_ext_prog(show_alt_config) + if not (altconfig['status'] == 0): + return + + if not altconfig['output']: + return + +# iprconfig -c show-alt-config +# Name Resource Path/Address Vendor Product ID Status +# ------ -------------------------- -------- ---------------- ----------------- +# sg9 0: IBM 57C7001SISIOA Operational +# sg0 0:0:0:0 IBM MBF2300RC Active +# sg1 0:1:0:0 IBM MBF2300RC Active +# sg2 0:2:0:0 IBM HUC106030CSS600 Active +# sg3 0:3:0:0 IBM HUC106030CSS600 Active +# sg4 0:4:0:0 IBM HUC106030CSS600 Active +# sg5 0:5:0:0 IBM HUC106030CSS600 Active +# sg7 0:8:0:0 IBM VSBPD6E4A 3GSAS Active +# sg8 0:8:1:0 IBM VSBPD6E4B 3GSAS Active + + for line in show_config['output'].splitlines(): + if "Enclosure" in line: + temp = re.split('\s+', line) + # temp[1] holds the PCI/SCSI location + pci, scsi = temp[1].split('/') + for line in altconfig['output'].splitlines(): + if scsi in line: + temp = line.split(' ') + # temp[0] holds device name + self.add_cmd_output("iprconfig -c " + "query-ses-mode %s" % (temp[0],)) -- 1.9.3 From bmr at redhat.com Mon Dec 15 17:50:28 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Mon, 15 Dec 2014 17:50:28 +0000 Subject: [sos-devel] [PATCH] postgresql: avoid to crash with numerical passwords In-Reply-To: <20141114142210.GE11622@localhost.localdomain> References: <1415972145-16252-1-git-send-email-sbonazzo@redhat.com> <20141114142210.GE11622@localhost.localdomain> Message-ID: <20141215175027.GC31314@localhost.localdomain> On Fri, Nov 14, 2014 at 02:22:11PM +0000, Bryn M. Reeves wrote: > On Fri, Nov 14, 2014 at 02:35:45PM +0100, Sandro Bonazzola wrote: > > Issue: https://github.com/sosreport/sos/issues/433 > > Change-Id: I99fd0fe913e3f99826f98cb2b8cf7a460ecb5c4c > > Signed-off-by: Sandro Bonazzola > > --- > > sos/plugins/postgresql.py | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/sos/plugins/postgresql.py b/sos/plugins/postgresql.py > > index 350131f..0a51074 100644 > > --- a/sos/plugins/postgresql.py > > +++ b/sos/plugins/postgresql.py > > @@ -51,7 +51,7 @@ class PostgreSQL(Plugin): > > # is no need to save and restore environment variables if the user > > # decided to pass the password on the command line. > > if self.get_option("password") is not False: > > - os.environ["PGPASSWORD"] = self.get_option("password") > > + os.environ["PGPASSWORD"] = str(self.get_option("password")) > > This is just working around the problem; the type of the return value > of get_option() should not depend on the value passed on the command > line. Unfortunately it does; the current behaviour is to attempt to convert each plugin option to an integer (in the global code) and only pass it along as a string if that fails. Changing this would mean re-working a lot of plugins that now take numeric arguments or introducing a type field to the plugin option tuple (which is already a bit long..). If we do go down that route it needs to happen after the option_list cleanup (e.g. removing the fast/slow thing - #274). So I think at least for now your fix is the correct one; postgresql knows that it wants to work with strings here and until we have better plugin option infratructure I think this is the best solution. Cheers, Bryn. From bmr at redhat.com Mon Dec 15 17:55:31 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Mon, 15 Dec 2014 17:55:31 +0000 Subject: [sos-devel] [PATCH] postgresql: avoid to crash with numerical passwords In-Reply-To: <20141215175027.GC31314@localhost.localdomain> References: <1415972145-16252-1-git-send-email-sbonazzo@redhat.com> <20141114142210.GE11622@localhost.localdomain> <20141215175027.GC31314@localhost.localdomain> Message-ID: <20141215175530.GD31314@localhost.localdomain> On Mon, Dec 15, 2014 at 05:50:28PM +0000, Bryn M. Reeves wrote: > So I think at least for now your fix is the correct one; postgresql > knows that it wants to work with strings here and until we have > better plugin option infratructure I think this is the best > solution. Pushed: commit 067508239b7e1237f696fbe4a2b1484e6f720ebd Author: Sandro Bonazzola Date: Mon Dec 15 17:53:57 2014 +0000 [postgresql] avoid to crash with numerical passwords Signed-off-by: Sandro Bonazzola Signed-off-by: Bryn M. Reeves Cheers, Bryn. From sbonazzo at redhat.com Tue Dec 16 06:54:54 2014 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Tue, 16 Dec 2014 07:54:54 +0100 Subject: [sos-devel] [PATCH] postgresql: avoid to crash with numerical passwords In-Reply-To: <20141215175530.GD31314@localhost.localdomain> References: <1415972145-16252-1-git-send-email-sbonazzo@redhat.com> <20141114142210.GE11622@localhost.localdomain> <20141215175027.GC31314@localhost.localdomain> <20141215175530.GD31314@localhost.localdomain> Message-ID: <548FD73E.6030705@redhat.com> Il 15/12/2014 18:55, Bryn M. Reeves ha scritto: > On Mon, Dec 15, 2014 at 05:50:28PM +0000, Bryn M. Reeves wrote: >> So I think at least for now your fix is the correct one; postgresql >> knows that it wants to work with strings here and until we have >> better plugin option infratructure I think this is the best >> solution. > > Pushed: > > commit 067508239b7e1237f696fbe4a2b1484e6f720ebd > Author: Sandro Bonazzola > Date: Mon Dec 15 17:53:57 2014 +0000 > > [postgresql] avoid to crash with numerical passwords > > Signed-off-by: Sandro Bonazzola > Signed-off-by: Bryn M. Reeves > > Cheers, > Bryn. > Thanks! -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com From aruna at linux.vnet.ibm.com Tue Dec 16 07:33:35 2014 From: aruna at linux.vnet.ibm.com (Aruna Balakrishnaiah) Date: Tue, 16 Dec 2014 13:03:35 +0530 Subject: [sos-devel] [PATCH] [iprconfig] add plugin for IBM Power RAID adapters In-Reply-To: <20141215173423.GB31314@localhost.localdomain> References: <20141016110015.11503.22082.stgit@aruna-ThinkPad-T420> <545AFF77.70607@linux.vnet.ibm.com> <20141215173423.GB31314@localhost.localdomain> Message-ID: <548FE04F.70504@linux.vnet.ibm.com> Bryn, Tested. Works fine, you can merge. Regards, Aruna On Monday 15 December 2014 11:04 PM, Bryn M. Reeves wrote: > Capture information which helps in better understanding of > IBM Power RAID storage adapter configuration. Since iprconfig > is specific to power adding the relevant commands in powerpc plugin. > > Signed-off-by: Aruna Balakrishnaiah > Signed-off-by: Bryn M. Reeves > --- > sos/plugins/iprconfig.py | 110 +++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 110 insertions(+) > create mode 100644 sos/plugins/iprconfig.py > > diff --git a/sos/plugins/iprconfig.py b/sos/plugins/iprconfig.py > new file mode 100644 > index 0000000..b77c192 > --- /dev/null > +++ b/sos/plugins/iprconfig.py > @@ -0,0 +1,110 @@ > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > + > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > + > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write to the Free Software > +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. > + > +# This plugin enables collection of logs for Power systems > + > +import os > +import re > +from sos.plugins import Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin > +from sos.utilities import is_executable > + > + > +class IprConfig(Plugin, RedHatPlugin, UbuntuPlugin, DebianPlugin): > + """IBM Power RAID storage adapter configuration information > + """ > + > + plugin_name = 'iprconfig' > + > + def check_enabled(self): > + arch = self.policy().get_arch() > + return arch == "ppc64" and is_executable("iprconfig") > + > + def setup(self): > + self.add_cmd_output([ > + "iprconfig -c show-config", > + "iprconfig -c show-alt-config", > + "iprconfig -c show-arrays", > + "iprconfig -c show-jbod-disks", > + "iprconfig -c show-ioas", > + ]) > + > + show_ioas = self.call_ext_prog("iprconfig -c show-ioas") > + if not show_ioas['status'] == 0: > + return > + > + devices = [] > + if show_ioas['output']: > + p = re.compile('sg') > + for line in show_ioas['output'].splitlines(): > + temp = line.split(' ') > + # temp[0] holds the device name > + if p.search(temp[0]): > + devices.append(temp[0]) > + > + for device in devices: > + self.add_cmd_output("iprconfig -c show-details %s" % (device,)) > + > + # Look for IBM Power RAID enclosures (iprconfig lists them) > + show_config = self.call_ext_prog("iprconfig -c show-config") > + if not show_config['status'] == 0: > + return > + > + if not show_config['output']: > + return > + > +# iprconfig -c show-config > +# Name PCI/SCSI Location Description Status > +# ------ ------------------------- ------------------------- ----------------- > +# 0005:60:00.0/0: PCI-E SAS RAID Adapter Operational > +# sda 0005:60:00.0/0:0:0:0 Physical Disk Active > +# sdb 0005:60:00.0/0:1:0:0 Physical Disk Active > +# sdc 0005:60:00.0/0:2:0:0 Physical Disk Active > +# sdd 0005:60:00.0/0:3:0:0 Physical Disk Active > +# sde 0005:60:00.0/0:4:0:0 Physical Disk Active > +# sdf 0005:60:00.0/0:5:0:0 Physical Disk Active > +# 0005:60:00.0/0:8:0:0 Enclosure Active > +# 0005:60:00.0/0:8:1:0 Enclosure Active > + > + show_alt_config = "iprconfig -c show-alt-config" > + altconfig = self.call_ext_prog(show_alt_config) > + if not (altconfig['status'] == 0): > + return > + > + if not altconfig['output']: > + return > + > +# iprconfig -c show-alt-config > +# Name Resource Path/Address Vendor Product ID Status > +# ------ -------------------------- -------- ---------------- ----------------- > +# sg9 0: IBM 57C7001SISIOA Operational > +# sg0 0:0:0:0 IBM MBF2300RC Active > +# sg1 0:1:0:0 IBM MBF2300RC Active > +# sg2 0:2:0:0 IBM HUC106030CSS600 Active > +# sg3 0:3:0:0 IBM HUC106030CSS600 Active > +# sg4 0:4:0:0 IBM HUC106030CSS600 Active > +# sg5 0:5:0:0 IBM HUC106030CSS600 Active > +# sg7 0:8:0:0 IBM VSBPD6E4A 3GSAS Active > +# sg8 0:8:1:0 IBM VSBPD6E4B 3GSAS Active > + > + for line in show_config['output'].splitlines(): > + if "Enclosure" in line: > + temp = re.split('\s+', line) > + # temp[1] holds the PCI/SCSI location > + pci, scsi = temp[1].split('/') > + for line in altconfig['output'].splitlines(): > + if scsi in line: > + temp = line.split(' ') > + # temp[0] holds device name > + self.add_cmd_output("iprconfig -c " > + "query-ses-mode %s" % (temp[0],)) From bmr at redhat.com Tue Dec 16 12:42:55 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Tue, 16 Dec 2014 12:42:55 +0000 Subject: [sos-devel] [PATCH v3] Add plugin for Kimchi In-Reply-To: <1418335940-25000-1-git-send-email-christy@linux.vnet.ibm.com> References: <1418335940-25000-1-git-send-email-christy@linux.vnet.ibm.com> Message-ID: <20141216124254.GE3495@localhost.localdomain> On Thu, Dec 11, 2014 at 04:12:20PM -0600, Christy Perez wrote: > This plugin gathers kimchi logs and configuration files. > > Kimchi is an HTML5-based virtualization-management platform > For more information, please visit: > https://github.com/kimchi-project/kimchi/wiki > > Signed-off-by: Christy Perez Hi Christy, Thanks for revising the plugin - I think this looks good now. One minor thing - the two options you're using ('all_logs' and 'log_size') are actually global options now so there's no need for plugins to declare them individually in their option_list. You can find more info on the feature here: https://github.com/sosreport/sos/wiki/Plugin-options It's a trivial fixup though so I'll go ahead and remove the option_list and get this committed & pushed. Regards, Bryn. > --- > + option_list = [('logsize', 'maximum size (MiB) of each log file', '', 15), > + ('all_logs', 'collect all logs, regardless of size', '', > + False) > + ] > + From bmr at redhat.com Tue Dec 16 12:48:02 2014 From: bmr at redhat.com (Bryn M. Reeves) Date: Tue, 16 Dec 2014 12:48:02 +0000 Subject: [sos-devel] [PATCH v3] Add plugin for Kimchi In-Reply-To: <20141216124254.GE3495@localhost.localdomain> References: <1418335940-25000-1-git-send-email-christy@linux.vnet.ibm.com> <20141216124254.GE3495@localhost.localdomain> Message-ID: <20141216124801.GF3495@localhost.localdomain> On Tue, Dec 16, 2014 at 12:42:55PM +0000, Bryn M. Reeves wrote: > On Thu, Dec 11, 2014 at 04:12:20PM -0600, Christy Perez wrote: > > This plugin gathers kimchi logs and configuration files. > > > > Kimchi is an HTML5-based virtualization-management platform > > For more information, please visit: > > https://github.com/kimchi-project/kimchi/wiki > > > > Signed-off-by: Christy Perez > > Hi Christy, > > Thanks for revising the plugin - I think this looks good now. One minor > thing - the two options you're using ('all_logs' and 'log_size') are > actually global options now so there's no need for plugins to declare > them individually in their option_list. You can find more info on the > feature here: > > https://github.com/sosreport/sos/wiki/Plugin-options > > It's a trivial fixup though so I'll go ahead and remove the option_list > and get this committed & pushed. Pushed, thanks! commit 3aabd18b2666dbbeb86871ac4b9f025e468d5818 Author: Christy Perez Date: Tue Dec 16 12:46:27 2014 +0000 [kimchi] add new plugin This plugin gathers kimchi logs and configuration files. Kimchi is an HTML5-based virtualization-management platform For more information, please visit: https://github.com/kimchi-project/kimchi/wiki Signed-off-by: Christy Perez Signed-off-by: Bryn M. Reeves From christy at linux.vnet.ibm.com Tue Dec 16 15:22:04 2014 From: christy at linux.vnet.ibm.com (Christy Perez) Date: Tue, 16 Dec 2014 09:22:04 -0600 Subject: [sos-devel] [PATCH v3] Add plugin for Kimchi In-Reply-To: <20141216124801.GF3495@localhost.localdomain> References: <1418335940-25000-1-git-send-email-christy@linux.vnet.ibm.com> <20141216124254.GE3495@localhost.localdomain> <20141216124801.GF3495@localhost.localdomain> Message-ID: <54904E1C.9020107@linux.vnet.ibm.com> On 12/16/2014 06:48 AM, Bryn M. Reeves wrote: > On Tue, Dec 16, 2014 at 12:42:55PM +0000, Bryn M. Reeves wrote: >> On Thu, Dec 11, 2014 at 04:12:20PM -0600, Christy Perez wrote: >>> This plugin gathers kimchi logs and configuration files. >>> >>> Kimchi is an HTML5-based virtualization-management platform >>> For more information, please visit: >>> https://github.com/kimchi-project/kimchi/wiki >>> >>> Signed-off-by: Christy Perez >> >> Hi Christy, >> >> Thanks for revising the plugin - I think this looks good now. One minor >> thing - the two options you're using ('all_logs' and 'log_size') are >> actually global options now so there's no need for plugins to declare >> them individually in their option_list. You can find more info on the >> feature here: >> >> https://github.com/sosreport/sos/wiki/Plugin-options >> >> It's a trivial fixup though so I'll go ahead and remove the option_list >> and get this committed & pushed. Thanks so much! > > Pushed, thanks! > > commit 3aabd18b2666dbbeb86871ac4b9f025e468d5818 > Author: Christy Perez > Date: Tue Dec 16 12:46:27 2014 +0000 > > [kimchi] add new plugin > > This plugin gathers kimchi logs and configuration files. > > Kimchi is an HTML5-based virtualization-management platform > For more information, please visit: > https://github.com/kimchi-project/kimchi/wiki > > Signed-off-by: Christy Perez > Signed-off-by: Bryn M. Reeves > > From sma102874 at gmail.com Fri Dec 19 04:26:23 2014 From: sma102874 at gmail.com (Stephen Ma) Date: Thu, 18 Dec 2014 20:26:23 -0800 Subject: [sos-devel] [RESEND_PATCH] [openvswitch] Add flow information from OVS bridges Message-ID: <1418963183-2739-1-git-send-email-stephen.ma@hp.com> Add ovs-ofctl dump-flows output to openvswitch plugin report. Signed-off-by: Stephen Ma --- sos/plugins/openvswitch.py | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/sos/plugins/openvswitch.py b/sos/plugins/openvswitch.py index 3611671..e92d9f4 100644 --- a/sos/plugins/openvswitch.py +++ b/sos/plugins/openvswitch.py @@ -30,7 +30,30 @@ class OpenVSwitch(Plugin): # The '-t 5' adds an upper bound on how long to wait to connect # to the Open vSwitch server, avoiding hangs when running sosreport. - self.add_cmd_output("ovs-vsctl -t 5 show") + vsctl_file = self.get_cmd_output_now("ovs-vsctl -t 5 show") + self.get_bridge_flows(vsctl_file) + + def get_bridge_flows(self, vsctl_file): + bridges = self.get_bridge_names(vsctl_file) + for bridge in bridges: + self.add_cmd_output("ovs-ofctl dump-flows " + bridge) + + def get_bridge_names(self, vsctl_file): + """Return a list for which items are bridge name according to the + output of ovs-vsctl show stored in vsctl_file. + """ + out = [] + try: + vsctl_out = open(vsctl_file).read() + except Exception: + return out + + for line in vsctl_out.splitlines(): + tokens = line.split() + if len(tokens) == 2 and tokens[0] == "Bridge": + out.append(tokens[1]) + + return out class RedHatOpenVSwitch(OpenVSwitch, RedHatPlugin): -- 1.9.1 From stephen.ma at hp.com Mon Dec 1 15:53:32 2014 From: stephen.ma at hp.com (Ma, Stephen B.) Date: Mon, 01 Dec 2014 15:53:32 -0000 Subject: [sos-devel] [PATCH] [plugins/openstack_neutron] Add netns dumps for DVR namespaces In-Reply-To: <20141201150227.GA29725@localhost.localdomain> References: <1416639978-3173-1-git-send-email-stephen.ma@hp.com> <20141125115712.GB31857@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D57671BE6@G9W0344.americas.hpqcorp.net> <20141128110352.GA3703@localhost.localdomain> <36AF4B62444F4B4FB04EDD1FDAE1CF1D576725A9@G9W0344.americas.hpqcorp.net> <20141201150227.GA29725@localhost.localdomain> Message-ID: <36AF4B62444F4B4FB04EDD1FDAE1CF1D57672CB2@G9W0344.americas.hpqcorp.net> Of course *something else* or *some administrator* can create a namespace with starting with "fip", "snat", "qrouter", or "qdhcp". But that is highly unlikely. Only neutron creates and destroy these namespaces. Do you need any help in looking into how add checks to see whether neutron is running? Do I need to resubmit my patch again? -----Original Message----- From: Bryn M. Reeves [mailto:bmr at redhat.com] Sent: Monday, December 01, 2014 7:02 AM To: Ma, Stephen B. Cc: sos-devel at redhat.com Subject: Re: [sos-devel] [PATCH] [plugins/openstack_neutron] Add netns dumps for DVR namespaces On Fri, Nov 28, 2014 at 06:09:44PM +0000, Ma, Stephen B. wrote: > Hi Bryn, > > The fip and snat namespaces are created by the neutron-l3-agent when it is running in the dvr or dvr_snat mode. They are not used outside of neutron. That's still not what I mean ;) There is nothing to stop *something else* creating namespaces with those prefixes; therefore we cannot assume that because we find those prefixes in use that we're looking at a Neutron DVR configuration. We should only assume that in the case that we already know that the system is running OpenStack Neutron. > One thing to note is that when Neutron is running on a machine, it does not necessary implies that qdhcp, qrouter, fip, or the snat namespaces are present on the machine. For example, neutron-l3-agent manages the qrouter, fip, and snat namespaces. When the agent crashed, these namespaces are left behind. So if you run sosreport at the time the agent is not running, I would still like to see the information on these 3 namespaces. The information would provide clues as to what is the problem. That's not a problem; we can collect data for them regardless of whether the agent is running or not. Regards, Bryn.