From kaushalshriyan at gmail.com Fri Dec 4 11:03:47 2015 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Fri, 4 Dec 2015 16:33:47 +0530 Subject: [sos-devel] sosreport grab /opt directory Message-ID: Hi, I am using sosreport on RHEL 6.6, is there a way to include /opt directory using sosreport where my application is running. I read the man sosreport and did not see any option to do it. Any help will be really appreciable. Please advise. Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmr at redhat.com Fri Dec 4 13:52:47 2015 From: bmr at redhat.com (Bryn M. Reeves) Date: Fri, 4 Dec 2015 13:52:47 +0000 Subject: [sos-devel] sosreport grab /opt directory In-Reply-To: References: Message-ID: <20151204135247.GA19464@hex.gsslab.fab.redhat.com> On Fri, Dec 04, 2015 at 04:33:47PM +0530, Kaushal Shriyan wrote: > I am using sosreport on RHEL 6.6, is there a way to include /opt directory > using sosreport where my application is running. I read the man sosreport > and did not see any option to do it. Any help will be really appreciable. Not really: /opt is a potentially very large (10s or 100s of GiB in some instances) directory tree and is one of the top-level FHS standard paths. We do not collect anything in sos as indiscriminately as this - both for privacy reasons (we do not know if we are collecting keys, passwords, or other secrets) and for the space consumed in doing so. If specific applications make use of known paths in /opt then we will consider making additions to plugins that support those applications. You could package up any additions you require from /opt using the tar tool - either as a separate archive or updating an existing sos tarball. Regards, Bryn. From bmr at redhat.com Fri Dec 4 15:54:53 2015 From: bmr at redhat.com (Bryn M. Reeves) Date: Fri, 4 Dec 2015 15:54:53 +0000 Subject: [sos-devel] release meeting Message-ID: <20151204155452.GB19464@hex.gsslab.fab.redhat.com> Hi Folks, I'd like to hold a quick release meeting for 3.3 early next week. We are overdue on the release and I'd like to get a handle on the current open issues that are strictly required before we can get this out. There is some added urgency here in that a flaw was recently reported in sos's handling of temporary files (CVE-2015-7529) that can allow a local user to obtain archive content and in some circumstances execute arbitrary commands with administrative privileges. There are currently 28 open issues with a milestone of 3.3 set: https://github.com/sosreport/sos/issues?q=is%3Aopen+is%3Aissue+milestone%3A3.3 Many of these however are ongoing work that has been bumped from a previous release milestone: most of these can be moved out to >= 3.4. We'll hold a short meeting (~30m or so) on IRC to discuss any issues that are urgent for 3.3 - if you are interested in attending please reply off-list or swing by #sosreport on Freenode - Monday or Tuesday next week are pretty clear for me but we can be flexible on times. If you're unable to attend and have an open issue that you're keen to see in the next release please reply to the list or update the issue on GitHub: https://github.com/sosreport/sos/issues/ Regards, Bryn. From kaushalshriyan at gmail.com Fri Dec 4 16:20:29 2015 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Fri, 4 Dec 2015 21:50:29 +0530 Subject: [sos-devel] sosreport grab /opt directory In-Reply-To: References: <20151204135247.GA19464@hex.gsslab.fab.redhat.com> Message-ID: Hi Bryn, is there a full or expanded form of sos in sosreport? Regards, Kaushal On Fri, Dec 4, 2015 at 9:46 PM, Kaushal Shriyan wrote: > > On Fri, Dec 4, 2015 at 7:22 PM, Bryn M. Reeves wrote: > >> On Fri, Dec 04, 2015 at 04:33:47PM +0530, Kaushal Shriyan wrote: >> > I am using sosreport on RHEL 6.6, is there a way to include /opt >> directory >> > using sosreport where my application is running. I read the man >> sosreport >> > and did not see any option to do it. Any help will be really >> appreciable. >> >> Not really: /opt is a potentially very large (10s or 100s of GiB in some >> instances) directory tree and is one of the top-level FHS standard paths. >> >> We do not collect anything in sos as indiscriminately as this - both for >> privacy reasons (we do not know if we are collecting keys, passwords, or >> other secrets) and for the space consumed in doing so. >> >> If specific applications make use of known paths in /opt then we will >> consider making additions to plugins that support those applications. >> >> You could package up any additions you require from /opt using the >> tar tool - either as a separate archive or updating an existing sos >> tarball. >> >> Regards, >> Bryn. >> >> > Thanks Bryn for the explanation and much appreciated. I will appreciate if > there is a way to handle this specific use case using plugins or some > elegant and efficient mechanism that way it would be a awesome tool for us > to push it more than 1000 servers in our Infrastructure. > > Please advise. > > Regards, > > Kaushal > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaushalshriyan at gmail.com Fri Dec 4 16:16:58 2015 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Fri, 4 Dec 2015 21:46:58 +0530 Subject: [sos-devel] sosreport grab /opt directory In-Reply-To: <20151204135247.GA19464@hex.gsslab.fab.redhat.com> References: <20151204135247.GA19464@hex.gsslab.fab.redhat.com> Message-ID: On Fri, Dec 4, 2015 at 7:22 PM, Bryn M. Reeves wrote: > On Fri, Dec 04, 2015 at 04:33:47PM +0530, Kaushal Shriyan wrote: > > I am using sosreport on RHEL 6.6, is there a way to include /opt > directory > > using sosreport where my application is running. I read the man sosreport > > and did not see any option to do it. Any help will be really appreciable. > > Not really: /opt is a potentially very large (10s or 100s of GiB in some > instances) directory tree and is one of the top-level FHS standard paths. > > We do not collect anything in sos as indiscriminately as this - both for > privacy reasons (we do not know if we are collecting keys, passwords, or > other secrets) and for the space consumed in doing so. > > If specific applications make use of known paths in /opt then we will > consider making additions to plugins that support those applications. > > You could package up any additions you require from /opt using the > tar tool - either as a separate archive or updating an existing sos > tarball. > > Regards, > Bryn. > > Thanks Bryn for the explanation and much appreciated. I will appreciate if there is a way to handle this specific use case using plugins or some elegant and efficient mechanism that way it would be a awesome tool for us to push it more than 1000 servers in our Infrastructure. Please advise. Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From sghosh at redhat.com Tue Dec 1 20:55:16 2015 From: sghosh at redhat.com (SGhosh) Date: Tue, 1 Dec 2015 15:55:16 -0500 Subject: [sos-devel] runtime plugin detection? Message-ID: <565E0934.8010206@redhat.com> Is there a way apps to install their own sosreport plugin during app installation time and not be carried in the sos package? -subhendu From ebeuerlein at cloudpassage.com Tue Dec 8 18:34:21 2015 From: ebeuerlein at cloudpassage.com (Eddie Beuerlein) Date: Tue, 8 Dec 2015 13:34:21 -0500 Subject: [sos-devel] Issue with 3.2 on Amazon AMI Message-ID: Hi, I have downloaded the github zip file and I have been unable to get it to work. I am tried make install, make rpm, etc. The end result is always: $ sosreport -l sosreport (version 3.2) no valid plugins found Please help! Eddie -- Eddie Beuerlein Sr. Security Analyst | CloudPassage 571-989-2884 ebeuerlein at cloudpassage.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmr at redhat.com Wed Dec 9 13:00:50 2015 From: bmr at redhat.com (Bryn M. Reeves) Date: Wed, 9 Dec 2015 13:00:50 +0000 Subject: [sos-devel] runtime plugin detection? In-Reply-To: <565E0934.8010206@redhat.com> References: <565E0934.8010206@redhat.com> Message-ID: <20151209130050.GB19230@hex.gsslab.fab.redhat.com> Hi Subhendu, Sorry for the slow response - your message was caught in the non-members moderation filter (I've added you to the whitelist now although replies will only get through if you're cc'ed on the message). On Tue, Dec 01, 2015 at 03:55:16PM -0500, SGhosh wrote: > Is there a way apps to install their own sosreport plugin during app > installation time and not be carried in the sos package? Right now this is problematic and is something that we very strongly discourage - the problem is that as the API coupling between sos and plugins is still quite tight it is impossible to guarantee compatibility and deliver the best user experience while allowing out-of-tree plugins (in some respect it's similar to the kernel module problem). We have a development roadmap to allow this but it is unlikely to be complete before the next major release. Generally though we can accomodate most needs via upstream plugins - the only cases I can think of that couldn't be handle were site-local changes - these are a legitimate case for out-of-tree plugins (e.g. an organisation's own plugin to collect site-specific data) and since they are managed by local administrators or developers are an easier testing problem than plugins distributed in arbitrary packages. If you're interested in the steps we are planning to take to allow out-of-tree plugins I'd be happy to write up a more complete description (the short version is that we're aiming to make virtually all plugins be effectively a configuration file: a simple list of commands to run and files to collect). Regards, Bryn. From bmr at redhat.com Wed Dec 9 17:08:26 2015 From: bmr at redhat.com (Bryn M. Reeves) Date: Wed, 9 Dec 2015 17:08:26 +0000 Subject: [sos-devel] Issue with 3.2 on Amazon AMI Message-ID: <20151209170826.GD19230@hex.gsslab.fab.redhat.com> On Tue, 8 Dec 2015 13:34:21 -0500, Eddie Beuerlein wrote: > Hi, > I have downloaded the github zip file and I have been unable to get it to work. I am tried make install, make rpm, etc. The end result is always: What distribution is the AMI using (i.e.: Amazon Linux, RHEL, Ubuntu etc.)? For RHEL or Ubuntu you are best of using the distribution provided sos packages as these will be appropriately configured for the image's python version and installation paths. I do not know if Amazon includes a build of sos in their own Amazon Linux AMIs (although it would be easy to do so as afaik it is based on RHEL). > $ sosreport -l > > sosreport (version 3.2) > > no valid plugins found There are several possible causes for this error. With the information about your installation it's not really possible to say definitively which one applies: - No policy exists for the installed operating system Sos uses a policy framework to detect the platform on which it is running and to provide access to system-specific resources such as package management. The plugins that can execute are tied to a particular policy (or set of policies) and if no policy matches then no plugins can run. - There are no valid python plugin files in the sos.plugins package in the system defined PYTHONPATH. Normally this is something like /usr/lib/pythonx.y/site-packages (dist-packages on Debian / Ubuntu like distributions). - An sos.conf exists that disables all valid plugins (unlikely). At a guess I suspect your problem is the first of these: the Amazon image doesn't contain the paths that any of the existing LinuxPolicy classes expect so it ends up with a valid plugins list that is the empty set. We can fix this by creating a new policy class for this distribution - normally this means reading the '/etc/*-release' file and matching strings in either the file name or content - e.g. on Red Hat we look for '/etc/redhat-release'. If there's a '*-release' file present in your images we can start to work on a policy if you can send the full file name and an example of the content on your systems. Longer-term and once we have something working you may want to ask Amazon to include this in their builds. Regards, Bryn.