[sos-devel] New plugin "gfs2.py" for collecting lockdumps for DLM, GFS2

[Shane Bradley]

Thanks for the response. I will take a look at this again and maybe a better approach would be to just have end user run multiple iterations of sosreport:
# sosreport ...; sleep 30; sosreport ....; sleep 30; sosreport...

I do agree with what you said, currently the plugin feels a bit hacky in a way because of libs I need do not exist. I will revisit this patch and see what I can do based on your feedback.
----
Shane Bradley
Senior Software Maintenance Engineer (Cluster HA, GFS, GFS2)
Red Hat Global Support Services 
Red Hat, Inc.

On Oct 31, 2013, at 7:16 AM, Bryn M. Reeves <bmr at redhat.com> wrote:

> On Mon, 26 Aug 2013 15:44:38, Shane Bradley wrote:
>> This plugin "gfs2.py" will capture GFS2 and DLM lockdumps. The cluster.py plugin already had code for GFS1, but it only capture 1 set of lockdumps. When debugging lockdumps for DLM, GFS1, or GFS2 capturing a single lockdump is usually not enough. You really need to capture multiple instances of DLM, GFS1, or GFS2 to see if what has changed in the lockdumps or threads. Capturing multiple lockdump iterations helps to diagnose lock contention.  A filesystem might appear hung when in fact it is just moving very slow which you would not be able to tell with a single instances of the lockdumps.
>> 
>> The plugin gfs2.py adds two options that were previously not in cluster.py. 
>> * lockdump_count: number of iterations/runs of capturing lockdump data
>> * lockdump_wait: wait time between capturing each lockdump
>> 
>> This gives sosreport the ability to capture lockdumps in a more useful manner. We have been using a script that we developed called gfs2_lockcapture, but I believe that adding the same ability to sosreport is a better fit and provide a standard way of capturing this data.
>> - https://git.fedorahosted.org/cgit/gfs2-utils.git/tree/gfs2/scripts/gfs2_lockcapture
> 
> Is this shipped in any package that we'd expect to be installed on a
> regular gfs2 installation?
> 
> Our approach to date when components have complex needs for data
> collection (or require multi-step actions that need to be kept in-sync
> with the requirements of the other component) has been to lean on
> externally provided scripts and to use those to collect this data.
> 
> Could that be an option here?
> 
>> When enabling the gfs2 plugin it is best to only run a small set of plugins that will not interfere with the GFS2 filesystems. For example, capturing filesys data could cause problems with GFS2 in surprising ways. Here is the command that I have found that captures all the information needed when reviewing GFS2 lockdumps:
> 
> This is a bit tricky; ideally we want all options to be usable in any
> combination. Special purpose options that require the user to know that
> they shouldn't allow certain plugins to run can be confusing and lead to
> false alarm bug reports.
> 
> If the plugin was enabled by default and collected a minimal amount of
> general data (that's normally expected to be low-risk and to not have
> any significant side-effects) then I think I'd be more comfortable
> having this sort of functionality as an option.
> 
> I'd like to have sos be able to gather something that's useful for
> general debugging on systems using gfs2 without any special options - is
> this feasible?
> 
> I think it might be a good approach to get something basic that deals
> with simple cases first and to then add more complex debugging
> assistance like lockdumps at a later time - especially as this seems
> like it may need some changes to core sos to make it practical (we're
> not really set up for long-running data collection tasks right now).
> 
> Regards,
> Bryn.
> 
> _______________________________________________
> sos-devel mailing list
> sos-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/sos-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/sos-devel/attachments/20131031/4570cb31/attachment.htm>