[PATCH 3/3] utrace-based ftrace "process" engine, v2

[Ingo Molnar]

* Andrew Morton <akpm at linux-foundation.org> wrote:

> [...]  Let's fix systemtap?

Yes, it needs to be fixed.

The main issue i see is that no kernel developer i work with on a 
daily basis uses SystemTap - and i work with a lot of people. Yes, i 
could perhaps name two or three people from lkml using it, but its 
average penetration amongst kernel folks is essentially zero.

Was any critical analysis done why that penetration is so absymally 
low for a tool with such a promise and with years of availability, 
and what are the measures planned to address those problems?

To me personally there are two big direct usability issues with 
SystemTap:

 1) It relies on DEBUG_INFO for any reasonable level of utility.
    Yes, it will limp along otherwise as well, but most of the
    actual novel capabilities depend on debuginfo. Which is an
    acceptable constraint for enterprise usage where kernels are
    switched every few months and having a debuginfo package is not
    a big issue. Not acceptable for upstream kernel development. It 
    also puts way too trust into the compiler generating 1GB+ of 
    debuginfo correctly. I want to be able to rely on tools all the 
    time and thus i want tools to have some really simple and 
    predictable foundations.

 2) It's not upstream and folks using it seem to insist on not 
    having it upstream ;-) This 'distance' to upstream seems to have 
    grown during the past few years - instead of shrinking. As a 
    result it simply does not matter and there's no know-how and no 
    visibility of it upstream.

If these fundamental problems are addressed then i'd even argue for 
the totality of SystemTap to be aimed upstreamed (including the 
scripting language, etc.), because for something this fundamental 
there's just no good reason not to have a turn-key solution there.

Plus then there should be a (steadily growing) library of utility 
scripts in the kernel proper as well.

Anything less does not make much sense IMO. Having a separate tool 
will reduce efficiency, increases the latency of fixes and 
enhancements and creates ABI-like expectations - which are all 
counter-productive to good instrumentation.

These are the aspects of SystemTap that i have to say were never 
done right, and these are the aspects of SystemTap that need to 
change most. Putting utrace upstream now will just make it more 
convenient to have SystemTap as a separate entity - without any of 
the benefits. Do we want to do that? Maybe, but we could do better i 
think.

	Ingo