[Libvirt-cim] debugging providers with gdb

[Jay Gagnon]

So we all know that it's annoying when your provider is running in sfcb 
and it segfaults, because using gdb on a threaded app is not so fun.  
Well, I re-visited the debugging sfcb section of sfcb's README, and I 
guess something sunk in this time because I was able to put it into 
practice.  This seemed like the kind of thing to mention here for 
posterity.  Turns out it's pretty simple.  (FYI, not sure how much of 
this requires you to be root, I just did the easy way out and was root 
for all of it.)

The first step is to recompile sfcb with debugging symbols:
CFLAGS=-g ./configure
make clean && make && make install

Then start up sfcbd and tell it which provider you want to debug:
SFCB_PAUSE_PROVIDER=Virt_VSMigrationService sfcbd # (of course put your 
provider's name in here)

Now do whatever it is you need to do to get your provider to run 
(wbemcli call, etc.), and you'll see something like this:
-#- Pausing for provider: Virt_VSMigrationService -pid: 13943

Now you can attach gdb.  Open up a new terminal and run:
gdb sfcbd 13943 # whatever pid you just got

You're now attached to the thread; I'll summarize what state everything 
is in.  After gdb attaches, it pauses the thread.  The thread itself is 
in a loop that checks if a local variable (debug_break) is zero.  If it 
is zero, the thread sleeps for five seconds and checks again.  We need 
to get out of that loop so our provider can actually run.  So we set a 
breakpoint at the check.  The check is in providerDrv.c; in sfcb-1.2.5, 
it is line 2577.  To double-check, just do a grep:

grep -n debug_break providerDrv.c
2569:   int debug_break = 0;
2577:      if (debug_break) break;

That if statement is the one we need.  So at the gdb prompt, set the 
breakpoint, then use "continue" to start the thread.  You should hit the 
breakpoint very quickly:

(gdb) break providerDrv.c:2577
Breakpoint 1 at 0x1c1c36: file providerDrv.c, line 2577.
(gdb) continue
Continuing.
[Switching to Thread -1208785216 (LWP 13943)]

Breakpoint 1, processProviderInvocationRequests (name=0x9e06300 
"Virt_VSMigrationService") at providerDrv.c:2577
2577               if (debug_break) break;

Now we can set debug_break to nonzero so it will break out of the sleep 
loop.  This is also a convenient place to set breakpoints in your 
provider if you wanted to.  If you're just waiting for a segfault to hit 
so you can do a backtrace, you shouldn't have to worry about that.  For 
the example, we'll just get out of the loop:

(gdb) set debug_break=1
(gdb) continue

And now your provider is running in gdb.  For a little help with what 
you can do with gdb, I found this to be a pretty good resource:
http://www.delorie.com/gnu/docs/gdb/gdb_toc.html#SEC_Contents

-- 

-Jay