[Crash-utility] PATCH 00/10] teach crash to work with "live" ramdump

[Dave Anderson]

Hi Oleg,

Just to (maybe) clarify my first reply...

As I see it, this facility is simply another LIVE_SYSTEM memory source,
of which there currently are /dev/mem, /proc/kcore and /dev/crash.  The
essential difference between them is the pc->readmem plugin:

  /dev/mem -> read_dev_mem
  /proc/kcore -> read_proc_kcore
  /dev/crash -> read_memory_device

The CRASHBUILTIN stuff and related stuff you ran into is only there because
live system analysis typically does not require a crash command line option, 
so crash has to figure what to do when a user just enters "crash".  

In this QEMU/KVM live system case, it would be much easier to initialize things
based upon the crash command line option that specifies it.

Dave


----- Original Message -----
> 
> 
> ----- Original Message -----
> > Hi Dave,
> > 
> > Recently I used crash-tool for the first time and I was pleasantly
> > surprised,
> > it really looks like a very useful and handy debugging tool ;)
> > 
> > And I was surprised again when I figured out that it can be used to debug
> > the
> > live system on the same machine. Cool!
> > 
> > Now I am wondering if we can teach it to debug the live guests runnning
> > under
> > qemu/kvm. This looks certainly possible, qemu supports gdb remote protocol.
> > 
> > But. this obviously needs more work. And. Afaics crash-tool needs some
> > fixes
> > anyway. See 01/10-07/10, but probably it needs more changes, so far I only
> > tried to audit task.c and kernel.c. I tried to document every change, but I
> > am very new to this code so I can be easily wrong.
> > 
> > So can't we teach it to work with live/raw RAM dumpfiles for the start? See
> > 09/10 and 10/10.
> > 
> > With these changes I can run qemu-kvm with the
> > 
> > 	-object memory-backend-file,id=MEM,size=128m,mem-path=/tmp/MEM,share=on
> > 	-numa node,memdev=MEM
> > 
> > options and then do
> > 
> > 	$ crash path-to-guests-vmlinux live:/tmp/MEM at 0
> > 
> > to debug the live guest, No need to dump-guest-memory + restart
> > /usr/bin/crash
> > which can be slow.
> > 
> > What do you think?
> > 
> > Oleg.
> > 
> >  defs.h    |  1 +
> >  filesys.c |  8 ++++----
> >  kernel.c  |  7 +++----
> >  main.c    | 11 +++++++++++
> >  ramdump.c | 49 ++++++++++++++++++++++++++++++++++---------------
> >  task.c    | 13 ++++++-------
> >  tools.c   |  2 +-
> >  7 files changed, 60 insertions(+), 31 deletions(-)
>  
> 
> Hi Oleg,
> 
> This is going to take some time for me to digest.  But without my studying it
> at all, there are a couple things that immediately come to mind.
> 
> First, a couple of nits about the files being modified.
> 
> I see that your overloading the ramdump.c file, but the ramdump facility was
> proposed and added by companies that use a firmware-based facility for
> dumping
> raw ARM/ARM64/MIPS RAM into one or more ramdump files, typically because they
> do/did not have kdump capability with their embedded hardware.  I really
> don't
> feel comfortable conflating that facility with a remote/live QEMU/KVM memory
> source.  Those companies support it, and so any changes that you've added to
> ramdump.c should be accomplished elsewhere.
> 
> Also, we don't want to get this confused in any way with the REMOTE_xxx
> stuff.  The remote.c file and remote access facility was deprecated many
> years ago, but recently resurrected in crash-7.0.4 by Verizon specifically
> for use like so:
> 
>        - Resurrection of the remote analysis capability for use with the
>          "xen-crashd" daemon running on a Xen Dom0 host, which communicates
>          with a paused or shutdown DomU guest kernel.  The daemon can be
>          accessed like so:
> 
>            $ crash localhost:5001,/dev/xenmem vmlinux
> 
>          (dslutz at verizon.com
> 
> I don't think you are modifying their "resurrected" usage, but just an FYI
> since
> you refer to it in your patches.
> 
> >From a crash utility perspective, this QEMU/KVM live memory source is a
> >completely
> new live memory source, and as such should have its own source file and/or
> set of
> functions, as is the case with the other actual dumpfile types, or live
> memory
> sources /proc/kcore, /dev/mem and /dev/crash.
> 
> Now, more to the point...
> 
> Anyway, this is a great idea, and one I've pondered in the past, but is there
> a
> concrete reason that it could not be a simple matter of plugging in a new
> function
> into pc->readmem()?  Currently there are 20+ pluggable readmem functions that
> exist
> for supporting all of the supported dumpfile types and live memory sources.
> Regardless of how it's accomplished, the plug-in basically just needs to be
> able to
> take a physical address and a count (all guaranteed to be contained within a
> page),
> and access/return it.  How it accomplishes that task is hidden from the
> mainstream
> crash code.
> 
> Thanks,
>   Dave
> 
> 
> 
>  
> 
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>