[Crash-utility] how to use crash utility to parse the binary memory dump

[Dave Anderson]

----- Original Message -----
> 
> Hi Dave,
> 
> one thing I want to correct:
> I have my own read_ramdump function,
> 
> write function is still the same as of write_kdump
> 
> 
> by the way
> 
> why do we need write function ?

If /dev/mem is used as the live memory source, then it is possible to
modify memory.  It's theoretically possible to use it with ELF dumpfiles,
but the vmcore is opened O_RDONLY and cmd_wr() doesn't allow it.

> 
> Currently I am using pc- >flags as of kdump itself, which I feel is not
> right.  (need to derive own set of flags for ramdump I suppose).
>
> moreover my thinking is following:
> 
> 1) implement a framework which provide hooks to implement conversion of raw ramdump to ELF.

Right -- and then the decision needs to be made whether it's necessary to create
a special dumpfile type, or simply create a kdump ELF clone.  More on that below...

> 2) this hooks could be implemented by ramdump32/ramdump64 or even anybody else.
> 3) have own set of flags and do not rely on KDUMP flags because this has nothing to do with Kdump.

You could continue to use the KDUMP flag given that you are essentially
creating a clone of a kdump ELF vmcore.  That is what is done with KVM
guest vmcores that are created with "virsh dump --memory-only".  The
KDUMP flag is set, but because that facility also creates an extra
ELF note with the name string "QEMU", the QEMU_MEM_DUMP flag is set
in pc->flags2.  That extra flag could be used if a differentiator is
needed.  You could do the same kind of thing.

On the other hand, if you do think its best to create a new dumpfile-type flag,
then some existing pc->flags #define (which is currently full) needs to be moved
to pc->flags2, and your new flag could take its place.  That's necessary because
the MEMORY_SOURCES and DUMPFILE_TYPES require that their bit-flags contents are 
in pc->flags.

> 4) have its own read and write function.

Unless you have a compelling need for it, the write function be a placeholder
that just returns WRITE_ERROR.  Or just don't bother setting pc->writemem
during initialization -- it will never get called anyway.

Thanks,
  Dave


 
> please provide your inputs.
> 
> Regards,
> Oza.
> 
> 
> 
> 
> 
> On Friday, 30 May 2014 9:58 AM, paawan oza <paawan1982 at yahoo.com> wrote:
>  
> 
> 
> 
> Hi Dave,
> 
> 1) what I talked about is pure 32bit (not with PAE, but should not be a
> problem with PAE)
> 64 bit ELF headers could also be in place.
> 
> but my first and important point is: get 32bit implementation right (rest
> will fall after that)
> in other words, the design/framework has to be made in such a way that
> existing crash utility absorbs both ELF32 and 64 and even any other
> debugging format or not to mention for long term but 128 bit ELF format as
> well :)
> kdump should be capable of reading it I think.
> 
> 2) OK, so then after you've created the temporary ELF file, crash internally
> would end up setting the pc->readmem() pointers to read_kdump()?  Or do
> you have your own readmem() function in the new ramdump.c file?
> 
> 
> currently I am setting function point to kdump, so I dont have separate
> readmem function.
> 
> 3) Clarify one more thing please: is the temporary file simply a little
> standalone ELF
> header that describes what's in the ddr.bin file?  Or do you concatenate the
> ELF
> header and the ddr.bin file into a single, temporary, ELF vmcore file that
> is essentially a clone
>  of an ELF kdump dumpfile?
> 
> 
> it looks more closer to the former one which you mentioned.
> standalone ELF object. (Vmcore is ELF format anyway, so should be similar)
> not sure about kdump format though.
> 
> 
> PS: It might take some time to post the patch because I wrote it in hurry to
> make it work at the first point:but now I have to think with the design
> perspective and more on long term perspective to make this framework
> scalable and get easily accommodated in crash utility architecture.
> 
> Regards,
> Oza.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Thursday, 29 May 2014 7:09 PM, Dave Anderson <anderson at redhat.com> wrote:
>  
> 
> 
> 
> 
> ----- Original Message -----
> > Hi Dave,
> > 
> > please find all my comments/clarifications/answers inline below.
> > 
> > 
> > Since these raw DDR dumps seem to be an existing feature, it certainly
> > seems worth implementing support
>  for them.
> > 
> > I'm curious as to how these RAM dumps are currently used -- are there other
> > tools that use them somehow?
> > 
> > Oza: there could be various tools which organisation might be writing to
> > extract some logs, because they know the physical address
> > where their private/public logs are stored.
> > but none of the tools match the ability of crash utility as we all agree.
> > 
> > Are these DDR RAM dumps specific to embedded 32-bit ARM machines?
> > 
> > Oza: yes it depends, if it  is taken on armv7 then 32bit, if
>  it is armv8,
> > then 64 bit.
> 
> Are you thinking of supporting this feature for ARM64 as well?
> 
> Also, by any chance are you working with 32-bit ARM with PAE?  Currently
> there is no support in the crash utility for ARM/PAE, and I have been
> hoping/waiting for ARM developers on this list to develop and post patches
> supporting it.  Although it's unclear whether the kdump facility would
> even work as-is with PAE, if this feature were available, it would be an
> an alternative to kdump.  Anyway, if/when ARM/PAE support is put in place,
> it would require the usage of 64-bit ELF
>  headers for the 32-bit ARM/PAE dumps.
> But I'm getting ahead of myself...   ;-)
> 
> > 
> >   
> > Your feature sounds like a two-stage process:
> > 
> > (1) invoke crash utility -- passing it the base physical address of the
> >      contiguous RAM dump, and the RAM dump file name(s) -- and then crash
> >      creates a single ELF vmcore by pre-pending an ELF header and
> >      concatenating
> >      the dump file names.
> > (2) invoke crash utility with vmlinux and
>  newly-created vmcore.
> > 
> > Oza: yes it is 2 stage process, in the first stage crash utility will
> > prepare
> > something which crash utility itself can understand.
> > which is nothing but ELF format, which every debugger typically understands
> > (such as gdb)
> > so I do that conversion at the first place.
> > once that conversion is in place, crash utility core code takes it over and
> > do the rest of the job.
> > 
> > But you mention "generate object files" above.  Do you generate more than
> > one
> > file?
> > 
> > Oza: what I meant by object file is ELF file (ELF is something which is
> > ready to be loaded/linked)
> > this object file is nothing but one temporary file which crash utility will
> > create...
> > so if you give
> > 
> > crash ./vmlinux ./ramdump.bin kernel_base_address
> > it woudl have created ramdump_ARM_ELF32  (temporary file) which crash
> > utility will understand).
> > after you exit crash, the temporary file would be deleted.
> > 
> > Is the newly-created vmcore subsequently recognized as a netdump or kdump
> > ELF vmcore?  (i.e, handled by existing code in netdump.c)
> > 
> > Oza: it would be recognized as kdump (because I am currently using 'flags'
> > of
> > kdump), this is just to be aligned with crash internal implementations.
> > 
> > but nontheless it is a separate ELF file
> 
> OK, so then after you've created the temporary ELF file, crash internally
> would end up setting the pc->readmem() pointers to read_kdump()?  Or do
> you have your own readmem() function in the new ramdump.c file?
> 
> > 
> > Or does it create a new ELF-like dumpfile type that is handled in your new
> > ramdump.c file?
> > Oza: it is
>  separate ELF file.
> > 
> > Could it be done in one step?  In other words, something like:
> > 
> >   $ crash vmlinux --ddr 80000000 ddr1.bin [ddr2.bin ...]
> > 
> > 
> > Oza: this is precisely what is hapepning.
> > 
> > 
> > where there would be a "virtual" ELF header created that could be used
> > during the crash session?  (perhaps with an optional "-o outputfile"
> > command line option to create/save it as an ELF vmcore)
> > 
> > Oza: virtual ELF
>  header, and one more virtual program header would be created
> > in temporary file which, the budle, called ELF32.
> > optional command line option could be created as well. but currently we
> > take
> > bin file only.
> 
> Clarify one more thing please: is the temporary file simply a little
> standalone ELF
> header that describes what's in the ddr.bin file?  Or do you concatenate the
> ELF
> header and the ddr.bin file into a single, temporary, ELF vmcore file that
> is essentially a clone of an ELF kdump dumpfile?
> 
> Also, I suggested the "-o outputfile" option -- after which you could simply
> enter
> "crash vmlinux outputfile" -- because your implementation requires that the
> user must be aware of the base physical address value.
> 
> Anyway, this all sounds good, so please post a patch.  And can you also make
> one of
> these dumpfiles available to me to download?
> 
> Thanks,
> 
>   Dave