[Crash-utility] [RFC 0/4] Add support to extract hardware device dumps from vmcore

Surendra Mobiya surendra at chelsio.com
Tue Apr 16 10:13:36 UTC 2019


On Monday, April 04/15/19, 2019 at 21:16:49 +0530, Dave Anderson wrote:
> 
> 
> ----- Original Message -----
> > Hi Dave,
> > 
> > On Saturday, April 04/13/19, 2019 at 00:39:09 +0530, Dave Anderson wrote:
> > > 
> > > Hi Surenda,
> > > 
> > > Great -- I've been looking forward for this patch set to arrive.
> > > 
> > > A couple things...
> > > 
> > > First, I'm going to need two sample vmcores (one ELF, one compressed kdump)
> > > along with the associated vmlinux.  You can contact me off-list with
> > > details
> > > on how we can arrange a transfer.
> > > 
> > > Second, I do not want to add a new command.  I rarely do so -- in fact,
> > > the only times since the original crash utility was released was in 2012,
> > > when the "ipcs" and "tree" commands were added in crash-6.0.7 and
> > > crash-6.0.8.
> > > New functionality is typically added as an option to an existing command.
> > > And in this case, the natural location to put it is in the existing "dev"
> > > command, and the devdump_extract() function can be moved into dev.c.
> > > (FWIW, you can add your Chelsio copyright at the top of that file)
> > > 
> > 
> > Ok, thanks for the suggestion. We will move the logic to dev command,
> > instead.
> > 
> > We're thinking along the lines of following sample commands:
> > 
> > Display the available device dumps
> > crash> dev -v
> > INDEX                         NAME                      OFFSET      SIZE
> > 0                      cxgb4_0000:02:00.4               0x278       33558464
> > 1                      cxgb4_0000:03:00.4               0x2001278   33558464
> > 
> > Extract device dump at specified index
> > crash> devdump -v 0 -f device_dump_0.bin
> > 33558464 bytes copied from 0x278 to device_dump_0.bin
> > 
> > Let us know your thoughts.
> 
> Hi Surenda,
> 
> I've got your sample files -- thanks for them, I really appreciate it.
> 
> Here are my thoughts...
> 
> You probably want to make "-V" display the list of available device dumps in the
> vmcore, and make "-v <index>" select a singular device for dumping.
>

Thanks for the suggestion. We will follow this approach.

> I also have a question re: the note contents.  Is it up to the individual device
> as to what format the dump contents are made up of?  Are they always binary
> dumps, or could a device dump ASCII log data or something to that effect?
> 

They are generally binary in nature. However, it is really up to 
the individual device to decide the format on how to save its
dump contents.

> I ask because I see that you are calling display_memory() using these arguments:
> 
>   void
>   display_memory_from_file_offset(ulonglong addr, long count, void *opt)
>   {
>           display_memory(addr, count, DISPLAY_RAW | ASCII_ENDLINE | HEXADECIMAL,
>                          FILEADDR, opt);
>   }
> 
> Since you are using DISPLAY_RAW, display_memory() will simply copy the
> note data unmodified directly to the file, and the ASCII_ENDLINE and 
> HEXADECIMAL arguments are ignored.  So it's not clear why you added them?  

You are correct. It should have been only DISPLAY_RAW. We will fix in
next version.

> But since you did, I'm now wondering whether it would be useful for
> a user to optionally dump a human-readable HEXADECIMAL/ASCII_ENDLINE 
> formatted display of the data to the screen?  If so, then perhaps if 
> the "-v index" option is used alone *without* a file specified,  
> why not just do a translated device dump to the screen?  

We notice that rd command can manipulate the output using below
options:
rd [-adDxN][-8|-16|-32|-64]

Let us know if it is recommended to duplicate above options from rd to
dev command for manipulating screen output.

We are thinking, dev -v should only be used for doing "full" raw dump
to file. Since dev -V already shows offset and size of dump, rd -f
can be used for more control over manipulating the dump output to
screen or file.

What do you think?

> 
> So to summarize, my suggestions would be:
> 
>   dev -V                        dump the list of device dumps
>   dev -v <index> [ -r file ]    select and display one device dump, either in
>                                 human-readable format to the screen by default,
>                                 or optionally copy it raw to a file
> 
> Thanks,
>   Dave
> 
> 

Thanks,
Surendra

> 
> 
> > 
> > > Third, there are some aesthetic changes that should be made in order
> > > to have the display use a more traditional output format like those
> > > used by other commands, e.g., with a single header with NAME, OFFSET and
> > > SIZE columns.
> > > 
> > 
> > Ok, We will fix this in next version.
> > 
> > > Other than that, this looks good on paper!
> > > 
> > > Thanks,
> > >   Dave
> > > 
> > 
> > Thanks
> > Surendra
> > 
> > 
> > > 
> > > ----- Original Message -----
> > > > When kernel panic happens and kdump crash kernel is loaded, device
> > > > drivers enabled in the kdump crash kernel collect device specific
> > > > snapshot of the hardware/firmware state of their underlying devices.
> > > > These snapshots are exported as ELF notes with note type NT_VMCOREDD
> > > > (i.e., 0x700) in vmcore [1].
> > > > 
> > > > This series of patches enhance crash utility to analyze and
> > > > extract these hardware specific device dumps from vmcore using
> > > > a new "devdump" command.
> > > > 
> > > > Patches 1 and 2 enhance help -D to parse NT_VMCOREDD ELF notes
> > > > in ELF vmcore and KDUMP vmcore, respectively.
> > > > 
> > > > Patches 3 and 4 implement devdump command to analyze and extract
> > > > hardware specific device dumps from ELF vmcore and KDUMP vmcore,
> > > > respectively.
> > > > 
> > > > Suggestions and feedback will be much appreciated.
> > > > 
> > > > Thanks,
> > > > Surendra
> > > > 
> > > > [1] https://lkml.org/lkml/2018/5/2/190
> > > > 
> > > > Surendra Mobiya (4):
> > > >   parse NT_VMCOREDD ELF notes in ELF vmcore
> > > >   parse NT_VMCOREDD ELF notes in KDUMP vmcore
> > > >   add devdump command to extract NT_VMCOREDD from ELF vmcore
> > > >   enhance devdump command to extract NT_VMCOREDD from KDUMP vmcore
> > > > 
> > > >  Makefile      |   4 +--
> > > >  defs.h        |  16 +++++++++
> > > >  devdump.c     | 114
> > > >  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  diskdump.c    |  71 +++++++++++++++++++++++++++++++++++-
> > > >  global_data.c |   1 +
> > > >  help.c        |  23 ++++++++++++
> > > >  memory.c      |   7 ++++
> > > >  netdump.c     |  89 +++++++++++++++++++++++++++++++++++++++++++--
> > > >  netdump.h     |   3 ++
> > > >  vmcore.h      |  36 +++++++++++++++++++
> > > >  10 files changed, 359 insertions(+), 5 deletions(-)
> > > >  create mode 100644 devdump.c
> > > >  create mode 100644 vmcore.h
> > > > 
> > > > --
> > > > 1.8.3.1
> > > > 
> > > > 
> > 




More information about the Crash-utility mailing list