[Crash-utility] crash seek error, failed to read vmcore file

[Dave Anderson]

----- "Pavan Naregundi" <pavan at linux.vnet.ibm.com> wrote:

> Hi Everyone,
> 
> We are facing a problem while analysing the vmcore on PPC64 systems
> running SLES11 SP1. 
> 
> ===============
> please wait... (gathering kmem slab cache data)                       
> 
> crash: seek error: kernel virtual address: c0000000af715480  type: "kmem_cache buffer"
> 
> crash: unable to initialize kmem slab cache subsystem
> 
> please wait... (gathering module symbol data)
> WARNING: cannot access vmalloc'd module memory
> 
> please wait... (gathering task table data)
> crash: cannot read pid_hash upid
> 
> crash: cannot read pid_hash upid
> 
> crash: cannot read pid_hash upid
> =====================
> 
> Version: crash-5.0.3
> 
> Command used:
> #crash vmlinux-2.6.32.10-0.4.99.25.62005-ppc64.debug
> vmlinux-2.6.32.10-0.4.99.25.62005-ppc64 vmcore
> 
> Attaching the output of the above command.. 
> 
> 
> Using crash -d8 for above command,
> ========================
> <snip>
> <readmem: c00000000134ffa0, KVADDR, "memory section", 32, (FOE), 122f94b0>
>     addr: c00000000134ffa0  paddr: 134ffa0  cnt: 32
> <readmem: c00000000134ffc0, KVADDR, "memory section", 32, (FOE), 122f94b0>
>     addr: c00000000134ffc0  paddr: 134ffc0  cnt: 32
> <readmem: c00000000134ffe0, KVADDR, "memory section", 32, (FOE), 122f94b0>
>     addr: c00000000134ffe0  paddr: 134ffe0  cnt: 32
> NOTE: page_hash_table does not exist in this kernel
> 
> please wait... (gathering kmem slab cache data)
> <readmem: c0000000012fc718, KVADDR, "cache_chain", 8, (FOE), fffeff7f108>
>     addr: c0000000012fc718  paddr: 12fc718  cnt: 8
>   GETBUF(248 -> 1)
>   FREEBUF(1)
>   GETBUF(10344 -> 1)
> <readmem: c000000000d8af90, KVADDR, "kmem_cache buffer", 10344, (FOE), 1082f5d8>
>     addr: c000000000d8af90  paddr: d8af90  cnt: 10344
>     GETBUF(248 -> 2)
>     FREEBUF(2)
>   FREEBUF(1)
>   GETBUF(10344 -> 1)
> <readmem: c0000000af715480, KVADDR, "kmem_cache buffer", 10344, (ROE), 1082f5d8>
>     addr: c0000000af715480  paddr: af715480  cnt: 10344
> 
> crash: seek error: kernel virtual address: c0000000af715480  type: "kmem_cache buffer"
>   FREEBUF(1)
> 
> crash: unable to initialize kmem slab cache subsystem
> =================================
> 
> NOTE: Crash was able to read a vmcore on the same system that was
> manually generated using: echo c > /proc/sysrq-trigger.

The cause for seek errors depends upon the type
of dumpfile.

You didn't mention which type of dumpfile the vmcore
is, so I'll presume that it's either an ELF-format
kdump or a compressed kdump created by makedumpfile.

If it's an ELF-format kdump, seek errors are returned
by the read_netdump() function in netdump.c.  If the
ELF header indicates that the physical address is contained
within one of the PT_LOAD segments, it calculates the 
vmcore file offset from that, and simply does this:

        if (lseek(nd->ndfd, offset, SEEK_SET) == -1)
                return SEEK_ERROR;

But that's highly unlikely to fail, even if the lseek
offset is beyond the end of the file.  And if it went 
beyond the end of the vmcore file, the subsequent read()
would fail, and return a READ_ERROR instead.  Also, if
none of the ELF header PT_LOAD segments contain the requested
physical address, it also would have returned a READ_ERROR.

So presuming that it's a compressed kdump, the seek error 
most likely comes from here in read_diskdump() in diskdump.c:

        if ((pfn >= dd->header->max_mapnr) || !page_is_ram(pfn))
                return SEEK_ERROR;

where the requested physical address pfn values are larger
than the max_mapnr value advertised in the header.

When you do any "crash -d# ...", the dumpfile header will
be dumped first.  What does that show?

Dave