[Crash-utility] 64 GiB hugemem vmcore can't be analyzed

[Kurtis D. Rader]

The following vmcore was manually induced ([sysrq-d]) from a freshly
booted hugemem kernel on a system with 64 GiB of memory:

    -rw-r--r-- 1 krader krader 68719480832 2007-03-09 12:18:44 /vmware/vmcore

Running crash on it

    crash -d255 System.map-2.4.21-47.0.1.ELhugemem /vmware/vmcore \
    ./usr/lib/debug/boot/vmlinux-2.4.21-47.ELhugemem.debug

Fails with

    crash: ./usr/lib/debug/boot/vmlinux-2.4.21-47.ELhugemem.debug: 
    no text and data contents

Crash -d255 output is below. Is there any hope for getting this to
work?  We've got a customer whose system is exhibiting both hangs and
panics. Until now the system hasn't been configured for crash dumps. This
dump was obtained under controlled conditions to validate that we can get
a complete dump.  The dump process appeared to complete without error.
The size of the vmcore looks reasonable. From the vmcore dmesg buffer:

    <6>disk_dump: Maximum block size: 16384
    <6>disk_dump: total blocks required: 16515376 (header 3 + bitmap 512 + memory 16514861)
    <6>SysRq : Crashing the kernel by request

crash 4.0-3.17
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006  Fujitsu Limited
Copyright (C) 2006  VA Linux Systems Japan K.K.
Copyright (C) 2005  NEC Corporation
Copyright (C) 1999, 2002  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
crash: diskdump: dump does not have panic dump header
vmcore_data: 
                  flags: 5 (NETDUMP_LOCAL|NETDUMP_ELF32)
                   ndfd: 4
                    ofp: 48a3a4c0
            header_size: 4096
   num_pt_load_segments: 1
     pt_load_segment[0]:
            file_offset: 1000
             phys_start: 0
               phys_end: 0
              zero_fill: 0
             elf_header: 8405700
                  elf32: 8405700
                notes32: 8405734
                 load32: 8405754
                  elf64: 0
                notes64: 0
                 load64: 0
            nt_prstatus: 8405774
            nt_prpsinfo: 8405814
          nt_taskstruct: 84058a0
            task_struct: 75ff6000
              page_size: 4096
           switch_stack: 0
         xen_kdump_data: (unused)
       num_prstatus_notes: 1
       nt_prstatus_percpu: 08405774


Elf32_Ehdr:
                e_ident: \177ELF
      e_ident[EI_CLASS]: 1 (ELFCLASS32)
       e_ident[EI_DATA]: 1 (ELFDATA2LSB)
    e_ident[EI_VERSION]: 1 (EV_CURRENT)
      e_ident[EI_OSABI]: 0 (ELFOSABI_SYSV)
 e_ident[EI_ABIVERSION]: 0
                 e_type: 4 (ET_CORE)
              e_machine: 3 (EM_386)
              e_version: 1 (EV_CURRENT)
                e_entry: 0
                e_phoff: 34
                e_shoff: 0
                e_flags: 0
               e_ehsize: 34
            e_phentsize: 20
                e_phnum: 2
            e_shentsize: 0
                e_shnum: 0
             e_shstrndx: 0
Elf32_Phdr:
                 p_type: 4 (PT_NOTE)
               p_offset: 116 (74)
                p_vaddr: 0
                p_paddr: 0
               p_filesz: 344 (158)
                p_memsz: 0 (0)
                p_flags: 0 ()
                p_align: 0
Elf32_Phdr:
                 p_type: 1 (PT_LOAD)
               p_offset: 4096 (1000)
                p_vaddr: c0000000
                p_paddr: 0
               p_filesz: 0 (0)
                p_memsz: 0 (0)
                p_flags: 7 (PF_X|PF_W|PF_R)
                p_align: 4096
Elf32_Nhdr:
               n_namesz: 4 ("CORE")
               n_descsz: 144
                 n_type: 1 (NT_PRSTATUS)
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 023b3ee0 00000001 
                         00000001 00000063 00000006 00000063 
                         75ff7f7c 00000068 00000068 00000000 
                         00000000 ffffffff 021d2010 00000060 
                         00010002 75ff7eac 00000068 00000000 
Elf32_Nhdr:
               n_namesz: 4 ("CORE")
               n_descsz: 124
                 n_type: 3 (NT_PRPSINFO)
                         00005200 00000000 00000000 00000000 
                         00000000 00000000 00000000 696c6d76 
                         0078756e 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 00000000 
                         00000000 00000000 00000000 
Elf32_Nhdr:
               n_namesz: 4 ("CORE")
               n_descsz: 8
                 n_type: 4 (NT_TASKSTRUCT)
                         75ff6000 00000000 
Elf32_Nhdr:
               n_namesz: 4 ("CORE")
               n_descsz: 4
                 n_type: 70000001 (NT_DISKDUMP)
                         00000000 
crash: ./usr/lib/debug/boot/vmlinux-2.4.21-47.ELhugemem.debug: no text and data contents

crash: the use of a System.map file requires that the accompanying namelist
argument is a kernel file built with the -g CFLAG.  The namelist argument
supplied in this case is a debuginfo file, which must be accompanied by the
kernel file from which it was derived.

-- 
Kurtis D. Rader, Linux level 3 support  email: krader at us.ibm.com
IBM Integrated Technology Services      DID: +1 503-578-3714
15350 SW Koll Pkwy, MS DES1-01          service: 800-IBM-SERV
Beaverton, OR 97006-6063                http://www.ibm.com