[Crash-utility] Cannot open dump when using 2G/2G memory split with 2.6.27.21 SMP x86 kernel

Tero Pirkkanen tp at pirkkanen.net
Wed May 27 11:12:05 UTC 2009


Hello

I've been trying to open crashdump produced by 2.6.27.21 SMP kernel
with 2G memory split without luck.
I tried to generate the dump by copying from /proc/vmcore and with
makedumpfile 1.3.3. Both without success.
The crash works fine with uniprocessor kernel as well as SMP kernel
using standard memory split. Architecture is 32bit x86 and the crash
was generated inside vmware server 2.0.

Is this supported configuration? Have anybody tried crash dump with 2G
split in recent 2.6.x kernels?

I'll attach some debug output from both non-working smp dump and
working UP dump.
The kernel with debug symbols uses different flavor in name (smp-syms)
than the image without those (smp), so crash always warns that kernel
and dump do not match.

------- output from non-working SMP dump (dumped directly from /proc/vmcore)
# crash -d7 ....
crash 4.0-8.9
....
vmcore_data:
                 flags: a0 (KDUMP_LOCAL|KDUMP_ELF32)
                   ndfd: 3
                    ofp: f7eba4c0
            header_size: 540
   num_pt_load_segments: 4
     pt_load_segment[0]:
            file_offset: 21c
             phys_start: 0
               phys_end: a0000
              zero_fill: 0
     pt_load_segment[1]:
            file_offset: a021c
             phys_start: 100000
               phys_end: 1000000
              zero_fill: 0
     pt_load_segment[2]:
            file_offset: fa021c
             phys_start: 2800000
               phys_end: fef0000
              zero_fill: 0
     pt_load_segment[3]:
            file_offset: e69021c
             phys_start: ff00000
               phys_end: 10000000
              zero_fill: 0
             elf_header: 9a266f8
                  elf32: 9a266f8
                notes32: 9a2672c
                 load32: 9a2674c
                  elf64: 0
                notes64: 0
                 load64: 0
            nt_prstatus: 9a267cc
            nt_prpsinfo: 0
          nt_taskstruct: 0
            task_struct: 0
              page_size: 0
           switch_stack: 0
         xen_kdump_data: (unused)
       num_prstatus_notes: 2
               vmcoreinfo: 0
          size_vmcoreinfo: 0
       nt_prstatus_percpu:
        09a267cc 09a26870
.... skip to end
<readmem: 8053f700, KVADDR, "kernel_config_data", 32768, (ROE), a5f17c0>
WARNING: could not find MAGIC_START!
<readmem: 807348c0, KVADDR, "cpu_possible_map", 4, (ROE), 8430680>
cpu_possible_map: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31
<readmem: 806c478c, KVADDR, "cpu_present_map", 4, (ROE), 8430680>
cpu_present_map: 3 7 8 9 10 14 18 19 21 22 31
<readmem: 806c42a0, KVADDR, "cpu_online_map", 4, (ROE), 8430680>
cpu_online_map: 8 9 14 18 19 21 22 31
<readmem: 807506e0, KVADDR, "xtime", 8, (FOE), 8408d38>
<readmem: 80679e84, KVADDR, "init_uts_ns", 390, (ROE), 840931c>
crash: cannot determine base kernel version
<readmem: 8053a000, KVADDR, "accessible check", 4, (ROE|Q), ff96df04>
<readmem: 8053a000, KVADDR, "readstring characters", 1499, (ROE|Q), ff96cf00>
linux_banner:
&y �&y �&y �(z �]z �jz ��z �a� �f� ��� �
� ��� �9� �� �0� �6� �<� �Њ
�B� �H� �N� �
crash: s3/boot/vmlinux-2.6.27.21+smp-syms and
kerneldump-20090526-140502 do not match

------- output from working UP crash (this was generated with makedumpfile)
compressed kdump: header->utsname.machine:
diskdump_data:
          filename: (null)
             flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
               dfd: 3
               ofp: 0
      machine_type: 3 (EM_386)

            header: 94c96f8
           signature: "KDUMP   "
      header_version: 2
             utsname:
               sysname:
              nodename:
               release:
               version:
               machine:
            domainname:
           timestamp:
                tv_sec: 0
               tv_usec: 0
              status: 0 ()
          block_size: 4096
        sub_hdr_size: 1
       bitmap_blocks: 4
           max_mapnr: 65536
    total_ram_blocks: 0
       device_blocks: 0
      written_blocks: 0
         current_cpu: 0
             nr_cpus: 1
      tasks[nr_cpus]: 0

        sub_header: 0 (n/a)

  sub_header_kdump: 94ca700
           phys_base: 0
          dump_level: 1 (0x1) (DUMP_EXCLUDE_ZERO)

       data_offset: 6000
        block_size: 4096
       block_shift: 12
            bitmap: 94cb708
        bitmap_len: 16384
   dumpable_bitmap: 94cf710
              byte: 0
               bit: 0
   compressed_page: 94e3768
         curbufptr: 0
 page_cache_hdr[0]:
            pg_flags: 0 ()
             pg_addr: 0
           pg_bufptr: 94d3760
        pg_hit_count: 0
 page_cache_hdr[1]:
            pg_flags: 0 ()
             pg_addr: 0
           pg_bufptr: 94d4760
        pg_hit_count: 0
 page_cache_hdr[2]:
            pg_flags: 0 ()
             pg_addr: 0
           pg_bufptr: 94d5760
.... skipped rest of page_cache stuff
    page_cache_buf: 94d3760
       evict_index: 0
         evictions: 0
          accesses: 0
      cached_reads: 0
       valid_pages: 94d3718
... skipped some stuff
<readmem: 804ed6c0, KVADDR, "kernel_config_data", 32768, (ROE), a009250>
WARNING: could not find MAGIC_START!
<readmem: 8066cce8, KVADDR, "cpu_possible_map", 4, (ROE), 8430680>
cpu_possible_map: 0
<readmem: 8066ccec, KVADDR, "cpu_present_map", 4, (ROE), 8430680>
cpu_present_map: 0
<readmem: 8066cce4, KVADDR, "cpu_online_map", 4, (ROE), 8430680>
cpu_online_map: 0
<readmem: 806e09d0, KVADDR, "xtime", 8, (FOE), 8408d38>
<readmem: 80626004, KVADDR, "init_uts_ns", 390, (ROE), 840931c>
base kernel version: 2.6.27
<readmem: 804e8000, KVADDR, "accessible check", 4, (ROE|Q), ff87ce14>
<readmem: 804e8000, KVADDR, "readstring characters", 1499, (ROE|Q), ff87be10>

WARNING: cannot find matching kernel version in
u/boot/vmlinux-2.6.27.21+up-syms file:

verify_namelist:
/proc/version:
Linux version 2.6.27.21+up (root at x) (gcc version 4.1.2 20061115
(prerelease) (Debian 4.1.1-21)) #1 Fri May 15 13:56:19 UTC 2009
utsname version: #1 Fri May 15 13:56:19 UTC 2009
u/boot/vmlinux-2.6.27.21+up-syms:

<readmem: 802577e0, KVADDR, "x86_omit_frame_pointer", 8, (ROE), ff87dfd8>
<readmem: 806e93ec, KVADDR, "high_memory", 4, (FOE), 840b504>
<readmem: 806e93e0, KVADDR, "mem_map", 4, (FOE), 840b50c>
<readmem: 806e940c, KVADDR, "vmlist", 4, (FOE), ff87d974>
<readmem: 8f8028a4, KVADDR, "first vmlist addr", 4, (ROE), ff87d970>
<readmem: 8066cd5c, KVADDR, "totalram_pages", 4, (FOE), 840b514>
.... skipped lot of output
      KERNEL: u/boot/vmlinux-2.6.27.21+up-syms
    DUMPFILE: kerneldump-20090527-075410  [PARTIAL DUMP]
        CPUS: 1
        DATE: Wed May 27 10:53:54 2009
<readmem: 8062b8f8, KVADDR, "jiffies_64", 8, (FOE), ff87d9c8>
crash: convert_time: 283566 (453ae)
      UPTIME: 00:04:43
<readmem: 806e05e0, KVADDR, "avenrun array", 12, (FOE), ff87d9c0>
LOAD AVERAGE: 0.14, 0.14, 0.06
       TASKS: 141
    NODENAME: tp3
     RELEASE: 2.6.27.21+up
     VERSION: #1 Fri May 15 13:56:19 UTC 2009
     MACHINE: i686  <readmem: 806bff50, KVADDR, "cpu_khz", 4, (FOE), ff87d9e0>
(2658 Mhz)
      MEMORY: 255.5 MB
       PANIC: NOTE: GETBUF request > MAX_CACHE_SIZE: 65536
"[  136.195328] SysRq : Trigger a crashdump"
         PID: 3208
     COMMAND: "bash"
        TASK: 8f1be0c0  [THREAD_INFO: 88d44000]
         CPU: 0
     <readmem: 80630790, KVADDR, "task_state_array", 4, (ROE), ff87d998>
<readmem: 805b2fd4, KVADDR, "readstring characters", 44, (ROE|Q), ff87c370>
...
------- some output from working SMP standard memory split (but PAE
enabled) crash
mcore_data:
                  flags: a0 (KDUMP_LOCAL|KDUMP_ELF32)
                   ndfd: 3
                    ofp: f7f074c0
            header_size: 540
   num_pt_load_segments: 4
     pt_load_segment[0]:
            file_offset: 21c
             phys_start: 0
               phys_end: a0000
              zero_fill: 0
     pt_load_segment[1]:
            file_offset: a021c
             phys_start: 100000
               phys_end: 1000000
              zero_fill: 0
....
            elf_header: 99eb6f8
                  elf32: 99eb6f8
                notes32: 99eb72c
                 load32: 99eb74c
...
<readmem: c053e700, KVADDR, "kernel_config_data", 32768, (ROE), a5b67f8>
crash: CONFIG_NR_CPUS: 32
crash: CONFIG_HZ: 100
<readmem: c07318c0, KVADDR, "cpu_possible_map", 4, (ROE), 842fea0>
cpu_possible_map: 0 1
<readmem: c06c378c, KVADDR, "cpu_present_map", 4, (ROE), 842fea0>
cpu_present_map: 0 1
<readmem: c06c32a0, KVADDR, "cpu_online_map", 4, (ROE), 842fea0>
cpu_online_map: 0 1
...
PANIC: NOTE: GETBUF request > MAX_CACHE_SIZE: 65536
"[   31.044652] SysRq : Trigger a crashdump"
         PID: 2745
....
---------- end of snips

Thanks in advance,

Tero




More information about the Crash-utility mailing list