[Crash-utility] Re: Re: Re: crash and sles 9 dumps (Dave Anderson)

Daniel Li dli at virtualiron.com
Tue Aug 28 22:51:50 UTC 2007


It seems the problem is not one with guest dump, but the version of SLES.

After upgrading my NATIVE SLES 9 system to SP 3, exactly the same 
problem happened while trying to use 'crash' on the live system, with a 
debug linux kernel ('vmlinux.dbg' below) built on the same system from 
matching 'kernel-source' package. (During this upgrade, the linux kernel 
changed from 2.6.5-7.97-smp to 2.6.5-7.244-smp, the same as that on the 
guest.)

Has anyone else seen this?

tst037:/boot # ./crash vmlinux-2.6.5-7.244-smp vmlinux.dbg /dev/mem

crash 4.0-4.6
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: could not find MAGIC_START!
      KERNEL: vmlinux-2.6.5-7.244-smp
DEBUG KERNEL: vmlinux.dbg (2.6.5-7.244-default)
    DUMPFILE: /dev/mem
        CPUS: 4
        DATE: Tue Aug 28 10:57:37 2007
      UPTIME: 213503982284 days, 23:42:22
LOAD AVERAGE: 0.00, 0.01, 0.22
       TASKS: 104
    NODENAME: tst037
     RELEASE: 2.6.5-7.244-smp
     VERSION: #1 SMP Mon Dec 12 18:32:25 UTC 2005
     MACHINE: x86_64  (2793 Mhz)
      MEMORY: 1015808 GB
         PID: 14432
     COMMAND: "crash"
        TASK: 1012fbb0db8  [THREAD_INFO: 5]
         CPU: 3
       STATE: (unknown)

crash> bt
PID: 14432  TASK: 1012fbb0db8       CPU: 3   COMMAND: "crash"
(active)
crash> ps
   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
 >     0      0   0  ffffffff803d2800  RU   0.0 4398751044160 
4389851194656  [200^L]
 >     0     -1   1     10037e1e9a0    RU   0.0 4398425096768 
8800221827680  [200^L]
 >     0     -1   2     10037e1e080    RU   0.0 4398425096768 
8800221827680  [200^L]
      0     -1   3     100bff6b2d0    RU   0.0 4398425096768 
8800221827680  [200^L]
      0      0   3  ffffffff803d2808  ??   0.0       0      0  [swapper]
      1      0   3     10037e1f2c8    ??   0.0     640    320  init
      2     -1   3     100bff6a9b8    ??   0.0       0      0  [migration/0]
      3     -1   3     100bff6a098    ??   0.0       0      0  [ksoftirqd/0]
      4     -1   3     100bfe552e8    ??   0.0       0      0  [migration/1]
      5     -1   3     100bfe549c8    ??   0.0       0      0  [ksoftirqd/1]
      6     -1   3     100bfe540a8    ??   0.0       0      0  [migration/2]
      7     -1   3     100bfe5f2f8    ??   0.0       0      0  [ksoftirqd/2]
      8     -1   3     100bfe5e9d8    ??   0.0       0      0  [migration/3]
      9     -1  3     100bfe5e0b8    ??   0.0       0      0  [ksoftirqd/3]
     10     -1   3     1000a7ef308    ??   0.0       0      0  [events/0]
     11     -1   3     1000a7ee9e8    ??   0.0       0      0  [events/1]
     12     -1   3     1000a7ee0c8    ??   0.0       0      0  [events/2]
     13     -1   3     1000a7f9318    ??   0.0       0      0  [events/3]
     14     -1   3     1000a7f89f8    ??   0.0       0      0  [khelper]
     15     -1   3     1000a7f80d8    ??   0.0       0      0  [kacpid]
     82     -1   3     10037c7f328    ??   0.0       0      0  [kblockd/0]
     83     -1   3     10037c7ea08    ??   0.0       0      0  [kblockd/1]
     84     -1   3     10037c7e0e8    ??   0.0       0      0  [kblockd/2]
     85     -1   3     10037d0d338    ??   0.0       0      0  [kblockd/3]
    110     -1   3     10037d0ca18    ??   0.0       0      0  [pdflush]
    111     -1   3     10037d5b358    ??   0.0       0      0  [pdflush]
    112     -1   3     10037d36108    ??   0.0       0      0  [kswapd0]
    113     -1   3     10037d63368    ??   0.0       0      0  [aio/0]
    114     -1   3     10037d5aa38    ??   0.0       0      0  [aio/1]
    115     -1   3     10037d62a48    ??   0.0       0      0  [aio/2]
    116     -1   3     10037d5a118    ??   0.0       0      0  [aio/3]
   1355     -1   3     10037dbaa68    ??   0.0       0      0  [kseriod]
   2050     -1   3     1013fde0238    ??   0.0       0      0  [scsi_eh_0]
   2224     -1   3     1013fde5488    ??   0.0       0      0  [reiserfs/0]
   2225     -1   3     1013fc2d448    ??   0.0       0      0  [reiserfs/1]
   2226     -1   3     1013fc954d8    ??   0.0       0      0  [reiserfs/2]
   2227     -1   3     10037dcc168    ??   0.0       0      0  [reiserfs/3]
   4282     -1   3     10132400a58    ??   0.0       0      0  [khubd]
   4865     -1   3     10132a2f398    ??   0.0    2560    608  hwscand
   6086     -1   3     10037dbea98    ??   0.0    2512    692  dhcpcd
   7143     -1   3     10132eb9338    ??   0.0    2484    620  irqbalance
   8276     -1   3     1012dff89e8    ??   0.0    5716    924  syslogd
   8282     -1   3     1012d1ff678    ??   0.0    2748   1120  klogd
   8340     -1   3     1012d2129c8    ??   0.0    4580    724  portmap
   8342     -1   3     1013971b2d8    ??   0.0   21460   2336  slpd
   8343     -1   3     1012dc17358    ??   0.1   22396   3696  resmgrd
   8612     -1   3     1012197f398    ??   0.1   31732   7452  cupsd
   8625     -1   3     1012f8ed3c8    ??   0.1   30476   3176  sshd
   8744     -1   3     1012dc16118    ??   0.1   23640   3268  powersaved
   8797     -1   3     1012dff80c8    ??   0.2   99996   7952  slapd
   8820     -1   3     10132a2e158    ??   0.2   99996   7952  slapd
   8982     -1   3     1012156aa48    ??   0.2   99996   7952  slapd
   9523     -1   3     1012e03e0f8    ??   0.0   20104   2376  master
   9539     -1   3     10129c60328    ??   0.1   22640   3996  qmgr
   9555     -1   3     1012d66ccf8    ??   0.2   99996   7952  slapd
   9558     -1   3     1012b6c0338    ??   0.2   99996   7952  slapd
   9656     -1   3     10132eb8a18    ??   0.0   17884   2040  cron
   9661     -1   3     1012f8ecaa8    ??   0.1   35180   4028  nscd
   9662     -1   3     1012cc595e8    ??   0.1   35180   4028  nscd
   9663     -1   3     10037dbf3b8    ??   0.1   35180   4028  nscd
   9664     -1   3     1012ebf75a8    ??   0.1   35180   4028  nscd
   9665     -1   3     10122aa6bc8    ??   0.1   35180   4028  nscd
   9666     -1   3     1013fcdd598    ??   0.1   35180   4028  nscd
   9667     -1   3     10122aa62a8    ??   0.1   35180   4028  nscd
   9773     -1   3     10129c60c48    ??   0.0    8936    980  kdm
   9780     -1   3     10122a3f3b8    ??   0.5   40900  24864  X
   9783     -1   3     1012dcdcd98    ??   0.1   35220   3400  kdm
   9832     -1   3     1013a841678    ??   0.0    2720    768  mingetty
   9833     -1   3     10037dcca88    ??   0.0    2720    768  mingetty
   9834     -1   3     1013a840d58    ??   0.0    2720    768  mingetty
   9835     -1   3     1012d66c3d8    ??   0.0    2720    768  mingetty
   9836     -1   3     1012fa8e428    ??   0.0    2720    768  mingetty
   9837     -1   3     1013fcdcc78    ??   0.0    2720    768  mingetty
   9865     -1   3     10132a2ea78    ??   0.0    8000   1808  kde
   9898     -1   3     101347db548    ??   0.0    6288    932  gpg-agent
   9899     -1   3     1012a4203f8    ??   0.0   21196   2256  ssh-agent
   9923     -1   3     10129542378    ??   0.3   60192  13740  kdeinit
   9926     -1   3     1013971a098    ??   0.3   60972  13780  kdeinit
   9928     -1   3     10132eb80f8    ??   0.3   63436  15452  kdeinit
   9931     -1   3     1012156a128    ??   0.4   78752  21480  kdeinit
   9939     -1   3     101226cf2c8    ??   0.2   35848   8404  artsd
  10009     -1   3     1012d034a08    ??   0.5   94832  27044  kdeinit
  10010     -1   3     101223d1688    ??   0.0    2480    484  kwrapper
  10012     -1   3     101226ce088    ??   0.4   70116  20084  kdeinit
  10013     -1   3     1012b8f8d88    ??   0.4   78556  23520  kdeinit
  10015     -1   3     101347dac28    ??   0.4   70120  22384  kdeinit
  10017     -1   3     101237a14f8    ??   0.4   69476  20816  kdeinit
  10019     -1   3     1012e03ea18    ??   0.5   79308  25180  kdeinit
  10021     -1   3     101295435b8    ??   0.5   86836  25992  kdeinit
  10022     -1   3     10134215378    ??   0.3   62000  14332  kdeinit
  10026     -1   3     1013fde0b58    ??   0.4   75996  23024  kdeinit
  10030     -1   3     10037dbe178    ??   0.5   77916  26192  kdeinit
  10031     -1   3     1013f34aad8    ??   0.4   72728  22036  kpowersave
  10032     -1   3     101295963e8    ??   0.5   79788  26072  suseplugger
  10033     -1   3     1013fde1478    ??   0.4   71832  22452  susewatcher
  10035     -1   3     1012156b368    ??   0.4   75900  23428  kdeinit
  10040     -1   3     1013a840438    ??   0.5   79788  26072  suseplugger
  10050     -1   3     10037dbb388    ??   0.5   81420  25236  kdeinit
  10051     -1   3     1012fbb0498    ??   0.1    8420   2760  bash
  10087     -1   3     1013f34b3f8    ??   0.1   38132   3560  sshd
  10090     -1   3     1012a421638    ??   0.1    9324   2976  bash
  10196     -1   3     1012fbb16d8    ??   0.4   64700  19300  kdesktop_lock
  14432     -1   3     1012fbb0db8    ??   2.4   74380 125620  crash
  14437     -1   3     1012d66d618    ??   0.0    4284   1252  crash
  25011     -1   3     1012b8f96a8    ??   0.0   16920   2180  pickup
crash> bt 1012d1ff678
PID: 8282   TASK: 1012d1ff678       CPU: 3   COMMAND: "klogd"
bt: invalid kernel virtual address: 11  type: "stack contents"
bt: read of stack at 11 failed
crash> task 1012d1ff678
PID: 8282   TASK: 1012d1ff678       CPU: 3   COMMAND: "klogd"
struct task_struct {
  state = 1104534274048,
  thread_info = 0x11,
  usage = {
    counter = 320
  },
  flags = 0,
  ptrace = 498216206335,
  lock_depth = 120,
  prio = 0,
  static_prio = 1048832,
  run_list = {
    next = 0x200200,
    prev = 0x0
  },
  array = 0x11de3f11,
  sleep_avg = 6,
  interactive_credit = 252089242482,
  timestamp = 252089242482,
  last_ran = 0,
  activated = 0,
  policy = 18446744073709551615,
  cpus_allowed = 18446744073709551615,
  time_slice = 7,
  first_time_slice = 1,
  tasks = {
    next = 0x1012d212a58,
    prev = 0x1012dff8a78
crash> q





More information about the Crash-utility mailing list