[Crash-utility] crash: cannot gather a stable task list via pid_hash (500 retries)

[Dave Anderson]

Eugene Teo wrote:
> Hi Dave,
> 
> I tried to run crash on Fedora 8's kernel 2.6.24.3-12.fc8 x86_64, and
> it has errors that look like the following:
> 
> [...]
> crash: duplicate task in pid_hash: ffff81012f0811d0
> crash: duplicate task in pid_hash: ffff81012f0811d0
> crash: duplicate task in pid_hash: ffff81012f0811d0
> crash: duplicate task in pid_hash: ffff81012f0811d0
> crash: duplicate task in pid_hash: ffff81012f0811d0
> 
> crash: cannot gather a stable task list via pid_hash (500 retries)
> 
> I ran crash with -d7, and uploaded the log for debugging:
> http://hera.kernel.org/~eugeneteo/crash.log
> 
> Thanks,
> Eugene

Another thing that might work (as opposed to modifying the crash
sources), is to use the --active command line option:

# crash --active

crash 4.0-6.1
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

       KERNEL: /usr/lib/debug/lib/modules/2.6.24.3-12.fc8/vmlinux
     DUMPFILE: /dev/crash
         CPUS: 8
         DATE: Mon Mar 17 13:00:43 2008
       UPTIME: 01:36:46
LOAD AVERAGE: 0.03, 0.01, 0.00
        TASKS: 8
     NODENAME: hp-dl585g2-01.rhts.boston.redhat.com
      RELEASE: 2.6.24.3-12.fc8
      VERSION: #1 SMP Tue Feb 26 14:21:30 EST 2008
      MACHINE: x86_64  (2812 Mhz)
       MEMORY: 8 GB
          PID: 3417
      COMMAND: "crash"
         TASK: ffff81021fccb160  [THREAD_INFO: ffff810244124000]
          CPU: 0
        STATE: TASK_RUNNING (ACTIVE)

WARNING: only the active tasks on each cpu are being tracked

crash> ps
    PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
 >     0     -1   2  ffff81027ecb4000  RU   0.0       0      0  [swapper]
 >     0     -1   3  ffff81027ece68b0  RU   0.0       0      0  [swapper]
 >     0     -1   4  ffff81027ed17160  RU   0.0       0      0  [swapper]
 >     0     -1   5  ffff81027ed56000  RU   0.0       0      0  [swapper]
 >     0     -1   6  ffff81027ed8a8b0  RU   0.0       0      0  [swapper]
 >     0     -1   7  ffff81027edbd160  RU   0.0       0      0  [swapper]
 >  3417     -1   0  ffff81021fccb160  RU   0.8  177752  82104  crash
 >  3435   3417   1  ffff81027c4788b0  RU   0.0       4      4  crash
crash>

It's can be used in desperate situations where the pid_hash
bookkeeping is trashed.  It will only capture the active
tasks via the per-cpu runqueue data, and won't touch the
pid_hash at all.  The PPID may show -1 because the parent
doesn't "exist" with respect to crash, and there may be
other task-related command oddities.

Anyway, after doing that, you might be able to figure out what
the deal is with task ffff81012f0811d0.

Dave