<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=iso-8859-1" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18999"></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>Short update to my kernel-panic....</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>it turns out that this panic is caused by a bug in RHEL6
kernel (all 2.6.32-71.*) and happens</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>if somebody telnets to nlockmgr and simply press
<return> several times.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>newer kernels (in fedora 14) don't have this bug
anymore. does probably anybody know a</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>"bug number" or when this bug has been fixed in
kernel?</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>thanks a lot,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>christian</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011><FONT color=#0000ff
size=2 face=Arial>p.s.: i'm already in contact with redhat-support, but don't
have a fix up to now....</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=210154808-15022011></SPAN> </DIV><BR>
<BLOCKQUOTE
style="BORDER-LEFT: #0000ff 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px"
dir=ltr>
<DIV dir=ltr lang=de class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>Von:</B> rhelv6-list-bounces@redhat.com
[mailto:rhelv6-list-bounces@redhat.com] <B>Im Auftrag von </B>Masopust,
Christian<BR><B>Gesendet:</B> Mittwoch, 09. Februar 2011 12:01<BR><B>An:</B>
'Red Hat Enterprise Linux 6 (Santiago) discussion
mailing-list'<BR><B>Betreff:</B> [rhelv6-list] RHEL6 kernel
2.6.32-71.14.1.el6.x86_64 panic...<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>Hi
all,</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>some of my
RHEL6-systems are facing a kernel panic from time to time. 2 of them are huge
HP's (DL585 G7)</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>with 48cores and
128GB, one of them is an older Primergy RX300S2 (4 cores, 8GB). Some other
systems </FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>(also
Primergies) </FONT></SPAN><SPAN class=756185908-09022011><FONT size=2
face=Arial>run fine all the time...</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>some other
facts:</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>- all filesystems
ext4</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>- nfs4
enabled</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>- 3 bonding
devices, each having 2 physical devices</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>- 2 of the bonding
devices configured for jumbo frames (MTU=9000)</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>Here's the
console-log from one of the HP's:</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>------------[ cut
here ]------------<BR>kernel BUG at fs/inode.c:1333!<BR>invalid opcode: 0000
[#1] SMP<BR>last sysfs file:
/sys/devices/system/cpu/cpu47/cache/index2/shared_cpu_map<BR>CPU 4<BR>Modules
linked in: iptable_filter ip_tables nfs fscache fuse nfsd nfs_acl auth_rpcgss
exportfs autofs4 ipmi_devintf ipmi_si ipmi_msghandler<BR>lockd sunrpc bonding
ipv6 dm_mirror dm_region_hash dm_log uinput power_meter hwmon bnx2
amd64_edac_mod edac_core edac_mce_amd i2c_piix4 sg h<BR>pilo nx_nic(U) ext4
mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic pata_atiixp
ahci hpsa(U) radeon ttm drm_kms_helper drm<BR> i2c_algo_bit i2c_core
dm_mod [last unloaded: freq_table]</FONT></SPAN></DIV>
<DIV><FONT size=2 face=Arial></FONT> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>Modules linked in:
iptable_filter ip_tables nfs fscache fuse nfsd nfs_acl auth_rpcgss exportfs
autofs4 ipmi_devintf ipmi_si ipmi_msghandler<BR>lockd sunrpc bonding ipv6
dm_mirror dm_region_hash dm_log uinput power_meter hwmon bnx2 amd64_edac_mod
edac_core edac_mce_amd i2c_piix4 sg h<BR>pilo nx_nic(U) ext4 mbcache jbd2
sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic pata_atiixp ahci hpsa(U)
radeon ttm drm_kms_helper drm<BR> i2c_algo_bit i2c_core dm_mod [last
unloaded: freq_table]<BR>Pid: 3393, comm: lockd Tainted:
G W ----------------
2.6.32-71.14.1.el6.x86_64 #1 ProLiant DL585 G7<BR>RIP:
0010:[<ffffffff81186bf9>] [<ffffffff81186bf9>]
iput+0x69/0x70<BR>RSP: 0018:ffff88082b86fce0 EFLAGS: 00010246<BR>RAX:
0000000000000000 RBX: ffff8802fc8616c8 RCX: 000000000000c60e<BR>RDX:
ffff88202e13a901 RSI: ffffffffa0341de0 RDI: ffff8802fc8616c8<BR>RBP:
ffff88082b86fcf0 R08: 000000000002ac45 R09: 0000000000000000<BR>R10:
000000000000000f R11: 0000000000000000 R12: ffff880227b49c00<BR>R13:
ffffffffa034e060 R14: ffff88202e13a940 R15: 00000000fffffff5<BR>FS:
00007fac6a0247c0(0000) GS:ffff88002c240000(0000)
knlGS:00000000f77916c0<BR>CS: 0010 DS: 0018 ES: 0018 CR0:
000000008005003b<BR>CR2: 00007fac6a048000 CR3: 0000000c2da36000 CR4:
00000000000006e0<BR>DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000<BR>DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7:
0000000000000400<BR>Process lockd (pid: 3393, threadinfo ffff88082b86e000,
task ffff88082d9ab4e0)<BR>Stack:<BR> ffff88082b86fd40 ffff8802fc861680
ffff88082b86fd10 ffffffff813fdbf1<BR><0> ffff880227b49c00
ffff880227b49c00 ffff88082b86fd30 ffffffffa03351f8<BR><0>
ffff88082b86fd30 ffff880227b49c10 ffff88082b86fd60 ffffffffa0341e2c<BR>Call
Trace:<BR> [<ffffffff813fdbf1>]
sock_release+0x71/0x90<BR> [<ffffffffa03351f8>]
svc_sock_free+0x48/0x70 [sunrpc]<BR> [<ffffffffa0341e2c>]
svc_xprt_free+0x4c/0x70 [sunrpc]<BR> [<ffffffffa0341de0>] ?
svc_xprt_free+0x0/0x70 [sunrpc]<BR> [<ffffffff8125cb97>]
kref_put+0x37/0x70<BR> [<ffffffffa0340f29>] svc_xprt_put+0x19/0x20
[sunrpc]<BR> [<ffffffffa0341191>] svc_xprt_release+0xc1/0xe0
[sunrpc]<BR> [<ffffffffa03415bd>] svc_recv+0x2ed/0x830
[sunrpc]<BR> [<ffffffff8105c530>] ?
default_wake_function+0x0/0x20<BR> [<ffffffffa02f6291>]
lockd+0xc1/0x230 [lockd]<BR> [<ffffffffa02f61d0>] ? lockd+0x0/0x230
[lockd]<BR> [<ffffffff81091a76>]
kthread+0x96/0xa0<BR> [<ffffffff810141ca>]
child_rip+0xa/0x20<BR> [<ffffffff810919e0>] ?
kthread+0x0/0xa0<BR> [<ffffffff810141c0>] ?
child_rip+0x0/0x20<BR>Code: 38 48 c7 c0 f0 7c 18 81 48 85 d2 74 12 48 8b 42 20
48 c7 c2 f0 7c 18 81 48 85 c0 48 0f 44 c2 48 89 df ff d0 48 83 c4 08 5b c9 c3
<0f><BR>0b eb fe 0f 1f 00 55 48 89 e5 41 55 41 54 53 48 83 ec 08
0f<BR>RIP [<ffffffff81186bf9>] iput+0x69/0x70<BR> RSP
<ffff88082b86fce0><BR>˙Mounting proc filesystem<BR>Mounting sysfs
filesystem<BR>Creating /dev<BR>Creating initial device nodes<BR>Free
memory/Total memory (free %): 456164 / 495584 ( 92.0457 )<BR>Loading jbd2.ko
module<BR>Loading mbcache.ko module<BR>Loading ext4.ko module<BR>Loading
crc-t10dif.ko module<BR>Loading sd_mod.ko module<BR>Loading ata_generic.ko
module<BR>Loading exportfs.ko module<BR>Loading autofs4.ko module<BR>Loading
ipmi_msghandler.ko module<BR>Loading sunrpc.ko module<BR>Loading ipv6.ko
module<BR>Loading uinput.ko module<BR>Loading hwmon.ko module<BR>Loading
bnx2.ko module<BR>Loading edac_core.ko module<BR>Loading edac_mce_amd.ko
module<BR>Loading sg.ko module<BR>Loading hpilo.ko module<BR>Loading nx_nic.ko
module<BR>Loading cdrom.ko module<BR>Loading pata_acpi.ko module<BR>Loading
pata_atiixp.ko module<BR>Loading ahci.ko module<BR>Loading hpsa.ko
module<BR>hpsa 0000:03:00.0: controller message 03:00 timed out<BR>hpsa
0000:03:00.0: controller message 03:00 timed out<BR>hpsa 0000:03:00.0:
controller message 03:00 timed out<BR>hpsa 0000:44:00.0: controller message
03:00 timed out<BR>hpsa 0000:44:00.0: controller message 03:00 timed
out<BR>hpsa 0000:44:00.0: controller message 03:00 timed out<BR>Loading
i2c-core.ko module<BR>Loading dm-mod.ko module<BR>Loading nfs_acl.ko
module<BR>Loading auth_rpcgss.ko module<BR>Loading ipmi_devintf.ko
module<BR>Loading ipmi_si.ko module<BR>Loading lockd.ko
module<BR>Loadingpower_meter ACPI000D:00: Ignoring unsafe software power
cap!<BR> bonding.ko module<BR>Loading dm-log.ko module<BR>Loading
power_meter.ko module<BR>Loading amd64_edac_mod.ko module<BR>Loading
i2c-piix4.ko module<BR>Loading sr_mod.ko module<BR>Loading drm.ko
module<BR>Loading i2c-algo-bit.ko module<BR>Loading nfsd.ko module<BR>Loading
dm-region-hash.ko module<BR>Loading ttm.ko module<BR>Loading drm_kms_helper.ko
module<BR>Loading dm-mirror.ko module<BR>Loading radeon.ko module<BR>Waiting
for required block device discovery<BR>Waiting for 8 sdd-like
device(s)...Found<BR>Creating Block Devices<BR>Creating block device
loop0<BR>Creating block device loop1<BR>Creating block device
loop2<BR>Creating block device loop3<BR>Creating block device
loop4<BR>Creating block device loop5<BR>Creating block device
loop6<BR>Creating block device loop7<BR>Creating block device ram0<BR>Creating
block device ram1<BR>Creating block device ram10<BR>Creating block device
ram11<BR>Creating block device ram12<BR>Creating block device
ram13<BR>Creating block device ram14<BR>Creating block device
ram15<BR>Creating block device ram2<BR>Creating block device ram3<BR>Creating
block device ram4<BR>Creating block device ram5<BR>Creating block device
ram6<BR>Creating block device ram7<BR>Creating block device ram8<BR>Creating
block device ram9<BR>Creating block device sda<BR>Creating block device
sdb<BR>Creating block device sdc<BR>Creating block device sdd<BR>Creating
block device sr0<BR>mdadm: No arrays found in config file or
automatically<BR>Free memory/Total memory (free %): 432796 / 495584 ( 87.3305
)<BR>Saving to the local filesystem /dev/sdd1<BR>e2fsck 1.41.12
(17-May-2010)</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>Homes: recovering
journal<BR>Homes: clean, 9073003/164782080 files, 387571383/659105347
blocks<BR>Free memory/Total memory (free %): 427248 / 495584 ( 86.211
)<BR>Copying
data
: [ 2 %]<BR>Copying
data
: [100 %]<BR>Saving core complete<BR>Restarting
system.<BR></FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>Backtrace from
crash-dump utility shows:</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>GNU gdb (GDB)
7.0<BR>Copyright (C) 2009 Free Software Foundation, Inc.<BR>License GPLv3+:
GNU GPL version 3 or later <<A
href="http://gnu.org/licenses/gpl.html">http://gnu.org/licenses/gpl.html</A>><BR>This
is free software: you are free to change and redistribute it.<BR>There is NO
WARRANTY, to the extent permitted by law. Type "show copying"<BR>and
"show warranty" for details.<BR>This GDB was configured as
"x86_64-unknown-linux-gnu"...</FONT></SPAN></DIV>
<DIV> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial> KERNEL:
/usr/lib/debug/lib/modules/2.6.32-71.14.1.el6.x86_64/vmlinux<BR>
DUMPFILE: ./vmcore [PARTIAL
DUMP]<BR> CPUS:
48<BR> DATE: Wed Feb 9
09:30:52 2011<BR> UPTIME: 14 days,
13:57:19<BR>LOAD AVERAGE: 3.65, 3.39,
3.25<BR> TASKS: 1663<BR>
NODENAME: hydra.sie.siemens.at<BR> RELEASE:
2.6.32-71.14.1.el6.x86_64<BR> VERSION: #1 SMP Wed Jan
5 17:01:01 EST 2011<BR> MACHINE: x86_64 (2095
Mhz)<BR> MEMORY: 128
GB<BR> PANIC: "kernel BUG at
fs/inode.c:1333!"<BR> PID:
3393<BR> COMMAND:
"lockd"<BR> TASK:
ffff88082d9ab4e0 [THREAD_INFO:
ffff88082b86e000]<BR> CPU:
4<BR> STATE: TASK_RUNNING
(PANIC)</FONT></SPAN></DIV>
<DIV> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>crash>
bt<BR>PID: 3393 TASK: ffff88082d9ab4e0 CPU: 4
COMMAND: "lockd"<BR> #0 [ffff88082b86f9a0] machine_kexec at
ffffffff8103695b<BR> #1 [ffff88082b86fa00] crash_kexec at
ffffffff810b9068<BR> #2 [ffff88082b86fad0] oops_end at
ffffffff814cc6e0<BR> #3 [ffff88082b86fb00] die at
ffffffff8101733b<BR> #4 [ffff88082b86fb30] do_trap at
ffffffff814cbfb4<BR> #5 [ffff88082b86fb90] do_invalid_op at
ffffffff81014ee5<BR> #6 [ffff88082b86fc30] invalid_op at
ffffffff81013f5b<BR> [exception RIP:
iput+105]<BR> RIP: ffffffff81186bf9 RSP:
ffff88082b86fce0 RFLAGS: 00010246<BR> RAX:
0000000000000000 RBX: ffff8802fc8616c8 RCX:
000000000000c60e<BR> RDX: ffff88202e13a901 RSI:
ffffffffa0341de0 RDI: ffff8802fc8616c8<BR> RBP:
ffff88082b86fcf0 R8: 000000000002ac45 R9:
0000000000000000<BR> R10: 000000000000000f R11:
0000000000000000 R12: ffff880227b49c00<BR> R13:
ffffffffa034e060 R14: ffff88202e13a940 R15:
00000000fffffff5<BR> ORIG_RAX: ffffffffffffffff CS:
0010 SS: 0018<BR> #7 [ffff88082b86fcf8] sock_release at
ffffffff813fdbf1<BR> #8 [ffff88082b86fd18] svc_sock_free at
ffffffffa03351f8<BR> #9 [ffff88082b86fd38] svc_xprt_free at
ffffffffa0341e2c<BR>#10 [ffff88082b86fd68] kref_put at ffffffff8125cb97<BR>#11
[ffff88082b86fd88] svc_xprt_put at ffffffffa0340f29<BR>#12 [ffff88082b86fd98]
svc_xprt_release at ffffffffa0341191<BR>#13 [ffff88082b86fdc8] svc_recv at
ffffffffa03415bd<BR>#14 [ffff88082b86fe58] lockd at ffffffffa02f6291<BR>#15
[ffff88082b86fee8] kthread at ffffffff81091a76<BR>#16 [ffff88082b86ff48]
kernel_thread at ffffffff810141ca<BR>crash><BR></FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>any idea? any
hint? what else can i do to find the reason for these panics? how
to solve it?</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2 face=Arial>thanks a
lot,</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial>christian</FONT></SPAN></DIV>
<DIV><SPAN class=756185908-09022011><FONT size=2
face=Arial> </DIV></BLOCKQUOTE></FONT></SPAN></BODY></HTML>