[Linux-cachefs] kernel BUG at fs/fscache/operation.c:39!

[Fernando Santos]

Hi, 

 

I have a centos 7.1 (with kernel version 3.10.0-229.11.1.el7.x86_64) server
running stable for more than 70 days but recently crashed and rebooted
automatically. 

Part of the vmcore-dmesg.txt file which possible related to the crash
attached below: 

 

[6425317.842246] FS-Cache: Assertion failed

[6425317.842263] 0 > 0 is false

[6425317.842297] ------------[ cut here ]------------

[6425317.842316] kernel BUG at fs/fscache/operation.c:39!

[6425317.842337] invalid opcode: 0000 [#1] SMP 

[6425317.842356] Modules linked in: usb_storage iptable_filter ip_tables
xt_multiport xt_conntrack ipt_REJECT nf_conntrack_ipv4 nf_de

frag_ipv4 xt_connlimit nf_conntrack rpcsec_gss_krb5 nfsv4 dns_resolver nfsv3
nfs cachefiles fscache intel_powerclamp coretemp kvm crc

t10dif_pclmul crc32_pclmul dm_mirror dm_region_hash crc32c_intel dm_log
ghash_clmulni_intel aesni_intel dm_mod lrw gf128mul glue_help

er ablk_helper cryptd iTCO_wdt iTCO_vendor_support dcdbas lpc_ich shpchp
i7core_edac edac_core pcspkr mfd_core i2c_i801 ioatdma nfsd 

auth_rpcgss nfs_acl lockd sunrpc ext4 mbcache jbd2 sd_mod crc_t10dif
crct10dif_common ast syscopyarea sysfillrect sysimgblt drm_kms_h

elper igb ttm drm ptp ahci pps_core libahci dca i2c_algo_bit libata
megaraid_sas i2c_core [last unloaded: ip_tables]

[6425317.842750] CPU: 22 PID: 31861 Comm: amscore Not tainted
3.10.0-229.11.1.el7.x86_64 #1

[6425317.842795] Hardware name: Dell Inc.  C6100           /0D61XP, BIOS
1.70 12/19/2012

[6425317.842841] task: ffff880bf68c0b60 ti: ffff88038aa8c000 task.ti:
ffff88038aa8c000

[6425317.842888] RIP: 0010:[<ffffffffa0388bc8>]  [<ffffffffa0388bc8>]
fscache_enqueue_operation+0x128/0x180 [fscache]

[6425317.842956] RSP: 0000:ffff88183fd43c10  EFLAGS: 00010046

[6425317.842990] RAX: 000000000000000e RBX: ffff8814e753e900 RCX:
0000000000000006

[6425317.843035] RDX: 0000000000000000 RSI: 0000000000000046 RDI:
0000000000000046

[6425317.843073] RBP: ffff88183fd43c18 R08: 0000000000000096 R09:
000000000000049b

[6425317.843101] R10: 0000000000000157 R11: 0000000000000001 R12:
ffff880028adb3d0

[6425317.843129] R13: ffff88183ffc5bd8 R14: 0000000000000000 R15:
0000000000000003

[6425317.843157] FS:  00007f1ff3d7a700(0000) GS:ffff88183fd40000(0000)
knlGS:0000000000000000

[6425317.843188] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

[6425317.843211] CR2: 000000002bcc7120 CR3: 0000000e3ddc9000 CR4:
00000000000007e0

[6425317.843239] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000

[6425317.843267] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400

[6425317.843294] Stack:

[6425317.843304]  ffff881047d123c0 ffff88183fd43c38 ffffffffa03a1b15
0000000000000000

[6425317.843340]  ffff88183ffc5bf0 ffff88183fd43c80 ffffffff810a0758
000000016e242909

[6425317.843375]  ffff88183fd43cc8 ffff88183ffc5be8 0000000000000283
0000000000000003

[6425317.843409] Call Trace:

[6425317.843421]  <IRQ> 

[6425317.843432] 

[6425317.843444]  [<ffffffffa03a1b15>] cachefiles_read_waiter+0x85/0xd0
[cachefiles]

[6425317.843471]  [<ffffffff810a0758>] __wake_up_common+0x58/0x90

[6425317.843495]  [<ffffffff810a2379>] __wake_up+0x39/0x50

[6425317.843517]  [<ffffffff810981e0>] __wake_up_bit+0x40/0x60

[6425317.843541]  [<ffffffff81156bcb>] unlock_page+0x2b/0x30

[6425317.843567]  [<ffffffff81205dfe>] mpage_end_io+0x3e/0xb0

[6425317.844516]  [<ffffffff811fd2cd>] bio_endio+0x1d/0x30

[6425317.845460]  [<ffffffff812ad460>] blk_update_request+0x90/0x350

[6425317.846414]  [<ffffffff812ad73c>] blk_update_bidi_request+0x1c/0x80

[6425317.847360]  [<ffffffff812ada1f>] blk_end_bidi_request+0x1f/0x60

[6425317.848329]  [<ffffffff812ada70>] blk_end_request+0x10/0x20

[6425317.849254]  [<ffffffff813f9cd8>] scsi_io_completion+0x108/0x650

[6425317.850160]  [<ffffffff813eece3>] scsi_finish_command+0xb3/0x110

[6425317.851055]  [<ffffffff813f9adf>] scsi_softirq_done+0x12f/0x160

[6425317.851944]  [<ffffffff812b3fb0>] blk_done_softirq+0x90/0xc0

[6425317.852836]  [<ffffffff81077b2f>] __do_softirq+0xef/0x280

[6425317.853729]  [<ffffffff81615b9c>] call_softirq+0x1c/0x30

[6425317.854618]  [<ffffffff81015d95>] do_softirq+0x65/0xa0

[6425317.855500]  [<ffffffff81077ec5>] irq_exit+0x115/0x120

[6425317.856382]  [<ffffffff81041915>]
smp_call_function_single_interrupt+0x35/0x40

[6425317.857252]  [<ffffffff8161535d>]
call_function_single_interrupt+0x6d/0x80

[6425317.858095]  <EOI> 

[6425317.858105] 

[6425317.858915]  [<ffffffff816143ab>] ? sysret_audit+0x17/0x21

[6425317.859707] Code: 38 a0 31 c0 e8 a2 52 27 e1 48 c7 c7 e5 d3 38 a0 31 c0
e8 94 52 27 e1 48 63 73 44 31 d2 48 c7 c7 03 d4 38 a0 31

c0 e8 80 52 27 e1 <0f> 0b 48 c7 c7 e1 d3 38 a0 31 c0 e8 70 52 27 e1 48 c7 c7
e5 d3 

[6425317.861380] RIP  [<ffffffffa0388bc8>]
fscache_enqueue_operation+0x128/0x180 [fscache]

[6425317.862172]  RSP <ffff88183fd43c10>

 

There is seems to be a kernel update available in centos, version:
3.10.0-327.3.1.el7 .

Will this update can resolve my crash issue? Please advise. 

 

 

Thanks, 

Fernando