[Linux-cluster] Re: Hard lockups during file transfer to GNBD/GFS device
Benjamin Marzinski
bmarzins at redhat.com
Mon Oct 23 17:19:47 UTC 2006
On Mon, Oct 23, 2006 at 09:48:34AM -0400, David Brieck Jr. wrote:
> On 10/10/06, David Brieck Jr. <dbrieck at gmail.com> wrote:
> >On 9/29/06, David Brieck Jr. <dbrieck at gmail.com> wrote:
> >> On 9/28/06, David Teigland <teigland at redhat.com> wrote:
> >> >
> >> > Could you try it without multipath? You have quite a few layers there.
> >> > Dave
> >> >
> >> >
> >>
> >> Thanks for the response. I unloaded gfs, clvm, gnbd and multipath, the
> >> reloaded gnbd, clvm and gfs. It was only talking to one of the gnbd
> >> servers and without multipath. Here's the log from this crash. It
> >> seems to have more info in it.
> >>
> >> I'm kinda confused why it still has references to mulitpath though. I
> >> unloaded the multipath module so I'm not sure why it's still in there.
> >> SNIP
> >
> >Since I didn't hear back from anyone I decided to try things a little
> >differently. Instead of rsyncing on the local machine, I ran the rsync
> >from another cluster member who also mounts the same partition I was
> >trying to move things too.
> >
> >So instead of
> >
> >rsync -a /path/to/files/ /mnt/http/
> >
> >I used
> >
> >rsync -a root at 10.1.1.121::/path/to/files/ /mnt/http/
> >
> >and I didn't have a crash at all. Why would this not cause a problem
> >when the first one did? Is this more of an rsync problem maybe? I do
> >have 2 NFS exports, could those have been causing the problem?
> >
> >Thanks.
> >
>
> I am now seeing other crashes. Here's one from this weekend. It's huge
> and I'm not even sure how to see where things might have gone wrong.
> At the time of the crash the system was doing a backup to the gnbd
> server (if that helps). I'd also appreciate it if maybe someone could
> explain how to read this as far as where the problem actually started.
I'd bet that the problem started exactly where the crash says it did,
with a stack overflow. This is with running gfs on top of clvm on top of
multipath on to of gnbd, right? So for IO to complete, it needs to go through
gfs, device-mapper, and gnbd functions. It's possible that some functions in
those modules aren't incredibly efficient with their stack space usage (A
number of functions like this have been found and fixed in GFS over the years).
Since it's a pretty long function stack, it would take too much waste in
a couple of functions to put this over the edge.
Which means that this setup probably needs some testing, and functions need
auditing.
Just to make sure of some things:
This is not a crash on a gnbd server node. correct? More specifically, you
aren't running a gnbd client and server on the same node for the same device.
That is bad. On a gnbd server machine, you cannot gnbd_import the devices that
you just gnbd_exported from that machine, and there's no reason to anyway.
Is anything else running that would effect the gfs file system on this node?
You mentioned NFS earlier. Are the gnbd client machines using NFS to serve
up the gfs file system?
Are you running bonded ethernet? There's nothing wrong with that, it just adds
more functions to the stack. That is an amazingly ugly stack trace, and I'm
trying to figure out what all is on there.
> Thanks again.
>
> Oct 22 00:17:56 db1 kernel: do_IRQ: stack overflow: 476
> Oct 22 00:17:56 db1 kernel: [<02107c6b>] do_IRQ+0x49/0x1ae
> Oct 22 00:17:56 db1 kernel: [<022cb5ea>] _read_unlock_bh+0x3/0x8
> Oct 22 00:17:56 db1 kernel: [<f8fea2bd>] do_bindings+0x9d/0x1e3
> [iptable_nat]
> Oct 22 00:17:56 db1 kernel: [<f8fe91c6>] ip_nat_fn+0x1c6/0x1d4
> [iptable_nat]
> Oct 22 00:17:56 db1 kernel: [<0228227b>] nf_iterate+0x40/0x81
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<02282581>] nf_hook_slow+0x47/0xbc
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<022929b7>] ip_finish_output+0x1c5/0x1ce
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<022927e7>] dst_output+0xf/0x1a
> Oct 22 00:17:56 db1 kernel: [<022825bd>] nf_hook_slow+0x83/0xbc
> Oct 22 00:17:56 db1 kernel: [<02293093>] ip_queue_xmit+0x395/0x3f9
> Oct 22 00:17:56 db1 kernel: [<022927d8>] dst_output+0x0/0x1a
> Oct 22 00:17:56 db1 kernel: [<022927e7>] dst_output+0xf/0x1a
> Oct 22 00:17:56 db1 kernel: [<022825bd>] nf_hook_slow+0x83/0xbc
> Oct 22 00:17:56 db1 kernel: [<02293093>] ip_queue_xmit+0x395/0x3f9
> Oct 22 00:17:56 db1 kernel: [<022927d8>] dst_output+0x0/0x1a
> Oct 22 00:17:56 db1 kernel: [<0211df36>] find_busiest_group+0xdd/0x295
> Oct 22 00:17:56 db1 kernel: [<022a62a7>] tcp_v4_send_check+0x7c/0xb9
> Oct 22 00:17:56 db1 kernel: [<022a10b1>] tcp_transmit_skb+0x6bf/0x7d2
> Oct 22 00:17:56 db1 kernel: [<022a19be>] tcp_write_xmit+0x74/0x254
> Oct 22 00:17:56 db1 kernel: [<0229ef8e>] __tcp_data_snd_check+0x3d/0xaa
> Oct 22 00:17:56 db1 kernel: [<0229f471>] tcp_rcv_established+0x166/0x725
> Oct 22 00:17:56 db1 kernel: [<022a72ce>] tcp_v4_do_rcv+0x1b/0xe9
> Oct 22 00:17:56 db1 kernel: [<02274c39>] __release_sock+0x39/0x55
> Oct 22 00:17:56 db1 kernel: [<022751b8>] release_sock+0x1f/0x4f
> Oct 22 00:17:56 db1 kernel: [<02277e42>] sk_stream_wait_memory+0x113/0x1a8
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<02297ca1>] tcp_sendmsg+0xb5c/0xe4d
> Oct 22 00:17:56 db1 kernel: [<022b2f5d>] inet_sendmsg+0x38/0x42
> Oct 22 00:17:56 db1 kernel: [<0227221f>] sock_sendmsg+0xdb/0xf7
> Oct 22 00:17:56 db1 kernel: [<02286b85>] qdisc_restart+0x12/0x1b9
> Oct 22 00:17:56 db1 kernel: [<f88df37d>]
> bond_dev_queue_xmit+0x1f3/0x1fb [bonding]
> Oct 22 00:17:56 db1 kernel: [<f88e29bb>]
> bond_xmit_roundrobin+0xdb/0xe3 [bonding]
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<f89e95c8>] sock_xmit+0x11d/0x217 [gnbd]
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<022927e7>] dst_output+0xf/0x1a
> Oct 22 00:17:56 db1 kernel: [<022825bd>] nf_hook_slow+0x83/0xbc
> Oct 22 00:17:56 db1 kernel: [<02141db2>] mempool_alloc+0x7b/0x135
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<f89e9938>] __gnbd_send_req+0x276/0x2e7 [gnbd]
> Oct 22 00:17:56 db1 kernel: [<f89e9dc0>] do_gnbd_request+0xfa/0x143 [gnbd]
> Oct 22 00:17:56 db1 kernel: [<02220380>] __generic_unplug_device+0x2b/0x2d
> Oct 22 00:17:56 db1 kernel: [<02220397>] generic_unplug_device+0x15/0x21
> Oct 22 00:17:56 db1 kernel: [<02220992>] get_request_wait+0x51/0xb9
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<022213c1>] __make_request+0x2bf/0x46c
> Oct 22 00:17:56 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:56 db1 kernel: [<f88f623c>] multipath_map+0x64/0x7f
> [dm_multipath]
> Oct 22 00:17:56 db1 kernel: [<f88693ca>] __map_bio+0x35/0xb0 [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<f88695cd>] __clone_and_map+0xc2/0x2f0
> [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<f88698a2>] __split_bio+0xa7/0x103 [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<f8869a19>] dm_request+0x11b/0x130 [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:56 db1 kernel: [<0215e848>] bio_clone+0x84/0x9c
> Oct 22 00:17:56 db1 kernel: [<f88693ca>] __map_bio+0x35/0xb0 [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<f88695cd>] __clone_and_map+0xc2/0x2f0
> [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<f88698a2>] __split_bio+0xa7/0x103 [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<f8869a19>] dm_request+0x11b/0x130 [dm_mod]
> Oct 22 00:17:56 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:56 db1 kernel: [<0215e848>] bio_clone+0x84/0x9c
> Oct 22 00:17:56 db1 kernel: [<f8f2c3c1>] diaper_make_request+0x8b/0x91
> [gfs]
> Oct 22 00:17:56 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<022217d6>] submit_bio+0xca/0xd2
> Oct 22 00:17:56 db1 kernel: [<0215e681>] bio_alloc+0x100/0x168
> Oct 22 00:17:56 db1 kernel: [<0215e038>] submit_bh+0x141/0x166
> Oct 22 00:17:56 db1 kernel: [<0215e0b9>] ll_rw_block+0x5c/0x72
> Oct 22 00:17:56 db1 kernel: [<f8f2d658>] gfs_dreread+0x65/0xd6 [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f2d5a4>] gfs_dread+0x20/0x36 [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f2e4ec>] gfs_get_meta_buffer+0xb1/0x250
> [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f2a647>] gfs_block_map+0x242/0x3de [gfs]
> Oct 22 00:17:56 db1 kernel: [<022cad75>] __cond_resched+0x14/0x39
> Oct 22 00:17:56 db1 kernel: [<f8f49cd6>] get_block+0x2a/0x6d [gfs]
> Oct 22 00:17:56 db1 kernel: [<0215cd70>] __block_prepare_write+0x165/0x3ec
> Oct 22 00:17:56 db1 kernel: [<0215d682>] block_prepare_write+0x16/0x23
> Oct 22 00:17:56 db1 kernel: [<f8f49cac>] get_block+0x0/0x6d [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4a258>] gfs_prepare_write+0x122/0x135
> [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f49cac>] get_block+0x0/0x6d [gfs]
> Oct 22 00:17:56 db1 kernel: [<02140ec8>]
> generic_file_buffered_write+0x218/0x533
> Oct 22 00:17:56 db1 kernel: [<02126347>] current_fs_time+0x44/0x4c
> Oct 22 00:17:56 db1 kernel: [<0214156c>]
> __generic_file_aio_write_nolock+0x389/0x3b7
> Oct 22 00:17:56 db1 kernel: [<021415d3>]
> generic_file_aio_write_nolock+0x39/0x7f
> Oct 22 00:17:56 db1 kernel: [<02141736>]
> generic_file_write_nolock+0x84/0x99
> Oct 22 00:17:56 db1 kernel: [<f8f38055>] gfs_glock_nq+0xe3/0x116 [gfs]
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<f8f58bac>] gfs_trans_begin_i+0xfd/0x15a [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4c0fc>] do_do_write_buf+0x2a6/0x452 [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4c3c3>] do_write_buf+0x11b/0x15e [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4b31c>] walk_vm+0xd7/0x100 [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4c4a7>] __gfs_write+0xa1/0xbb [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4c2a8>] do_write_buf+0x0/0x15e [gfs]
> Oct 22 00:17:56 db1 kernel: [<f8f4c4cc>] gfs_write+0xb/0xe [gfs]
> Oct 22 00:17:56 db1 kernel: [<0215a52f>] vfs_write+0xb6/0xe2
> Oct 22 00:17:56 db1 kernel: [<0215a5f9>] sys_write+0x3c/0x62
> Oct 22 00:17:56 db1 kernel: =======================
> Oct 22 00:17:56 db1 kernel: Unable to handle kernel paging request at
> virtual address 000104e6
> Oct 22 00:17:56 db1 kernel: printing eip:
> Oct 22 00:17:56 db1 kernel: 02105fc0
> Oct 22 00:17:56 db1 kernel: *pde = 00004001
> Oct 22 00:17:56 db1 kernel: Oops: 0000 [#1]
> Oct 22 00:17:56 db1 kernel: SMP
> Oct 22 00:17:56 db1 kernel: Modules linked in: ipt_REDIRECT
> iptable_nat ip_conntrack iptable_mangle iptable_filter ip_tables
> mptctl mptbase dell_rbu parport_
> pc lp parport autofs4 i2c_dev i2c_core lock_dlm(U) gfs(U)
> lock_harness(U) dm_round_robin gnbd(U) dlm(U) cman(U) sunrpc
> ipmi_devintf ipmi_si ipmi_msghandler m
> d5 ipv6 dm_mirror dm_multipath button battery ac joydev uhci_hcd
> ehci_hcd hw_random e1000 bonding(U) floppy sg ext3 jbd dm_mod
> megaraid_mbox megaraid_mm sd_m
> od scsi_mod
> Oct 22 00:17:56 db1 kernel: CPU: 35683031
> Oct 22 00:17:56 db1 kernel: EIP: 0060:[<02105fc0>] Not tainted VLI
> Oct 22 00:17:56 db1 kernel: EFLAGS: 00010093 (2.6.9-42.0.2.ELhugemem)
> Oct 22 00:17:56 db1 kernel: EIP is at show_trace+0x11/0x6b
> Oct 22 00:17:56 db1 kernel: eax: 00010ffd ebx: 000104e6 ecx:
> a9dae200 edx: 022d65b8
> Oct 22 00:17:56 db1 kernel: esi: 000104e6 edi: 00010000 ebp:
> 00000000 esp: a9dae200
> Oct 22 00:17:56 db1 kernel: ds: 007b es: 007b ss: 0068
> Oct 22 00:17:56 db1 kernel: Process (pid: 122215904,
> threadinfo=a9dae000 task=a9dae094)
> Oct 22 00:17:56 db1 kernel: Stack: a9dae000 00005480 02385480 021060a4
> 0000001c 02107c6b 022d6d40 000001dc
> Oct 22 00:17:56 db1 kernel: c4dbbea0 000000a9 00000000 c4dbbe00
> c4dbbea0 00000000 fffecca1 00000000
> Oct 22 00:17:56 db1 kernel: c4dbbea0 f8fee558 c4dbbe00 c4dbbea0
> 00000000 f8fee558 c4db007b 0000007b
> Oct 22 00:17:56 db1 kernel: Call Trace:
> Oct 22 00:17:56 db1 kernel: [<021060a4>] dump_stack+0x11/0x13
> Oct 22 00:17:56 db1 kernel: [<02107c6b>] do_IRQ+0x49/0x1ae
> Oct 22 00:17:56 db1 kernel: [<022cb5ea>] _read_unlock_bh+0x3/0x8
> Oct 22 00:17:56 db1 kernel: [<f8fea2bd>] do_bindings+0x9d/0x1e3
> [iptable_nat]
> Oct 22 00:17:56 db1 kernel: [<f8fe91c6>] ip_nat_fn+0x1c6/0x1d4
> [iptable_nat]
> Oct 22 00:17:56 db1 kernel: [<0228227b>] nf_iterate+0x40/0x81
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<02282581>] nf_hook_slow+0x47/0xbc
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<022929b7>] ip_finish_output+0x1c5/0x1ce
> Oct 22 00:17:56 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:56 db1 kernel: [<022927e7>] dst_output+0xf/0x1a
> Oct 22 00:17:56 db1 kernel: [<022825bd>] nf_hook_slow+0x83/0xbc
> Oct 22 00:17:56 db1 kernel: [<02293093>] ip_queue_xmit+0x395/0x3f9
> Oct 22 00:17:56 db1 kernel: [<022927d8>] dst_output+0x0/0x1a
> Oct 22 00:17:56 db1 kernel: [<022927e7>] dst_output+0xf/0x1a
> Oct 22 00:17:56 db1 kernel: [<022825bd>] nf_hook_slow+0x83/0xbc
> Oct 22 00:17:56 db1 kernel: [<02293093>] ip_queue_xmit+0x395/0x3f9
> Oct 22 00:17:56 db1 kernel: [<022927d8>] dst_output+0x0/0x1a
> Oct 22 00:17:56 db1 kernel: [<0211df36>] find_busiest_group+0xdd/0x295
> Oct 22 00:17:56 db1 kernel: [<022a62a7>] tcp_v4_send_check+0x7c/0xb9
> Oct 22 00:17:56 db1 kernel: [<022a10b1>] tcp_transmit_skb+0x6bf/0x7d2
> Oct 22 00:17:56 db1 kernel: [<022a19be>] tcp_write_xmit+0x74/0x254
> Oct 22 00:17:56 db1 kernel: [<0229ef8e>] __tcp_data_snd_check+0x3d/0xaa
> Oct 22 00:17:56 db1 kernel: [<0229f471>] tcp_rcv_established+0x166/0x725
> Oct 22 00:17:56 db1 kernel: [<022a72ce>] tcp_v4_do_rcv+0x1b/0xe9
> Oct 22 00:17:56 db1 kernel: [<02274c39>] __release_sock+0x39/0x55
> Oct 22 00:17:56 db1 kernel: [<022751b8>] release_sock+0x1f/0x4f
> Oct 22 00:17:56 db1 kernel: [<02277e42>] sk_stream_wait_memory+0x113/0x1a8
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:56 db1 kernel: [<02297ca1>] tcp_sendmsg+0xb5c/0xe4d
> Oct 22 00:17:57 db1 kernel: [<022b2f5d>] inet_sendmsg+0x38/0x42
> Oct 22 00:17:57 db1 kernel: [<0227221f>] sock_sendmsg+0xdb/0xf7
> Oct 22 00:17:57 db1 kernel: [<02286b85>] qdisc_restart+0x12/0x1b9
> Oct 22 00:17:57 db1 kernel: [<f88df37d>]
> bond_dev_queue_xmit+0x1f3/0x1fb [bonding]
> Oct 22 00:17:57 db1 kernel: [<f88e29bb>]
> bond_xmit_roundrobin+0xdb/0xe3 [bonding]
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<f89e95c8>] sock_xmit+0x11d/0x217 [gnbd]
> Oct 22 00:17:57 db1 kernel: [<022929c0>] ip_finish_output2+0x0/0x18d
> Oct 22 00:17:57 db1 kernel: [<022927e7>] dst_output+0xf/0x1a
> Oct 22 00:17:57 db1 kernel: [<022825bd>] nf_hook_slow+0x83/0xbc
> Oct 22 00:17:57 db1 kernel: [<02141db2>] mempool_alloc+0x7b/0x135
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<f89e9938>] __gnbd_send_req+0x276/0x2e7 [gnbd]
> Oct 22 00:17:57 db1 kernel: [<f89e9dc0>] do_gnbd_request+0xfa/0x143 [gnbd]
> Oct 22 00:17:57 db1 kernel: [<02220380>] __generic_unplug_device+0x2b/0x2d
> Oct 22 00:17:57 db1 kernel: [<02220397>] generic_unplug_device+0x15/0x21
> Oct 22 00:17:57 db1 kernel: [<02220992>] get_request_wait+0x51/0xb9
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<022213c1>] __make_request+0x2bf/0x46c
> Oct 22 00:17:57 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:57 db1 kernel: [<f88f623c>] multipath_map+0x64/0x7f
> [dm_multipath]
> Oct 22 00:17:57 db1 kernel: [<f88693ca>] __map_bio+0x35/0xb0 [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<f88695cd>] __clone_and_map+0xc2/0x2f0
> [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<f88698a2>] __split_bio+0xa7/0x103 [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<f8869a19>] dm_request+0x11b/0x130 [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:57 db1 kernel: [<0215e848>] bio_clone+0x84/0x9c
> Oct 22 00:17:57 db1 kernel: [<f88693ca>] __map_bio+0x35/0xb0 [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<f88695cd>] __clone_and_map+0xc2/0x2f0
> [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<f88698a2>] __split_bio+0xa7/0x103 [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<f8869a19>] dm_request+0x11b/0x130 [dm_mod]
> Oct 22 00:17:57 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:57 db1 kernel: [<0215e848>] bio_clone+0x84/0x9c
> Oct 22 00:17:57 db1 kernel: [<f8f2c3c1>] diaper_make_request+0x8b/0x91
> [gfs]
> Oct 22 00:17:57 db1 kernel: [<022216fc>] generic_make_request+0x18e/0x19e
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<022217d6>] submit_bio+0xca/0xd2
> Oct 22 00:17:57 db1 kernel: [<0215e681>] bio_alloc+0x100/0x168
> Oct 22 00:17:57 db1 kernel: [<0215e038>] submit_bh+0x141/0x166
> Oct 22 00:17:57 db1 kernel: [<0215e0b9>] ll_rw_block+0x5c/0x72
> Oct 22 00:17:57 db1 kernel: [<f8f2d658>] gfs_dreread+0x65/0xd6 [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f2d5a4>] gfs_dread+0x20/0x36 [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f2e4ec>] gfs_get_meta_buffer+0xb1/0x250
> [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f2a647>] gfs_block_map+0x242/0x3de [gfs]
> Oct 22 00:17:57 db1 kernel: [<022cad75>] __cond_resched+0x14/0x39
> Oct 22 00:17:57 db1 kernel: [<f8f49cd6>] get_block+0x2a/0x6d [gfs]
> Oct 22 00:17:57 db1 kernel: [<0215cd70>] __block_prepare_write+0x165/0x3ec
> Oct 22 00:17:57 db1 kernel: [<0215d682>] block_prepare_write+0x16/0x23
> Oct 22 00:17:57 db1 kernel: [<f8f49cac>] get_block+0x0/0x6d [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4a258>] gfs_prepare_write+0x122/0x135
> [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f49cac>] get_block+0x0/0x6d [gfs]
> Oct 22 00:17:57 db1 kernel: [<02140ec8>]
> generic_file_buffered_write+0x218/0x533
> Oct 22 00:17:57 db1 kernel: [<02126347>] current_fs_time+0x44/0x4c
> Oct 22 00:17:57 db1 kernel: [<0214156c>]
> __generic_file_aio_write_nolock+0x389/0x3b7
> Oct 22 00:17:57 db1 kernel: [<021415d3>]
> generic_file_aio_write_nolock+0x39/0x7f
> Oct 22 00:17:57 db1 kernel: [<02141736>]
> generic_file_write_nolock+0x84/0x99
> Oct 22 00:17:57 db1 kernel: [<f8f38055>] gfs_glock_nq+0xe3/0x116 [gfs]
> Oct 22 00:17:57 db1 kernel: [<021204e9>] autoremove_wake_function+0x0/0x2d
> Oct 22 00:17:57 db1 kernel: [<f8f58bac>] gfs_trans_begin_i+0xfd/0x15a [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4c0fc>] do_do_write_buf+0x2a6/0x452 [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4c3c3>] do_write_buf+0x11b/0x15e [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4b31c>] walk_vm+0xd7/0x100 [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4c4a7>] __gfs_write+0xa1/0xbb [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4c2a8>] do_write_buf+0x0/0x15e [gfs]
> Oct 22 00:17:57 db1 kernel: [<f8f4c4cc>] gfs_write+0xb/0xe [gfs]
> Oct 22 00:17:57 db1 kernel: [<0215a52f>] vfs_write+0xb6/0xe2
> Oct 22 00:17:57 db1 kernel: [<0215a5f9>] sys_write+0x3c/0x62
> Oct 22 00:17:57 db1 kernel: =======================
> Oct 22 00:17:57 db1 kernel: Unable to handle kernel NULL pointer
> dereference at virtual address 00000001
> Oct 22 00:17:57 db1 kernel: printing eip:
> Oct 22 00:17:57 db1 kernel: 02105fc0
> Oct 22 00:17:57 db1 kernel: *pde = 00004001
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
More information about the Linux-cluster
mailing list