Xen issues - Was Xen network performance problem during kickstart.
Naoki
naoki at valuecommerce.com
Tue Feb 21 07:48:54 UTC 2006
On the guest that I was running the transfer tests on this popped up
sometime afterwards.
[root at localhost ~]# BUG: soft lockup detected on CPU#0!
Pid: 0, comm: swapper
EIP: 0061:[<c01040c7>] CPU: 0
EIP is at 0xc01040c7
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest) EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: c02ea000 EBP: 00000020 DS: 007b ES: 007b
CR0: 8005003b CR2: 003b4f34 CR3: 03389000 CR4: 00000600
[<c0105bf6>] xen_idle+0x59/0x68
[<c0105d09>] cpu_idle+0x8b/0xa4
[<c02ec708>] start_kernel+0x2ed/0x2f3
BUG: soft lockup detected on CPU#0!
Pid: 5028, comm: zcat
EIP: 0061:[<c0104227>] CPU: 0
EIP is at 0xc0104227
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00030000 EBX: 00000000 ECX: 00000000 EDX: c1cfbfbc
ESI: 00000000 EDI: c0110c11 EBP: 00000004 DS: 007b ES: 007b
CR0: 8005003b CR2: 0804eab0 CR3: 024be000 CR4: 00000600
[<c01f577c>] force_evtchn_callback+0xa/0xc
[<c0110cb8>] do_page_fault+0xa7/0x633
[<c0110c11>] do_page_fault+0x0/0x633
[<c0107c1b>] error_code+0x2b/0x30
BUG: soft lockup detected on CPU#0!
Pid: 5452, comm: makewhatis
EIP: 0073:[<002ccf13>] CPU: 0
EIP is at 0x2ccf13
ESP: 007b:bfa88b88 EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 09e81688 EBX: 0038dff4 ECX: 00000001 EDX: 00000000
ESI: 0038f120 EDI: 09e810c8 EBP: bfa88bb8 DS: 007b ES: 007b
CR0: 8005003b CR2: 09e70050 CR3: 01f15000 CR4: 00000600
BUG: soft lockup detected on CPU#0!
Pid: 1714, comm: makewhatis
EIP: 0061:[<c0104227>] CPU: 0
EIP is at 0xc0104227
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00030000 EBX: 00000000 ECX: 00000000 EDX: c3375fbc
ESI: 09e81ce8 EDI: c0110c11 EBP: 00000007 DS: 007b ES: 007b
CR0: 8005003b CR2: 09e81cf0 CR3: 02a3b000 CR4: 00000600
[<c01f577c>] force_evtchn_callback+0xa/0xc
[<c0110cb8>] do_page_fault+0xa7/0x633
[<c0110c11>] do_page_fault+0x0/0x633
[<c0107c1b>] error_code+0x2b/0x30
BUG: soft lockup detected on CPU#0!
Pid: 6859, comm: makewhatis
EIP: 0061:[<c0104347>] CPU: 0
EIP is at 0xc0104347
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000000 EBX: c1933de8 ECX: 00000001 EDX: 00000000
ESI: 00007ff0 EDI: c1d78004 EBP: c1d78004 DS: 007b ES: 007b
CR0: 8005003b CR2: b7f8ccdd CR3: 019d5000 CR4: 00000600
[<c0112232>] xen_pgd_pin+0x38/0x4a
[<c010ff59>] __pgd_pin+0x1d/0x31
[<c010ff85>] mm_pin+0x18/0x21
[<c0160fa6>] flush_old_exec+0x626/0x91e
[<c0157b4b>] vfs_read+0xf6/0x136
[<c016064d>] kernel_read+0x39/0x42
[<c017ceff>] load_elf_binary+0x307/0x15bc
[<c017d0a0>] load_elf_binary+0x4a8/0x15bc
[<c013f819>] get_page_from_freelist+0x99/0x396
[<c013f9cd>] get_page_from_freelist+0x24d/0x396
[<c01e4697>] copy_from_user+0x5c/0x90
[<c01602d4>] copy_strings+0x167/0x1c0
[<c017cbf8>] load_elf_binary+0x0/0x15bc
[<c01603e7>] search_binary_handler+0x90/0x21b
[<c0161eba>] do_execve+0x15f/0x1fc
[<c010621d>] sys_execve+0x2c/0x6d
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 28272, comm: gawk
EIP: 0061:[<c01493ec>] CPU: 0
EIP is at vma_adjust+0x1e6/0x3bc
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: c03c0650 EBX: c2ac3828 ECX: cdbd9fc4 EDX: 00000000
ESI: 00000000 EDI: c33da6fc EBP: 00000000 DS: 007b ES: 007b
CR0: 80050033 CR2: 0901f04c CR3: 0e2e7000 CR4: 00000600
[<c0264153>] _spin_unlock+0x6/0x8
[<c0147d88>] __handle_mm_fault+0x931/0x959
[<c0149975>] vma_merge+0xd1/0x155
[<c0149ec8>] do_brk+0x198/0x255
[<c014a4a9>] sys_brk+0xcf/0xfe
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 28426, comm: makewhatis
EIP: 0061:[<c0104345>] CPU: 0
EIP is at 0xc0104345
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 0000001a EBX: c1991d84 ECX: 80000001 EDX: 00000000
ESI: 00007ff0 EDI: c1d78604 EBP: c1d78a04 DS: 007b ES: 007b
CR0: 8005003b CR2: 002724f2 CR3: 019e9000 CR4: 00000600
[<c011227c>] xen_pgd_unpin+0x38/0x4a
[<c010ff9e>] __pgd_unpin+0x10/0x32
[<c010ffd8>] mm_unpin+0x18/0x21
[<c011012d>] _arch_exit_mmap+0x14c/0x155
[<c0148e99>] exit_mmap+0x19/0xee
[<c01178bc>] mmput+0x1e/0x93
[<c016106f>] flush_old_exec+0x6ef/0x91e
[<c0157b4b>] vfs_read+0xf6/0x136
[<c016064d>] kernel_read+0x39/0x42
[<c017ceff>] load_elf_binary+0x307/0x15bc
[<c017d0a0>] load_elf_binary+0x4a8/0x15bc
[<c013f819>] get_page_from_freelist+0x99/0x396
[<c013f9cd>] get_page_from_freelist+0x24d/0x396
[<c01e4697>] copy_from_user+0x5c/0x90
[<c01e4697>] copy_from_user+0x5c/0x90
[<c01602d4>] copy_strings+0x167/0x1c0
[<c017cbf8>] load_elf_binary+0x0/0x15bc
[<c01603e7>] search_binary_handler+0x90/0x21b
[<c0161eba>] do_execve+0x15f/0x1fc
[<c010621d>] sys_execve+0x2c/0x6d
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 29678, comm: makewhatis
EIP: 0061:[<c01530db>] CPU: 0
EIP is at poison_obj+0x1c/0x25
EFLAGS: 00000286 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 0000005a EBX: c18d9000 ECX: 00000000 EDX: 00001000
ESI: c0599ac0 EDI: c18da000 EBP: 000000d0 DS: 007b ES: 007b
CR0: 8005003b CR2: b7f8ccdd CR3: 068af000 CR4: 00000600
[<c01535bd>] cache_alloc_debugcheck_after+0x30/0xfb
[<c0153752>] kmem_cache_alloc+0xca/0xd5
[<c0164761>] getname+0x18/0x9f
[<c0164761>] getname+0x18/0x9f
[<c0164761>] getname+0x18/0x9f
[<c0166109>] __user_walk_fd+0xe/0x3a
[<c0156485>] sys_faccessat+0x92/0x126
[<c016fd6c>] mntput_no_expire+0x14/0x75
[<c013950c>] audit_syscall_entry+0x10e/0x133
[<c0109fb0>] do_syscall_trace+0x112/0x15e
[<c0156528>] sys_access+0xf/0x13
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 30063, comm: prelink
EIP: 0061:[<c01e52f9>] CPU: 0
EIP is at _raw_spin_lock+0x4f/0xd9
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000001 EBX: c6c04000 ECX: cf21cb78 EDX: 00000004
ESI: cf21cba4 EDI: cf21cba4 EBP: cf42a000 DS: 007b ES: 007b
CR0: 8005003b CR2: 00786000 CR3: 02d5f000 CR4: 00000600
[<c0264163>] _spin_lock+0x6/0x8
[<c02619c0>] packet_rcv_spkt+0x156/0x226
[<c020e96a>] netif_receive_skb+0x200/0x279
[<d115bc4e>] netif_poll+0x554/0x741 [xennet]
[<c0210205>] net_rx_action+0xcd/0x201
[<c011e382>] __do_softirq+0x70/0xef
[<c011e441>] do_softirq+0x40/0x67
[<c010927b>] do_IRQ+0x22/0x28
[<c01f5c52>] evtchn_do_upcall+0x63/0x9d
[<c0107c4c>] hypervisor_callback+0x2c/0x34
BUG: soft lockup detected on CPU#0!
Pid: 30063, comm: prelink
EIP: 0061:[<c02135d0>] CPU: 0
EIP is at neigh_lookup+0x33/0xa3
EFLAGS: 00000286 Not tainted (2.6.15-1.1955_FC5guest)
EAX: c03df0e8 EBX: 00000001 ECX: df8982df EDX: 00000000
ESI: c02b50e0 EDI: cc74d030 EBP: 00000004 DS: 007b ES: 007b
CR0: 8005003b CR2: b7da4000 CR3: 02d5f000 CR4: 00000600
[<c02449fa>] arp_process+0x46b/0x536
[<c015327e>] cache_free_debugcheck+0x19a/0x1bc
[<c0153298>] cache_free_debugcheck+0x1b4/0x1bc
[<c020b182>] kfree_skbmem+0x65/0x69
[<c015364e>] cache_alloc_debugcheck_after+0xc1/0xfb
[<c0153298>] cache_free_debugcheck+0x1b4/0x1bc
[<c020b182>] kfree_skbmem+0x65/0x69
[<c0154dab>] kmem_cache_free+0x3f/0x9c
[<c0244e3c>] arp_rcv+0x10a/0x149
[<c020e995>] netif_receive_skb+0x22b/0x279
[<d115bc4e>] netif_poll+0x554/0x741 [xennet]
[<c0210205>] net_rx_action+0xcd/0x201
[<c011e382>] __do_softirq+0x70/0xef
[<c011e441>] do_softirq+0x40/0x67
[<c010927b>] do_IRQ+0x22/0x28
[<c01f5c52>] evtchn_do_upcall+0x63/0x9d
[<c0107c4c>] hypervisor_callback+0x2c/0x34
[<c026007b>] packet_ioctl+0xbb/0xd3
[<c01f577c>] force_evtchn_callback+0xa/0xc
[<c013f486>] free_hot_cold_page+0xae/0x181
[<c013f573>] __pagevec_free+0x1a/0x24
[<c0141b1f>] release_pages+0x11f/0x162
[<c014200b>] __pagevec_release+0x18/0x23
[<c014255a>] truncate_inode_pages_range+0xca/0x278
[<c014271d>] truncate_inode_pages+0x15/0x1c
[<d1058945>] reiserfs_delete_inode+0x40/0x10a [reiserfs]
[<c0153298>] cache_free_debugcheck+0x1b4/0x1bc
[<c01641b2>] vfs_rename+0x3bf/0x3cf
[<d1058905>] reiserfs_delete_inode+0x0/0x10a [reiserfs]
[<c016d72b>] generic_delete_inode+0xa7/0x119
[<c016b78a>] dentry_iput+0x75/0x90
[<c016c090>] dput+0x151/0x169
[<c0165b7a>] sys_renameat+0x167/0x1bf
[<c0156c5b>] sys_utime+0x109/0x125
[<c0153298>] cache_free_debugcheck+0x1b4/0x1bc
[<c013933a>] audit_syscall_exit+0x27a/0x33e
[<c013950c>] audit_syscall_entry+0x10e/0x133
[<c0109fb0>] do_syscall_trace+0x112/0x15e
[<c0165be3>] sys_rename+0x11/0x15
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 30997, comm: ld-linux.so.2
EIP: 0061:[<c0153469>] CPU: 0
EIP is at check_poison_obj+0x39/0x15d
EFLAGS: 00000293 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 000000a5 EBX: ced5cdac ECX: ced5cdb0 EDX: ced5cd6b
ESI: c05881c0 EDI: 0000002a EBP: 00000080 DS: 007b ES: 007b
CR0: 8005003b CR2: b7f52000 CR3: 0b20d000 CR4: 00000600
[<c016dfaa>] file_update_time+0x2d/0x86
[<c01535af>] cache_alloc_debugcheck_after+0x22/0xfb
[<c0153ed7>] __kmalloc_track_caller+0xfa/0x104
[<c0157d07>] sys_writev+0x3b/0x97
[<c015769f>] do_readv_writev+0x5b/0x242
[<c0157d07>] sys_writev+0x3b/0x97
[<c01629be>] pipe_write+0x0/0x2a
[<c013950c>] audit_syscall_entry+0x10e/0x133
[<c0157d07>] sys_writev+0x3b/0x97
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 31017, comm: ld-linux.so.2
EIP: 0073:[<002481d5>] CPU: 0
EIP is at 0x2481d5
ESP: 007b:bfef94fc EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 007a50dc EBX: 00258fd8 ECX: b7ff22d0 EDX: 000001ab
ESI: 00444ae8 EDI: 00000218 EBP: bfef9564 DS: 007b ES: 007b
CR0: 8005003b CR2: 0078d2ec CR3: 08272000 CR4: 00000600
BUG: soft lockup detected on CPU#0!
Pid: 0, comm: swapper
EIP: 0061:[<c01040c7>] CPU: 0
EIP is at 0xc01040c7
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: c02ea000 EBP: 00000020 DS: 007b ES: 007b
CR0: 8005003b CR2: 00388304 CR3: 02d5f000 CR4: 00000600
[<c0105bf6>] xen_idle+0x59/0x68
[<c0105d09>] cpu_idle+0x8b/0xa4
[<c02ec708>] start_kernel+0x2ed/0x2f3
BUG: soft lockup detected on CPU#0!
Pid: 31079, comm: ld-linux.so.2
EIP: 0073:[<002484c3>] CPU: 0
EIP is at 0x2484c3
ESP: 007b:bfe382cc EFLAGS: 00000202 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000000 EBX: 00258fd8 ECX: 00000061 EDX: 0026cbb3
ESI: 6cbb5430 EDI: f0000000 EBP: bfe383dc DS: 007b ES: 007b
CR0: 8005003b CR2: 00256798 CR3: 0767d000 CR4: 00000600
BUG: soft lockup detected on CPU#0!
Pid: 30063, comm: prelink
EIP: 0061:[<c01e4597>] CPU: 0
EIP is at __copy_to_user_ll+0x3e/0xe2
EFLAGS: 00000216 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 5362646d EBX: cb830000 ECX: 00000ec0 EDX: 70720065
ESI: cb830140 EDI: 09f4766c EBP: c6c05ebc DS: 007b ES: 007b
CR0: 8005003b CR2: 00ad1000 CR3: 02d5f000 CR4: 00000600
[<c013b397>] file_read_actor+0x60/0xd4
[<c013bc74>] do_generic_mapping_read+0x1b0/0x490
[<c013c67a>] __generic_file_aio_read+0x16b/0x1b3
[<c013b337>] file_read_actor+0x0/0xd4
[<c013d8c7>] generic_file_read+0xad/0xc3
[<c0156043>] do_filp_open+0x2d/0x35
[<c012b17f>] autoremove_wake_function+0x0/0x3a
[<c0264163>] _spin_lock+0x6/0x8
[<c013950c>] audit_syscall_entry+0x10e/0x133
[<c0157af7>] vfs_read+0xa2/0x136
[<c0158015>] sys_pread64+0x43/0x5b
[<c0107a75>] syscall_call+0x7/0xb
BUG: soft lockup detected on CPU#0!
Pid: 31583, comm: ld-linux.so.2
EIP: 0061:[<c016dfde>] CPU: 0
EIP is at file_update_time+0x61/0x86
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: cfd4ddbc EBX: cfd4dd68 ECX: 38e79400 EDX: 43fa1ca6
ESI: 00000000 EDI: 0000000a EBP: c02a3900 DS: 007b ES: 007b
CR0: 8005003b CR2: 00256798 CR3: 0969e000 CR4: 00000600
[<c016299a>] pipe_writev+0x365/0x389
[<c0157794>] do_readv_writev+0x150/0x242
[<c01629be>] pipe_write+0x0/0x2a
[<c0157d07>] sys_writev+0x3b/0x97
[<c0107a75>] syscall_call+0x7/0xb
INIT: version 2.86 reloading
BUG: soft lockup detected on CPU#0!
Pid: 0, comm: swapper
EIP: 0061:[<c01040c7>] CPU: 0
EIP is at 0xc01040c7
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: c02ea000 EBP: 00000020 DS: 007b ES: 007b
CR0: 8005003b CR2: 08652a74 CR3: 0fdde000 CR4: 00000600
[<c0105bf6>] xen_idle+0x59/0x68
[<c0105d09>] cpu_idle+0x8b/0xa4
[<c02ec708>] start_kernel+0x2ed/0x2f3
BUG: soft lockup detected on CPU#0!
Pid: 0, comm: swapper
EIP: 0061:[<c01040c7>] CPU: 0
EIP is at 0xc01040c7
EFLAGS: 00200246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: c02ea000 EBP: 00000020 DS: 007b ES: 007b
CR0: 8005003b CR2: b7da4000 CR3: 0de0c000 CR4: 00000600
[<c0105bf6>] xen_idle+0x59/0x68
[<c0105d09>] cpu_idle+0x8b/0xa4
[<c02ec708>] start_kernel+0x2ed/0x2f3
BUG: soft lockup detected on CPU#0!
Pid: 0, comm: swapper
EIP: 0061:[<c01040c7>] CPU: 0
EIP is at 0xc01040c7
EFLAGS: 00200246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: c02ea000 EBP: 00000020 DS: 007b ES: 007b
CR0: 8005003b CR2: 0016c9f8 CR3: 0de0c000 CR4: 00000600
[<c0105bf6>] xen_idle+0x59/0x68
[<c0105d09>] cpu_idle+0x8b/0xa4
[<c02ec708>] start_kernel+0x2ed/0x2f3
BUG: soft lockup detected on CPU#0!
Pid: 1507, comm: sh
EIP: 0061:[<c015346c>] CPU: 0
EIP is at check_poison_obj+0x3c/0x15d
EFLAGS: 00000246 Not tainted (2.6.15-1.1955_FC5guest)
EAX: 00000fa5 EBX: c9a6e000 ECX: c9a6e000 EDX: c9a6e06b
ESI: c0599ac0 EDI: 000000a3 EBP: 00001000 DS: 007b ES: 007b
CR0: 8005003b CR2: 080976c0 CR3: 05223000 CR4: 00000600
[<c01535af>] cache_alloc_debugcheck_after+0x22/0xfb
[<c0153752>] kmem_cache_alloc+0xca/0xd5
[<c0164761>] getname+0x18/0x9f
[<c0164761>] getname+0x18/0x9f
[<c0164761>] getname+0x18/0x9f
[<c01061fd>] sys_execve+0xc/0x6d
[<c0107a75>] syscall_call+0x7/0xb
More information about the fedora-devel-list
mailing list