[Crash-utility] ARM64 (odroid-c2) crash fails to read live kernel
Markham thomas
markham.thomas at gmail.com
Tue Mar 8 17:05:55 UTC 2016
I have the new Odroid-C2 arm64 cortex-a53 board and have been trying to get
crash to work against the live kernel.
I think the key error is this:
linux_banner:
crash: /lib/modules/3.14.29+/build/vmlinux and /dev/mem do not match!
They should match as I built the kernel myself and verified the vmlinux in
/lib/modules is the one
I'm booted on. What concerns me is that it does not appear to be able to
read anything
from the vmlinux file:
<read_dev_mem: addr: ffffffc001c0dbac paddr: 2c0dbac cnt: 390>
utsname:
sysname: (not printable)
nodename:
release: J
version: (not printable)
machine: r
domainname:
base kernel version: 0.1.4
If I elfdump or objdump the vmlinux and grep banner I can see the symbol:
root at odroid64-pre:~/linux# readelf --syms vmlinux | grep banner
74463: ffffffc00186a090 149 OBJECT GLOBAL DEFAULT 4 linux_banner
75496: ffffffc00186a028 100 OBJECT GLOBAL DEFAULT 4 linux_proc_banner
root at odroid64-pre:~/linux# eu-nm -a vmlinux | grep banner
linux_banner
|ffffffc00186a090|GLOBAL|OBJECT |0000000000000095|
version.c:43|.rodata
linux_proc_banner
|ffffffc00186a028|GLOBAL|OBJECT |0000000000000064|
version.c:47|.rodata
I pulled the crash source and built it native on the arm64 box.
If I could get a pointer on where to start with debugging this it would
help (i.e. which error to focus on first)
===
The full dump of crash startup is below:
root at odroid64-pre:~/linux# /root/crash-7.1.4/crash -d 64
crash 7.1.4
Copyright (C) 2002-2015 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
find_booted_kernel: search for [Linux version 3.14.29+ (root at odroid64-pre)
(gcc version 5.3.1 20160225 (Ubuntu/Linaro 5.3.1-10ubuntu2) ) #1 SMP
PREEMPT Tue Mar 8 01:06:35 CST 2016]
searchdirs[8]: /usr/lib/debug/lib/modules/3.14.29+/
searchdirs[0]: /usr/src/linux/
searchdirs[1]: /boot/
searchdirs[2]: /boot/efi/redhat
searchdirs[3]: /boot/efi/EFI/redhat
searchdirs[4]: /
searchdirs[5]: /lib/modules/3.14.29+/build/
searchdirs[6]: /usr/src/redhat/BUILD/kernel-3.14.29/linux/
searchdirs[7]: /usr/src/redhat/BUILD/kernel-3.14.29/linux-3.14.29/
mount_points[0]: / (c46630)
mount_points[1]: /sys (c46650)
mount_points[2]: /proc (c46670)
mount_points[3]: /dev (c46690)
mount_points[4]: /dev/pts (c466b0)
mount_points[5]: /run (c466d0)
mount_points[6]: / (c466f0)
mount_points[7]: /sys/kernel/security (c46710)
mount_points[8]: /dev/shm (c46740)
mount_points[9]: /run/lock (c46760)
mount_points[10]: /sys/fs/cgroup (c46780)
mount_points[11]: /sys/fs/cgroup/systemd (c467b0)
mount_points[12]: /sys/fs/cgroup/devices (c467f0)
mount_points[13]: /sys/fs/cgroup/cpuset (c46830)
mount_points[14]: /sys/fs/cgroup/cpu,cpuacct (c46870)
mount_points[15]: /sys/fs/cgroup/blkio (c468b0)
mount_points[16]: /sys/fs/cgroup/debug (c468e0)
mount_points[17]: /sys/fs/cgroup/perf_event (c46910)
mount_points[18]: /sys/fs/cgroup/freezer (c46950)
mount_points[19]: /sys/fs/cgroup/net_cls (c46990)
mount_points[20]: /proc/sys/fs/binfmt_misc (c469d0)
mount_points[21]: /dev/mqueue (c46a10)
mount_points[22]: /sys/kernel/debug (c46a30)
mount_points[23]: /dev/hugepages (c46a60)
mount_points[24]: /run/rpc_pipefs (c46a90)
mount_points[25]: /sys/kernel/config (c46ac0)
mount_points[26]: /media/boot (c46af0)
mount_points[27]: /run/cgmanager/fs (c46b10)
mount_points[28]: /run/user/118 (c46b40)
mount_points[29]: /run/user/118/gvfs (c46b70)
mount_points[30]: /sys/fs/fuse/connections (c46ba0)
mount_points[31]: /run/user/0 (c46be0)
find_booted_kernel: check: /lib/modules/3.14.29+/build/vmlinux
find_booted_kernel: found: /lib/modules/3.14.29+/build/vmlinux
get_live_memory_source: /dev/mem
/proc/version:
Linux version 3.14.29+ (root at odroid64-pre) (gcc version 5.3.1 20160225
(Ubuntu/Linaro 5.3.1-10ubuntu2) ) #1 SMP PREEMPT Tue Mar 8 01:06:35 CST 2016
/lib/modules/3.14.29+/build/vmlinux:
Linux version 3.14.29+ (root at odroid64-pre) (gcc version 5.3.1 20160225
(Ubuntu/Linaro 5.3.1-10ubuntu2) ) #1 SMP PREEMPT Tue Mar 8 01:06:35 CST 2016
readmem: read_dev_mem() -> /dev/mem
VA_BITS: 39
using 1000000 as phys_offset
gdb /lib/modules/3.14.29+/build/vmlinux
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu"...
GETBUF(248 -> 0)
GETBUF(1500 -> 1)
FREEBUF(1)
FREEBUF(0)
<readmem: ffffffc001874510, KVADDR, "kernel_config_data", 32768, (ROE),
17c47c0>
<read_dev_mem: addr: ffffffc001874510 paddr: 2874510 cnt: 2800>
<read_dev_mem: addr: ffffffc001875000 paddr: 2875000 cnt: 4096>
<read_dev_mem: addr: ffffffc001876000 paddr: 2876000 cnt: 4096>
<read_dev_mem: addr: ffffffc001877000 paddr: 2877000 cnt: 4096>
<read_dev_mem: addr: ffffffc001878000 paddr: 2878000 cnt: 4096>
<read_dev_mem: addr: ffffffc001879000 paddr: 2879000 cnt: 4096>
<read_dev_mem: addr: ffffffc00187a000 paddr: 287a000 cnt: 4096>
<read_dev_mem: addr: ffffffc00187b000 paddr: 287b000 cnt: 4096>
<read_dev_mem: addr: ffffffc00187c000 paddr: 287c000 cnt: 1296>
WARNING: could not find MAGIC_START!
GETBUF(248 -> 0)
FREEBUF(0)
GETBUF(8 -> 0)
<readmem: ffffffc00186fd80, KVADDR, "cpu_possible_mask", 8, (FOE),
7ffe1dfbd0>
<read_dev_mem: addr: ffffffc00186fd80 paddr: 286fd80 cnt: 8>
<readmem: 1600000102, KVADDR, "possible", 8, (ROE), bf8ae8>
crash: invalid kernel virtual address: 1600000102 type: "possible"
WARNING: cannot read cpu_possible_map
<readmem: ffffffc00186fd70, KVADDR, "cpu_present_mask", 8, (FOE),
7ffe1dfbd0>
<read_dev_mem: addr: ffffffc00186fd70 paddr: 286fd70 cnt: 8>
<readmem: 189a, KVADDR, "present", 8, (ROE), bf8ae8>
crash: invalid kernel virtual address: 189a type: "present"
WARNING: cannot read cpu_present_map
<readmem: ffffffc00186fd78, KVADDR, "cpu_online_mask", 8, (FOE), 7ffe1dfbd0>
<read_dev_mem: addr: ffffffc00186fd78 paddr: 286fd78 cnt: 8>
<readmem: 13a48, KVADDR, "online", 8, (ROE), bf8ae8>
crash: invalid kernel virtual address: 13a48 type: "online"
WARNING: cannot read cpu_online_map
<readmem: ffffffc00186fd68, KVADDR, "cpu_active_mask", 8, (FOE), 7ffe1dfbd0>
<read_dev_mem: addr: ffffffc00186fd68 paddr: 286fd68 cnt: 8>
<readmem: 1600000102, KVADDR, "active", 8, (ROE), bf8ae8>
crash: invalid kernel virtual address: 1600000102 type: "active"
WARNING: cannot read cpu_active_map
FREEBUF(0)
GETBUF(248 -> 0)
FREEBUF(0)
GETBUF(248 -> 0)
FREEBUF(0)
<readmem: ffffffc001d61238, KVADDR, "timekeeper xtime_sec", 8, (ROE),
7ffe1dfc98>
<read_dev_mem: addr: ffffffc001d61238 paddr: 2d61238 cnt: 8>
xtime timespec.tv_sec: 5f044158000e9068: (null)
<readmem: ffffffc001c0dbac, KVADDR, "init_uts_ns", 390, (ROE), b9606c>
<read_dev_mem: addr: ffffffc001c0dbac paddr: 2c0dbac cnt: 390>
utsname:
sysname: (not printable)
nodename:
release: J
version: (not printable)
machine: r
domainname:
base kernel version: 0.1.4
<readmem: ffffffc00186a090, KVADDR, "accessible check", 8, (ROE|Q),
7ffe1df350>
<read_dev_mem: addr: ffffffc00186a090 paddr: 286a090 cnt: 8>
<readmem: ffffffc00186a090, KVADDR, "read_string characters", 1499,
(ROE|Q), 7ffe1df6c8>
<read_dev_mem: addr: ffffffc00186a090 paddr: 286a090 cnt: 1499>
/proc/version:
Linux version 3.14.29+ (root at odroid64-pre) (gcc version 5.3.1 20160225
(Ubuntu/Linaro 5.3.1-10ubuntu2) ) #1 SMP PREEMPT Tue Mar 8 01:06:35 CST 2016
linux_banner:
crash: /lib/modules/3.14.29+/build/vmlinux and /dev/mem do not match!
Usage:
crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form)
crash [OPTION]... [NAMELIST] (live system form)
Enter "crash -h" for details.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/crash-utility/attachments/20160308/ce3a06c4/attachment.htm>
More information about the Crash-utility
mailing list