<div dir="ltr"><div dir="ltr">On Wed, Jun 29, 2022 at 2:40 PM qianli zhao <<a href="mailto:zhaoqianligood@gmail.com">zhaoqianligood@gmail.com</a>> wrote: </div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,Lianbo Thank you for your feedback. My kernel version is 5.10.59, and this issue is happened when the</blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> ramdumps without vmcoreinfo. </blockquote><div> </div><div>Thank you for the explanation.</div><div> </div><div>It is important to point this(*without vmcoreinfo*) out in the patch log. That can help understand this issue. Can you add it to the patch log?</div><div> </div><div>In addition, I just got a dump file generated by the "virsh dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors </div><div>with or without this patch, can you also double check? Maybe it is a similar issue.</div><div> </div><div>[root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep vabits_actual=48 crash 8.0.1++ Copyright (C) 2002-2022 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2022 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. NOTE: setting vabits_actual to: 48 WARNING: kimage_voffset cannot be determined from the dumpfile. Try using the command line option: --machdep kimage_voffset=<addr> GNU gdb (GDB) 10.2 Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <<a href="http://gnu.org/licenses/gpl.html">http://gnu.org/licenses/gpl.html</a>> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "aarch64-unknown-linux-gnu". Type "show configuration" for configuration details. Find the GDB manual and other documentation resources online at: <<a href="http://www.gnu.org/software/gdb/documentation/">http://www.gnu.org/software/gdb/documentation/</a>>. For help, type "help". Type "apropos word" to search for commands related to "word"... crash: read error: kernel virtual address: ffff8000119cee00 type: "possible" WARNING: cannot read cpu_possible_map crash: read error: kernel virtual address: ffff8000119cf208 type: "present" WARNING: cannot read cpu_present_map crash: read error: kernel virtual address: ffff8000119cec00 type: "online" WARNING: cannot read cpu_online_map crash: read error: kernel virtual address: ffff8000119cf408 type: "active" WARNING: cannot read cpu_active_map crash: read error: kernel virtual address: ffff8000120c97a8 type: "shadow_timekeeper xtime_sec" crash: read error: kernel virtual address: ffff8000119e1668 type: "init_uts_ns" crash: vmlinux and dump.core do not match! Usage: crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form) crash [OPTION]... [NAMELIST] (live system form) Enter "crash -h" for details. </div><div> </div><div>Thanks.</div><div>Lianbo</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> I guess your scene may have vmcore,the following logic helps to correctly locate MODULES/VMALLOC ranges with right kernel version. 279 ms->kimage_end = (sp ? sp->value : 0); 280 281 if (ms->struct_page_size && (r = arm64_get_va_range(ms))) {------------->//If there is a vmcore, it will go to the following codes 282 /* We can get all the MODULES/VMALLOC/VMEMMAP ranges now.*/ 283 ms->modules_vaddr = r->modules_vaddr; 284 ms->modules_end = r->modules_end - 1; 285 ms->vmalloc_start_addr = r->vmalloc_start_addr; 286 ms->vmalloc_end = r->vmalloc_end - 1; 287 ms->vmemmap_vaddr = r->vmemmap_vaddr; 288 ms->vmemmap_end = r->vmemmap_end - 1; 289 } else if (ms->VA_BITS_ACTUAL) {------------>//unable to locate the version and other information through vmcore } lijiang <<a href="mailto:lijiang@redhat.com" target="_blank">lijiang@redhat.com</a>> 于2022年6月29日周三 14:21写道： > > Hi, Qianli > On Tue, Jun 28, 2022 at 9:55 AM lijiang <<a href="mailto:lijiang@redhat.com" target="_blank">lijiang@redhat.com</a>> wrote: >> >> Hi, Kazu and Qianli >> >> On Tue, Jun 28, 2022 at 9:17 AM HAGIO KAZUHITO(萩尾　一仁) <<a href="mailto:k-hagio-ab@nec.com" target="_blank">k-hagio-ab@nec.com</a>> wrote: >>> >>> Hi Qianli, >>> >>> thanks for the patch and explanation. I was off. >>> >>> On 2022/06/27 11:24, qianli zhao wrote: >>> > Hi,Kazu >>> > >>> > Would you like to help review this patch? >>> >>> Sure, I think I can review it this week. >>> >>> Lianbo, can you possibly reproduce and test this? >> >> >> OK, I will have a look and give feedback later. > > > Could you please point out the kernel version? I tried it with the latest kernel and did not reproduce this issue when disabling the kaslr feature(# CONFIG_RANDOMIZE_BASE is not set or nokaslr) > > crash > help -m > .. > vmalloc_start_addr: ffff800008000000 > vmalloc_end: fffffbffefffffff > modules_vaddr: ffff800000000000 > modules_end: ffff800007ffffff > vmemmap_vaddr: fffffc0000000000 > vmemmap_end: fffffdffffffffff > ... > > And the following information is dumped from the kernel(commit: 941e3e791269) > ... > [ 0.000000] Virtual kernel memory layout: > [ 0.000000] modules : 0xffff800000000000 - 0xffff800008000000 ( 128 MB) > [ 0.000000] vmalloc : 0xffff800008000000 - 0xfffffbfff0000000 (126975 GB) > ... > [ 0.000000] vmemmap : 0xfffffc0000000000 - 0xfffffe0000000000 ( 2048 GB maximum) > [ 0.000000] 0xfffffc000000bc00 - 0xfffffc023df40000 ( 9183 MB actual) > > I'm wondering if that can be only reproduced on the old kernel, right? Or did I miss anything else? > > Thanks. > Lianbo > >> >>> >>> Kazu >>> >>> > >>> > qianli zhao <<a href="mailto:zhaoqianligood@gmail.com" target="_blank">zhaoqianligood@gmail.com</a>> 于2022年6月24日周五 10:56写道： >>> > >>> >> >>> >> Hi,all >>> >> >>> >> Here's some explanation for this patch >>> >> >>> >> Without patch: >>> >> Consider the following scenario >>> >> ->arm64_init(PRE_GDB) >>> >> case PRE_GDB: >>> >> ... >>> >> 292 } else if (ms->VA_BITS_ACTUAL) { >>> >> 293 ms->modules_vaddr = >>> >> (st->_stext_vmlinux & TEXT_OFFSET_MASK) - >>> >> ARM64_MODULES_VSIZE;-->//ms->modules_vaddr=0xfffffffff8000000 >>> >> 294 ms->modules_end = >>> >> ms->modules_vaddr + ARM64_MODULES_VSIZE >>> >> -1;--->//ms->modules_end=0xffffffffffffffff >>> >> 295 ms->vmalloc_start_addr = >>> >> ms->modules_end + 1;--->//ms->vmalloc_start_addr=0 >>> >> 296 } else { >>> >> .... >>> >> } >>> >> arm64_calc_kimage_voffset(); >>> >> ..... >>> >> >>> >> Since arm64_calc_kimage_voffset() depends on vmalloc_start_addr, >>> >> kimage_voffset cannot be calculated correctly. >>> >> >>> >> st->_stext_vmlinux can be initialized in numeric_forward(),just set >>> >> st->_stext_vmlinux to UNINITIALIZED. >>> >> >>> >> ============ >>> >> log as below: >>> >> >>> >> $ ~/crash/crash/crash vmlinux DDRCS0.bin@0x80000000 --machdep vabits_actual=48 >>> >> >>> >> crash 8.0.1++ >>> >> Copyright (C) 2002-2022 Red Hat, Inc. >>> >> Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation >>> >> Copyright (C) 1999-2006 Hewlett-Packard Co >>> >> Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited >>> >> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. >>> >> Copyright (C) 2005, 2011, 2020-2022 NEC Corporation >>> >> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. >>> >> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. >>> >> Copyright (C) 2015, 2021 VMware, Inc. >>> >> This program is free software, covered by the GNU General Public License, >>> >> and you are welcome to change it and/or distribute copies of it under >>> >> certain conditions. Enter "help copying" to see the conditions. >>> >> This program has absolutely no warranty. Enter "help warranty" for details. >>> >> >>> >> NOTE: setting vabits_actual to: 48 >>> >> >>> >> WARNING: kimage_voffset cannot be determined from the dumpfile. >>> >> Try using the command line option: --machdep kimage_voffset=<addr> >>> >> GNU gdb (GDB) 10.2 >>> >> Copyright (C) 2021 Free Software Foundation, Inc. >>> >> License GPLv3+: GNU GPL version 3 or later <<a href="http://gnu.org/licenses/gpl.html" rel="noreferrer" target="_blank">http://gnu.org/licenses/gpl.html</a>> >>> >> This is free software: you are free to change and redistribute it. >>> >> There is NO WARRANTY, to the extent permitted by law. >>> >> Type "show copying" and "show warranty" for details. >>> >> This GDB was configured as "--host=x86_64-pc-linux-gnu >>> >> --target=aarch64-elf-linux". >>> >> Type "show configuration" for configuration details. >>> >> Find the GDB manual and other documentation resources online at: >>> >> <<a href="http://www.gnu.org/software/gdb/documentation/" rel="noreferrer" target="_blank">http://www.gnu.org/software/gdb/documentation/</a>>. >>> >> >>> >> For help, type "help". >>> >> Type "apropos word" to search for commands related to "word"... >>> >> >>> >> crash: read error: kernel virtual address: ffff80001083d4a0 type: >>> >> "kernel_config_data" >>> >> WARNING: cannot read kernel_config_data >>> >> crash: read error: kernel virtual address: ffff80001170e798 type: "possible" >>> >> WARNING: cannot read cpu_possible_map >>> >> crash: read error: kernel virtual address: ffff80001170e7a8 type: "present" >>> >> WARNING: cannot read cpu_present_map >>> >> crash: read error: kernel virtual address: ffff80001170e788 type: "online" >>> >> WARNING: cannot read cpu_online_map >>> >> crash: read error: kernel virtual address: ffff80001170e7c0 type: "active" >>> >> WARNING: cannot read cpu_active_map >>> >> crash: read error: kernel virtual address: ffff8000122e00f0 type: >>> >> "shadow_timekeeper xtime_sec" >>> >> crash: read error: kernel virtual address: ffff80001171dc04 type: "init_uts_ns" >>> >> crash: vmlinux and /var/tmp/ramdump_elf_m2ivkg do not match! >>> >> >>> >> Usage: >>> >> >>> >> crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form) >>> >> crash [OPTION]... [NAMELIST] (live system form) >>> >> >>> >> Enter "crash -h" for details. >>> >> >>> >> Qianli Zhao <<a href="mailto:zhaoqianligood@gmail.com" target="_blank">zhaoqianligood@gmail.com</a>> 于2022年6月24日周五 00:14写道： >>> >>> >>> >>> From: Qianli Zhao <<a href="mailto:qianli.zhao@horizon.ai" target="_blank">qianli.zhao@horizon.ai</a>> >>> >>> >>> >>> Setting st->_stext_vmlinux to UNINITIALIZED to search for "_stext" from the vmlinux >>> >>> Without the patch, if we do not enable kaslr, will get the wrong >>> >>> MODULES/VMALLOC ranges, cause parsing dump failure >>> >>> >>> >>> Signed-off-by: Qianli Zhao <<a href="mailto:qianli.zhao@horizon.ai" target="_blank">qianli.zhao@horizon.ai</a>> >>> >>> --- >>> >>> arm64.c | 3 +++ >>> >>> 1 file changed, 3 insertions(+) >>> >>> >>> >>> diff --git a/arm64.c b/arm64.c >>> >>> index 0f615cf..4458a66 100644 >>> >>> --- a/arm64.c >>> >>> +++ b/arm64.c >>> >>> @@ -149,6 +149,9 @@ arm64_init(int when) >>> >>> >>> >>> ms = machdep->machspec; >>> >>> >>> >>> + if (ms->VA_BITS_ACTUAL) >>> >>> + st->_stext_vmlinux = UNINITIALIZED; >>> >>> + >>> >>> if (!ms->kimage_voffset && STREQ(pc->live_memsrc, "/dev/crash")) >>> >>> ioctl(pc->mfd, DEV_CRASH_ARCH_DATA, &ms->kimage_voffset); >>> >>> >>> >>> -- >>> >>> 2.17.1 >>> >>> </blockquote></div></div>