[Crash-utility] Crash utility with RT patchset
HAGIO KAZUHITO(萩尾 一仁)
k-hagio-ab at nec.com
Mon Jul 19 06:04:10 UTC 2021
-----Original Message-----
> Hi Kazu,
>
> I worked on this for a couple days. I had asked below question in earlier mail:
>
> > "Can anyone suggest whether any change is required in either crash utility or
> RT Linux to make use of crash utility to analyze ramdump with RT Linux?"
>
> I was referring to the Linux 5.4 RT patchset (I had mentioned 5.4.61-rt37),
> I realized that crash does not (need to) support RT Linux. It should
> just support the mainline kernel(s).
It seems that actually there has been no patches for RT kernels for a
long time, but..
>
> > If it still fails, what do you see with -d8 option?
> I see kernel configuration(.config) being dumped, correct linux banner and
> lot of readmem/read_ramdump prints until failure is seen.
..also seems it does not break completely.
> Sharing the last few lines from the log, if it helps, can share full if needed:
Thanks. Not sure but the full is possibly helpful.
>
> [--Start of Log--]
> <readmem: ffffff8019e52100, KVADDR, "fill_task_struct", 4224, (ROE), 464bd90>
> <read_ramdump: addr: ffffff8019e52100 paddr: 99e52100 cnt: 3840>
> read_ramdump: addr: ffffff8019e52100 paddr: 99e52100 cnt: 3840 offset: 19e52100
> <read_ramdump: addr: ffffff8019e53000 paddr: 99e53000 cnt: 384>
> read_ramdump: addr: ffffff8019e53000 paddr: 99e53000 cnt: 384 offset: 19e53000
> <readmem: ffffff8019e52100, KVADDR, "fill_thread_info", 40, (ROE), 4c92290>
> <read_ramdump: addr: ffffff8019e52100 paddr: 99e52100 cnt: 40>
> read_ramdump: addr: ffffff8019e52100 paddr: 99e52100 cnt: 40 offset: 19e52100
> FREEBUF(1)
> GETBUF(8 -> 1)
> <readmem: fffffff00af339f8, KVADDR, "cpu_online_map", 8, (ROE), d21fc0>
> <read_ramdump: addr: fffffff00af339f8 paddr: 819339f8 cnt: 8>
> read_ramdump: addr: fffffff00af339f8 paddr: 819339f8 cnt: 8 offset: 19339f8
> crash: get_cpus_online: online: 8
> FREEBUF(1)
> please wait... (determining panic task)
> crash: get_netdump_panic_task: failed
>
> crash: invalid kernel virtual address: fffffff791ff5a5f type: "64-bit KVADDR"
> [--End of Log--]
hmm, I'm not sure what is being read and why it's invalid from the
last error message. Probably you need to debug it somehow.
Just out of curiosity, what happens with crash --no_panic option?
It skips some collection of data.
>
> I had an additional question, just out of curiosity:
> After the latest printk ring buffer implementation was added to crash 7.3.0;
> does crash still work with the old printk implementation?
> For ex., is the latest crash 7.3.0 expected to work for dmesg with
> mainline kernel 5.4?
Yes, it should work with the old implementations. Please see dump_log(),
which determines which implementation is used in the kernel.
Thanks,
Kazu
>
>
> Regards,
> Abhishek
>
> On Fri, Jul 16, 2021 at 12:04 PM HAGIO KAZUHITO(萩尾 一仁)
> <k-hagio-ab at nec.com> wrote:
> >
> > Hi Abhishek,
> >
> > -----Original Message-----
> > > I see the same issue with crash-7.3.0 as well.
> > >
> > > crash: invalid kernel virtual address: fffffff791ff5a5f type: "64-bit KVADDR"
> >
> > Do you see the same issue with the latest commit f53b73e8380bc?
> > If it still fails, what do you see with -d8 option?
> >
> > Thanks,
> > Kazu
> >
> > >
> > >
> > > Regards,
> > > Abhishek
> > >
> > > On Fri, Jul 9, 2021 at 2:20 PM Abhishek Shah <abrs612 at gmail.com> wrote:
> > > >
> > > > Hi Kazu,
> > > >
> > > > I am trying crash from master branch, top commit: d6b4f36d6b22
> > > > ("Handle task_struct state member changes for kernels >= 5.14-rc1")
> > > > I will give it a try with the released crash-7.3.0 and update.
> > > >
> > > >
> > > > Regards,
> > > > Abhishek
> > > >
> > > > On Fri, Jul 9, 2021 at 1:26 PM HAGIO KAZUHITO(萩尾 一仁) <k-hagio-ab at nec.com> wrote:
> > > > >
> > > > > Hi Abhishek,
> > > > >
> > > > > Does it happens with the released crash-7.3.0 ?
> > > > > Recently we had four patches for arm64 after the release,
> > > > > I'd like to know whether it's a regression or not to narrow down.
> > > > >
> > > > > Thanks,
> > > > > Kazu
> > > > >
> > > > > -----Original Message-----
> > > > > > Hi Ron,
> > > > > >
> > > > > > > I guess the address convention seems x8664 when looking at 0xffffffxxxxxxxx type sort of 64
> bit
> > > virtual
> > > > > > address.
> > > > > >
> > > > > > arm64 also has a similar range of virtual addresses (see
> > > > > > "Documentation/arm64/memory.rst" in Linux),
> > > > > > currently my target is using 39-bits (as specified in crash command
> > > > > > used below) and 4 kB page table.
> > > > > >
> > > > > > I see below address range being used, when giving "-d 1" to enable debug prints:
> > > > > >
> > > > > > crash: get_cpus_present: present: 8
> > > > > > hypervisor: (undetermined)
> > > > > > ikconfig: 1616 valid configs.
> > > > > > SECTION_SIZE_BITS: 30
> > > > > > CONFIG_ARM64_VA_BITS: 39
> > > > > > VA_BITS_ACTUAL: 39
> > > > > > (calculated) VA_BITS: 39
> > > > > > PAGE_OFFSET: ffffff8000000000
> > > > > > VA_START: ffffffc000000000
> > > > > > modules: ffffffc008000000 - ffffffc00fffffff
> > > > > > vmalloc: ffffffc010000000 - fffffffebffeffff
> > > > > > kernel image: fffffff009680000 - fffffff00b194000
> > > > > > vmemmap: fffffffeffe00000 - ffffffffffffffff
> > > > > >
> > > > > >
> > > > > > Also ,There is no issue with crash (compiled for arm64) or vmlinux (arm64) type.
> > > > > >
> > > > > > > Either that or possibly the offset at which you are trying to load.
> > > > > > Are you referring to kaslr offset? Without that in place, I get a
> > > > > > vmlinux/ramdump mismatch error.
> > > > > > So I am confident that it is correct.
> > > > > >
> > > > > > Can anyone suggest whether any change is required in either crash utility or
> > > > > > RT Linux to make use of crash utility to analyze ramdump with RT Linux?
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > Abhishek
> > > > > >
> > > > > > On Thu, Jul 8, 2021 at 7:49 AM Ron HM <gpl.positron at gmail.com> wrote:
> > > > > > >
> > > > > > > Either that or possibly the offset at which you are trying to load.
> > > > > > >
> > > > > > > On Thu, 8 Jul 2021, 9:28 am Ron HM, <gpl.positron at gmail.com> wrote:
> > > > > > >>
> > > > > > >> Hi Abhishek,
> > > > > > >> I guess the address convention seems x8664 when looking at 0xffffffxxxxxxxx type sort of 64
> bit
> > > virtual
> > > > > > address.
> > > > > > >>
> > > > > > >> ------
> > > > > > >> crash: invalid kernel virtual address: fffffff791ff5a5f type: "64-bit KVADDR"
> > > > > > >> -----
> > > > > > >>
> > > > > > >> For arm64 bit the addressing would more be sort of 0xCCAbxxxxxxx sorts. Due to the convention
> of
> > > memory
> > > > > > addressing start addresses.
> > > > > > >>
> > > > > > >> As a first check confirm if your crash and vmlinuz file is x8664 type
> > > > > > >> # file vmlinux
> > > > > > >> # file crash
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, 7 Jul 2021, 3:53 pm Abhishek Shah, <abrs612 at gmail.com> wrote:
> > > > > > >>>
> > > > > > >>> Hi All,
> > > > > > >>>
> > > > > > >>> I was wondering if there is any change required in either crash utility or
> > > > > > >>> RT Linux to make use of crash utility to analyze ramdump with RT Linux.
> > > > > > >>>
> > > > > > >>> I am using crash 7.3.0++ and Linux - 5.4.61-rt37 on arm64 target.
> > > > > > >>> I see the below error:
> > > > > > >>> ../crash DDR0.BIN at 0x80000000,DDR1.BIN at 0x100000000 vmlinux
> > > > > > >>> --machdep vabits_actual=39 --kaslr 0x2ff9600000
> > > > > > >>> ......
> > > > > > >>> crash: invalid kernel virtual address: fffffff791ff5a5f type: "64-bit KVADDR"
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Regards,
> > > > > > >>> Abhishek
> > > > > > >>>
> > > > > > >>> --
> > > > > > >>> Crash-utility mailing list
> > > > > > >>> Crash-utility at redhat.com
> > > > > > >>> https://listman.redhat.com/mailman/listinfo/crash-utility
> > > > > > >>>
> > > > > > > --
> > > > > > > Crash-utility mailing list
> > > > > > > Crash-utility at redhat.com
> > > > > > > https://listman.redhat.com/mailman/listinfo/crash-utility
> > > > > >
> > > > > > --
> > > > > > Crash-utility mailing list
> > > > > > Crash-utility at redhat.com
> > > > > > https://listman.redhat.com/mailman/listinfo/crash-utility
> > > > >
> > > > >
> > > > > --
> > > > > Crash-utility mailing list
> > > > > Crash-utility at redhat.com
> > > > > https://listman.redhat.com/mailman/listinfo/crash-utility
> > > > >
> > >
> > >
> > > --
> > > Crash-utility mailing list
> > > Crash-utility at redhat.com
> > > https://listman.redhat.com/mailman/listinfo/crash-utility
> >
> > --
> > Crash-utility mailing list
> > Crash-utility at redhat.com
> > https://listman.redhat.com/mailman/listinfo/crash-utility
>
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://listman.redhat.com/mailman/listinfo/crash-utility
More information about the Crash-utility
mailing list