[Crash-utility] linux_banner has garbage

[Eshak]

Thank you for the guidance Dave.

I have two questions regarding runq.

1. Could you please let me know how the active task has spent more time
than uptime on some CPUs ?

crash> runq -m

  CPU 0: [0 00:23:29.808]  PID: 529    TASK: ffff88079d0d1e40  COMMAND:
"kworker/u141:1"

  CPU 1: [1 12:10:42.840]  PID: 0      TASK: ffff88079df48000  COMMAND:
"swapper/1"

  CPU 2: [1 12:10:42.841]  PID: 0      TASK: ffff88079df4bc80  COMMAND:
"swapper/2"

  CPU 3: [1 12:10:42.841]  PID: 0      TASK: ffff88079df4dac0  COMMAND:
"swapper/3"

  CPU 4: [1 12:10:42.841]  PID: 0      TASK: ffff88079df49e40  COMMAND:
"swapper/4"

  CPU 5: [1 12:10:42.841]  PID: 0      TASK: ffff88079df58000  COMMAND:
"swapper/5"

crash> sys

      KERNEL: ./usr/lib/debug/usr/lib/modules/4.14.19-coreos/vmlinux

    DUMPFILE: gt-user2-gmt-612746ca.vmss

        CPUS: 70

        DATE: Wed Feb 21 14:53:20 2018

      UPTIME: 1 days, 11:52:25

LOAD AVERAGE: 70.70, 30.98, 12.88

       TASKS: 2312

    NODENAME: gt-user2-gmt.com

     RELEASE: 4.14.19-coreos

     VERSION: #1 SMP Wed Feb 14 03:18:05 UTC 2018

     MACHINE: x86_64  (2094 Mhz)

      MEMORY: 60 GB

       PANIC: ""

crash>

2. Is there a way to find out why some CPUs have time lag in run queue ?

  CPU 32: 0.00 secs

  CPU 65: 0.00 secs

  CPU 54: 0.00 secs

   CPU 0: 0.01 secs

  CPU 16: 84.22 secs

  CPU 66: 268.75 secs

  CPU 58: 268.75 secs

  CPU 57: 268.75 secs

  CPU 43: 268.75 secs

  CPU 20: 268.75 secs

   CPU 7: 268.75 secs

crash>

I'm struggling to find out why my VM hung(unresponsive to ping/ssh and
couple of CPUs at 100% utilization).

-Eshak

On Thu, Feb 22, 2018 at 6:27 AM, Dave Anderson <anderson at redhat.com> wrote:

> ----- Original Message -----
> > Hello Dave,
> >
> > I got a kernel freeze yesterday and am able to successfully open the
> memory
> > image using crash utility.
> >
> > crash> sys
> >       KERNEL: ./usr/lib/debug/usr/lib/modules/4.14.19-coreos/vmlinux
> >     DUMPFILE: gt-Server02-gmt-612746ca.vmss
> >         CPUS: 70
> >         DATE: Wed Feb 21 14:53:20 2018
> >       UPTIME: 1 days, 11:52:25
> > LOAD AVERAGE: 70.70, 30.98, 12.88
> >        TASKS: 2312
> >     NODENAME: gt-Server02-gmt.com
> >      RELEASE: 4.14.19-coreos
> >      VERSION: #1 SMP Wed Feb 14 03:18:05 UTC 2018
> >      MACHINE: x86_64  (2094 Mhz)
> >       MEMORY: 60 GB
> >        PANIC: ""
> > crash>
> >
> > Could you please guide me about couple of things I should check in case
> of
> > a kernel freeze before diving in deep to find the root cause ?
>
> I'm not sure what you mean by a "kernel freeze", but typically something
> would complain about a hard or soft lockup in the system log.  So I would
> first run "log" to see if there's anything of interest.  Run "bt -a" on
> the active tasks to see if the active tasks are contesting for something,
> or work your way through "foreach bt" to see what the tasks of interest are
> doing/waiting on.  It would seem that some task has taken control of
> something,
> a lock, or counter, or whatever, and many other tasks have blocked waiting
> for its release.  So there's probably a common theme among the blocked
> tasks
> that might give you a clue.
>
> Dave
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/crash-utility/attachments/20180222/ac77e4b7/attachment.htm>