[Crash-utility] [PATCH] runq: make tasks in throttled cfs_rqs/rt_rqs displayed

Dave Anderson anderson at redhat.com
Thu Nov 1 21:09:16 UTC 2012



----- Original Message -----

> > > My suggestion re: a new -g flag would have it applicable to both CFS and RT
> > > run queues (even though the RT groups are currently being shown).  I was
> > > thinking that it might be safest to have "runq" (with no argument) to just
> > > display a simple list of tasks, and only show group information with -g.
> > > Are you suggesting that -g would only apply to throttled cfs_rqs/rt_rqs?
> > > 
> > 
> > nop, '-g' is ok to show all rqs with group information.
> > 
> > I rewrote the patch by adding option '-g' to runq and not changing any behaviour of
> > the runq (with no argument).
> > 
> > TODO:
> > 1. The help info about the -g option.
> > 
> > I still have a suggestion: when we dump cfs_rq, we don't dump it hierarchically,
> > but for the current rt_rq, we dump it hierarchically with group info.
> > So could I make another patch to remove the rt_rq hierarchy, just make it
> > displayed identical with cfs_rq display.(For -g will display group info for both rt_rq
> > and cfs_rq)
> 
> Yes, that's exactly what I was suggesting.
> 
> I worry about the large number of kernel structure.member declarations that the
> command depends upon, because if just one of them changes, it breaks the command.
> So it would be preferable if the "runq" command (with no arguments) would only use
> a minimal set of structure.member offsets.
> 
> Thanks,
>   Dave

And to follow up, I'm still running tests (and will do so overnight) on your latest
patch, but I immediately seen this on any 2.6.30, 2.6.31 or 2.6.32 kernel, and on
some 2.6.36 and 2.6.38 kernels, where "runq -g" fails like this:

  crash> runq -g
  runq: invalid kernel virtual address: 0  type: "dentry"
  crash>
  
And we certainly want to keep the group information separate from
the normal "runq" command.  Here on a "live" 4 cpu Fedora 3.6.3 kernel,
the command output exceeds 1000 lines!  I'm pretty sure that most
people will *not* want to see all of this:

  crash> runq -g
  CPU 0
    CURRENT: PID: 0      TASK: ffffffff81c13420  COMMAND: "swapper/0"
    RT PRIO_ARRAY: ffff88021e213e28
       [100] GROUP RT PRIO_ARRAY: ffff88020db22000 <system>
             [100] GROUP RT PRIO_ARRAY: ffff8801f8180000 <udisks2.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e11800 <colord-sane.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802116f0000 <colord.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f1fd000 <cups.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f51800 <upower.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f994c800 <accounts-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212ae1000 <prefdm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8180800 <atd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9abe000 <crond.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9aba000 <autofs.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f54000 <sm-client.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7ee6000 <sendmail.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7e21800 <nscd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7d00800 <wpa_supplicant.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f6a000 <ypbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f6d800 <nfs-lock.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e12800 <rpcbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8d7a800 <libvirtd.service>
                   [100] GROUP RT PRIO_ARRAY: ffff8801f9944000 <libvirt>
                         [100] GROUP RT PRIO_ARRAY: ffff8801f7e22800 <lxc>
                               [no tasks queued]
                         [100] GROUP RT PRIO_ARRAY: ffff88021091b000 <qemu>
                               [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f4e000 <sshd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f4a800 <ksmtuned.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a7c000 <dbus.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a78800 <rsyslog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99cd800 <avahi-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99c9800 <system-setup-keyboard.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9975800 <systemd-logind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2cb800 <irqbalance.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f98c6800 <abrt-xorg.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9885800 <NetworkManager.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee39800 <acpid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee3d800 <gpm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f40d800 <mcelog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211ed9800 <abrt-oops.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f403000 <abrtd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f400800 <smartd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020dc47800 <auditd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211a55000 <iscsid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802108ab800 <bluetooth.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f4a000 <fsck at .service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211edc000 <udev.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212adc000 <systemd-journald.service>
                   [no tasks queued]
    CFS RB_ROOT: ffff880215806068
       GROUP CFS RB_ROOT: ffff8801ed860728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ed861a28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dab6228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dd98e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef1ec828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8800a07d6d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2ad928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f11fe228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f180b628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ee24528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e110828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5fe28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e161c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f503b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5b128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ff7c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d972828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5f328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e115c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5d128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8802129c2b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d6a028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f7d3e328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822ed28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbcd28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbde28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822ec28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d77d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dd59328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7d528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbdb28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f59428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7e128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ee78b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d983f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ed4028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b92428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b03128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd4928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a71628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a70028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dcfe128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ec9b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd5328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d924028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e69228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dd59428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211f26f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e68528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f75e828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f75e528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef0e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d972f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e2aac28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e110428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0cc28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ee1428 <system>
          GROUP CFS RB_ROOT: ffff88020ed3f928 <udisks2.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f814c028 <colord-sane.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd0eb28 <colord.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88021041b328 <cups.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd98228 <upower.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210aab628 <accounts-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6d728 <prefdm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6c028 <atd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d01e28 <crond.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d01728 <autofs.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d980228 <sm-client.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8f5dd28 <sendmail.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d77c28 <nscd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e152f28 <wpa_supplicant.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d6bb28 <ypbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d943b28 <nfs-lock.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211ec8428 <rpcbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d981728 <libvirtd.service>
             GROUP CFS RB_ROOT: ffff8801f82b6828 <libvirt>
                GROUP CFS RB_ROOT: ffff8801f828d828 <lxc>
                   [no tasks queued]
                GROUP CFS RB_ROOT: ffff8801f82b7628 <qemu>
                   [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc232d28 <sshd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a70428 <ksmtuned.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ddf7128 <dbus.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a77728 <rsyslog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd53e28 <avahi-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcf6728 <system-setup-keyboard.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e68c28 <systemd-logind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e3f928 <irqbalance.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f638828 <abrt-xorg.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210800c28 <NetworkManager.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcfe928 <acpid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982628 <gpm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ff68d28 <mcelog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982428 <abrt-oops.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d983928 <abrtd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211a5b728 <smartd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210af7528 <auditd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f9c28 <iscsid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e2aa328 <bluetooth.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d942828 <fsck at .service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211a5a128 <udev.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8328 <systemd-journald.service>
             [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020a844628
          [no tasks queued]
  
  CPU 1
    CURRENT: PID: 1122   TASK: ffff8801f8e18000  COMMAND: "Xorg"
    RT PRIO_ARRAY: ffff88021e253e28
       [100] GROUP RT PRIO_ARRAY: ffff88020db21800 <system>
             [100] GROUP RT PRIO_ARRAY: ffff8801f8187800 <udisks2.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e10800 <colord-sane.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802116f5800 <colord.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f1fc800 <cups.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f50000 <upower.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f994c000 <accounts-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212ae3000 <prefdm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8181000 <atd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9abe800 <crond.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9aba800 <autofs.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f52000 <sm-client.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7ee5800 <sendmail.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7e20800 <nscd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7d01000 <wpa_supplicant.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f69800 <ypbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f6e000 <nfs-lock.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e13000 <rpcbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8d7b000 <libvirtd.service>
                   [100] GROUP RT PRIO_ARRAY: ffff8801f9943000 <libvirt>
                         [100] GROUP RT PRIO_ARRAY: ffff8801f7e23000 <lxc>
                               [no tasks queued]
                         [100] GROUP RT PRIO_ARRAY: ffff88021091a800 <qemu>
                               [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212bdf800 <sshd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f4b000 <ksmtuned.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a7c800 <dbus.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a78000 <rsyslog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99ce000 <avahi-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99ca000 <system-setup-keyboard.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9976000 <systemd-logind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2cd000 <irqbalance.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f98c7000 <abrt-xorg.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9886000 <NetworkManager.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee39000 <acpid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee3d000 <gpm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f40d000 <mcelog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211ed9000 <abrt-oops.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f401000 <abrtd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f404800 <smartd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2c8000 <auditd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211a53000 <iscsid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802108ac800 <bluetooth.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f4d800 <fsck at .service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211edc800 <udev.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212adc800 <systemd-journald.service>
                   [no tasks queued]
    CFS RB_ROOT: ffff880215806068
       GROUP CFS RB_ROOT: ffff880211ee1b28 <system>
          GROUP CFS RB_ROOT: ffff8801f7f6d528 <prefdm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ed3fa28 <udisks2.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f814dc28 <colord-sane.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd0ff28 <colord.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88021041a928 <cups.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd98128 <upower.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210aab928 <accounts-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6d228 <atd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d01228 <crond.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d00c28 <autofs.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d980528 <sm-client.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8f5c228 <sendmail.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d76b28 <nscd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e153028 <wpa_supplicant.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d6bc28 <ypbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d942d28 <nfs-lock.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc177928 <rpcbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d981d28 <libvirtd.service>
             GROUP CFS RB_ROOT: ffff8801f82b6128 <libvirt>
                GROUP CFS RB_ROOT: ffff8801f828cb28 <lxc>
                   [no tasks queued]
                GROUP CFS RB_ROOT: ffff8801f82b7228 <qemu>
                   [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc232f28 <sshd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a70928 <ksmtuned.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ddf7828 <dbus.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a77828 <rsyslog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd52a28 <avahi-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcf7428 <system-setup-keyboard.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e68328 <systemd-logind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e3eb28 <irqbalance.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f639b28 <abrt-xorg.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210800b28 <NetworkManager.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcff828 <acpid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d983828 <gpm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ff69d28 <mcelog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d983728 <abrt-oops.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d983d28 <abrtd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211a5ab28 <smartd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210af7228 <auditd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8e28 <iscsid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e2aa928 <bluetooth.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d943828 <fsck at .service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e95ba28 <udev.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8228 <systemd-journald.service>
             [no tasks queued]
       [120] PID: 27130  TASK: ffff8801c2c3ae20  COMMAND: "kworker/1:2"
       GROUP CFS RB_ROOT: ffff8801ed860028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ed860828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dab6e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2d4e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef1ed928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8800a07d7f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2aca28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f11ff228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f180b828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ee24f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e111428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5fd28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e160028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f502528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5b528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ff7d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ddf7728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5e828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e114e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5c028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8802129c2c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d6a828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f7d3f028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822f028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbce28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbd028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822eb28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d76e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f58628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7c728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbd828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f59528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7e228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ee78928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d983028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ed5d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b93928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b03f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd5f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a71e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e2aad28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dcfe328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ec8728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd4628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d925328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e69628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dd58028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211f26a28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e68f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef0c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef1328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef1028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d972728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e2aaa28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0c228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0c728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020a844328
          [no tasks queued]
  
  CPU 2
    CURRENT: PID: 0      TASK: ffff880212958000  COMMAND: "swapper/2"
    RT PRIO_ARRAY: ffff88021e293e28
       [100] GROUP RT PRIO_ARRAY: ffff88020db22800 <system>
             [100] GROUP RT PRIO_ARRAY: ffff8801f7ee7000 <udisks2.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e10000 <colord-sane.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802116f7000 <colord.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f1fb800 <cups.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f53800 <upower.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f994b800 <accounts-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212ae4000 <prefdm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8181800 <atd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9abf000 <crond.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9abb000 <autofs.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f56000 <sm-client.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7ee5000 <sendmail.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7e21000 <nscd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7d01800 <wpa_supplicant.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f69000 <ypbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f6e800 <nfs-lock.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e13800 <rpcbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8d7b800 <libvirtd.service>
                   [100] GROUP RT PRIO_ARRAY: ffff8801f8079000 <libvirt>
                         [100] GROUP RT PRIO_ARRAY: ffff8801f7e23800 <lxc>
                               [no tasks queued]
                         [100] GROUP RT PRIO_ARRAY: ffff88021091a000 <qemu>
                               [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020dc47000 <sshd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f49800 <ksmtuned.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a7d000 <dbus.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a79000 <rsyslog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99ce800 <avahi-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99ca800 <system-setup-keyboard.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9976800 <systemd-logind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2cd800 <irqbalance.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f98c7800 <abrt-xorg.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9886800 <NetworkManager.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee38800 <acpid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee3c800 <gpm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f40c800 <mcelog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211ed8800 <abrt-oops.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f401800 <abrtd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f405000 <smartd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2c8800 <auditd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211a55800 <iscsid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020eee2000 <bluetooth.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020e634000 <fsck at .service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211edd000 <udev.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212add000 <systemd-journald.service>
                   [no tasks queued]
    CFS RB_ROOT: ffff880215806068
       GROUP CFS RB_ROOT: ffff8801ed861928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ed861c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dab6928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2d5028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef1edf28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8800a07d6428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2ada28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f11fe428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f180b128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ee25728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e110728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5f928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e160b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e114428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5b028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ff7e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822ff28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5f628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e115f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5c828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8802129c2028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d6a928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f7d3f928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822f428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbcf28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbd228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822e928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d76328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f58428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7ca28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbc128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f59628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7e328
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a73228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801fc3e8628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ed5c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b93128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b03b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211a5a528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a71f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f02e528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dcfeb28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ec9428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd5828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d924c28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e68228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d9f3d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211f27928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e68928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e111e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef0f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef1d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d973228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e2ab128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0d028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0d628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ee1628 <system>
          GROUP CFS RB_ROOT: ffff88020ed3fb28 <udisks2.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f814cb28 <colord-sane.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd0fc28 <colord.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88021041a228 <cups.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd99628 <upower.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210aaba28 <accounts-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6df28 <prefdm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6d428 <atd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d00628 <crond.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d00428 <autofs.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d981b28 <sm-client.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8f5c328 <sendmail.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d77a28 <nscd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8e99428 <wpa_supplicant.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d6bd28 <ypbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d943528 <nfs-lock.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc232328 <rpcbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d980b28 <libvirtd.service>
             GROUP CFS RB_ROOT: ffff8801f82b6328 <libvirt>
                GROUP CFS RB_ROOT: ffff8801f828de28 <lxc>
                   [no tasks queued]
                GROUP CFS RB_ROOT: ffff8801f82b6928 <qemu>
                   [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc233128 <sshd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a70728 <ksmtuned.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ddf6528 <dbus.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a77928 <rsyslog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd53028 <avahi-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcf6e28 <system-setup-keyboard.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e3f528 <systemd-logind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e3e428 <irqbalance.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f638128 <abrt-xorg.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210800028 <NetworkManager.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcfec28 <acpid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982828 <gpm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ff69028 <mcelog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982c28 <abrt-oops.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982728 <abrtd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211a5a228 <smartd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8f28 <auditd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f9728 <iscsid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e2ab328 <bluetooth.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d942528 <fsck at .service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8a28 <udev.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8528 <systemd-journald.service>
             [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020a844f28
          [no tasks queued]
  
  CPU 3
    CURRENT: PID: 4733   TASK: ffff8801eed74530  COMMAND: "crash"
    RT PRIO_ARRAY: ffff88021e2d3e28
       [100] GROUP RT PRIO_ARRAY: ffff88020db23000 <system>
             [100] GROUP RT PRIO_ARRAY: ffff8801f7ee0000 <udisks2.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e17000 <colord-sane.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802116f1000 <colord.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f1fa000 <cups.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f53000 <upower.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f994f800 <accounts-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212ae4800 <prefdm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8182000 <atd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9abf800 <crond.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9abb800 <autofs.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f56800 <sm-client.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7ee4800 <sendmail.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7e20000 <nscd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f7d02000 <wpa_supplicant.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f68800 <ypbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8f6f000 <nfs-lock.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8e14000 <rpcbind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f8d7c000 <libvirtd.service>
                   [100] GROUP RT PRIO_ARRAY: ffff880211ec1000 <libvirt>
                         [100] GROUP RT PRIO_ARRAY: ffff8801f7e24000 <lxc>
                               [no tasks queued]
                         [100] GROUP RT PRIO_ARRAY: ffff88021091c800 <qemu>
                               [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8802108ab000 <sshd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211f4e800 <ksmtuned.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a7d800 <dbus.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9a79800 <rsyslog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99cf000 <avahi-daemon.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f99cb000 <system-setup-keyboard.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9977000 <systemd-logind.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2ce000 <irqbalance.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9948000 <abrt-xorg.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801f9887000 <NetworkManager.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee38000 <acpid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020ee3c000 <gpm.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f408000 <mcelog.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211ed8000 <abrt-oops.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f403800 <abrtd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020f405800 <smartd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff8801fc2c9000 <auditd.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211a53800 <iscsid.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020eee1000 <bluetooth.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff88020e632800 <fsck at .service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880211edd800 <udev.service>
                   [no tasks queued]
             [100] GROUP RT PRIO_ARRAY: ffff880212add800 <systemd-journald.service>
                   [no tasks queued]
    CFS RB_ROOT: ffff880215806068
       GROUP CFS RB_ROOT: ffff8801ed860b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2b7b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dcff028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2d4528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef1ecf28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8800a07d7b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801ef2ad528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f11ff828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f180a128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020ee24928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e111028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5fa28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e161a28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a73928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5ae28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ff7f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822fe28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880210a5e428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e115b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f5d228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8802129c2228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d6b128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f7d3ed28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822f628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbc028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbd128
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f822e828
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8d76228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7d228
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7cb28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8fbc428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8f59728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f8e7e428
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd4a28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801fc3e8b28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ed5628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b92f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9b03e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211a5bb28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff8801f9a71528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f02ef28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020dcfea28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ec9728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020edd5e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d925e28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e69f28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ed5928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211f26528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211e69d28
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e111928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ef1628
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020f75e728
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020d972528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020e2aa028
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0d528
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020db0c928
          [no tasks queued]
       GROUP CFS RB_ROOT: ffff880211ee1f28 <system>
          GROUP CFS RB_ROOT: ffff88020ed3fc28 <udisks2.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f52de28 <colord-sane.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd0fe28 <colord.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f006128 <cups.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd99728 <upower.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210aab228 <accounts-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6c328 <prefdm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f7f6d028 <atd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d00228 <crond.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d01828 <autofs.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f638c28 <sm-client.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8f5cc28 <sendmail.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d76928 <nscd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8e99128 <wpa_supplicant.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f8d6be28 <ypbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e152b28 <nfs-lock.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc233b28 <rpcbind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d980428 <libvirtd.service>
             GROUP CFS RB_ROOT: ffff8801f82b6428 <libvirt>
                GROUP CFS RB_ROOT: ffff8801f828c928 <lxc>
                   [no tasks queued]
                GROUP CFS RB_ROOT: ffff8801f82b6b28 <qemu>
                   [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801fc233328 <sshd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a70c28 <ksmtuned.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ddf7a28 <dbus.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8801f9a77a28 <rsyslog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dd53b28 <avahi-daemon.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcf7628 <system-setup-keyboard.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e3fb28 <systemd-logind.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211e3f428 <irqbalance.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020f638528 <abrt-xorg.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210801428 <NetworkManager.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020dcff128 <acpid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d983e28 <gpm.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020ff68628 <mcelog.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982a28 <abrt-oops.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d982128 <abrtd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880211a5a028 <smartd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020de77828 <auditd.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff880210af7a28 <iscsid.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020e2ab828 <bluetooth.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff88020d943d28 <fsck at .service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f9128 <udev.service>
             [no tasks queued]
          GROUP CFS RB_ROOT: ffff8802116f8828 <systemd-journald.service>
             [no tasks queued]
       GROUP CFS RB_ROOT: ffff88020a844128
          [no tasks queued]
  crash> 

In fact, it's difficult to actually find a real *task* that is on
a run queue from among all of the empty "[no tasks queued]" groups!

Dave






















More information about the Crash-utility mailing list