[Crash-utility] crash-5.0: Segmentation fault with x86_64_get_active_set

ville.mattila at stonesoft.com ville.mattila at stonesoft.com
Fri Jan 15 08:50:01 UTC 2010


crash-utility-bounces at redhat.com wrote on 14.01.2010 16:08:41:

> From:
> 
> Dave Anderson <anderson at redhat.com>
> 
> To:
> 
> ----- "ville mattila" <ville.mattila at stonesoft.com> wrote:
> 
> > Hello,
> > 
> > I get segementation fault from our 64-bit kernel crash
> > This crash is caused by "echo c > /proc/sys-trigger".
> > The reason seems to be that the x86_64_cpu_pda_init is
> > not called at least gdb do not break there.
> > 
> > Here is a little patch that fixes it. Everyting seems to
> > work correctly. I'll provide more info if needed.
> > 
> > 
> > --- crash-5.0.0/x86_64.c 2010-01-06 21:38:27.000000000 +0200
> > +++ crash-5.0.0-64bit/x86_64.c 2010-01-14 08:24:13.679603706 +0200
> > @@ -6325,6 +6325,12 @@ x86_64_get_active_set(void)
> > 
> > ms = machdep->machspec;
> > 
> > + if (!ms->current) {
> > + error(INFO, "%s: Cannot get active set, ms->current is NULL\n",
> > + __func__);
> > + return;
> > + }
> > +
> 
> That patch just masks the real problem.
> 
> What kernel version is it?
> 
> If it's 2.6.30 or later, then x86_64_per_cpu_init() should
> be called, otherwise x86_64_cpu_pda_init() is called.  And
> whichever one that gets called should allocate the array.
> 
> 2.6.30 or later kernels should show:
> 
>    crash> struct x8664_pda
>    struct: invalid data structure reference: x8664_pda
>    crash>
> 
> and they will use x86_64_per_cpu_init().
> 
> Kernels prior to 2.6.30 should show:
> 
>   crash> struct x8664_pda
>   struct x8664_pda {
>       struct task_struct *pcurrent;
>       long unsigned int data_offset;
>       long unsigned int kernelstack;
>       long unsigned int oldrsp;
>       long unsigned int debugstack;
>       int irqcount;
>       int cpunumber;
>       char *irqstackptr;
>       int nodenumber;
>       unsigned int __softirq_pending;
>       unsigned int __nmi_count;
>       int mmu_state;
>       struct mm_struct *active_mm;
>       unsigned int apic_timer_irqs;
>   }
>   SIZE: 128
>   crash>
> 
> and they will use x86_64_cpu_pda_init().
> 
> If you're having trouble with gdb, can you put some fprintf(fp, ...)
> calls in the relevant function and find out why it isn't doing
> the calloc() call?

 
   Yes I thought so. This is a customized 2.6.31.7 kernel.org
   kernel. This is a UP configuration e.g. CONFIG_SMP is n.
   I think the problem is that the PER_CPU_OFF is not set.

   Breakpoint 1, x86_64_per_cpu_init () at x86_64.c:797
797             if (!(kt->flags & PER_CPU_OFF))
(gdb) bt
#0  x86_64_per_cpu_init () at x86_64.c:797
#1  0x00000000004cb1c9 in x86_64_init (when=3) at x86_64.c:325
#2  0x000000000045f0cc in main_loop () at main.c:546
#3  0x0000000000544869 in captured_command_loop (data=0xca29e0) at 
./main.c:226
#4  0x0000000000542aa4 in catch_errors (func=0x544860 
<captured_command_loop>, 
    func_args=0x0, errstring=0x7f949c "", mask=<value optimized out>)
    at exceptions.c:520
#5  0x00000000005442f6 in captured_main (data=<value optimized out>)
    at ./main.c:924
#6  0x0000000000542aa4 in catch_errors (func=0x543900 <captured_main>, 
    func_args=0x7fff2be0e3c0, errstring=0x7f949c "", 
    mask=<value optimized out>) at exceptions.c:520
#7  0x00000000005436ef in gdb_main_entry (argc=<value optimized out>, 
    argv=<value optimized out>) at ./main.c:939
#8  0x000000000045fe8e in main (argc=3, argv=0x7fff2be0e518) at main.c:517
(gdb) p/x kt->flags
$2 = 0x102000


Btw, the "struct" command caused another segementation fault.
Here is gdb bt:

(gdb) bt
#0  0x00007f74b3524a92 in strcmp () from /lib/libc.so.6
#1  0x0000000000534284 in lookup_partial_symtab (name=0x120e3c0 
"x8664_pda")
    at symtab.c:276
#2  0x00000000005344ed in lookup_symtab (name=0x120e3c0 "x8664_pda")
    at symtab.c:228
#3  0x000000000060019d in c_lex () at c-exp.y:2149
#4  0x00000000006008f5 in c_parse_internal () at c-exp.c.tmp:1468
#5  0x00000000006022dd in c_parse () at c-exp.y:2225
#6  0x000000000055f614 in parse_exp_in_context (stringptr=0x7fffbc2f2260, 
    block=<value optimized out>, comma=<value optimized out>, 
    void_context_p=0, out_subexp=0x0) at parse.c:1094
#7  0x000000000055f924 in parse_expression (string=0x7fffbc2f2950 
"x8664_pda")
    at parse.c:1144
#8  0x000000000053291b in gdb_command_funnel (req=0xca2c00) at 
symtab.c:4992
#9  0x00000000004c1740 in gdb_interface (req=0xca2c00) at 
gdb_interface.c:407
#10 0x00000000004e9dca in datatype_info (name=0xb618a7 "x8664_pda", 
    member=0x0, dm=0x7fffbc2f3620) at symbols.c:4146
#11 0x00000000004eb1ee in arg_to_datatype (s=0xb618a7 "x8664_pda", 
    dm=0x7fffbc2f3620, flags=524290) at symbols.c:4867
#12 0x00000000004efa1b in cmd_datatype_common (flags=2048) at 
symbols.c:4664
#13 0x000000000045efd9 in exec_command () at main.c:644
#14 0x000000000045f1fa in main_loop () at main.c:603
#15 0x00000000005452a9 in captured_command_loop (data=0x120e3c0)
    at ./main.c:226
#16 0x00000000005434e4 in catch_errors (func=0x5452a0 
<captured_command_loop>, 
    func_args=0x0, errstring=0x7f9d7c "", mask=<value optimized out>)
    at exceptions.c:520
#17 0x0000000000544d36 in captured_main (data=<value optimized out>)
    at ./main.c:924
#18 0x00000000005434e4 in catch_errors (func=0x544340 <captured_main>, 
    func_args=0x7fffbc2f38b0, errstring=0x7f9d7c "", 
    mask=<value optimized out>) at exceptions.c:520
#19 0x000000000054412f in gdb_main_entry (argc=<value optimized out>, 
    argv=<value optimized out>) at ./main.c:939
#20 0x000000000045fece in main (argc=3, argv=0x7fffbc2f3a08) at main.c:517
(gdb) frame 1
#1  0x0000000000534284 in lookup_partial_symtab (name=0x120e3c0 
"x8664_pda")
    at symtab.c:276
276         if (FILENAME_CMP (name, pst->filename) == 0)
(gdb) p name
$4 = 0x120e3c0 "x8664_pda"
(gdb) p pst
$5 = (struct partial_symtab *) 0x14d6040
(gdb) p pst->filename
$6 = 0x0
(gdb) p *pst
$7 = {next = 0x0, filename = 0x0, fullname = 0x0, dirname = 0x0, 
  objfile = 0x0, section_offsets = 0x0, textlow = 0, texthigh = 0, 
  dependencies = 0x0, number_of_dependencies = 0, globals_offset = 0, 
  n_global_syms = 0, statics_offset = 0, n_static_syms = 0, symtab = 0x0, 
  read_symtab = 0, read_symtab_private = 0x0, readin = 0 '\0'}
(gdb) 


I fixed it with the patch below:
-- crash-5.0.0/gdb-7.0/gdb/symtab.c     2010-01-15 10:41:00.919973440 
+0200
+++ crash-5.0.0-64bit/gdb-7.0/gdb/symtab.c      2010-01-15 
10:19:21.436128740 +0200
@@ -256,7 +256,7 @@ got_symtab:
 struct partial_symtab *
 lookup_partial_symtab (const char *name)
 {
-  struct partial_symtab *pst;
+  struct partial_symtab *pst = NULL;
   struct objfile *objfile;
   char *full_path = NULL;
   char *real_path = NULL;
@@ -273,7 +273,7 @@ lookup_partial_symtab (const char *name)
 
   ALL_PSYMTABS (objfile, pst)
   {
-    if (FILENAME_CMP (name, pst->filename) == 0)
+    if (pst->filename && FILENAME_CMP (name, pst->filename) == 0)
       {
        return (pst);
       }
@@ -311,7 +311,7 @@ lookup_partial_symtab (const char *name)
   if (lbasename (name) == name)
     ALL_PSYMTABS (objfile, pst)
     {
-      if (FILENAME_CMP (lbasename (pst->filename), name) == 0)
+      if (pst->filename && FILENAME_CMP (lbasename (pst->filename), name) 
== 0)
        return (pst);
     }
 
> 
> Either that, or if you can make the vmlinux/vmcore pair available
> for me to download, I can look at it.

  I'll arrange this if the above information is not enough.

 - Ville
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/crash-utility/attachments/20100115/039540f3/attachment.htm>


More information about the Crash-utility mailing list