[Crash-utility] Why module's global symbol cannot be displayed in crash? [ARM]

[Dave Anderson]

----- Original Message -----
> On Thu, Mar 21, 2013 at 11:04:16AM -0400, Dave Anderson wrote:
> > 
> > 
> > ----- Original Message -----
> > 
> > > After your fix, the module could show module address now.
> > > However I don't know whether this showing is correct or not...
> > > For while I want to check the module's defined global variant like
> > > below, I just find it is not being mapped yet...
> > > But this variant definition is very straightforward, like:
> > > int cctdev_major =   0;
> > > 
> > > 
> > > crash> sym cctdev_major
> > > bf16564d (B) cctdev_major
> > > crash> vtop -k bf16564d
> > > VIRTUAL   PHYSICAL
> > > bf16564d  (not mapped)
> > > 
> > > PAGE DIRECTORY: c0004000
> > >   PGD: c0006fc4 => 1f3cdc11
> > >   PMD: c0006fc4 => 1f3cdc11
> > >   PTE: 1f3cd594 => 0
> > > 
> > > I don't know what is going wrong there, and I am planning to manually
> > > print out symbols' address before trigger the dump, and to see
> > > whether they could be aligned.
> > > 
> > > Do you have some better idea how to fix it?...
> > 
> > No, not really, I'm not an ARM guy...
> > 
> > But it's possible/probable that the "vtop" translation on kernel module
> > virtual (vmalloc) addresses may not be working correctly.  I also noted
> > yesterday that "vtop" on user-space virtual addresses fails pretty miserably
> > most of the time.  Both arm_kvtop() and arm_uvtop() both end up calling the
> > common arm_vtop() function, so I'm guessing that it's the culprit.
> 
> AFAICT, arm_uvtop() should do the translation depending whether the address
> falls to the kernel virtual address or userspace. There's a check:
> 
> 	if (is_kernel_thread(tc->task) && IS_KVADDR(uvaddr))
> 
> and I believe that IS_KVADDR() is enough here as it checks both the module and
> kernel virtual memory limits.
> 
> I asked Luc whether he could provide his last file set to me as well.
> Hope they will shed some light into this.
> 
> Dave,

If for some reason you can't get them, I can make them available to you.  
And Lei Wen can also give you a sample dumpfile from his environment.
 
> Are you able to access module symbols on ARM dump (the one that Luc provided)?
> Or is it failing completely?

I *think* so...

This module text disassembly looks right:

crash> dis usbnet_suspend
0xbf000ae8 <usbnet_suspend>:    push    {r3, r4, r5, lr}
0xbf000aec <usbnet_suspend+4>:  add     r0, r0, #32
0xbf000af0 <usbnet_suspend+8>:  mov     r5, r1
0xbf000af4 <usbnet_suspend+12>: bl      0xc01b8264 <dev_get_drvdata>
0xbf000af8 <usbnet_suspend+16>: ldrb    r3, [r0, #36]   ; 0x24
0xbf000afc <usbnet_suspend+20>: mov     r4, r0
0xbf000b00 <usbnet_suspend+24>: add     r2, r3, #1
0xbf000b04 <usbnet_suspend+28>: cmp     r3, #0
0xbf000b08 <usbnet_suspend+32>: strb    r2, [r0, #36]   ; 0x24
0xbf000b0c <usbnet_suspend+36>: bne     0xbf000bdc <usbnet_suspend+244>
0xbf000b10 <usbnet_suspend+40>: mrs     r3, CPSR
0xbf000b14 <usbnet_suspend+44>: orr     r3, r3, #128    ; 0x80
0xbf000b18 <usbnet_suspend+48>: msr     CPSR_c, r3
0xbf000b1c <usbnet_suspend+52>: mov     r0, #1
0xbf000b20 <usbnet_suspend+56>: bl      0xc0015f40 <add_preempt_count>
0xbf000b24 <usbnet_suspend+60>: ldr     r3, [r4, #200]  ; 0xc8
0xbf000b28 <usbnet_suspend+64>: cmp     r3, #0
0xbf000b2c <usbnet_suspend+68>: beq     0xbf000b70 <usbnet_suspend+136>
0xbf000b30 <usbnet_suspend+72>: tst     r5, #1024       ; 0x400
0xbf000b34 <usbnet_suspend+76>: beq     0xbf000b70 <usbnet_suspend+136>
0xbf000b38 <usbnet_suspend+80>: mrs     r3, CPSR
...

This (r) data looks OK:

crash> p smsc95xx_netdev_ops
smsc95xx_netdev_ops = $8 = {
  ndo_init = 0, 
  ndo_uninit = 0, 
  ndo_open = 0xbf000514 <usbnet_open>, 
  ndo_stop = 0xbf000bec <usbnet_stop>, 
  ndo_start_xmit = 0xbf001a60 <usbnet_start_xmit>, 
  ndo_select_queue = 0, 
  ndo_change_rx_flags = 0, 
  ndo_set_rx_mode = 0, 
  ndo_set_multicast_list = 0xbf008abc <smsc95xx_set_multicast>, 
  ndo_set_mac_address = 0xc025d854 <eth_mac_addr>, 
  ndo_validate_addr = 0xc025d6f8 <eth_validate_addr>, 
  ndo_do_ioctl = 0xbf00926c <smsc95xx_ioctl>, 
  ndo_set_config = 0, 
  ndo_change_mtu = 0xbf000de0 <usbnet_change_mtu>, 
  ndo_neigh_setup = 0, 
  ndo_tx_timeout = 0xbf000d4c <usbnet_tx_timeout>, 
  ndo_get_stats64 = 0, 
  ndo_get_stats = 0, 
  ndo_vlan_rx_add_vid = 0, 
  ndo_vlan_rx_kill_vid = 0, 
  ndo_set_vf_mac = 0, 
  ndo_set_vf_vlan = 0, 
  ndo_set_vf_tx_rate = 0, 
  ndo_get_vf_config = 0, 
  ndo_set_vf_port = 0, 
  ndo_get_vf_port = 0, 
  ndo_setup_tc = 0, 
  ndo_add_slave = 0, 
  ndo_del_slave = 0, 
  ndo_fix_features = 0, 
crash>

But the user-space vtop is clearly wrong:

crash> vm
PID: 1495   TASK: c1ef1380  CPU: 0   COMMAND: "bash"
   MM       PGD      RSS    TOTAL_VM
c30cd1e0  c1de4000  1484k    2940k  
  VMA       START      END    FLAGS  FILE
c1e9ae90      8000     c2000 8001875  /bin/bash
c1e9aee8     c9000     ce000 8101877  /bin/bash
c1e9af40     ce000     d3000 100077  
c2fc27b0   1247000   1268000 100077  
c2fc2650  4001c000  4001d000 100077  
c1e9af98  40038000  40055000 8000875  /lib/ld-linux.so.3
c2fc20d0  4005c000  4005d000 8100875  /lib/ld-linux.so.3
c2fc2758  4005d000  4005e000 8100877  /lib/ld-linux.so.3
...


crash> vtop 8000
VIRTUAL   PHYSICAL
8000      8000    

PAGE DIRECTORY: c1de4000
  PGD: c1de4000 => 412
  PMD: c1de4000 => 412
 PAGE:        0  (1MB)


  VMA       START      END    FLAGS  FILE
c1e9ae90      8000     c2000 8001875  /bin/bash

crash> vtop 4005d000
VIRTUAL   PHYSICAL
4005d000  4005d000

PAGE DIRECTORY: c1de4000
  PGD: c1de5000 => 40000412
  PMD: c1de5000 => 40000412
 PAGE: 40000000  (1MB)


  VMA       START      END    FLAGS  FILE
c2fc2758  4005d000  4005e000 8100877  /lib/ld-linux.so.3

crash> 

Dave