[Crash-utility] is crash 7.1.8 meant to support CONFIG_DEBUG_INFO_SPLIT=y / CONFIG_DEBUG_INFO_DWARF4=y kernel builds?

Thu Feb 23 11:17:48 UTC 2017

I guess I  answered my own question - the answer is : NO  ??
I guess a more modern *.dwo format is being used that is not
understood by the version of bfd used by gdb-7.6 .
Debugging crash with gdb 7.11.1 shows BFD is getting
the section offsets wrong :

$ gdb --args crash $BLD/linux-4.10/vmlinux /proc/kcore
GNU gdb (GDB) 7.11.1
...
(gdb) b dwarf2read.c:3972
(gdb) c
Breakpoint 2, error_check_comp_unit_head
(header=header at entry=0x7fffffffd078,
abbrev_section=abbrev_section at entry=0x28c91b0, section=0x28c9290,
section=0x28c9290) at dwarf2read.c:3972
3972	    error (_("Dwarf Error: wrong version in compilation unit header "
(gdb) where
#0  error_check_comp_unit_head (header=header at entry=0x7fffffffd078,
abbrev_section=abbrev_section at entry=0x28c91b0, section=0x28c9290,
section=0x28c9290) at dwarf2read.c:3972
#1  0x00000000006f1106 in read_and_check_comp_unit_head
(header=header at entry=0x7fffffffd078, section=section at entry=0x28c9290,
abbrev_section=abbrev_section at entry=0x28c91b0, info_ptr=0x7ffff7f4704b
"",
    info_ptr at entry=0x7ffff7f47040 "\001", is_debug_types_section=0) at
dwarf2read.c:4018
#2  0x00000000006f11f2 in init_cutu_and_read_dies_no_follow
(this_cu=this_cu at entry=0x7fffffffd270,
abbrev_section=abbrev_section at entry=0x28c91b0,
dwo_file=dwo_file at entry=0x28c91a0,
    die_reader_func=die_reader_func at entry=0x6f5110
<create_dwo_debug_info_hash_table_reader>,
data=data at entry=0x7fffffffd240) at dwarf2read.c:4829
#3  0x00000000006f34e8 in create_dwo_debug_info_hash_table
(dwo_file=0x28c91a0) at dwarf2read.c:8394
#4  open_and_init_dwo_file (comp_dir=<optimized out>,
dwo_name=0x7ffff7f4ecd4 "arch/x86/kernel/head64.dwo") at
dwarf2read.c:8978
#5  lookup_dwo_cutu (dwo_name=dwo_name at entry=0x7ffff7f4ecd4
"arch/x86/kernel/head64.dwo", comp_dir=<optimized out>,
signature=signature at entry=13748681403860065761,
is_debug_types=is_debug_types at entry=0, this_unit=0x1f277d0)
    at dwarf2read.c:9182
#6  0x00000000006f410f in lookup_dwo_comp_unit
(signature=13748681403860065761, comp_dir=<optimized out>,
dwo_name=0x7ffff7f4ecd4 "arch/x86/kernel/head64.dwo",
this_cu=0x1f277d0) at dwarf2read.c:9237
#7  init_cutu_and_read_dies (this_cu=this_cu at entry=0x1f277d0,
abbrev_table=abbrev_table at entry=0x0,
use_existing_cu=use_existing_cu at entry=0, keep=keep at entry=0,
    die_reader_func=die_reader_func at entry=0x702420
<process_psymtab_comp_unit_reader>, data=data at entry=0x7fffffffd4ac) at
dwarf2read.c:4656
#8  0x00000000006f8cc8 in process_psymtab_comp_unit
(this_cu=0x1f277d0, want_partial_unit=0) at dwarf2read.c:5053
#9  0x0000000000709e44 in dwarf2_build_psymtabs_hard
(objfile=<optimized out>) at dwarf2read.c:5548
#10 dwarf2_build_psymtabs (objfile=0x1639970) at dwarf2read.c:3855
#11 0x000000000067bc0e in require_partial_symbols
(objfile=objfile at entry=0x1639970, verbose=verbose at entry=0) at
psymtab.c:92
#12 0x0000000000682644 in read_symbols
(objfile=objfile at entry=0x1639970, add_flags=add_flags at entry=4) at
symfile.c:862
#13 0x00000000006819dc in syms_from_objfile_1 (add_flags=4,
num_offsets=0, offsets=0x0, addrs=<optimized out>, objfile=0x1639970)
at symfile.c:1045
#14 syms_from_objfile (objfile=0x1639970, addrs=<optimized out>,
offsets=0x0, num_offsets=0, add_flags=4) at symfile.c:1063
#15 0x0000000000681f14 in symbol_file_add_with_addrs_or_offsets
(abfd=abfd at entry=0x162c590, add_flags=add_flags at entry=4,
addrs=addrs at entry=0x0, flags=flags at entry=0, parent=parent at entry=0x0,
num_offsets=0, offsets=0x0) at symfile.c:1168
#16 0x0000000000682085 in symbol_file_add_from_bfd
(abfd=abfd at entry=0x162c590, add_flags=add_flags at entry=4,
addrs=addrs at entry=0x0, flags=flags at entry=0, parent=parent at entry=0x0)
at symfile.c:1258
#17 0x00000000006820c8 in symbol_file_add
(name=name at entry=0x7fffffffdd99 "/usr/build/linux/linux-4.10/vmlinux",
add_flags=4, addrs=addrs at entry=0x0, flags=flags at entry=0) at
symfile.c:1274
#18 0x0000000000682113 in symbol_file_add_main_1 (args=0x7fffffffdd99
"/usr/build/linux/linux-4.10/vmlinux", from_tty=0, flags=0) at
symfile.c:1300
#19 0x00000000006a6359 in catch_command_errors (command=0x682140
<symbol_file_add_main>, arg=arg at entry=0x7fffffffdd99
"/usr/build/linux/linux-4.10/vmlinux", from_tty=from_tty at entry=0,
mask=mask at entry=6) at exceptions.c:584
#20 0x00000000006a8e1d in captured_main
(data=data at entry=0x7fffffffd8b0) at main.c:948
#21 0x00000000006a6285 in catch_errors (func=func at entry=0x6a7d90
<captured_main>, func_args=func_args at entry=0x7fffffffd8b0,
errstring=errstring at entry=0x8f412a "", mask=mask at entry=6) at
exceptions.c:557
#22 0x00000000006a8fbb in gdb_main (args=args at entry=0x7fffffffd8b0) at
main.c:1079
#23 0x00000000006a9001 in gdb_main_entry (argc=<optimized out>,
argv=argv at entry=0x7fffffffda18) at main.c:1099
#24 0x00000000004f59af in gdb_main_loop (argc=<optimized out>,
argc at entry=3, argv=argv at entry=0x7fffffffda18) at gdb_interface.c:76
#25 0x0000000000464d3d in main (argc=3, argv=0x7fffffffda18) at main.c:707

(gdb) p *header
$3 = {length = 1, version = 0, addr_size = 0 '\000', signed_addr_p = 0
'\000', abbrev_offset = {sect_off = 2890530816}, offset_size = 4,
initial_length_size = 4, offset = {sect_off = 0}, first_die_offset =
{cu_off = 11}}

we genuinely have a bad header here, because bfd seems to have got the
section offsets wrong:

If I do:

$ objdump -fh head64.o

head64.o:     file format elf64-x86-64
architecture: i386:x86-64, flags 0x00000011:
HAS_RELOC, HAS_SYMS
start address 0x0000000000000000

Sections:
...
Idx Name          Size      VMA               LMA               File off  Algn
 11 .debug_info   00000034  0000000000000000  0000000000000000  00000593  2**0
                  CONTENTS, RELOC, READONLY, DEBUGGING
 12 .debug_abbrev 0000001d  0000000000000000  0000000000000000  000005c7  2**0
                  CONTENTS, READONLY, DEBUGGING
...

So the .debug_info section we are looking at should begin at file
offset 0x593 = 1427, right ?

But the bfd has the section offsets wrong :

(gdb) p *section->asection
$40 = {name = 0x28c2ba3 ".debug_info.dwo", id = 139, index = 0, next =
0x28c2f70, prev = 0x0, flags = 41224, user_set_vma = 1, linker_mark =
0, linker_has_input = 0, gc_mark = 0, compress_status = 0,
segment_mark = 0, sec_info_type = 0,
  use_rela_p = 1, sec_flg0 = 0, sec_flg1 = 0, sec_flg2 = 0, sec_flg3 =
0, sec_flg4 = 0, sec_flg5 = 0, vma = 0, lma = 0, size = 28569, rawsize
= 0, compressed_size = 0, relax = 0x0, relax_count = 0, output_offset
= 0,
  output_section = 0x0, alignment_power = 0, relocation = 0x0,
orelocation = 0x0, reloc_count = 0, filepos = 64, rel_filepos = 0,
line_filepos = 0, userdata = 0x28c53d8, contents = 0x0, lineno = 0x0,
lineno_count = 0, entsize = 0,
  kept_section = 0x0, moving_line_filepos = 0, target_index = 0,
used_by_bfd = 0x28c2c10, constructor_chain = 0x0, owner = 0x2465c10,
symbol = 0x28c2ce8, symbol_ptr_ptr = 0x28c2f38, map_head = {link_order
= 0x0, s = 0x0}, map_tail = {
    link_order = 0x0, s = 0x0}}

'filepos=64' does not correspond to any section header offset or section offset
in the file .

Oh well, I'll just have to rebuild kernel without
CONFIG_DEBUG_INFO_SPLIT=y / CONFIG_DEBUG_INFO_DWARF4=y
.
This is a shame - the massive decrease in overall build size makes
building kernels
a much less daunting  task on my limited storage machine .
I hope crash's bfd is upgraded to handle the modern .dwo format from gcc 5.4.0 +
binutils-2.27 soon.
Is crash ever going to migrate to gdb-7.11.1+ ?

Regards,
Jason

On 23/02/2017, Jason Vas Dias <jason.vas.dias at gmail.com> wrote:
> Hi -
> I have these kernel config options set for a successful kernel build :
>
> $ grep DEBUG_INFO .config
> CONFIG_DEBUG_INFO=y
> # CONFIG_DEBUG_INFO_REDUCED is not set
> CONFIG_DEBUG_INFO_SPLIT=y
> CONFIG_DEBUG_INFO_DWARF4=y
>
> This splits debug_info sections into separate ${X}.dwo files for each
> kernel
> object produced.
>
> I modified several files and did a 'make bzImage' ,  which succeeded,
> and installed the kernel, and tried to run crash-7.1.8 to inspect a
> few things, but it says:
> <quote><pre>
> $ crash vmlinuz /proc/kcore
> ....
> gdb called without error_hook: Dwarf Error: wrong version in
> compilation unit header (is 0, should be 2, 3, or 4) [in module
> /usr/build/linux/linux-4.10/arch/x86/kernel/head64.dwo]
> Dwarf Error: wrong version in compilation unit header (is 0, should be
> 2, 3, or 4) [in module
> /usr/build/linux/linux-4.10/arch/x86/kernel/head64.dwo]
>
> crash: vmlinux: no debugging data available
> </pre></quote>
>
> But the files still exist from the build:
> -rw-r--r-- 1 root root 47784 Feb 21 20:07
> /usr/build/linux/linux-4.10/arch/x86/kernel/head64.dwo
> -rw-r--r-- 1 root root 17072 Feb 21 20:07
> /usr/build/linux/linux-4.10/arch/x86/kernel/head64.o
> in the same directory as the head64.c file .
>
> I thought the whole point of 'vmlinux' was that it contained the debug_info
> ?
> Do I need to re-link a vmlinux.dbg with all the *.dwo files
> corresponding to each '*.o' file vmlinux contains , and vmlinux?
> If so, then I don't see much point in the 'CONFIG_DEBUG_INFO_SPLIT=y'
> option. Shouldn't a 'vmlinux.dwo' file be produced, containing all
> concatenated
> debug_info sections for 'vmlinux' ?
> I guess crash just doesn't support
> CONFIG_DEBUG_INFO_SPLIT=y / CONFIG_DEBUG_INFO_DWARF4=y ?
>
> Sorry for the newbie type question.  Please respond to :
>   jason.vas.dias at gmail.com
> I'm not a member of the list.
>
> Thanks & Regards,
> Jason
>