[Crash-utility] Using crash with structure layout randomized kernel

[Hatayama, Daisuke]

> -----Original Message-----
> From: crash-utility-bounces at redhat.com
> [mailto:crash-utility-bounces at redhat.com] On Behalf Of Dave Anderson
> Sent: Thursday, January 25, 2018 12:23 AM
> To: Cao, Jin/曹 晋 <caoj.fnst at cn.fujitsu.com>
> Cc: crash-utility at redhat.com
> Subject: Re: [Crash-utility] Using crash with structure layout randomized
> kernel
> 
> 
> 
> ----- Original Message -----
> >
> >
> > On 01/23/2018 11:19 PM, Dave Anderson wrote:
> > >
> > >
> > > ----- Original Message -----
> > >> Hi Dave,
> > >>
> > >>     Recently I was trying crash tool with kdump dumpfile & structure
> > >> layout randomized kernel[*](), and it fails without any surprise. After
> > >> looking into the different errors crash reports, I can confirm it is a
> > >> result from randomized structure layout.
> > >>
> > >> So my questions is, do you ever consider supporting this feature[*] in
> > >> crash?
> > >>   If yes, do you have any plan & technique evaluation about it?
> > >>   If no, what's the reason?
> > >>
> > >> [*]https://lwn.net/Articles/722293/
> > >> --
> > >> Sincerely,
> > >> Cao jin
> > >
> > > I was under the impression that the structure layout was done at
> > > compile-time, and that the vmlinux file's debuginfo data would
> > > represent the randomized layout.  And that being the case, the
> > > inconvenience would be that the crash session would show the
> > > randomized layout, while the associated source code would show
> > > the original layout.
> > >
> >
> > BTW, I don't have any compiler knowledge before, just from these two
> > days learning, I feel you are right at "vmlinux file's debuginfo data
> > would represent the randomized layout".
> >
> > But when I debug, it seem not like what it should be. I have two file
> > pairs, randomized and non-randomized one. I print some member offset of
> > structure tagged with __randomize_layout after MEMBER_OFFSET_INIT, like
> > this one:
> >
> > (gdb) p offset_table->task_struct_state
> > $1 = 8
> > (gdb) p offset_table->task_struct_exit_state
> > $2 = 2164
> > (gdb) p offset_table->task_struct_pid
> > $3 = 2264
> > (gdb) p offset_table->task_struct_comm
> > $4 = 2744
> > (gdb) p offset_table->task_struct_next_task
> > $5 = -1
> > (gdb) p offset_table->task_struct_processor
> > $6 = -1
> > (gdb) p offset_table->task_struct_p_pptr
> > $7 = -1
> > (gdb) p offset_table->task_struct_parent
> > $8 = 2288
> >
> > Under both file pairs, these offset value are the same, so, I think that
> > is why I have the impression that debuginfo has the original structure
> > layout. I guess this is one kind of "MEMBER_OFFSET() no longer work"?
> 
> 
> It appears that way.  A better manner would be to simply run
> gdb on the 2 vmlinux files, i.e., run "gdb vmlinux", and then
> print out one of the required data structures that has the
> "__randomize_layout" compiler annotation, such as the mm_struct:
> 
>   struct mm_struct {
>   	struct vm_area_struct *mmap;		/* list of VMAs */
>   	struct rb_root mm_rb;
>   	u32 vmacache_seqnum;                   /* per-thread vmacache */
> 
>   ... [ cut ] ...
> 
>   } __randomize_layout;
> 
> In each gdb session, run the command "ptype struct mm_struct",
> and compare the two outputs.  If they are the identical, then there
> is a major problem.
> 
> A fundamental underpinning of the crash utility is the ability
> to gather correct structure member offsets from the embedded gdb
> module, which gets them from the debuginfo data of the vmlinux file.
> If the structure layout is identical in both the randomized
> and non-randomized vmlinux files, then the crash utility will
> be pretty much useless.
> 
> I agree with Linus Torvalds with respect to this feature, where in
> the article that you referenced, he says:
> 
>    Nevertheless, Torvalds is unimpressed by structure randomization,
>    calling it "security theater". The fact that distributions need to
>    publish the randomization seeds for module-building meant it did
>    not provide as big of a security feature as advertised. Torvalds
>    however did add: "So it's imnsho a pretty questionable security
>    thing. It's likely most useful for one-off 'special secure
>    installations' than mass productions.
> 
> I have long believed that the constant onslaught of new "security"
> features pushed into the kernel by Kees Cook will eventually kill
> the capability of using the crash utility to debug the kernel.  So
> far we've been able to work around or adapt to previous security
> related changes, but this particular feature looks to be a crash
> killer.
> 

Also, the structure randomization trivially causes KABI breakage;
KABI is a term of RHEL. See https://access.redhat.com/solutions/444773?tour=6 for example.

When I tried it on v4.14-rc6 a couple months ago, I saw 213 of the 794 symbols,
26.8%, in the KABI on RHEL7.4 got broken.

The structure randomization appears incompatible with KABI in nature.
The randomization typically targets primary structure types in the kernel
such as task_struct, mm_struct and so on that are likely to be targeted
by attackers, while such primary structure types are also likely to be
KABI protected because modules often use them.

So, this feature is for special kernel as Linus said, not for
distribution kernels to which vendors and individuals provide their
kernel modules that require to maintain compatibility across
multiple releases.

Thanks.
HATAYAMA, Daisuke