[Crash-utility] Using crash with structure layout randomized kernel

[Cao jin]

On 01/25/2018 05:35 PM, Petr Tesarik wrote:
> On Thu, 25 Jan 2018 16:37:01 +0800
> Cao jin <caoj.fnst at cn.fujitsu.com> wrote:
> 
>> On 01/24/2018 11:23 PM, Dave Anderson wrote:
>>>
>>>
>>> ----- Original Message -----  
>>>>
>>>>
>>>> On 01/23/2018 11:19 PM, Dave Anderson wrote:  
>>>>>
>>>>>
>>>>> ----- Original Message -----  
>>>>>> Hi Dave,
>>>>>>
>>>>>>     Recently I was trying crash tool with kdump dumpfile & structure
>>>>>> layout randomized kernel[*](), and it fails without any surprise. After
>>>>>> looking into the different errors crash reports, I can confirm it is a
>>>>>> result from randomized structure layout.
>>>>>>
>>>>>> So my questions is, do you ever consider supporting this feature[*] in
>>>>>> crash?
>>>>>>   If yes, do you have any plan & technique evaluation about it?
>>>>>>   If no, what's the reason?
>>>>>>
>>>>>> [*]https://lwn.net/Articles/722293/
>>>>>> --
>>>>>> Sincerely,
>>>>>> Cao jin  
>>>>>
>>>>> I was under the impression that the structure layout was done at
>>>>> compile-time, and that the vmlinux file's debuginfo data would
>>>>> represent the randomized layout.  And that being the case, the
>>>>> inconvenience would be that the crash session would show the
>>>>> randomized layout, while the associated source code would show
>>>>> the original layout.
>>>>>  
>>>>
>>>> BTW, I don't have any compiler knowledge before, just from these two
>>>> days learning, I feel you are right at "vmlinux file's debuginfo data
>>>> would represent the randomized layout".
>>>>
>>>> But when I debug, it seem not like what it should be. I have two file
>>>> pairs, randomized and non-randomized one. I print some member offset of
>>>> structure tagged with __randomize_layout after MEMBER_OFFSET_INIT, like
>>>> this one:
>>>>
>>>> (gdb) p offset_table->task_struct_state
>>>> $1 = 8
>>>> (gdb) p offset_table->task_struct_exit_state
>>>> $2 = 2164
>>>> (gdb) p offset_table->task_struct_pid
>>>> $3 = 2264
>>>> (gdb) p offset_table->task_struct_comm
>>>> $4 = 2744
>>>> (gdb) p offset_table->task_struct_next_task
>>>> $5 = -1
>>>> (gdb) p offset_table->task_struct_processor
>>>> $6 = -1
>>>> (gdb) p offset_table->task_struct_p_pptr
>>>> $7 = -1
>>>> (gdb) p offset_table->task_struct_parent
>>>> $8 = 2288
>>>>
>>>> Under both file pairs, these offset value are the same, so, I think that
>>>> is why I have the impression that debuginfo has the original structure
>>>> layout. I guess this is one kind of "MEMBER_OFFSET() no longer work"?  
>>>
>>>
>>> It appears that way.  A better manner would be to simply run
>>> gdb on the 2 vmlinux files, i.e., run "gdb vmlinux", and then
>>> print out one of the required data structures that has the
>>> "__randomize_layout" compiler annotation, such as the mm_struct:
>>>
>>>   struct mm_struct {
>>>   	struct vm_area_struct *mmap;		/* list of VMAs */
>>>   	struct rb_root mm_rb;
>>>   	u32 vmacache_seqnum;                   /* per-thread vmacache */
>>>
>>>   ... [ cut ] ...
>>>
>>>   } __randomize_layout;
>>>
>>> In each gdb session, run the command "ptype struct mm_struct",
>>> and compare the two outputs.  If they are the identical, then there
>>> is a major problem.
>>>   
>>
>> I compared the output of "ptype struct uts_namespace"(and also
>> mm_struct), it seems there does is a major problem, the two outputs are
>> the same:(
>>
>>> A fundamental underpinning of the crash utility is the ability
>>> to gather correct structure member offsets from the embedded gdb
>>> module, which gets them from the debuginfo data of the vmlinux file. 
>>> If the structure layout is identical in both the randomized
>>> and non-randomized vmlinux files, then the crash utility will
>>> be pretty much useless.
>>>   
>>
>> Seems horrible thing happens.
>>
>>> I agree with Linus Torvalds with respect to this feature, where in 
>>> the article that you referenced, he says:
>>>
>>>    Nevertheless, Torvalds is unimpressed by structure randomization, 
>>>    calling it "security theater". The fact that distributions need to 
>>>    publish the randomization seeds for module-building meant it did
>>>    not provide as big of a security feature as advertised. Torvalds
>>>    however did add: "So it's imnsho a pretty questionable security 
>>>    thing. It's likely most useful for one-off 'special secure 
>>>    installations' than mass productions. 
>>>   
>>> I have long believed that the constant onslaught of new "security"
>>> features pushed into the kernel by Kees Cook will eventually kill
>>> the capability of using the crash utility to debug the kernel.  So 
>>> far we've been able to work around or adapt to previous security
>>> related changes, but this particular feature looks to be a crash 
>>> killer.
>>>   
>>
>> Yes, also feels that it is not easy to adapt or work around since the
>> randomized structure layout info is not reflect in the vmlinux's
>> debuginfo. I don't know gcc internals, so a blind guess is that it is
>> done in this way on purpose for security, so attackers won't bother to
>> seek the random seed, but just using gdb to find out the structure layout.
> 
> I'm not an expert either, but I feel that this is a gcc bug. If anyone
> is concerned about security, they needn't publish any debuginfo at all,
> rather than giving useless debuginfo.
> 

Ah, yes, I think you have a point here.

> Have you tried reporting this behaviour upstream as abug?

No, but it is good suggestion for me, thanks Petr.

> Petr T
> 
> 
> .
> 

-- 
Sincerely,
Cao jin