[olpc-software] graceful handling of out-of-memory conditions

[msevior at physics.unimelb.edu.au]

> On Tue, Mar 28, 2006 at 01:11:09PM +1000, Martin Sevior wrote:
>> Is it possible to provide a signal to a process upon receiving a an Out
>> Of Memory rather than just returning a NULL pointer? While returning
>
> The C standard is pretty clear. Handle malloc failure yourself. If you
> want a signal from it then wrap malloc and do it yourself. A simple
>
> 	if(foo == NULL) raise(SIGBLAH)
>
> or in C++ throw an exception.
>
>> NULL pointers will almost certainly cause AbiWord to segfault it would
>> nicer to know that the segfault was actually an OOM event rather than a
>
> Thats worrying. So Abiword handles untrusted documents but doesn't do NULL
> pointer checks which on some machines lets you scribble on low memory
> pages
> and build exploits.
>
> I hope thats not what you are implying.
>

Well we have many NULL pointer checks and our import classes are designed
to have have no buffer overruns. Our string classes are designed to grow
as needed. That said, we're generally not security experts so any advice
you have on potential exploits would be appreciated.

What we generally don't do is:

SomeClass * p = new SomeClass(...);
if(!p)
{
 UT_DEBUGMSG(("Eek! Out of Memory TODO \n"));
}

Instead we have code like:

SomeClass * p = new SomeClass(...);
p->SomeMethod(...);

The original authors of AbiWord put in such checks but in practice we've
never seen AbiWord fail this way so they have fallen out of favour.

I guess we could grep through the code and put in an OOM dump after every
such call. I was hoping we could avoid this by instead handling some
specific signal.


>> delete classes (to release memory) and to give an appropriate document
>> name, if the latter it make more sense to dump the document as quickly
>> as possible.
>
> Very hard from a signal handler because you may only use signal safe
> library
> functions.
>

OK Once again my own ignorance shows. How can I find out what these are?

Thanks!

Martin

> Alan
>
>
>