[libvirt] heisenbug in command.c

[Serge Hallyn]

On 03/16/2012 03:52 PM, Jim Paris wrote:
> Serge Hallyn wrote:
>> On 03/16/2012 11:50 AM, Eric Blake wrote:
>>> On 03/16/2012 10:36 AM, Serge Hallyn wrote:
>>>> Hi,
>>>>
>>>> It seems I've run into quite the heisenbug, reported at
>>>> https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/922628
>>>>
>>>> It manifests itself as virPidWait returning status=4 for iptables (which
>>>> should never exit with status=4).
>>>
>>> Maybe iptables isn't documented as exiting with $? of 4, but that's what
>>> is happening.  The libvirt code in question is quite clear that it
>>> grabbed an accurate exit status from the child process.
>>>
>>
>> Well, yes.  I figured that either (1) iptables actually got -EINTR
>> from the kernel and passed that along as its exit code, or (2)
>> something went wrong with memory being overwritten in libvirt,
>> however unlikely. Stranger things have happened.  If (1), I was
>> wondering if it was being ignored on purpose.
>
> Why do you bring up EINTR at all?  Just because EINTR is 4?

Yup.

 >  That
> seems very much unrelated.

I didn't think so, but looks like you're right :)

> This is from iptables:
>
> enum xtables_exittype {
>          OTHER_PROBLEM = 1,
>          PARAMETER_PROBLEM,
>          VERSION_PROBLEM,
>          RESOURCE_PROBLEM,
>          XTF_ONLY_ONCE,
>          XTF_NO_INVERT,
>          XTF_BAD_VALUE,
>          XTF_ONE_ACTION,
> };
>
> So it looks like iptables is returning RESOURCE_PROBLEM (which could
> explain why it's intermittent).

That makes a lot more sense then, yes.  When scripting a bunch of 
parallel iptables rules adds (followed by waits), I do once in awhile get

iptables: Resource temporarily unavailable.

(and sometimes a -EINVAL message)

though 'wait' still says status was 0.  I've never gotten 4.  The next 
run then succeeds.

This still however sounds like src/util/iptables.c might ought to re-run 
the command if it gets 4.

thanks,
-serge