[libvirt] [Xen-devel] [xen-unstable test] 35257: regressions - FAIL

[Ian Campbell]

On Fri, 2015-02-27 at 11:51 -0700, Jim Fehlig wrote:
> > 2015-02-23 20:13:15.845+0000: 2133: error :
> > virFirewallValidateBackend:193 : direct firewall backend requested,
> > but /sbin/ebtables is not available: No such file or directory
> 
> Odd, since ebtables was found when building
> 
> checking for ebtables... /sbin/ebtables
> 
> But AFAICT, that wont prevent libvirtd from starting.

The build host and the runtime host will likely be different (or at
least reinstalled).

The base set of packages should be the same, but the build one will
install a bunch of libfoo-dev while the runtime host will only get
libfoo. Perhaps some libfoo-dev is pulling in ebtables somehow while
just libfoo is not. I'll have a look next week. I think its probably
non-critical to the error here.

> > I think these are just spurious.
> >
> >         2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
> >         
> >
> >         2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
> >         2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
> >         
> > As are these two.
> >         
> >         2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
> >
> > Has these OOM messages resulted in libvirtd exiting?
> 
> No, I don't think so.  The related code is
> 
> int
> virFirewallApply(virFirewallPtr firewall)
> {
>     size_t i, j;
>     int ret = -1;
> 
>     virMutexLock(&ruleLock);
> 
>     if (!firewall || firewall->err == ENOMEM) {
>         virReportOOMError();
>         goto cleanup;
>     ...
> }
> 
> I suspect 'firewall' is null, so OOM error is reported and the function
> returns -1.  But I also don't see this preventing libvirtd from
> starting.  I've cc'd the libvirt list for verification that these errors
> won't prevent libvirtd from starting.

I'm pretty sure libvirtd did successfully start, since we have
successfully done a guest start and stop.

The failing step is a second guest start, so it seems like libvirtd has
either crashed or exited.

I suppose these messages are from start of day and therefore
red-herrings wrt the reason libvirtd went away.

> >  I don't see any
> > evidence of a crash elsewhere in the logs (i.e. no "process segfaulted"
> > in dmesg, no OOM killing going on etc).
> >
> > We don't seem to collect dom0 freemem info, but that most likely
> > wouldn't help given the libvirtd process has exited.
> >
> >   
> > Any ideas where to look next?
> 
> Can you access the test environment and try starting libvirtd in the
> foreground?  Or enable debug log level in /etc/libvirt/libvirtd.conf?

The test env will have been recycled, I could try and replicate it
manually, but I think to start with I should arrange for the test env to
have more logging enabled, in the hopes that if it happens again we get
more information. I had some question around this in my reply Wei in
this thread at <1425042785.14641.188.camel at citrix.com>.

Cheers,
Ian.