[libvirt-users] error : virHashForEach:597 : Hash operation not allowed during iteration

[Michal Privoznik]

On 04/04/2018 10:36 AM, Vincent Bernat wrote:
> Hey!
> 
> On many of our servers, we often have the following error:
> 
> error : virHashForEach:597 : Hash operation not allowed during iteration
> 
> When querying the list of domains, this means libvirt will silently
> return 0 domain (most uses of virHashForEach don't handle the return
> code).
> 
> We are using a hook, but the hook doesn't query libvirt back, so it
> should be safe from this point of view. However, we have several sources
> querying libvirt at regular interval (once every 5 seconds from two
> different sources). I am unable to reproduce the problem with just
> running "virsh list" in parallel.
> 
> We are using libvirt 3.6.

Is this plain 3.6.0 or do you have any patches on top of it (e.g. some
in-house patches)?

> 
> Is it an expected constraint to not be able to reliably query libvirt
> from several sources? Have fixes for this problem been pushed since 3.6?
> 

Dunno, this is the first time I hear about this issue. Maybe you can try
to set a break point on virHashIterationError() and when it's hit get
stacktrace of all threads 't a a bt'. That might shed more light into
the issue. Smells like we are not locking somewhere properly.

Also, you might want to try the latest release of 4.2.0 and see if the
bug still reproduces.

Michal