[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Libguestfs] Thoughts on nbdkit automatic reconnection



On Thu, Sep 19, 2019 at 01:42:25PM +0200, Nenad Peric wrote:
> I have an update on the networking issue:
> - After the deep dive into the logs of the firewall by customer's security
> team, it turns out that even though there were some disconnections, the
> time-stamps do not match.

I wouldn't necessarily expect them to match, although it would depend
on how exactly the disconnect happens.  For example:

 - Firewall sends TCP FIN packets to both ends:

   => Our client would immediately respond to this, so timestamps
      should be very close (within seconds).

 - Firewall drops the entry in its connections table but does not send
   anything to either side:

    => Our client which is sending TCP keepalives (or rather, the
       Linux kernel is) would notice the next time it sends a
       keepalive, which would be bounced back and cause the connection
       to close.

    => Delay could be as long as the keepalive period, so that depends
       on a bunch of stuff such as VDDK client settings and kernel
       settings.

>  This means that we got the disconnected by something else (ESXi or
> conversion host perhaps)
> - As we mentioned in the chat briefly, there could be general keep-alive
> issues on both RHEL (conversion host) and ESXi side.
>  We changed the keep-alive settings in RHEL, but could not find the
> equvalent in VMware as of yet.
> - I found on a few spots that there are some vddk (vixDiskLib.nfc*)
> settings which can configure NFC keep-alives and timeouts, but I do not
> understand it deeply enough to see if anything would help.

Yes there's a VDDK setting (vixDiskLib.nfc.ReadTimeoutMs).  We don't
set it at the moment so AIUI it should default to between 6 and 45
seconds depending on the version of VDDK.

> Whatever may be the cause, a retry filter would most likely solve the
> problem.

Yes I think so too.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]