Migration with "--p2p --tunnelled" hanging in v6.9.0

[Daniel P. Berrangé]

On Wed, Nov 25, 2020 at 05:20:41PM +0000, Daniel P. Berrangé wrote:
> On Wed, Nov 25, 2020 at 05:00:20PM +0000, Daniel P. Berrangé wrote:
> > On Wed, Nov 25, 2020 at 04:36:39PM +0000, Daniel P. Berrangé wrote:
> > > On Wed, Nov 25, 2020 at 04:49:14PM +0100, Christian Ehrhardt wrote:
> > > > I found that the same vol-download vs 127.0.0.1 gives the same results.
> > > > That in turn makes it easier to gather results as we only need one system.
> > > 
> > > Yep, that's useful, I'm able to reproduce this problem myself too
> > > now. Will do some local tests and report back...
> > 
> > Sigh, the problem is way too many reallocs, repeatedly growing and shrinking
> > the buffer we use for I/O.
> > 
> > I guess we never noticed this awfulness in the virsh console code it was
> > copied from, as the data volumes are lower.
> > 
> > Switching to a fixed size buffer makes it massively faster. I'll prep a
> > patch asap.
> 
> Actually, it is also important to have a bigger buffer. Using a 1 MB buffer
> makes all the difference in throughput.

I've sent two patches to improve the performance and managed to test
vol-download with our ssh helper to beat netcat. I've not had a chance
to test with tunnelled migration yet though, so if you want to try the
patches with your migration test scenario, that'd be useful confirmation


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|