<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Tue, Apr 10, 2018 at 6:53 PM Richard W.M. Jones <<a href="mailto:rjones@redhat.com">rjones@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Tue, Apr 10, 2018 at 03:25:47PM +0000, Nir Soffer wrote:<br>
> On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones <<a href="mailto:rjones@redhat.com" target="_blank">rjones@redhat.com</a>><br>
> wrote:<br>
><br>
> > On Tue, Apr 10, 2018 at 02:07:33PM +0000, Nir Soffer wrote:<br>
> > > This makes sense if the device is backed by a block device on oVirt side,<br>
> > > and the NBD support efficient zeroing. But in this case the device is<br>
> > backed<br>
> > > by an empty sparse file on NFS, and oVirt does not support yet efficient<br>
> > > zeroing, we just write zeros manually.<br>
> > ><br>
> > > I think should be handled on virt-v2v plugin side. When zeroing a file<br>
> > raw<br>
> > > image,<br>
> > > you can ignore zero requests after the highest write offset, since the<br>
> > > plugin<br>
> > > created a new image, and we know that the image is empty.<br>
> > ><br>
> > > When the destination is a block device we cannot avoid zeroing since a<br>
> > block<br>
> > > device may contain junk data (we usually get dirty empty images from our<br>
> > > local<br>
> > > xtremio server).<br>
> ><br>
> > (Off topic for qemu-block but ...) We don't have enough information<br>
> > at our end to know about any of this.<br>
> ><br>
><br>
> Can't use use this logic in the oVirt plugin?<br>
><br>
> file based storage -> skip initial zeroing<br>
> block based storage -> use initial zeroing<br>
><br>
> Do you think that publishing disk capabilities in the sdk will solve this?<br>
<br>
The plugin would have to do some complicated gymnastics. It would<br>
have to keep track of which areas of the disk have been written and<br>
ignore NBD_CMD_WRITE_ZEROES for other areas, except if block-based<br>
storage is being used. And so yes we'd also need the imageio API to<br>
publish that information to the plugin.<br>
<br>
So it's possible but not trivial.<br></blockquote><div><br></div><div>I think this should be fixed in qemu-img. The current zero optimization may</div><div>have insignificant improvement when the backend supports fast zeroing </div><div>(e.g fallocate, blkzeroout) by minimizing number of syscalls, but it has severe</div><div>performance issue when the desntiation does not support fast zero</div><div>(e.g. NFS < 4.2).</div><div><br></div><div>We plan to improve zero performance in 4.2.z, but this will not solve the issue</div><div>on NFS < 4.2.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
By the way I think we're slowly reimplementing NBD in the imageio API.<br></blockquote><div><br></div><div>You can also look at it as - slowly showing that HTTP can replace</div><div>NBD :-)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Dan Berrange pointed out earlier on that it might be easier if imageio<br>
just exposed NBD, or if we found a way to tunnel NBD requests over web<br>
sockets (in the format case nbdkit would not be needed, in the latter<br>
case nbdkit could act as a bridge).<br></blockquote><div><br></div><div>I'm not the more complex web sockets are needed for moving images</div><div>around. web sockets are good when the clients is a browser, and you need</div><div>to pass lot of small messages. If you have a real socket you don't need</div><div>web sockets. And the overhead of http headers is insignificant when you</div><div>move lot of data around.</div><div><br></div><div>Exposing NBD using the same authentication mechanism can be interesting,</div><div>but only if clients have an easy way to use this.</div><div><br></div><div>Will this allow client (e.g. backup vender) to download and upload images</div><div>using qemu-img?</div><div><br></div><div> qemu-img <- nbd-> ovirt-imageio <- ndb -> qemu/ndb-server</div><div><br></div><div>We also need streaming support - with http you can stream the data from </div><div>anywhere, while qemu-img needs a file. I don't think that backup vendors</div><div>or users will be happy to implement nbdkit plugins.</div><div><br></div><div>If qemu-img would support input and ouptut from stdin/stdout instead of</div><div>files, this model sounds much more interesting.</div><div><br></div><div>Nir</div></div></div>