[Libguestfs] [PATCH v7 6/6] v2v: Add -o rhv-upload output mode (RHBZ#1557273).

[Nir Soffer]

On Thu, Mar 29, 2018 at 5:19 PM Nir Soffer <nirsof at gmail.com> wrote:

> On Thu, Mar 29, 2018 at 1:59 PM Richard W.M. Jones <rjones at redhat.com>
> wrote:
>
>> I found another problem which is sort of related to this thread.  I
>> hit a time-out waiting for the disk to be unlocked after creation:
>>
>>
>> https://github.com/libguestfs/libguestfs/commit/8081f54105bd990233f166170890192c1fd7d1f3#diff-5ca47c29ae13efa3959b8b28cf4dbd31R112
>> line 112
>>
>> The disk size in this case is 40 GB and we currently wait 5 minutes.
>>
>> On the ovirt-node there's an ‘fallocate’ process (still) running.
>> It's taken 23 minutes so far.
>>
>
> This code is using posix_fallocate and it can be very slow on NFS < 4.2,
> writing one byte to every 4k block.
>

Actually it is worse - today we:

1. create virtual_size bytes volume
2. preallocate entire image using posix_fallocate, possibly writing
   one byte to every 4k sector
3. write virtual_size bytes of data to volume

The preallocation is mostly useless. It can save you time if there
is not enough storage to copy the volume, but you have to pay for
something that practically never happen for every volume you create.

I have this patch:
https://gerrit.ovirt.org/#/c/86282/

With this you will be able to create a raw preallocated
volume without preallocating anything using initial_size=0,
avoiding the slow creation you experience now.

The file will be allocated by uploading the data into the volume.

This will require also changes on engine side, currently we don't
send initial_size to vdsm for file storage.

It would be nice if you file a performance bug for this issue.

Nir


>
>
>> Because we didn't reach a satisfactory conclusion of the whole
>> sparse/format business, I am currently setting sparse = False
>> (format is always raw in my testing):
>>
>>
>> https://github.com/libguestfs/libguestfs/commit/8081f54105bd990233f166170890192c1fd7d1f3#diff-5ca47c29ae13efa3959b8b28cf4dbd31R92
>> line 92
>>
>> I guess this is the cause of this problem in some sense.  Is there
>> something we can do?  I'm still unclear how to fix the sparse/format
>> mess, it seems unsolvable from the client side because we don't have
>> sufficient information to make a decision.
>>
>
> I don't think always using sparse=False is a good solution. We have an
> issue
> only with block storage, so there is no need to force sparse=False for
> every
> kind of storage.
>
> I think the behavior should be:
>
> - if the user wants raw image (sparse=False) on file domain we can warn
> that
> creating preallocated disk on file storage can take lot of time, and we
> should
> wait until the disk is ready. Or we can always use thin for upload to file
> based
> storage, this is what we for uploading from the UI.
>
> - if the user wants thin image on block storage, we must allocate the
> entire
> image when we create a disk. In 4.2.4 we will support engine API to reduce
> an image to the optimal size, and you will be able to call this when you
> complete
> the upload.
>
> - if the user wants thin image on file storage, or raw image on block
> storage,
> we don't have any issue
>
> In 4.3 we plan to improve engine extend volume api so it will update the
> ticket
> size when extending a disk. With this you will be able to create a thin
> disk on
> block storage and extend it when needed.
>
> Nir
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20180329/2b0bcd10/attachment.htm>