[Thincrust-devel] issues with zip packaging for appliances

[Perry Myers]

Perry Myers wrote:
> I've been playing with the -p zip option to appliance-creator and ran 
> into an issue that I'm not sure how we can solve...
> 
> If the output format of a disk image is raw there are two sizes, the 
> logical size and the actual size.  So if I create a 20GB disk partition 
> in the kickstart, but only use 1.5 GB the file appears to be 20GB but 
> actually only takes 1.5GB on disk.
> 
> When appliance-creator attempts to add this disk image to the zip 
> archive it fails because the logical size of the file is > 4GB which is 
> the limitation of an individual file in a zip archive.  You can pass the 
> allowZip64=True parameter to Zip in python to allow the zip64 extensions 
> to remove this limitation, but then the zip file can no longer be 
> extracted using the standard unzip command line tools (also not sure if 
> native Windows zip can handle this)
> 
> It will probably be fairly common that appliance images will have 
> logical sizes of > 4GB, so zip files are really only useful for 
> packaging appliances if the image is qcow compressed instead of raw.  If 
> the images are qcow, then the logical=actual and will generally be < 4GB.
> 
> We can consider using another archiving format like tar, but then we 
> can't  extract appliances on Windows with the default tools there.  We 
> can force qcow compression for zip packaging but that limits our 
> hypervisor usage.

Other random interesting datapoints...

for the oVirt Appliance, if we have the output format as raw the image 
size is 1.5GB since it is sparse

with the output format as qcow2 the image only drops to 1.2GB

if I take that qcow2 image and put it in a zip file the resulting zip file 
is around 400MB

if I zip the raw 1.5GB image it's also around 400MB (maybe a little bit 
more but not much)  This only worked with the Zip64 extensions turned on 
though

So for distribution of appliances qcow2 does not really get you any 
savings.  You'd still have to use a compressed package format.  We've been 
doing that up until now by putting the image into an RPM which compresses it.

So the fundamental question here is... what packaging format can we use 
that will have good compression, is natively supported on a wide variety 
of platforms (Unix, Windows, Linux) and will hold raw images that are 
fairly large.

Perry