[Pulp-list] Anyone using VDO to save space in /var/lib/pulp?

[Mike DePaulo]

Hi Kodiak,

I've had the discussion over deduplication before (I can't find it though),
and I think the agreement was I would run a test & publish the results at
some point.

Basically, Pulp already duplicates completely identical content files such
as RPMS & debs. VDO would provide dedup at the (4KB) block layer.

However, RPMs, debs, etc contain compression like xz within them. And they
do solid compression (like "data.tar.xz" with debs), not a per-file
compression. Compression interferes with most of the effects of
deduplication:
http://thestoragealchemist.com/blog/2010/04/comression-deduplication-oil-water-or-milk-cookies

Other content types may be uncompressed. I *think* docker images typically
are uncompressed. They would certainly benefit.

-Mike

On Mon, May 13, 2019 at 2:37 PM Kodiak Firesmith <kfiresmith at gmail.com>
wrote:

> Just curious if anyone is using VDO block dedupe since it went into
> production support in RHEL 7.5.  Playing with it at Summit got me thinking
> about use cases, which of course made me think of Pulp.
> Thanks,
>  - Kodiak
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190513/fb1b795b/attachment.htm>