[linux-lvm] Data deduplication for Linux : lessfs

[Roy Sigurd Karlsbakk]

On 24. juni. 2009, at 21.25, Mark Ruijter wrote:

> Hi Roy,
>>
>> It's a good idea, but given the current traffic on the lessfs  
>> mailing list, I'm not sure if much work is done. I have been a  
>> member of that list since June 1 and haven't received more than one  
>> message, which was the one I wrote myself.
>>
>
> Almost all the traffic is on the forum - open discussion.
> Only one person posted to the mailing list. ;-)

Why??
Mailing lists are so much easier to use. Instead of visiting a bunch  
of websites, they all sit in my mailbox.

>> If done smartly, this may perhaps be possible, but the problem is  
>> the filesystem's metadata. Is this going to be dedup'ed? How much  
>> will this take? A simple backup will update atime on all the files  
>> backed up, and although atime isn't always wanted or needed, the  
>> problem occurs elsewhere.
>
> Typically the meta data on production systems is approx 10%~20% of  
> the deduplicated stored data.
> Stored data is on my systems 40x less then the data written to the  
> filesystem.


The problems with metadata is not that they take up a lot of space,  
but that they are updated so regularly. As Greg Freemyer pointed out,  
relatime will help a lot, but still, deduplicating metadata may take  
up a serious amount of time because of the frequent updates.

roy
--
Roy Sigurd Karlsbakk
(+47) 97542685
roy at karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres  
intelligibelt. Det er et elementært imperativ for alle pedagoger å  
unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de  
fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.