Repository Snapshot storage format

[Atul Aggarwal]

Hello everybody,

I would like to discuss the way the snapshots are stored on the disk.

Repository snapshots are snapshots of the published directory tree which are
taken at regular interval of time and are given UUID for reference by
client.

Since we have ruled out fuse for backend, I am thinking of very trivial
procedure for storing these snapshots on disk. For first snapshot, we will
save the complete tree and for further snapshots we will save the files
which are changed only and link all existing files (which has not been
changed since last snapshot) to previous snapshot. For tagging purpose,
another link will be created in the tag directory which will point to the
snapshot which is tagged.
Pro of this method is it is very easy to implement and understand.
Con is that while deleting snapshot we need to check whether the file is not
referenced in any other snapshot/ tag. Same is the case with the tagging.

To check this con, the process of linking can be changed as whenever we have
a new snapshot, instead of creating a link in the new snapshot, we may move
the file to new snapshot and create a link in the older snapshot. In this
way we may delete the old snapshot which bothering the breaking of new
snapshot.

I also thought of using diff in the storage but it seems to be a bad idea as
we will be mostly dealing with binary files and it will create load on the
server on the cost of some disk space. Also using diff extra computation
needs to be done while adding new snapshot and removing snapshots. Using
some version control system will unnecessary load on the system without much
benefit of space on large repository tree.

Ideas mentioned on
https://fedorahosted.org/InstantMirror/#InstantMirrorArchitectureDesignunder
*Repository Snapshots, Views, and Tags* looks good enough regarding
Snapshots.

The idea about the snapshot storage may seem trivial in nature but it seems
to fulfill our requirements.
Any better idea that I should look into or comments are welcome.

Regards,

Atul Aggarwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/instantmirror-list/attachments/20090521/1b72929e/attachment.htm>