Ideas about snapshot migration.

Atul Aggarwal searchatul at gmail.com
Wed Jul 15 19:58:27 UTC 2009


I am thinking about how to migrate from one snapshot to another without much
overhead. I thought it to be easy task but as I start thinking about it in
details, it really seams complicated and requires discussion and thoughts
before finalizing one idea.
I couldn't thought much about it. Here are some pieces of ideas. I am trying
to club some ideas from irc.

When creating new torrents, files must be padded to match the multiple of
segmentation size of torrent. In this way, we can preserve the hash value
for the piece allocated in the torrent file. So each file (segments of file
in torrent) has unique hash value which does not change if the file doesn't
change between the snapshots.
Another problem is how to determine if the file has not been changed (and
file from previous snapshot is downloaded), we may add a hash value to the
end of file with some proper formatting. Now whenever a request for a file
arrives, we check the hash value from the torrent file and check if that
file+hash already exists, if yes pass the file otherwise download the file
and pass it.
So my idea is that we have one common directory for all the snapshots, place
the file+hash into the directory. So a directory say dir will contains :
dir/a.tar-snap1
dir/a.tar-snap3
If the snapshot 2 file a is same as in snapshot 1. (here snap1 is the hash
value of file in the snapshot1).

Suppose a user browse through older snapshot (this will be possible as older
torrent file along with repo will be stored), hash value will come handy.
Also if user defines like setting a limit on harddisk, we may be able to
remove older snapshots like browse through older torrent file, and delete
files with older torrent file hash (checking with torrent file just newer
than removing torrent, if it is not using that file).

Does these ideas make some sense? Please provide feedback, suggestion for
improvement or total new idea. I will try to add more information as soon as
things become clear to me.

-- 
Regards,

Atul Aggarwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/instantmirror-list/attachments/20090716/fef66ccd/attachment.htm>


More information about the instantmirror-list mailing list