archive-teleconstructor.py - use your previous downloads to save bandwidth getting newer releases

[David Timms]

Hi All,

Firstly, if you feel this is not the appropriate list, can you please 
suggest another ;)

As mentioned in a few places, and with thanks to responses received from 
a few people who provided an "iso-info" run for iso files I didn't have 
yet, I have got to an operational state of a program I'm calling: 
archive-teleconstructor.py

The idea is to save a heap of download time and capacity when you know 
that a heap of files included in a download archive {iso at this stage} 
are already present on your disk or mountable media. The second 
requirement is that the download is presented in a format that allows 
for partial downloading and detection of correct data before actual 
download. This means rsync and bittorrent {tested with azureus}.

It is written in python and included as an attachment, along with some 
iso-info files made by Bruno for fedora 7 dvd and rescue discs.

For example, the F-7-i386-rescuecd.iso can be constructed from the 
content of F-7-i386-DVD.iso, and an rsync that took less than a minute 
{my connection speed might be reasonable, but none of the mirrors I have 
tested with are very fast at the moment}. The stats for that process showed:
   error: verify_files: could not find local files for: 3 files
   verify_files: found local files for: 22 files

I used rsync to finish the download. Comparing the constructed iso 
before and after the rsync completed the download:
     overall statistics: identical:different  101,499,602: 9,518 {99.9905%}
What this says is that after construct only 9518 bytes where different 
from the final file {which was tested again with sha1sum}.

In fact, it would be easy to include such a program on the DVD.iso, that 
builds the rescue cd on demand {with very little extra data required - 
unlike jigdo that seems to need large {250MB} template files for DVD 
sized downloads}.

To get the F-7-i386-DVD.iso in the first place, I provided as local 
files the mounted f7t3 and f7t4 isos, and my most recent development 
download folder. Because it is quite difficult at the moment to get a 
responsive rsync server, I completed the download with azureus. The 
completion jumped from 1% to about 49% after running the program.

For instructions see the --help and the example near the bottom of the code.

I would love to hear your criticisms or kudos for anything about the 
code or it's operation as it stands, especially since it's my first 
python program, many thanks, DaveT.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: archive-teleconstructor.py
Type: text/x-python
Size: 23738 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20070602/24f25d41/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: F-7-i386-DVD.iso.iso-info.txt.bz2
Type: application/x-bzip
Size: 30035 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20070602/24f25d41/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: F-7-i386-rescuecd.iso.iso-info.txt.bz2
Type: application/x-bzip
Size: 869 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20070602/24f25d41/attachment-0001.bin>