Best practice for bulk data transfer

Andrew linux.als at gmail.com
Fri Feb 17 08:00:10 UTC 2006


I just need to transfer a couple hundred thousand files off a **REALLY**
slow server to a faster server and to be able to document that file x on
slow server is identical to file x on faster server. It is not a forensics
problem per se -- but I do need to prove/document the process -- so I'm
thinking md5sums.

/** begin simplification **/

Procedure 1: The slow option

[slow server] calc md5sums > list && nc list > faster_server
[slow server] rsync files to faster at server over a 10T crossover cable
[faster server] check md5sums against list

Procedure 2: The faster option (repeated for each hard drive)

[slow server] halt
pull hard drives off slow_server and place in removable tray on
faster_server
[faster server] dd if= /dev/slowdrive ibs=4096 conv=notruc,noerror >
slowdrive.img
[faster server] mount slowdrive.img && calc md5sums > list
[faster server] rsync /mnt/slowdrive/ /newdir/
[faster server] cd /newdir/ && check md5sums against list

/** end simplification **/

-The slow server is 6 years old -- no cdrom, no floppy :-)
-Data corruption is totally unacceptable.

Questions:
1) Theory question: will Option 2 produce identical results to Option 1 only
quicker ??
2) Practical question: is Option 2 "too risky" given the age of  the drives
?? I have this fear of possibly harming a hard drive no matter how careful
.. is my fear justified or baseless ??
3) The better idea is to ... I'm always up for the better idea !!!

Thanks again and best wishes to everyone.

Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/fedora-list/attachments/20060217/7f7e953b/attachment-0001.htm>


More information about the fedora-list mailing list