[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Incremental & scheduled copy of large data set over the net

I am wondering if anyone can help me with this scenario.

Suppose I want to copy large data set over the net (about 20 GB). it's a bunch 
of files with directories and subdirectories, so basically I'm trying to copy 
a directory tree.
Now, I don't want to copy the whole thing in one sit since it's large. I want 
to do it incrementally during off-peak hours, so, say everyday between 2-3 AM 
I would copy 500 MB, and then the next day continue with another ~500MB, and 
the next day, etc, etc, until it's completed. The content of source directory 
will change, although not by much at all, so this needs to be able to keep 
trakc of that and sync with the changes.

I have a shell access (SSH) on both machines, the source machine, and the 
target machine. My local machine is the target machine. So I think something 
like 'rsync' should probably work, combine with something else for scheduling 
(cron maybe). But I am not sure how to stop after the first ~500 MB is 
transferred, and continue the next day (of course the number 500MB does not 
have to be exact, it could be a rough rounding, because it's probably easier 
to stop after one file is completed, than stop in the middle of the file).

So, any ideas on how to do this?

Thanks in advance for any help.

Reuben D. Budiardja

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]