[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: optimesing NFS Server for e.g. remote Backup

Tony Molloy wrote:
On Thursday 16 February 2006 07:45, Götz Reinicke wrote:


I have some RHEL4 servers and want to set up a central backup system
with a "big" harddisk. I found a very nice shell-script, doing
incremental backups using tar.

Any ideas what coul be wrong? The Backups are about 10 GB +- up to 100
GB in the future.

With that amount of data to backup don't use tar, use rsync instead. Rsync is much faster as it only copies the parts of files which have changed. An excellent backup system using rsync to try is BackupPC < backuppc.sourceforge.net >. It does full and incremental backups using rsync.

Some time ago, I tried backing up our server over the Internet using rsync, and even after discussion with the developers and updating to the latest version, it was way too slow.

It took enormous amounts of RAM and swap (fortunately, depite making swap several times RAM it didn't induce thrashing), and never completed in a reasonable time.

Now, I have a devious script that creates a compressed ISO using mkzftree and mkisofs and then uses rsync to update a remote copy over ADSL. The longest time to update the remote copy is a whisker over the hour which we think entirely reasonable.

It takes around five minutes to copy over the local LAN.

Here is the script: watch for line-wrap.
mail:~# cat /root/bin/bu.image
#!/bin/bash -e
RSO="--recursive --links --hard-links --perms --owner --group --devices --times"
RSO="${RSO} --sparse --one-file-system --rsh=/usr/bin/ssh --delete "
RSO="${RSO} --delete-excluded --relative --stats "
RSO="${RSO} --numeric-ids --timeout=7200 ${EXC}"
trap '/etc/init.d/postfix start' EXIT
/etc/init.d/postfix stop
[ -d /mnt/backup ] || mkdir /mnt/backup
umount /mnt/backup || :
rm -fr ${Image} /var/tmp/backup
dd if=/dev/zero of=${Image} count=0 bs=1024 \
ls -shog ${Image}
mke2fs -Fq ${Image}
mount -o loop ${Image} /mnt/backup/
sfdisk >/etc/disktab -d /dev/hda
tar clC / --exclude=backup.img --exclude=/tmp --exclude=/mnt\/* --exclude=/var/lock --exclude=swapfil\* \ --exclude=/var/autofs --exclude=lost+found --exclude=/var/tmp --exclude=/var/local --exclude=squid-cache \
        --exclude=/var/spool/cyrus/mail-backup \
        / /boot /home  /var \
        | buffer -m $((2*1024*1024)) -p 75\
        | tar xpC /mnt/backup || { df -h ; exit ; }
ls -go /mnt/backup
rm ${Image}
/etc/init.d/postfix start && trap -
mkzftree --one-filesystem /mnt/backup/ /var/tmp/backup
df -h
umount /mnt/backup/
mkisofs -R -z -quiet -nobak -o /var/tmp/backup-${HOSTNAME}.iso /var/tmp/backup
df -h /var/tmp
# scp -p /var/tmp/backup-${HOSTNAME}.iso backup.office.lan:/var/local/backups || : time rsync ${RSO} /var/tmp/backup-*.iso backup.office.lan:/var/local/backups ||:
time rsync ${RSO} /var/tmp/backup-*.iso


I do the mucking around with tar because mkzftree doesn't do much in the way of filtering the source filesystem.

The resultant image is easily read by Linux but on other operating systems one needs to run mkzftree to unpack it.



-- spambait
1aaaaaaa computerdatasafe com au  Z1aaaaaaa computerdatasafe com au
Tourist pics http://portgeographe.environmentaldisasters.cds.merseine.nu/

do not reply off-list

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]