Understanding how dd works

[Malcolm Kay]

On Wed, 25 Jun 2008 08:57 pm, Dan Track wrote:
> Hi
> 
> I've got a xen vm file called test, if I copy it with dd I get the following
> dd if=/opt/xen/test of=/opt/test-vm.img bs=4096
> du -s /opt/xen/test = 1934112
> du -s /opt/test-vm.img = 26240040
> 
> My question is why is the test-vm.img larger in size than the original?

du does not report file size but disk blocks used.
If you 'ls -l /opt/xen/test /opt/test-vm.img' you will find both files are the same
size. Yes this is somewhat pedantic; but see what du claims for the disk usage
of a file having a size of one byte.

However 'ls -s /opt/xen/test /opt/test-vm.img' will report the disk blocks usage
which will be different.

As others have suggested, a file which is partially empty or having large regions 
of zeros might be stored sparsely. But the sparse representation is not unique,
but must depend on the allocation size for the file system; nor do all file systems
have the capability of managing files sparsely. If you copy a sparse file from
one file system to another using a copy method with a 'handle sparse files efficienty'
option then chances are that the disk usage will change even while both source and
destination remain sparse.

On Wed, 25 Jun 2008 10:49 pm, Patrick O'Callaghan wrote:
> Note also that if you're not careful you can backup a file or even a
> filesystem that you can't restore because it's too big, especially if
> copying it to some medium (e.g. a tape drive or non-UNIX disk system)
> that can't handle sparse files.

Mostly this should not be too much of a problem because one would usually
archive in some way and one depends on the archiver to manage the sparse files.
GNU tar has a --sparse option. So with the --sparse option a tar archived set of 
files should fit back where they came from.

Malcolm