EXT2 vs. EXT3: mount w/sync or fdatasync

[brian stone]

>>nothing better than benchmarking your app with different

IO performance is always a consideration, but for this application reliability is much more important.  

I am looking for the most reliable way of dumping files to disk.  We I call close(), I need to know that the data is one disk.  It doesn't need to be the highest performance method, just the most reliable.

>>Unless your filesystem is constantly busy

It is constantly busy.  Each file system manages around 10 millions files across a TB.  Each day, an average of 500,000 files totaling 100G are throw away while the same amount is generated.  Its a constant cycle.  The point is, these are very active file systems. 

I have already seen EXT3 corrupt its superblock(s) after a disk failure, using data=ordered.  Trying different superblocks didn't work, maybe -O sparse_super isn't the best idea.

No merit in EXT2 with fdatasync calls?

thanks for the response.

Andreas Dilger <adilger at clusterfs.com> wrote: On Mar 21, 2007  16:51 -0700, brian stone wrote:
> My application always needs to sync file data after writing. I don't want anything handing around in the kernel buffers. I am wondering what is the best method to accomplish this.

>  4. Do I use EXT3 in full journaled mode, where the data and metadata are journaled? In this case, is the journaled data sync'd or async'd? When the journal commits the data to the file system, is that sync'd or dumped into kernel buffers?
>  
>  5. Since I will always be syncing the data, does it make any sense to use EXT3? It feels like the EXT3 journal would be unnecessary.

In theory, ext3 + data=journal will give you the best performance, because
sync IO will always be linear IO to the journal.  Unless your filesystem is
constantly busy, then the writes to the filesystem can happen asynchronously
after being committed to the journal without danger of being lost.

That said, nothing better than benchmarking your app with different
filesystem options to see which one is best.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.



 
---------------------------------
Don't be flakey. Get Yahoo! Mail for Mobile and 
always stay connected to friends.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/ext3-users/attachments/20070322/19d2b4b5/attachment.htm>