Distributed file system

[Bill Rugolsky Jr.]

On Thu, Feb 24, 2005 at 12:54:57PM +0530, gaurav wrote:
>           I have around 50 machines in my lab, since user data is lying 
> at  central server (all home directory mounted here)....around 50 to 70% 
> storage of local machines is un used .. I was thinking if there was a 
> file system using which
> 
>   1. Is Distributed across all these  machines
>   2. Transparent to users (i.e for users can access thru normal path eg
>      /dist/tom/data )
>   3. Redundancy factor (Since files are distributed, if one part of
>      gets corrupt it should automatically restore)
>   4. ACL
>   5. Scalable
> 
> Is this my wild dream ...or stuff like already exists..if yes the pl 
> share.I was evaluating CODA but it is not ready for production release

OpenAFS, Lustre, GFS.  None of these do precisely what you want, but you
ought to evaluate them.

There are a number of experimental peer-to-peer filesystems based upon
distributed hashes.  Several are implemented in Java, e.g., OceanStore,
and have convenient Java APIs, but aren't readily usable as general-purpose
filesystems.

MIT has a (dormant?) project called Ivy that uses SFS and DHash/Chord to
implement a distributed peer-to-peer filesystem:

	http://www.pdos.lcs.mit.edu/ivy/

Ivy is log-structured, so reclaiming space of "deleted" files is a problem.

As with NFS, the semantics of most p2p filesystems is not POSIX, so there
are limits on what you can do with them.  But for archival storage, they're
great!

A fun/useful project for a bored student would be to port the various
user-space filesystems (CFS, SFS/Ivy, etc.) that use the NFS API (or
Java APIs), to the FUSE (Filesystems in Userspace, fuse.sourceforge.net)
API, which is currently in Andrew Morton's -mm kernel tree, awaiting
more testing before merging.

Regards,

	Bill Rugolsky