[Linux-cluster] Samba Technical thread

[Daniel Phillips]

Hi Chris!

On Tuesday 07 December 2004 00:56, Christopher R. Hertel wrote:
> There's a thread started on the Samba-Technical mailing list that has
> some discussion regarding cluster filesystems.  I'm learning fast,
> but I'm not the right one to answer this:
>  
> http://lists.samba.org/archive/samba-technical/2004-December/038326.h
>tml
>
> An excerpt:
>
>   The other network-like filesystems - Lustre, SANFS, GPFS, and
> RedHat's GFS do differ a little..  They differ in that they would
> attempt stricter posix semantics

A filesystem either complies with Posix semantics (understood to mean 
local filesystem semantics) or it doesn't, there is no "stricter".  The 
filesystems listed all do attempt to achieve local filesystem semantics 
for a filesystem shared by a cluster (and they all rely on shared disk 
access).

> and therefore view themselves as 
> "cluster" rather than "network" filesystems (an odd distinction ... 
> why shouldn't a network filesystem simply consider "cluster" in
> effect a mount option which would optimize for higher performance to
> nearby hosts in the cluster and stricter POSIX file semantics rather
> than relaxed "nfs file semantics").

Whoops, we've gotten off track mark here.  I think the author may have 
confused "cluster" with "distributed" filesystems (Coda, AFS) which do 
have a notion of "nearby", and support disconnected operation.  Cluster 
filesystems do not support disconnected operation: a disconnected 
machine can't see the shared filesystem at all, and any data it may 
have cached is invalid.

In return for giving up disconnected operation, cluster filesystems 
offer inexpensive scaling of bandwidth and compute power, and they 
offer local filesystem semantics.

> If they had a good standards 
> story with the IETF and were inkernel in 2.6, perhaps no one would
> care,

I'm not sure what he means.  The only standard that matters to a cluster 
filesystem is Posix/SUS.  Perhaps he is thinking about operating a 
shared filesystem over the internet, which is explicitly not a goal of 
any cluster filesystem that I know of.

> but it seems odd - when you can make AFS or CIFS or NFSv4 do 
> the same with rather more trivial changes.

Ahem. the task of adding local filesystem semantics to any existing 
network filesystem is far from trivial.  Also, these are all really 
stacking filesystems: each of them exports portions of a local 
filesystem.

A cluster filesystem is not stacked: it directly accesses a shared 
storage device without using some other filesystem and an intermediary.  
(Lustre actually uses a modified Ext3 on the storage device, however, 
this intermediate filesystem is used only to keep track of data blocks 
and does not provide namespace operations.)

So, cluster and network filesystems are very different animals.

> Somehow I think that the above doesn't quite capture what GFS is all
> about.

Indeed.  The importance of GFS to Samba is that you can stack a 
clustered Samba on top of it, though as we have discussed, GFS will 
need some hacking to support things like oplocks and weirdo Windows 
naming semantics.  Have we gotten to the point of discussing exactly 
how yet, or are we still digesting concepts?

> I'm not trying to start a flamewar, but I'd certainly like to 
> see someone provide a clearer explanation than I could do.
>
> Chris -)-----

Hope this helps, and that I haven't moosed anything up :-)

Regards,

Daniel