[Linux-cluster] NFSv4 with ACL-Support on GFS

[Marc Grimme]

On Monday 16 April 2007 17:05:59 Christian Brandes wrote:
> Hi Marc and Bob,
>
> >>> I would like to set up an active/active cluster of redundant NFSv4
> >>> servers with ACLs that have the same GFS file system exported at the
> >>> same time.
> >>>
> >>> At the moment I have a two node cluster of test servers with a GFS
> >>> exported over NFSv4, but I can not get or set ACLs with getfacl or
> >>> setfacl, which is the same behavior I had with samba.
> >>>
> >>> Some weeks ago I tried two SAMBA servers on GFS, though it was said not
> >>> to work in the Cluster FAQ.
> >>> It seemed to work -- but no ACLs.
> >>
> >> 1. Are you mounting gfs on the nfs server with the "-o acl" option?
> >> 2. Can you do setfacl and getfacl from the gfs host (i.e. not through
> >> nfs)? 3. On what version of the cluster software and gfs are you seeing
> >> this?
>
> Sorry for the late answers:
>
> 1. Yes, /gfs is mounted with "-o acl" but not the / filesystem and that
> is the point.
> I was not aware that it is not sufficient to mount the filesystem itself
> "-o acl" but also the root fs.
> But if you do it works with NFSv4 as well as with Samba.
> Now that I found out I took my Samba setup from some weeks ago and setup
> a Samba PDC and two BDCs that offer the same GFS at a time and I am very
> happy that it works. Finally I configured an IP-Cluster-Resource that is
> failed over and use that to mount SMB-shares e.g. home directories on
> SMB clients.
>
> 2. Yes, I can now and I could before / was "-o acl", too.
>
> > Hi Christian,
> > Yes but samba - without having "shared shares" in a multiple writer
> > configuration - works since the beginning even with acl. You need the -o
> > acl and everything should be fine.
>
> OK, but what is the problem with "shared shares"?
My understanding of "shared shares" is as follows:
If both, server A and B "export" share C on path D. Then if client Z accesses 
share C on server A and client Y accesses share C on server A and both write 
to the same file you'll run into problems with dataconsistency for the data 
in the file. That is because samba on server A and samba on server B don't 
have cache coherrency or some kind of locking. IMHO that is also a problem 
for NFS v3/4.
> Do 2 instances of Samba on 1 server, serving the same shares cause
> problems? Why do they do on GFS with different machines
> E.g. I nerver had problems accessing a combined Samba/NFS share by SMB
> an NFS at the same time.
If you have 2 instances of samba on 1 server you might end up with the same 
problem as state above. The same for NFS?! Only a fully disabled cache and 
synchronous writing might change things or even make concurrent access more 
stable.
Regards Marc.
>
> I am still going to try out different scenarios.
>
> Thanks and best regards
> 	Christian



-- 
Gruss / Regards,

Marc Grimme
Phone: +49-89 452 3538-14
http://www.atix.de/               http://www.open-sharedroot.org/

**
ATIX - Ges. fuer Informationstechnologie und Consulting mbH
Einsteinstr. 10 - 85716 Unterschleissheim - Germany

Registergericht: Amtsgericht München
Registernummer: HRB 131682
USt.-Id.: DE209485962

Geschäftsführung: Marc Grimme, Mark Hlawatschek, Thomas Merz