[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] Homebrew NAS Cluster

My setup sounds similar to yours but with a SAN for all the underlying storage.

I have a large FC SAN (might be cost prohibitive for you), and three physical (Dell PE1500s) servers. Two of them are running ESX 3.5 and one is running CentOS. The ESX Servers share a chunk of SAN using VMFS3. The rest of the san is shared by all three physical servers. I have a handful of virtual CentOS servers to which I've installed the shared SAN luns via raw device mapping (with the scsi bus' set in physical sharing mode).

I then put the physical and virtual CentOS machines in one GFS cluster to share the san (using a custom fence script). While this all works and is in production, the performance isn't what I'd like. Locking calls by the virtual centos machines really slow things down, especially when running samba on a vm. I think it's the nature of GFS being exacerbated by all the abstraction of ESX. It takes quite a bit of tuning.

The biggest caveat for ESX users is that putting a virtual machine's scsi bus in physical shared-bus mode, disables DRS and VMotion. You can't live migrate these machines. The HA feature still works well though.

Andrew A. Neuschwander, RHCE
Linux Systems/Software Engineer
College of Forestry and Conservation
The University of Montana
andrew ntsg umt edu - 406.243.6310

Stephen Nelson-Smith wrote:
Hi all,

I'm in the process of setting up a virtualisation farm which will have
50-60 virtual machines, running a wide range of web, application and
database applications, all on top of vmware vi3.

My budget won't stretch to a commercial NAS solution, so it's either a
SAN, which could get complicated and hard to manage with so many
nodes, or a home-brew NAS solution.

Has anyone done this, on the list?  I'm wondering what the catch is?
I'm thinking all I need to do is run NFS on top of a clustered
filesystem, and export to ESX.

I could use some pointers, gotchas, ideas and experiences.



Linux-cluster mailing list
Linux-cluster redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]