[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] virtualization on top of Red Hat cluster

Gunther Schlegel wrote:

Geoffrey wrote:

Before going into the actual issues we are running into, I thought we should first find out if there are others who are doing this, or attempting to do this.

This includes a separate virtual server for the following services: squid, ldap, mail server, dns, samba, email and a handful of Xservers/application servers.

The specific hardware includes:

8 Dell 19150 nodes w/ rh5.2 xen
Each node has 32GB of ram and 8 cores (xeon 2.66)

you need a 64-bit distro to have xen support more than 16GB RAM.

We do.

EMC San CX-310
2 Brocade 5000 fibre switches

Any feedback from anyone attempting or currently running a similar solution (virtualization on top of cluster) would be greatly appreciated.

We are running something like that for more than a year now (tests on rhel5.0, live on rhel5.1, now running rhel5.2).

45+ paravirtualized VMs on 2 clusters.
VMs are located on clustered logical volumes, not gfs.
gfs is used for /etc/xen, though.

Yes, we are using lvm as well.

performance is good, even with io-intensive apps inside the VMs.
stability is fair (but improving over time).

This is good to hear.

The main issue has always been the cluster loosing quorum without apparent reason. Improved after we added a Quorum Disk.

We are using Quorum disk as well.

Improved further after RH Support recommended a couple of not so well documented ;) configuration parameters, which also cannot be maintained with conga/luci/ricci or system-config-cluster. Improved even further after we complained again and RH came up with even more undocumented setting to solve the race-conditions we experienced when a node left the cluster (even intentionally).

Any possible way you can share these undocumented settings with us?

Another big issue was live migration, turned out that the bridge in Dom0 has a default forward delay of 15 seconds. We may have experienced that because we use a different xen network-script than delivered by RH. Though, the original one could not deal with neither bonding nor Vlans, so we had to do that. This has changed according to the rhel5.3 changelog, but we have not tested it yet.

We also had issues with the DomU time jumping around after live migration, but RH fixed that recently and as a workaround one could run ntpd inside the DomU.

best regards, Gunther


Linux-cluster mailing list
Linux-cluster redhat com

Until later, Geoffrey

Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety.
 - Benjamin Franklin

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]