[Linux-cluster] Cluster Suite v3 quorum devices over iSCSI

[Celso K. Webber]

Hello all,

I've configured a new cluster for a customer with the following details:
* using Red Hat Enterprise Linux ES v3 Update 6 (cannot update to U7 because
Adaptec's SAS controller driver officialy supports only RHEL3U6)
* using Cluster Suite v3U6 (latest packages, clumanager-1.2.28-1)
* 2-node cluster with IBM xSeries 366 servers
* storage is a NetApp FAS270c
* shared state (quorum) managed over an iSCSI LUN on the NetApp
* using Red Hat's latest release of iscsi-initiator-utils (v 3.6.3-3)
* storage volumes for services accessed through NFS on the NetApp
* access to the storage done on a separate VLAN with Linux 802.3ad channel
bonding configured, using Cisco switches with port configured for LACP link
aggregation
* NetApp also configured for access in the network using link aggregation

We had many problems with quorum formation under this environment. If we
booted both servers at the same time, quorum would be formed successfully. If
after that we rebooted one of the servers, it would not rejoin the cluster
until we stopped clumanager on both sides and restarted.

After a long interaction with an iSCSI and NetApp specialist, we could find
some references of a problem with iSCSI that required Cluser Suite v3 to use
broadcast heartbeating instead of multicast heartbeating:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=136553

Although the above Bugzilla is apparently resolved, we followed the directions
of changing to broadcast heartbeating and it solved our problem:

"Comment #4 From Lon Hohberger  	 on 2005-01-10 13:00 EST  	[reply]  	   	 

I'm pretty sure this is a duplicate of the following bug:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=114653

Basically, clumanager 1.2.9 reboots if it fails to join a multicast
group after a few seconds.  You should be able to fix this by changing
clumanager to use 'broadcast' instead of multicast heartbeating in the
GUI."


Question: is there a limitation or need to use broadcast heartbeating under an
iSCSI environment? Why multicast heartbeating didn't work here?


Thank you very much,

Celso.

--
Celso K. Webber <celso at webbertek.com.br>
(41) 3284-3035
(41) 8813-1919