[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] New fencing method



On Wed, May 21, 2008 at 01:47:20PM -0400, James Parsons wrote:
> Lon Hohberger wrote:
> >I'd recommend calling it something besides fence_snmp in the tree -
> >because other agents also use SNMP.  For example:
> >
> > fence_ethernet ?
> >
> Could you include some doc on how to use it, please? You can use one of 
> the existing agent man pages as a template.

Done and done.  I settled on fence_ifmib, since there's nothing
specific to ethernet about IF-MIB, and it could apply to many
different technologies.

Diff against today's git is attached.

-- 
Ross Vandegrift
ross kallisti us

"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
	--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
diff --git a/fence/agents/ifmib/README b/fence/agents/ifmib/README
new file mode 100644
index 0000000..3468581
--- /dev/null
+++ b/fence/agents/ifmib/README
@@ -0,0 +1,45 @@
+Intro:
+------
+This is an SNMP-based fencing agent for RHCS.  It was designed with the use-case
+of disabling ethernet ports on an iSCSI SAN, but could be used to disable any
+port on any SNMP v2c device that implementes the IF-MIB.
+
+The script requires PySNMP version 2 to be installed and working on all nodes
+in the cluster.  There are no requirements for any MIBs to be setup --- all of
+the required OIDs are hard-coded into the script.  Since the IF-MIB is an IETF
+standard, these identifiers are very widely supported and will not change.
+
+
+Typical usage:
+--------------
+To use this agen with the switch used on the iSCSI network, you'll require:
+   1) A managed switch running SNMP.
+   2) An SNMP community with write privileges.
+   3) Permission to send SNMP through any ACLs or firewalls from the nodes.
+   4) The ifIndex associated with the ports being used by the cluster nodes.
+
+Consider a three-node cluster composed of A, B, and C.  Each node has two
+network interfaces - one used for network and cluster communication, the second
+used for iSCSI traffic.  If A needs to be fenced, B and C will run this script
+to administratively disable the switchport for A's connection to the iSCSI
+storage.
+
+If you are using a single interface for cluster and iSCSI traffic, this will
+still work, but you will lose network connectivity to the fenced host.
+
+
+cluster.conf:
+-------------
+There is no GUI support for this fence agent at this time.  To use it, you will
+need something like this cluster.conf
+
+<fencedevice agent="fence_snmp" name="myswitch" comm="fencing" ipaddr="sw1"/>
+
+In a node's fencing methods, you'll include a line like this:
+
+<device name="myswitch" ifindex="43" option="off"/>
+
+This node will be fenced by disabling the port with ifIndex 43 on the host sw1.
+In SNMP speak, we set IF-MIB::ifAdminStatus.43 = down(2).
+
+
diff --git a/fence/agents/ifmib/fence_ifmib.py b/fence/agents/ifmib/fence_ifmib.py
new file mode 100644
index 0000000..3de9fc9
--- /dev/null
+++ b/fence/agents/ifmib/fence_ifmib.py
@@ -0,0 +1,214 @@
+#!/usr/bin/python
+# fence_ifmib.py: fabric fencing for RHCS based on setting a network interface
+# to admin down.  Intended to be used for iSCSI connections, can be used with
+# anything that supports the IF-MIB and SNMP v2c.
+#
+# Written by Ross Vandegrift <ross kallisti us>
+# Copyright (C) 2008 Ross Vandegrift
+#  This copyrighted material is made available to anyone wishing to use,
+#  modify, copy, or redistribute it subject to the terms and conditions
+#  of the GNU General Public License v.2.
+
+
+import os
+os.environ['PYSNMP_API_VERSION'] = 'v2'
+import sys, getopt, random, socket
+from pysnmp import role, v2c, asn1
+
+ifAdminStatus = '.1.3.6.1.2.1.2.2.1.7.'
+up = 1
+down = 2
+testing = 3
+
+def usage():
+    line = '\t%s\t%s'
+    print ''
+    print 'This script fences a node by sending a command via SNMP to set'
+    print 'ifAdminStatus to down.  It is designed to kill node access'
+    print 'to the shared storage.  It only supports SNMP v2c.'
+    print ''
+    print 'Usage: fence_ifmib [options]'
+    print line % ('-h', '\tPrint usage')
+    print line % ('-V', '\tRun verbosely')
+    print line % ('-c [private]', 'Write community string to use')
+    print line % ('-a [hostname]', 'IP/hostname of SNMP agent')
+    print line % ('-i [index]', 'ifIndex entry of the port ')
+    print line % ('-o [action]', 'One of down, up, or status')
+
+
+def vprint(v, s):
+    if v:
+        print s
+
+
+def parseargs():
+    try:
+        opt, arg = getopt.getopt (sys.argv[1:], 'hVc:v:a:i:o:')
+    except getopt.GetoptError, e:
+        print str (e)
+        usage ()
+        sys.exit (-1)
+
+    comm = ipaddr = ifindex = option = verbose = None
+
+    for o, a in opt:
+        if o == '-h':
+            usage ()
+            sys.exit (-1)
+        if o == '-V':
+            verbose = True
+        if o == '-c':
+            comm = a
+        if o == '-a':
+            ipaddr = a
+        if o == '-i':
+            try:
+                ifindex = int(a)
+            except:
+                sys.stderr.write ('fence_ifmib: ifIndex must be an integer\n')
+                usage ()
+                sys.exit (-1)
+        if o == '-o':
+            option = a
+            if option not in ('on', 'off', 'status'):
+                sys.stderr.write ('fence_ifmib: option must be one of on, off, or status\n')
+                usage ()
+                sys.exit (-1)
+
+    if comm == None or ipaddr == None or ifindex == None \
+            or option == None:
+        sts.stderr.write ('All args are madatory!\n')
+        usage ()
+        sys.exit (-1)
+
+    return (comm, ipaddr, ifindex, option, verbose)
+
+
+def parsestdin():
+    params = {}
+    for line in sys.stdin:
+        val = line.split('=')
+        if len (val) == 2:
+            params[val[0].strip ()] = val[1].strip ()
+
+    try:
+        comm = params['comm']
+    except:
+        sys.stdout.write ('fence_ifmib: Error reading community string\n')
+        sys.exit (1)
+
+    try:
+        ipaddr = params['ipaddr']
+    except:
+        sys.stdout.write ('fence_ifmib: Error reading destination IP/host\n')
+        sys.exit (1)
+
+    try:
+        ifindex = params['ifindex']
+    except:
+        sys.stdout.write ('fence_ifmib: Error reading ifindex\n')
+        sys.exit (1)
+
+    try:
+        option = params['option']
+    except:
+        option = 'off'
+
+    return (comm, ipaddr, ifindex, option)
+            
+
+def snmpget (host, comm, oid):
+    req = v2c.GETREQUEST ()
+    encoded_oids = map (asn1.OBJECTID().encode, [oid,])
+    req['community'] = comm
+    tr = role.manager ((host, 161))
+    rsp = v2c.RESPONSE ()
+    (rawrsp, src) = tr.send_and_receive (req.encode (encoded_oids=encoded_oids))
+    rsp.decode (rawrsp)
+    if rsp['error_status']:
+        raise IOError('SNMP error while reading')
+    oids = map (lambda x: x[0], map(asn1.OBJECTID ().decode, rsp['encoded_oids']))
+    vals = map (lambda x: x[0] (), map(asn1.decode, rsp['encoded_vals']))
+    return vals[0]
+
+
+def snmpset (host, comm, oid, type, value):
+    req = v2c.SETREQUEST (request_id=random.randint (1,2**16-1))
+    req['community'] = comm
+    tr = role.manager ((host, 161))
+    rsp = v2c.RESPONSE ()
+    encoded_oids = map (asn1.OBJECTID ().encode, [oid,])
+    encoded_vals = []
+    encoded_vals.append (eval ('asn1.' + type + '()').encode (value))
+    (rawrsp, src) = tr.send_and_receive (req.encode (encoded_oids=encoded_oids, encoded_vals=encoded_vals))
+    rsp.decode(rawrsp)
+    if rsp['error_status']:
+        raise IOError('SNMP error while setting')
+    oids = map (lambda x: x[0], map (asn1.OBJECTID().decode, rsp['encoded_oids']))
+    vals = map (lambda x: x[0] (), map (asn1.decode, rsp['encoded_vals']))
+    if vals[0] == value:
+        return vals[0]
+    else:
+        raise IOError('SNMP error while setting')
+
+
+def main():
+    if len (sys.argv) > 1:
+        (comm, host, index, option, verbose) = parseargs ()
+    else:
+        verbose = False
+        (comm, host, index, option) = parsestdin ()
+
+    try:
+        switch = socket.gethostbyname (host)
+    except socket.gaierror, err:
+        vprint (verbose, 'fence_ifmib: %s' % str (err[1]))
+        sys.exit(1)
+
+    if option == 'on':
+        value = up
+    elif option == 'off':
+        value = down
+    elif option == 'status':
+        value = None
+
+    if value:
+        # For option in (on, off) - write and verify
+        try:
+            r = snmpset (switch, comm, ifAdminStatus + str (index), 'INTEGER', value)
+        except:
+            sys.stderr.write ('fence_ifmib: Error during snmp write\n')
+            sys.exit (1)
+        
+        try:
+            s = int (snmpget (switch, comm, ifAdminStatus + str (index)))
+        except:
+            sys.stderr.write ('fence_ifmib: Error during fence verification\n')
+            sys.exit (1)
+
+        if s == value:
+            vprint (verbose, 'fence_ifmib: action %s sucessful' % option)
+            sys.exit (0)
+        else:
+            vprint (verbose, 'fence_ifmib: action %s failed' % option)
+            sys.exit (1)
+    else: # status
+        try: 
+            r = int (snmpget (switch, comm, ifAdminStatus + str (index)))
+        except:
+            sys.stderr.write ('fence_ifmib: Error during snmp read\n')
+            sys.exit (1)
+
+        if r == up:
+            vprint (verbose, 'fence_ifmib: Port is admin up')
+            sys.exit (0)
+        elif r == down:
+            vprint (verbose, 'fence_ifmib: Port is admin down')
+            sys.exit (2)
+        elif r == testing:
+            vprint (verbose, 'fence_ifmib: Port is admin testing')
+            sys.exit (2)
+
+
+if __name__ == '__main__':
+    main()
diff --git a/fence/man/fence_ifmib.8 b/fence/man/fence_ifmib.8
new file mode 100644
index 0000000..5b66aad
--- /dev/null
+++ b/fence/man/fence_ifmib.8
@@ -0,0 +1,69 @@
+.\"  Copyright (C) 2006-2007 Red Hat, Inc.  All rights reserved.
+.\"  
+.\"  This copyrighted material is made available to anyone wishing to use,
+.\"  modify, copy, or redistribute it subject to the terms and conditions
+.\"  of the GNU General Public License v.2.
+
+.TH fence_ifmib 8
+
+.SH NAME
+fence_ifmib - I/O Fencing agent for IF-MIB capable SNMP devices
+
+.SH SYNOPSIS
+.B 
+fence_ifmib
+[\fIOPTION\fR]...
+
+.SH DESCRIPTION
+fence_ifmib is an I/O Fencing agent which can be used with any IF-MIB capable
+SNMP device.  It was written with managed ethernet switches in mind, in order
+to fence iSCSI SAN connections.  However, there are many devices that support
+the IF-MIB interface.  The agent uses IF-MIB::ifAdminStatus to control the
+state of an interface.
+
+fence_ifmib accepts options on the command line as well as from stdin.  
+Fenced sends parameters through stdin when it execs the agent.  fence_ifmib 
+can be run by itself with command line options.  This is useful for testing.
+
+.SH OPTIONS
+.TP
+\fB-a\fP \fIIPaddress\fR
+IP address or hostname of the SNMP agent to be written.
+.TP
+\fB-h\fP 
+Print out a help message describing available options, then exit.
+.TP
+\fB-c\fP \fIcommunity\fR
+The write community string to be used in the request.
+.TP
+\fB-i\fP \fIiIindex\fR
+The ifIndex of the interface to be acted upon.  This will need to be determined
+manually prior to configuration.
+.TP
+\fB-o\fP \fIaction\fR
+The action required.  off (default), on, or status.  off sets ifAdminStatus
+down, on sets ifAdminStatus up, and status returns the current state.
+.TP
+\fB-V\fP
+Verbose.  Print informational messages to standard out.
+
+.SH STDIN PARAMETERS
+.TP
+\fIagent = < param >\fR
+This option is used by fence_node(8) and is ignored by fence_ifmib.
+.TP
+\fIipaddr = < hostname | ip >\fR
+IP address or hostname of the device.
+.TP
+\fIcomm = < param >\fR
+Write community string to be used in the request.
+.TP
+\fIifindex = < param >\fR
+The ifIndex of the interface to be acted upon.
+.TP
+\fIoption = < param >\fR
+The action required.  off (default), on, or status.
+.TP
+
+.SH SEE ALSO
+fence(8), fence_node(8)

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]