[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Cluster-devel] [fence-virt PATCH] fix handling of VIR_DOMAIN_EVENT_STARTED



Hi all,

I am using fence-virt.
When using a serial listener, fence_virtd does not receive a request
from VM which started after fence_virtd start.

Step 1) Start fence_virtd.
         # fence_virtd -d5 -F
Step 2) And then start VM.
         # virsh start srv-a1
Step 3) A request sent from the VM,
         # fence_virt -D/dev/ttyS1 -oXXX
        but fence_virtd not receive it.

Below, x3650f is a host, srv-XX is a VM.

[root x3650f ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.1 (Santiago)
[root x3650f ~]#
[root x3650f ~]# rpm -qa | egrep '^libvirt|^qemu' | sort
libvirt-0.8.7-18.el6.x86_64
libvirt-client-0.8.7-18.el6.x86_64
libvirt-devel-0.8.7-18.el6.x86_64
libvirt-java-0.4.7-1.el6.noarch
libvirt-java-devel-0.4.7-1.el6.noarch
libvirt-python-0.8.7-18.el6.x86_64
libvirt-qpid-0.2.22-6.el6.x86_64
qemu-img-0.12.1.2-2.160.el6.x86_64
qemu-kvm-0.12.1.2-2.160.el6.x86_64
[root x3650f ~]#

And I used following fence-virt.
http://fence-virt.git.sourceforge.net/git/gitweb.cgi?p=fence-virt/fence-virt;a=commit;h=53a2b293d12991beb6c2927f78586257bff5173a

[root x3650f ~]# tar xfz fence-virt-53a2b29.tar.gz && cd fence-virt-53a2b29 && ./build
[root x3650f ~]# cd && mkdir plugins && cd plugins
[root x3650f plugins]# ln -s /root/fence-virt-53a2b29/server/serial.so .
[root x3650f plugins]# ln -s /root/fence-virt-53a2b29/server/libvirt.so .
[root x3650f plugins]# ls -l
lrwxrwxrwx 1 root root 42 Sep 13 10:28 libvirt.so -> /root/fence-virt-53a2b29/server/libvirt.so
lrwxrwxrwx 1 root root 41 Sep 13 10:28 serial.so -> /root/fence-virt-53a2b29/server/serial.so
[root x3650f plugins]#

I attached /etc/fence_virt.conf and /etc/libvirt/qemu/srv-a1.xml,
please refer to it.

Step 1)
  Start fence_virtd.

  [root x3650f ~]# ./fence-virt-53a2b29/server/fence_virtd -d5 -F
  Background mode disabled
  Debugging threshold is now 5
  (snip)
  Backend plugin: libvirt
  Listener plugin: serial
  Searching /root/plugins for plugins...
  Searching for plugins in /root/plugins
  Loading plugin from /root/plugins/serial.so
  Failed to map backend_plugin_version
  Registered listener plugin serial 0.4
  Loading plugin from /root/plugins/libvirt.so
  Registered backend plugin libvirt 0.1
  2 plugins found
  Available backends:
      libvirt 0.1
  Available listeners:
      serial 0.4
  Debugging threshold is now 5
  Using qemu:///system
  Debugging threshold is now 5
  Got /var/lib/libvirt/qemu for uri
  Got serial for mode
  Libvirt event listener starting
   * Socket path: /var/lib/libvirt/qemu
   * Mode: Serial
  myEventAddHandleFunc:128: Add handle 6 1 0x37558aef80 0x7f68cc000920 (nil)
  myEventAddTimeoutFunc:160: Adding Timeout -1 0x37558a7f90 0x7f68cc000920
  event_thread:448 :: Registering domain event cbs
  myEventUpdateHandleFunc:140: Updated Handle 0 0
  myEventUpdateHandleFunc:140: Updated Handle 0 1
  myEventUpdateHandleFunc:140: Updated Handle 0 0
  myEventUpdateHandleFunc:140: Updated Handle 0 1
                                                   <- Step 2) I started VM here.
  myEventUpdateHandleFunc:140: Updated Handle 0 0
  myEventUpdateHandleFunc:140: Updated Handle 0 1
  Registered 621d8c61-1070-7aab-6158-3889d68470ab on 9
                                                   <- Step 3) I run fence_virt here.
  (snip)

Step 3)
  A request sent from the VM, but fence_virtd not receive it.

  [root srv-a1 ~]# ./fence-virt-53a2b29/client/fence_virt -D/dev/ttyS1 -olist
  Unknown response (255)
  [root srv-a1 ~]#

I attached the patch for this problem.

* Modified to send a SIGHUP to the primary thread from the secondary
  thread detects the start of the VM, and
  the primary thread is modified so as to select() the requests from
  the VM which started.
* Also, when four VM(s) are started simultaneously,
  poll() <in event_thread() at server/virt-serial.c> could not handle
  all the boot.
  Therefore, I revised it to call domainStarted()/domainStopped() in
  myDomainEventCallback1().

Regards,
Kazunori INOUE
diff -urN fence-virt-53a2b29/server/daemon_init.c mod/server/daemon_init.c
--- fence-virt-53a2b29/server/daemon_init.c	2011-08-11 09:53:47.000000000 +0900
+++ mod/server/daemon_init.c	2011-09-12 11:12:12.000000000 +0900
@@ -193,6 +193,8 @@
 	sigdelset(&set, SIGQUIT);
 	sigdelset(&set, SIGCHLD);
 
+	sigdelset(&set, SIGHUP);
+
 	return (sigprocmask(SIG_BLOCK, &set, NULL));
 }
 
diff -urN fence-virt-53a2b29/server/main.c mod/server/main.c
--- fence-virt-53a2b29/server/main.c	2011-08-11 09:53:47.000000000 +0900
+++ mod/server/main.c	2011-09-12 11:14:33.000000000 +0900
@@ -36,7 +36,8 @@
 void
 exit_handler(int sig)
 {
-	run = 0;
+	if (sig != SIGHUP)
+		run = 0;
 }
 
 
@@ -188,6 +189,7 @@
 	signal(SIGINT, exit_handler);
 	signal(SIGTERM, exit_handler);
 	signal(SIGQUIT, exit_handler);
+	signal(SIGHUP, exit_handler);
 
 	while (p->init(&backend_ctx, config) < 0) {
 		if (!wait_for_backend) {
diff -urN fence-virt-53a2b29/server/serial.c mod/server/serial.c
--- fence-virt-53a2b29/server/serial.c	2011-08-11 09:53:47.000000000 +0900
+++ mod/server/serial.c	2011-09-12 12:01:17.000000000 +0900
@@ -266,6 +266,8 @@
 
 	n = select(max+1, &rfds, NULL, NULL, timeout);
 	if (n < 0) {
+		if (errno == EINTR)
+			return 0;
 		perror("select");
 		return n;
 	}
diff -urN fence-virt-53a2b29/server/virt-serial.c mod/server/virt-serial.c
--- fence-virt-53a2b29/server/virt-serial.c	2011-08-11 09:53:47.000000000 +0900
+++ mod/server/virt-serial.c	2011-09-12 11:10:58.000000000 +0900
@@ -69,25 +69,6 @@
 };
 
 
-int
-myDomainEventCallback1(virConnectPtr conn,
-		       virDomainPtr dom, int event, int detail, void *opaque)
-{
-	struct domain_info *dinfo = (struct domain_info *) opaque;
-
-	if (event == VIR_DOMAIN_EVENT_STARTED ||
-	    event == VIR_DOMAIN_EVENT_STOPPED) {
-		virDomainRef(dom);
-		dinfo->dom = dom;
-		dinfo->event = event;
-	} else {
-		dinfo->event = VIR_DOMAIN_EVENT_UNDEFINED;
-	}
-
-	return 0;
-}
-
-
 /* EventImpl Functions */
 int
 myEventHandleTypeToPollEvent(virEventHandleType events)
@@ -413,15 +394,38 @@
 	char *uri;
 	char *path;
 	int mode;
+	pthread_t p_tid;
 };
 
 
+int
+myDomainEventCallback1(virConnectPtr conn,
+		       virDomainPtr dom, int event, int detail, void *opaque)
+{
+	struct event_args *args = (struct event_args *)opaque;
+
+	if (event == VIR_DOMAIN_EVENT_STARTED ||
+	    event == VIR_DOMAIN_EVENT_STOPPED) {
+		virDomainRef(dom);
+		if (event == VIR_DOMAIN_EVENT_STARTED) {
+			domainStarted(dom, args->path, args->mode);
+			virDomainFree(dom);
+			pthread_kill(args->p_tid, SIGHUP);
+		} else if (event == VIR_DOMAIN_EVENT_STOPPED) {
+			domainStopped(dom);
+			virDomainFree(dom);
+		}
+	}
+
+	return 0;
+}
+
+
 static void *
 event_thread(void *arg)
 {
 	struct event_args *args = (struct event_args *)arg;
 	virConnectPtr dconn = NULL;
-	struct domain_info dinfo;
 	int callback1ret = -1;
 	int sts;
 
@@ -450,11 +454,9 @@
 	registerExisting(dconn, args->path, args->mode);
 
 	/* Add 2 callbacks to prove this works with more than just one */
-	memset(&dinfo, 0, sizeof (dinfo));
-	dinfo.event = VIR_DOMAIN_EVENT_UNDEFINED;
 	callback1ret =
 	    virConnectDomainEventRegister(dconn, myDomainEventCallback1,
-					  &dinfo, NULL);
+					  arg, NULL);
 
 	if ((callback1ret == 0)) {
 		while (run) {
@@ -469,18 +471,6 @@
 				t_cb(t_timeout, t_opaque);
 			}
 
-			if (dinfo.event == VIR_DOMAIN_EVENT_STARTED) {
-				domainStarted(dinfo.dom, args->path, args->mode);
-				virDomainFree(dinfo.dom);
-				dinfo.dom = NULL;
-				dinfo.event = VIR_DOMAIN_EVENT_UNDEFINED;
-			} else if (dinfo.event == VIR_DOMAIN_EVENT_STOPPED) {
-				domainStopped(dinfo.dom);
-				virDomainFree(dinfo.dom);
-				dinfo.dom = NULL;
-				dinfo.event = VIR_DOMAIN_EVENT_UNDEFINED;
-			}
-
 			if (sts == 0) {
 				/* DEBUG0("Poll timeout"); */
 				continue;
@@ -546,6 +536,7 @@
 	}
 
 	args->mode = mode;
+	args->p_tid = pthread_self();
 	run = 1;
 
 	return pthread_create(&event_tid, NULL, event_thread, args);
fence_virtd {
	listener = "serial";
	backend = "libvirt";
	module_path = "/root/plugins-53a2b29";
}

listeners {
	serial {
		mode = "serial";
		path = "/var/lib/libvirt/qemu";
	}
}

backends {
	libvirt {
		uri = "qemu:///system";
	}
}

groups {
	group {
		uuid = "srv-a1";
		uuid = "srv-a2";
		uuid = "621d8c61-1070-7aab-6158-3889d68470ab";
		uuid = "4f0ed127-c8a3-4085-a0d0-a93231924202";
	}
        group {
		uuid = "srv-b1";
		uuid = "srv-b2";
		uuid = "d55103a7-eea7-ad9e-3ee9-8aa660afbe8d";
		uuid = "7243da51-2bda-413e-f172-4ed8cfee4297";
	}
}
<domain type='kvm'>
  <name>srv-a1</name>
  <uuid>621d8c61-1070-7aab-6158-3889d68470ab</uuid>
  <memory>2097152</memory>
  <currentMemory>2097152</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/guest1/srv-a1'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:ab:a1:01'/>
      <source bridge='br1'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:ab:a1:02'/>
      <source bridge='br2'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:ab:a1:03'/>
      <source bridge='br3'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:ab:a1:04'/>
      <source bridge='br4'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x10' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <serial type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/srva1'/>
      <target port='1'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]