Re: [libvirt] testsuite formatting bugs [was: [PATCH] Skip some xen tests if xend is not running]

[Matthias Bolte <matthias bolte googlemail com>]

2011/7/8 Eric Blake <eblake redhat com>:
> On 07/07/2011 04:21 PM, Eric Blake wrote:
>> Oh, and our testsuite has a cosmetic bug.  After applying your patch, I
>> see this during 'make check':
>>
>> TEST: xencapstest
>>       ..........                               10  OK
>> PASS: xencapstest
>> SKIP: reconnect
>> TEST: statstest
>>        0   FAIL
>> SKIP: statstest
>>
>> Bonus points for fixing up that output to say SKIP instead of FAIL and
>> to align it correctly (but that can be a separate patch).
>
> As long as we're investigating formatting errors, this one is also annoying:
>
> TEST: virsh-all
>      ........................................ 40
>      ........................................ 80
>      ........................................ 120
>      ....................................... 159 OK
> PASS: virsh-all
>
> We're obviously getting the logic wrong when there are 0 or when
> tests%40 == 39.

Here are two patches for this, plus one to use EXIT_AM_SKIP more.

-- 
Matthias Bolte
http://photron.blogspot.com

From cb39f79417dce294c654435c1a783065b4983adb Mon Sep 17 00:00:00 2001
From: Matthias Bolte <matthias bolte googlemail com>
Date: Sat, 9 Jul 2011 01:20:05 +0200
Subject: [PATCH] tests: Fix compressed test output padding logic

The current logic tries to count from 1 to 40 and ignores paddings
of 0 and 1 to 40. This doesn't work for counter + 1 mod 40 == 0 like
here for counter value 159

TEST: virsh-all
      ........................................ 40
      ........................................ 80
      ........................................ 120
      ....................................... 159 OK
PASS: virsh-all

Instead count from 0 to 39 to fix this.
---
 tests/test-lib.sh |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/test-lib.sh b/tests/test-lib.sh
index 768f96b..9eb6864 100644
--- a/tests/test-lib.sh
+++ b/tests/test-lib.sh
@@ -54,9 +54,9 @@ test_final()
   status=$2
 
   if test "$verbose" = "0" ; then
-    mod=`expr \( $counter + 1 \) % 40`
-    if test "$mod" != "0" && test "$mod" != "1" ; then
-      for i in `seq $mod 40`
+    mod=`expr $counter % 40`
+    if test "$mod" != "0" ; then
+      for i in `seq $mod 39`
       do
         printf " "
       done
-- 
1.7.4.1

From a1508239af921289cd6e357e8521ff42faf535bd Mon Sep 17 00:00:00 2001
From: Matthias Bolte <matthias bolte googlemail com>
Date: Sat, 9 Jul 2011 01:24:16 +0200
Subject: [PATCH] tests: Add the logic to skip the statstest to the right place

---
 tests/statstest.c |   31 ++++++++++++++++++++-----------
 1 files changed, 20 insertions(+), 11 deletions(-)

diff --git a/tests/statstest.c b/tests/statstest.c
index 18cdc5c..ebf9726 100644
--- a/tests/statstest.c
+++ b/tests/statstest.c
@@ -45,16 +45,6 @@ static int
 mymain(void)
 {
     int ret = 0;
-    int status;
-    virCommandPtr cmd;
-
-    /* skip test if xend is not running */
-    cmd = virCommandNewArgList("/usr/sbin/xend", "status", NULL);
-    if (virCommandRun(cmd, &status) != 0 || status != 0) {
-        virCommandFree(cmd);
-        return EXIT_AM_SKIP;
-    }
-    virCommandFree(cmd);
 
     /* Some of our tests deliberately test failure cases, so
      * register a handler to stop error messages cluttering
@@ -209,4 +199,23 @@ mymain(void)
     return(ret==0 ? EXIT_SUCCESS : EXIT_FAILURE);
 }
 
-VIRT_TEST_MAIN(mymain)
+/* Skipping the test in mymain is too late, it results in broken output.
+ * Therefore, expand VIRT_TEST_MAIN here manually to be able to skip at
+ * the right place. */
+int main(int argc, char **argv)
+{
+    int status;
+    virCommandPtr cmd;
+
+    /* skip test if xend is not running */
+    cmd = virCommandNewArgList("/usr/sbin/xend", "status", NULL);
+
+    if (virCommandRun(cmd, &status) != 0 || status != 0) {
+        virCommandFree(cmd);
+        return EXIT_AM_SKIP;
+    }
+
+    virCommandFree(cmd);
+
+    return virtTestMain(argc, argv, mymain);
+}
-- 
1.7.4.1

From 79e8a7e876722a67c37d930762e7a8b32701c6ca Mon Sep 17 00:00:00 2001
From: Matthias Bolte <matthias bolte googlemail com>
Date: Sat, 9 Jul 2011 01:24:44 +0200
Subject: [PATCH] tests: Use EXIT_AM_SKIP instead of 77 directly

---
 tests/esxutilstest.c     |    4 ++--
 tests/qemuhelptest.c     |    5 ++++-
 tests/qemuxml2argvtest.c |    5 ++++-
 tests/vmx2xmltest.c      |    4 ++--
 tests/xml2vmxtest.c      |    4 ++--
 5 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/tests/esxutilstest.c b/tests/esxutilstest.c
index 2097648..8241ecd 100644
--- a/tests/esxutilstest.c
+++ b/tests/esxutilstest.c
@@ -280,9 +280,9 @@ VIRT_TEST_MAIN(mymain)
 
 #else
 
-int main (void)
+int main(void)
 {
-    return 77; /* means 'test skipped' for automake */
+    return EXIT_AM_SKIP;
 }
 
 #endif /* WITH_ESX */
diff --git a/tests/qemuhelptest.c b/tests/qemuhelptest.c
index 119e771..102dd22 100644
--- a/tests/qemuhelptest.c
+++ b/tests/qemuhelptest.c
@@ -485,6 +485,9 @@ VIRT_TEST_MAIN(mymain)
 
 #else
 
-int main (void) { return (77); /* means 'test skipped' for automake */ }
+int main(void)
+{
+    return EXIT_AM_SKIP;
+}
 
 #endif /* WITH_QEMU */
diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
index ec1f4b5..f12d815 100644
--- a/tests/qemuxml2argvtest.c
+++ b/tests/qemuxml2argvtest.c
@@ -548,6 +548,9 @@ VIRT_TEST_MAIN(mymain)
 
 #else
 
-int main (void) { return (77); /* means 'test skipped' for automake */ }
+int main(void)
+{
+    return EXIT_AM_SKIP;
+}
 
 #endif /* WITH_QEMU */
diff --git a/tests/vmx2xmltest.c b/tests/vmx2xmltest.c
index 7396f39..92dc7d4 100644
--- a/tests/vmx2xmltest.c
+++ b/tests/vmx2xmltest.c
@@ -283,9 +283,9 @@ VIRT_TEST_MAIN(mymain)
 
 #else
 
-int main (void)
+int main(void)
 {
-    return 77; /* means 'test skipped' for automake */
+    return EXIT_AM_SKIP;
 }
 
 #endif /* WITH_VMX */
diff --git a/tests/xml2vmxtest.c b/tests/xml2vmxtest.c
index 66ed53d..f1e1b95 100644
--- a/tests/xml2vmxtest.c
+++ b/tests/xml2vmxtest.c
@@ -294,9 +294,9 @@ VIRT_TEST_MAIN(mymain)
 
 #else
 
-int main (void)
+int main(void)
 {
-    return 77; /* means 'test skipped' for automake */
+    return EXIT_AM_SKIP;
 }
 
 #endif /* WITH_VMX */
-- 
1.7.4.1