[libvirt] [test-API] RFC: Stabilization of libvirt-test-API
Guannan Ren
gren at redhat.com
Thu Apr 5 08:28:38 UTC 2012
On 04/04/2012 11:09 PM, Laine Stump wrote:
> On 03/29/2012 08:14 AM, Martin Kletzander wrote:
>> So here are the things I would like to do definitely (the optional
>> things follow later on):
>> - fix hard-coded options into real options (e.g. commit 65449e)
> Forgive me if I'm suggesting things that already exist - I haven't had
> time to go through all of the test-API code, but want to make sure this
> gets brought up sooner rather than later.
>
> Two things that I think are very important for a general purpose test
> setup are:
>
> 1) The ability to have multiple physical machines, network devices of
> various types (bridges, standard NICs, sr-iov NICs) in the test harness
> available via standard names (implying, of course, that the
> address/capabilities (and even the existence/non-existence) of at least
> one other machine be configurable in a global config file referenced by
> every test).
The env.cfg is the global config file that lists the
default data needed by testcases.
we could expand it for various requirements. The testsuit
neither manage these devices
or machines nor check if it is spelling-correct or present
on testing machine right now.
because it depends on testing environment , the tester
should check it before writing it in
env.cfg.
<snip>
##########################
#
# PCI device
# a PIC device to use for attach/detach/reset tests
# for example testpic = 00:19.0
testpic =
</snip>
>
> 2) The ability to mark some tests as requiring certain standard objects
> from the global configuration (e.g. the second machine, a bridge
> interface, sr-iov NICs) and to either skip the test when some required
> object is missing, or fail the test (the test itself would be marked
> either OPTIONAL or REQUIRED).
Currently, there is a file named BUGSKIP in the root of test-API
If certain testcase is written in it for a bug related
reason, the testcase
will be skipped during running time, we could improve it for
your purpose.
There are 70~80 testcases. Any number of this testcases
could be combined
in different order according to various testing
requirements. So there is not one
shaped running model to go through all of testcases(we could
have a basic one).
there are some testcase file that are mostly used in "cases"
folder in the root directory.
>
> This would allow us to have, for example, full testing of networking
> capabilities present in the test suite, but without raising an entry
> barrier for "random Joe" who only has a single machine, one NIC, etc.,
> but wants to run the test suite. When Joe ran the tests, each test that
> required multiple hosts (or sr-iov NICs or whatever esoteric piece of
> hardware/driver) and was designated as an "OPTIONAL" test, would be
> semi-silently skipped, but someone with a full complement of hardware
> who demanded a thorough and complete test could set a switch and
> guarantee that every single test would be run (or a failure saying, e.g.
> "object REMOTE_HOST required by this test is missing in config", or
> something like that).
The current data list in env.cfg is default and satisfied for
all of testcases. "random Joe" have to know which testcases he
want to
run, if he need the testcase which rely on two NICs , he need to
write MAC address of NICs to env.cfg before testing.
>
> Is there currently an allowance for these two items?
>
> --
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
More information about the libvir-list
mailing list