[libvirt] Building on Solaris 11 Express

Sun May 29 10:34:11 UTC 2011

2011/5/29 Richard Laager <rlaager at wiktel.com>:
> I would really like to use ZFS's zvols as storage for my guests. I
> really like ZFS's snapshots, send/receive, and checksumming features.
> (Compression and deduplication might be useful as well.)
>
> I'm roughly about half-way done writing a ZFS storage backend for
> libvirt. It behaves like the LVM backend. I've been writing it on Linux.
> However, ZFS on Linux is not yet production-ready.

Interesting :)

> So I tried building libvirt on Solaris 11 Express. The following
> outlines the trouble (and successes) I've had so far.

I assume your building from up-to-date git here?

> First, I had to prune down the features being built:
>
>        I added --without-xen to deal with the following. I wasn't
>        intending on using Xen, so this doesn't bother me personally.
>                configure: error: You must install the Xen development package to compile Xen driver with -lxenstore

That's strange, configure should automatically disable the Xen driver
if it can't find the required headers and libs. The only way I can see
from the logic in configure is that configure found libxenstore but
can't find the headers such as xen/xen.h xen/version.h xen/dom0_ops.h
xen/sys/privcmd.h xen/linux/privcmd.h.

You can check config.log for the results of those checks. On my Linux
system it looks like this:

configure:46331: checking for xs_read in -lxenstore
configure:46356: gcc -std=gnu99 -o conftest -g -ggdb -O0   conftest.c
-lxenstore  -ldl   >&5
configure:46356: $? = 0
configure:46365: result: yes
configure:46387: checking for xen/xen.h
configure:46387: gcc -std=gnu99 -c -g -ggdb -O0   conftest.c >&5
configure:46387: $? = 0
configure:46387: result: yes
configure:46387: checking for xen/version.h
configure:46387: gcc -std=gnu99 -c -g -ggdb -O0   conftest.c >&5
configure:46387: $? = 0
configure:46387: result: yes
configure:46387: checking for xen/dom0_ops.h
configure:46387: gcc -std=gnu99 -c -g -ggdb -O0   conftest.c >&5
configure:46387: $? = 0
configure:46387: result: yes
configure:46415: checking for xen/sys/privcmd.h
configure:46415: gcc -std=gnu99 -c -g -ggdb -O0   conftest.c >&5
configure:46415: $? = 0
configure:46415: result: yes
configure:46477: checking sys/inotify.h usability
configure:46477: gcc -std=gnu99 -c -g -ggdb -O0  conftest.c >&5
configure:46477: $? = 0
configure:46477: result: yes
configure:46477: checking sys/inotify.h presence
configure:46477: gcc -E  conftest.c
configure:46477: $? = 0
configure:46477: result: yes

configure found all libs and headers. In you case it probably found
libxenstore but at least one of the headers is missing, so one of the
results should be "no" in your case.

>        I added --without-openvz to deal with the following. Likewise,
>        I'm not using it.
>                  CC     libvirt_driver_openvz_la-openvz_driver.lo
>                openvz/openvz_driver.c:45:19: paths.h: No such file or directory
>                In file included from ../src/conf/domain_conf.h:40,
>                                 from openvz/openvz_conf.h:33,
>                                 from openvz/openvz_driver.c:56:
>                ../src/conf/nwfilter_conf.h:521: warning: parameter has incomplete type

configure should disable the OpenVZ driver on Solaris anyway, because
OpenVZ is Linux specific. Also paths.h isn't used in here anyway and
could be removed. I'll post patches for this.

>        I added --without-network to deal with the following. I may need
>        to actually fix this at some point.
>                  CC     libvirt_util_la-bridge.lo
>                util/bridge.c:39:20: paths.h: No such file or directory
>                util/bridge.c:42:55: linux/param.h: No such file or directory
>                util/bridge.c:43:55: linux/sockios.h: No such file or directory
>                util/bridge.c:44:55: linux/if_bridge.h: No such file or directory
>                util/bridge.c:45:55: linux/if_tun.h: No such file or directory
>                util/bridge.c: In function `ifSetInterfaceMac':
>                util/bridge.c:313: error: `SIOCGIFHWADDR' undeclared (first use in this function)
>                util/bridge.c:313: error: (Each undeclared identifier is reported only once
>                util/bridge.c:313: error: for each function it appears in.)
>                util/bridge.c:316: error: structure has no member named `ifr_hwaddr'
>                util/bridge.c:318: error: `SIOCSIFHWADDR' undeclared (first use in this function)
>                util/bridge.c: In function `ifGetMtu':
>                util/bridge.c:347: error: `SIOCGIFMTU' undeclared (first use in this function)
>                util/bridge.c: In function `ifSetMtu':
>                util/bridge.c:378: error: `SIOCSIFMTU' undeclared (first use in this function)
>                util/bridge.c: In function `brAddTap':
>                util/bridge.c:501: error: `IFF_TAP' undeclared (first use in this function)
>                util/bridge.c:501: error: `IFF_NO_PI' undeclared (first use in this function)
>                util/bridge.c:515: error: `TUNSETIFF' undeclared (first use in this function)
>                util/bridge.c:537: error: `TUNSETPERSIST' undeclared (first use in this function)
>                util/bridge.c: In function `brDeleteTap':
>                util/bridge.c:567: error: `IFF_TAP' undeclared (first use in this function)
>                util/bridge.c:567: error: `IFF_NO_PI' undeclared (first use in this function)
>                util/bridge.c:574: error: `TUNSETIFF' undeclared (first use in this function)
>                util/bridge.c:575: error: `TUNSETPERSIST' undeclared (first use in this function)
>                util/bridge.c: In function `brSetInterfaceUp':
>                util/bridge.c:612: error: `SIOCGIFFLAGS' undeclared (first use in this function)
>                util/bridge.c:620: error: `SIOCSIFFLAGS' undeclared (first use in this function)
>                util/bridge.c: In function `brGetInterfaceUp':
>                util/bridge.c:652: error: `SIOCGIFFLAGS' undeclared (first use in this function)

libvirt's virtual network is Linux specific but configure doesn't
disable it on non-Linux systems.

> Then, I had to fix a trivial code error. I've mailed the patch to this
> list with a subject of "[PATCH] Fix virExecWithHook Prototype".

I've applied your patch.

> At this point, libvirt builds and installs. I tested virsh against a
> remote system using: virsh -c qemu+ssh://REMOTE/system
>
> This worked except for the fact that it was trying to attach
> to /usr/local/var/run/libvirt-sock on the remote system due
> to /usr/local being the --prefix used with the *local* virsh was built.
> I'm not sure how that architectural issue should be addressed. For
> testing, I just made a /usr/local/var -> /var symlink on the remote
> system.
>
>
> However, libvirtd will not start locally (on Solaris) which has pretty
> much stopped my progress. I'm sure this must be something trivial. Is it
> getting a bad prefix? Is something other than libvirtd supposed to make
> the socket file? I did try making a /.libvirt directory for testing.
>
> # rm /usr/local/var/run/libvirtd.pid ; /usr/local/sbin/libvirtd -v
> 20:13:33.663: 1: info : libvirt version: 0.9.1
> 20:13:33.663: 1: error : qemudListenUnix:605 : Failed to bind socket to
> '@//.libvirt/libvirt-sock': No such file or directory

'@//.libvirt/libvirt-sock' should actually look like this
'@/home/<username>/.libvirt/libvirt-sock' as you're running libvirtd
as non-root it tries to open a UNIX socket in the home directory of
the user starting it. This path is build via this pattern:

  @<home-directory>/.libvirt/libvirt-sock

The @ is later on replace by NUL to indicate that it should go in to
the UNIX abstract namespace.

It seems that your home directory is set to '/'. libvirt uses
getpwuid_r to get the home directory. Maybe there is a Solaris
specific problem here. To work around this you can try running
libvirtd as root. In that case the UNIX socket will be opened at
/usr/local/var/run/libvirt/libvirt-sock in your case.

Matthias