[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[libvirt-users] libvirt v1.0.2 fails to boot LXC container, but v1.0.0 works



Hello.

tl;dr = v1.0.0 can boot my LXC containers, v.1.0.1 and v.1.0.2 fails.
Paraphrased error message: "lxcContainerMountProcFuse:616 : Failed to
mount ..../meminfo"

    I'd like to know if my host is misconfigured, or my domains, or
... why 1.0.2 and 1.0.1 are not working for me.

    I've been using libvirt for a while to manage QEMU instances.  I
have experimented with lxc.  Back in October of last year, I had some
working LXC containers.  I don't recall what version of libvirt I was
using at the time.

    I recently attempted to boot my containers, and they failed
(libvirt v1.0.2).  I then reverted to v1.0.1 and tried again.  Failed
with the same result (same error text, just different line numbers).
I then reverted to 1.0.0 and my containers boot up just fine.

    My host runs Gentoo Linux on an Inter core-i5 with 16G ram.  I
regularly (weekly) apply all available Gentoo updates.

    I strongly perfer to install all third-party software (including
libvirt) from Gentoo portage, not manually from source or git.  v1.0.2
is the most recent version available.



First, items common to all test cases:

*** GCC version
ostara ~ # gcc --version
gcc (Gentoo 4.6.3 p1.11, pie-0.5.2) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


*** Linux kernel
ostara ~ # uname -a
Linux ostara 3.6.11-gentoo #4 SMP PREEMPT Sat Jan 26 10:27:55 CST 2013
x86_64 Intel(R) Core(TM) i5 CPU 760 @ 2.80GHz GenuineIntel GNU/Linux

ostara ~ # zcat /proc/config.gz | egrep "CONFIG_(CGROUP|.*_NS|NAMESPACES)"
CONFIG_CGROUPS=y
CONFIG_CGROUP_DEBUG=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_SCHED=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_NS87410 is not set


*** Gento "USE flags" when building Libvirt:
ostara ~ # equery u libvirt | xargs echo
-audit -avahi +caps -firewalld -iscsi +libvirtd +lvm +lxc +macvtap
+nfs +nls -numa -openvz -parted +pcap -phyp -policykit -python +qemu
-rbd +sasl +udev +uml +vepa +virt-network -virtualbox -xen

***  libvirt config over-rides:
ostara ~ # egrep "^\w" < /etc/libvirt/libvirtd.conf
host_uuid = "ab8c50b8-2337-4a02-9274-00923fe8f476"

ostara ~ # egrep "^\w" < /etc/libvirt/libvirt.conf

ostara ~ # egrep "^\w" < /etc/conf.d/libvirtd
rc_need="net"
LIBVIRTD_OPTS="--listen"
LIBVIRTD_KVM_SHUTDOWN="managedsave"

*** Network when no domains (qemu or libvirt) are running:
ostara ~ # ifconfig -a | grep -e "^[a-z]"
br0: flags=4355<UP,BROADCAST,PROMISC,MULTICAST>  mtu 1500
br1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 16436

*** Network with libvirt running (libvirt manages "virbr0" for my qemu
instances.  It attaches to "br0").
ostara ~ # /etc/init.d/libvirtd start
 * Starting libvirtd ...
                                                        [ ok ]

ostara ~ # ifconfig -a | grep -e "^[a-z]"
br0: flags=4355<UP,BROADCAST,PROMISC,MULTICAST>  mtu 1500
br1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 16436
virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500


*** Mounts (cgroup is mounted)
ostara ~ # mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw,relatime)
udev on /dev type devtmpfs
(rw,nosuid,relatime,size=10240k,nr_inodes=2050547,mode=755)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
/dev/md3 on / type ext3 (rw,noatime,errors=continue,barrier=1,data=writeback)
tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
cgroup_root on /sys/fs/cgroup type tmpfs
(rw,nosuid,nodev,noexec,relatime,size=10240k,mode=755)
openrc on /sys/fs/cgroup/openrc type cgroup
(rw,nosuid,nodev,noexec,relatime,release_agent=/lib64/rc/sh/cgroup-release-agent.sh,name=openrc)
cpuset on /sys/fs/cgroup/cpuset type cgroup
(rw,nosuid,nodev,noexec,relatime,cpuset)
debug on /sys/fs/cgroup/debug type cgroup
(rw,nosuid,nodev,noexec,relatime,debug)
cpu on /sys/fs/cgroup/cpu type cgroup (rw,nosuid,nodev,noexec,relatime,cpu)
cpuacct on /sys/fs/cgroup/cpuacct type cgroup
(rw,nosuid,nodev,noexec,relatime,cpuacct)
memory on /sys/fs/cgroup/memory type cgroup
(rw,nosuid,nodev,noexec,relatime,memory)
devices on /sys/fs/cgroup/devices type cgroup
(rw,nosuid,nodev,noexec,relatime,devices)
freezer on /sys/fs/cgroup/freezer type cgroup
(rw,nosuid,nodev,noexec,relatime,freezer)
blkio on /sys/fs/cgroup/blkio type cgroup
(rw,nosuid,nodev,noexec,relatime,blkio)
perf_event on /sys/fs/cgroup/perf_event type cgroup
(rw,nosuid,nodev,noexec,relatime,perf_event)
/chroot/ssh-jails/jamiel/home/jamiel on /home/jamiel type none (rw,bind)
libvirt on /run/libvirt/lxc/dwj-lnx-dev type fuse (rw,nosuid,nodev)



*** My LXC containers:
ostara ~ # virsh -c lxc:// list --all
 Id    Name                           State
----------------------------------------------------
 -     dwj-lnx-dev                    shut off
 -     vm1                            shut off


*** Config from one container (itself is Gentoo Linux, sharing some
file-systems)
ostara ~ # virsh -c lxc:// dumpxml dwj-lnx-dev
<domain type='lxc'>
  <name>dwj-lnx-dev</name>
  <uuid>fbcd8c3a-9939-12b4-727d-5d3526bc448f</uuid>
  <memory unit='KiB'>500000</memory>
  <currentMemory unit='KiB'>500000</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/sbin/init</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/vm/lxc/dwj-lnx-dev'/>
      <target dir='/'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr/portage'/>
      <target dir='/usr/portage'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr/src'/>
      <target dir='/usr/src'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/home'/>
      <target dir='/home'/>
    </filesystem>
    <interface type='bridge'>
      <mac address='82:00:00:00:01:00'/>
      <source bridge='br0'/>
    </interface>
    <console type='pty'>
      <target type='lxc' port='0'/>
    </console>
  </devices>
</domain>


ostara ~ # ls -l /vm/lxc/dwj-lnx-dev
total 108
drwxr-xr-x  2 root root  4096 Oct 18 09:38 bin
drwxr-xr-x  2 root root  4096 Apr 27  2011 boot
drwxr-xr-x 10 root root 45056 Jan 26 10:52 dev
drwxr-xr-x 53 root root  4096 Feb  8 11:23 etc
drwxr-xr-x  2 root root  4096 Apr 27  2011 home
lrwxrwxrwx  1 root root     5 Oct 18 00:46 lib -> lib64
drwxr-xr-x  2 root root  4096 Oct 18 00:51 lib32
drwxr-xr-x 10 root root  4096 Oct 18 09:11 lib64
drwxr-xr-x  4 root root  4096 Apr 27  2011 mnt
drwxr-xr-x  4 root root  4096 May 22  2012 opt
drwxr-xr-x  2 root root  4096 Apr 27  2011 proc
drwx------  3 root root  4096 Oct 18 08:13 root
drwxr-xr-x  2 root root  4096 Oct 18 00:57 run
drwxr-xr-x  2 root root  4096 Oct 18 09:11 sbin
drwxr-xr-x  2 root root  4096 Apr 27  2011 sys
drwxrwxrwt  4 root root  4096 Feb  8 11:23 tmp
drwxr-xr-x 13 root root  4096 May  4  2011 usr
drwxr-xr-x 14 root root  4096 May 22  2012 var


************ v 1.0.0 works great!

ostara ~ # virsh --version
1.0.0

ostara ~ # virsh -c lxc:// list --all
 Id    Name                           State
----------------------------------------------------
 -     dwj-lnx-dev                    shut off
 -     vm1                            shut off

ostara ~ # virsh -c lxc:// start dwj-lnx-dev
Domain dwj-lnx-dev started

ostara ~ # virsh -c lxc:// list --all
 Id    Name                           State
----------------------------------------------------
 21364 dwj-lnx-dev                    running
 -     vm1                            shut off

*************  v 1.0.2 fails:

ostara ~ # /etc/init.d/libvirtd start
 * Caching service dependencies ...
                                                        [ ok ]
 * Starting libvirtd ...
                                                        [ ok ]
ostara ~ # virsh --version
1.0.2

ostara ~ # ls -l /var/lib/libvirt/lxc/
total 0

ostara ~ # virsh -c lxc:// start dwj-lnx-dev
error: Failed to start domain dwj-lnx-dev
error: internal error guest failed to start: PATH=/bin:/sbin
TERM=linux container=lxc-libvirt
container_uuid=fbcd8c3a-9939-12b4-727d-5d3526bc448f
LIBVIRT_LXC_UUID=fbcd8c3a-9939-12b4-727d-5d3526bc448f
LIBVIRT_LXC_NAME=dwj-lnx-dev /sbin/init
2013-02-08 18:09:28.402+0000: 1: info : libvirt version: 1.0.2
2013-02-08 18:09:28.402+0000: 1: error : lxcContainerMountProcFuse:616
: Failed to mount /.oldroot//var/run/libvirt/lxc/dwj-lnx-dev/meminfo
on /proc/meminfo: No such file or directory
2013-02-08 18:09:28.402+0000: 23867: info : libvirt version: 1.0.2
2013-02-08 18:09:28.402+0000: 23867: error : virLXCControllerRun:1468
: error receiving signal from container: Input/output error
2013-02-08 18:09:28.814+0000: 23867: error : virCommandWait:2287 :
internal error Child process (ip link del veth1) unexpected exit
status 1: Cannot find device "veth1"


ostara ~ # ls -l /var/lib/libvirt/lxc/
total 0

ostara ~ # mount | grep libvirt
libvirt on /run/libvirt/lxc/dwj-lnx-dev type fuse (rw,nosuid,nodev)


ostara ~ # ls -l /run/libvirt/lxc
total 0
drwxr-xr-x 2 root root 40 Feb  8 12:09 dwj-lnx-dev
srwx------ 1 root root  0 Feb  8 12:09 dwj-lnx-dev.sock
srwx------ 1 root root  0 Feb  8 11:22 vm1.sock

ostara ~ # ls -l /run/libvirt/lxc/dwj-lnx-dev
total 0

*** "veth1" (regarding the rror "Cannot find device")
*** "veth1" is still no created.  But I do know that when
*** using libvirt v1.0.0, and the container is running, the device
DOES exist (not shown above)

ostara ~ # ifconfig -a | egrep "^\w"
br0: flags=4355<UP,BROADCAST,PROMISC,MULTICAST>  mtu 1500
br1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 16436
virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

*** The last entry in "/var/log/libvirt/lxc/dwj-lnx-dev.log"
2013-02-08 18:09:27.771+0000: starting up
PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.6.3:/usr/x86_64-pc-linux-gnu/i686-pc-mingw32/gcc-bin/4.7.2
LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr /usr/libexec/libvirt_lxc
--name dwj-lnx-dev --console 20 --security=none --handshake 23
--background --veth veth1
PATH=/bin:/sbin TERM=linux container=lxc-libvirt
container_uuid=fbcd8c3a-9939-12b4-727d-5d3526bc448f
LIBVIRT_LXC_UUID=fbcd8c3a-9939-12b4-727d-5d3526bc448f
LIBVIRT_LXC_NAME=dwj-lnx-dev /sbin/init
2013-02-08 18:09:28.402+0000: 1: info : libvirt version: 1.0.2
2013-02-08 18:09:28.402+0000: 1: error : lxcContainerMountProcFuse:616
: Failed to mount /.oldroot//var/run/libvirt/lxc/dwj-lnx-dev/meminfo
on /proc/meminfo: No such file or directory
2013-02-08 18:09:28.402+0000: 23867: info : libvirt version: 1.0.2
2013-02-08 18:09:28.402+0000: 23867: error : virLXCControllerRun:1468
: error receiving signal from container: Input/output error
2013-02-08 18:09:28.814+0000: 23867: error : virCommandWait:2287 :
internal error Child process (ip link del veth1) unexpected exit
status 1: Cannot find device "veth1"


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]