[libvirt-users] libvrtd-1.1.0 crashes when attempting to start some (but not all) LXC containers

Dennis Jenkins dennis.jenkins.75 at gmail.com
Fri Jul 12 16:40:57 UTC 2013


Hello all,

    I have two issues:

1) I am unable to start a seemingly correct LXC domain (I cloned it from a
working domain).

2) I am able to crash "libvirtd" by attempting to start the cloned domain,
but starting the original works just fine.

    I humbly submit that item #2 is a bug - the "libvirtd" daemon should
never crash due to anything the "libvirt" client throws at it.  As for
item  #1, I'm not sure where I went wrong.  A full walk-through is below
(ending with a DIFF of the XML from the two domains).

    I created by original domain ("dwj-lnx-dev") a long time ago.  Today I
created the new domain ("dwj-hfax-dev") as follows:

1) Shutdown "dwj-lnx-dev"
2) Clone the root file system: "cd /vm/lxc/; cp -a dwj-lnx-dev
dwj-hfax-dev"  (2.5GB, ~5 min)
3) "libvirt -c lxc:/// dumpxml dwj-lnx-dev > a.xml"
4) ${EDITOR} a.xml
  a) changed MAC address, name, memory, source directory for "/"
5) "libvirt -c lxc:/// define a.xml"
6) Edit "/etc/bind/pri/*" and "/etc/dhcp/dhcpd.conf" on my host.


    It does not matter is "dwj-lnx-dev" is running or not.  Any attempt to
start "dwj-hfax-dev" will crash libvirtd.


    In the past I was asked to turn on some debugging and capture a
detailed log (
https://www.redhat.com/archives/libvirt-users/2013-May/msg00076.html).  I
will do this soon and post my results as a follow up.



ostara ~ # uname -a
Linux ostara 3.8.13-gentoo #1 SMP PREEMPT Mon Jun 3 17:10:56 CDT 2013
x86_64 Intel(R) Core(TM) i5 CPU 760 @ 2.80GHz GenuineIntel GNU/Linux


ostara ~ # equery l libvirt
 * Searching for libvirt ...
[IP-] [  ] app-emulation/libvirt-1.1.0-r1:0


ostara ~ # virsh -c lxc:/// version
Compiled against library: libvirt 1.1.0
Using library: libvirt 1.1.0
Using API: LXC 1.1.0
Running hypervisor: LXC 3.8.13


ostara ~ # /etc/init.d/libvirtd restart
 * Caching service dependencies
...
[ ok ]
 * Stopping libvirtd ...
 *  Shutting down network(s):
 *
default
[ ok ]
 * Starting libvirtd
...
[ ok ]


ostara ~ # virsh -c lxc:/// list --all
 Id    Name                           State
----------------------------------------------------
 -     dwj-hfax-dev                   shut off
 -     dwj-lnx-dev                    shut off
 -     vm1                            shut off

ostara ~ # virsh -c lxc:/// start dwj-lnx-dev
Domain dwj-lnx-dev started


ostara ~ # virsh -c lxc:/// list --all
 Id    Name                           State
----------------------------------------------------
 9441  dwj-lnx-dev                    running
 -     dwj-hfax-dev                   shut off
 -     vm1                            shut off


ostara ~ # virsh -c lxc:/// start dwj-hfax-dev
error: Failed to start domain dwj-hfax-dev
error: End of file while reading data: Input/output error
error: One or more references were leaked after disconnect from the
hypervisor
error: Failed to reconnect to the hypervisor


ostara ~ # virsh -c lxc:/// list --all
error: failed to connect to the hypervisor
error: no valid connection
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock':
Connection refused


ostara ~ # ls -l /var/run/libvirt/libvirt-sock
srwx------ 1 root root 0 Jul 12 11:21 /var/run/libvirt/libvirt-sock


ostara ~ # ps axfw | grep libvirt
 9997 pts/2    S+     0:00                      \_ grep --colour=auto
libvirt
 8446 ?        S      0:00 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf
 9441 ?        Ss     0:00 /usr/libexec/libvirt_lxc --name dwj-lnx-dev
--console 19 --security=none --handshake 23 --background --veth veth1


ostara ~ # /etc/init.d/libvirtd restart
 * Stopping libvirtd ...
 * start-stop-daemon: no matching processes
found
[ ok ]
 * Starting libvirtd
...
[ ok ]


ostara ~ # ps axfw | grep libvirt
10130 pts/2    S+     0:00                      \_ grep --colour=auto
libvirt
 8446 ?        S      0:00 /usr/sbin/dnsmasq
--conf-file=/var/lib/libvirt/dnsmasq/default.conf
 9441 ?        Ss     0:00 /usr/libexec/libvirt_lxc --name dwj-lnx-dev
--console 19 --security=none --handshake 23 --background --veth veth1
10033 ?        Sl     0:00 /usr/sbin/libvirtd -d --listen


ostara ~ # virsh -c lxc:/// list --all
 Id    Name                           State
----------------------------------------------------
 9441  dwj-lnx-dev                    running
 -     dwj-hfax-dev                   shut off
 -     vm1                            shut off


ostara ~ # virsh -c lxc:/// dumpxml dwj-hfax-dev
<domain type='lxc'>
  <name>dwj-hfax-dev</name>
  <uuid>681410de-7b56-41bd-b38d-3c66ce97e7b3</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/sbin/init</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/vm/lxc/dwj-hfax-dev'/>
      <target dir='/'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr/portage'/>
      <target dir='/usr/portage'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr/src'/>
      <target dir='/usr/src'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/home'/>
      <target dir='/home'/>
    </filesystem>
    <interface type='bridge'>
      <mac address='82:00:00:00:01:01'/>
      <source bridge='br0'/>
      <target dev='veth0'/>
    </interface>
    <console type='pty'>
      <target type='lxc' port='0'/>
    </console>
  </devices>
  <seclabel type='none'/>
</domain>


ostara ~ # virsh -c lxc:/// dumpxml dwj-lnx-dev
<domain type='lxc' id='9441'>
  <name>dwj-lnx-dev</name>
  <uuid>fbcd8c3a-9939-12b4-727d-5d3526bc448f</uuid>
  <memory unit='KiB'>500000</memory>
  <currentMemory unit='KiB'>500000</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/sbin/init</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/vm/lxc/dwj-lnx-dev'/>
      <target dir='/'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr/portage'/>
      <target dir='/usr/portage'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/usr/src'/>
      <target dir='/usr/src'/>
    </filesystem>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/home'/>
      <target dir='/home'/>
    </filesystem>
    <interface type='bridge'>
      <mac address='82:00:00:00:01:00'/>
      <source bridge='br0'/>
      <target dev='veth0'/>
    </interface>
    <console type='pty' tty='/dev/pts/3'>
      <source path='/dev/pts/3'/>
      <target type='lxc' port='0'/>
      <alias name='console0'/>
    </console>
  </devices>
  <seclabel type='none'/>
</domain>


ostara ~ # virsh -c lxc:/// dumpxml dwj-lnx-dev > lnx.xml


ostara ~ # virsh -c lxc:/// dumpxml dwj-hfax-dev > hfax.xml


ostara ~ # diff lnx.xml hfax.xml
1,6c1,6
< <domain type='lxc' id='9441'>
<   <name>dwj-lnx-dev</name>
<   <uuid>fbcd8c3a-9939-12b4-727d-5d3526bc448f</uuid>
<   <memory unit='KiB'>500000</memory>
<   <currentMemory unit='KiB'>500000</currentMemory>
<   <vcpu placement='static'>2</vcpu>
---
> <domain type='lxc'>
>   <name>dwj-hfax-dev</name>
>   <uuid>681410de-7b56-41bd-b38d-3c66ce97e7b3</uuid>
>   <memory unit='KiB'>4194304</memory>
>   <currentMemory unit='KiB'>4194304</currentMemory>
>   <vcpu placement='static'>4</vcpu>
21c21
<       <source dir='/vm/lxc/dwj-lnx-dev'/>
---
>       <source dir='/vm/lxc/dwj-hfax-dev'/>
37c37
<       <mac address='82:00:00:00:01:00'/>
---
>       <mac address='82:00:00:00:01:01'/>
41,42c41
<     <console type='pty' tty='/dev/pts/3'>
<       <source path='/dev/pts/3'/>
---
>     <console type='pty'>
44d42
<       <alias name='console0'/>


(After reseting everything, and attempting to boot hfax with dev offline,
libvirtd still crashes)

ostara ~ # virsh -c lxc:/// list --all
 Id    Name                           State
----------------------------------------------------
 -     dwj-hfax-dev                   shut off
 -     dwj-lnx-dev                    shut off
 -     vm1                            shut off



ostara ~ # virsh -c lxc:/// start dwj-hfax-dev
error: Failed to start domain dwj-hfax-dev
error: End of file while reading data: Input/output error
error: One or more references were leaked after disconnect from the
hypervisor
error: Failed to reconnect to the hypervisor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20130712/70b9ce2d/attachment.htm>


More information about the libvirt-users mailing list