[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] libvirtd crashing with network bridge configuration



Hi Daniel,

On Thu, Dec 18, 2008 at 1:47 PM, Daniel P. Berrange <berrange redhat com> wrote:
> ps ax shows the following output right after the service is started:
>
>  3605 ?        S      0:00 /usr/sbin/libvirtd --daemon
>  3624 ?        Z      0:00 [libvirtd] <defunct>
>
> If I don't enable any network at  /etc/libvirt/qemu/networks/autostart, then
> there's no error.

This suggest libvirtd itself is *not* crashing. Some process that
libvirtd runs is dieing - not sure which though.

Please kill all libvirtd  instances, and making sure you have compiled
with debugging info turned on (ie, '-g' compile flag), then run

 # valgrind /usr/sbin/libvirtd

And also try

 LIBVIRT_DEBUG=1 /usr/sbin/libvirtd

and send the output for both of these.

The output of these seperately, and combined (total 3 files) are attached. Guessing that there might a compiler flag problem, here's what our default compiler uses while compiling packages:

cflags = -mtune=generic -march=i686 -O2 -pipe -fomit-frame-pointer -fstack-protector -D_FORTIFY_SOURCE=2
cxxflags = -mtune=generic -march=i686 -O2 -pipe -fomit-frame-pointer -fstack-protector -D_FORTIFY_SOURCE=2
host = i686-pc-linux-gnu                                                                                 
jobs = -j1
ldflags = -Wl,-O1 -Wl,-z,relro -Wl,--hash-style=gnu

A typical compile line for libvirt includes the following command, except the -g debug flag that we added:

 gcc -DHAVE_CONFIG_H -I. -I.. -I../gnulib/lib -I../gnulib/lib -I../include -I../include -I../qemud -I/usr/include/libxml2 -DLIBDIR=\"/usr/lib\" -DBINDIR=\"/usr/libexec\" -DSBINDIR=\"/usr/sbin\" -DSYSCONF_DIR=\"/etc\" -DLOCALEBASEDIR=\"/usr/share/locale\" -DLOCAL_STATE_DIR=\"/var\" -DGETTEXT_PACKAGE=\"libvirt\" -Wall -Wformat -Wformat-security -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wextra -Wshadow -Wcast-align -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Winline -Wredundant-decls -Wno-sign-compare -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fasynchronous-unwind-tables -mtune=generic -march=i686 -O2 -pipe -fomit-frame-pointer -fstack-protector -D_FORTIFY_SOURCE=2 -g -MT libvirt_driver_xen_la-xen_internal.lo -MD -MP -MF .deps/libvirt_driver_xen_la-xen_internal.Tpo -c xen_internal.c  -fPIC -DPIC -o .libs/libvirt_driver_xen_la-xen_interna! l.o


Thanks again for any insight and please don't hesitate if you need further info from my side.

Best Regards,


Emre
voyager emre # LIBVIRT_DEBUG=1 /usr/sbin/libvirtd 
DEBUG: libvirt.c: virInitialize (register drivers)
DEBUG: libvirt.c: virRegisterDriver (registering Test as driver 0)
DEBUG: libvirt.c: virRegisterNetworkDriver (registering Test as network driver 0)
DEBUG: libvirt.c: virRegisterStorageDriver (registering Test as storage driver 0)
DEBUG: libvirt.c: virRegisterDriver (registering Xen as driver 1)                
DEBUG: libvirt.c: virRegisterDriver (registering OPENVZ as driver 2)             
DEBUG: libvirt.c: virRegisterDriver (registering remote as driver 3)             
DEBUG: libvirt.c: virRegisterNetworkDriver (registering remote as network driver 1)
DEBUG: libvirt.c: virRegisterStorageDriver (registering remote as storage driver 1)
DEBUG: libvirt.c: virRegisterDeviceMonitor (registering remote as device driver 0) 
DEBUG: libvirt.c: virRegisterNetworkDriver (registering Network as network driver 2)
DEBUG: libvirt.c: virRegisterStorageDriver (registering storage as storage driver 2)
DEBUG: libvirt.c: virRegisterDeviceMonitor (registering halDeviceMonitor as device driver 1)
DEBUG: libvirt.c: virRegisterDriver (registering QEMU as driver 4)
DEBUG: libvirt.c: virRegisterDriver (registering LXC as driver 5)
DEBUG: libvirt.c: virRegisterDriver (registering UML as driver 6)
DEBUG: util.c: virRun (/usr/sbin/brctl setfd virbr0 0)
DEBUG: util.c: virRun (/usr/sbin/brctl stp virbr0 on)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol tcp --destination-port 67 --jump ACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol udp --destination-port 67 --jump ACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol tcp --destination-port 53 --jump ACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol udp --destination-port 53 --jump ACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --in-interface virbr0 --jump REJECT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --out-interface virbr0 --jump REJECT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --in-interface virbr0 --out-interface virbr0 --jump ACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --source 192.168.122.0/255.255.255.0 --in-interface virbr0 --jumpACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --destination 192.168.122.0/255.255.255.0 --out-interface virbr0 --match state --state ESTABLISHED,RELATED --jump ACCEPT)
DEBUG: util.c: virRun (/sbin/iptables --table nat --insert POSTROUTING --source 192.168.122.0/255.255.255.0 --destination ! 192.168.122.0/255.255.255.0 --jump MASQUERADE)
DEBUG: util.c: virExec (dnsmasq --keep-in-foreground --strict-order --bind-interfaces --pid-file  --conf-file  --listen-address 192.168.122.1 --except-interface lo --dhcp-leasefile=/var/lib/libvirt/dhcp-default.leases --dhcp-range 192.168.122.2,192.168.122.254)
libvir: error : internal error cannot execute binary 'dnsmasq': No such file or directory
DEBUG: lxc_container.c: lxcContainerAvailable (clone call returned Cannot allocate memory, container support is not enabled)
voyager emre # LIBVIRT_DEBUG=1 valgrind /usr/sbin/libvirtd 
==3206== Memcheck, a memory error detector.                
==3206== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==3206== Using LibVEX rev 1854, a library for dynamic binary translation.
==3206== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.       
==3206== Using valgrind-3.3.1, a dynamic binary instrumentation framework.
==3206== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.  
==3206== For more details, rerun with: -v                                 
==3206==                                                                  
DEBUG: libvirt.c: virInitialize (register drivers)                        
DEBUG: libvirt.c: virRegisterDriver (registering Test as driver 0)        
DEBUG: libvirt.c: virRegisterNetworkDriver (registering Test as network driver 0)
DEBUG: libvirt.c: virRegisterStorageDriver (registering Test as storage driver 0)
DEBUG: libvirt.c: virRegisterDriver (registering Xen as driver 1)                
DEBUG: libvirt.c: virRegisterDriver (registering OPENVZ as driver 2)             
DEBUG: libvirt.c: virRegisterDriver (registering remote as driver 3)             
DEBUG: libvirt.c: virRegisterNetworkDriver (registering remote as network driver 1)
DEBUG: libvirt.c: virRegisterStorageDriver (registering remote as storage driver 1)
DEBUG: libvirt.c: virRegisterDeviceMonitor (registering remote as device driver 0) 
DEBUG: libvirt.c: virRegisterNetworkDriver (registering Network as network driver 2)
DEBUG: libvirt.c: virRegisterStorageDriver (registering storage as storage driver 2)
DEBUG: libvirt.c: virRegisterDeviceMonitor (registering halDeviceMonitor as device driver 1)
DEBUG: libvirt.c: virRegisterDriver (registering QEMU as driver 4)                          
DEBUG: libvirt.c: virRegisterDriver (registering LXC as driver 5)                           
DEBUG: libvirt.c: virRegisterDriver (registering UML as driver 6)                           
==3206== Warning: noted but unhandled ioctl 0x89a0 with no size/direction hints             
==3206==    This could cause spurious value errors to appear.                               
==3206==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.   
DEBUG: util.c: virRun (/usr/sbin/brctl setfd virbr0 0)                                      
DEBUG: util.c: virRun (/usr/sbin/brctl stp virbr0 on)                                       
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol tcp --destination-port 67 --jump ACCEPT)                                                                                                                                   
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol udp --destination-port 67 --jump ACCEPT)                                                                                                                                   
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol tcp --destination-port 53 --jump ACCEPT)                                                                                                                                   
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert INPUT --in-interface virbr0 --protocol udp --destination-port 53 --jump ACCEPT)                                                                                                                                   
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --in-interface virbr0 --jump REJECT)                              
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --out-interface virbr0 --jump REJECT)                             
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --in-interface virbr0 --out-interface virbr0 --jump ACCEPT)       
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --source 192.168.122.0/255.255.255.0 --in-interface virbr0 --jump ACCEPT)                                                                                                                                 
DEBUG: util.c: virRun (/sbin/iptables --table filter --insert FORWARD --destination 192.168.122.0/255.255.255.0 --out-interface virbr0 --match state --state ESTABLISHED,RELATED --jump ACCEPT)                                                                                 
DEBUG: util.c: virRun (/sbin/iptables --table nat --insert POSTROUTING --source 192.168.122.0/255.255.255.0 --destination ! 192.168.122.0/255.255.255.0 --jump MASQUERADE)                                                                                                      
DEBUG: util.c: virExec (dnsmasq --keep-in-foreground --strict-order --bind-interfaces --pid-file  --conf-file  --listen-address 192.168.122.1 --except-interface lo --dhcp-leasefile=/var/lib/libvirt/dhcp-default.leases --dhcp-range 192.168.122.2,192.168.122.254)           
libvir: error : internal error cannot execute binary 'dnsmasq': No such file or directory                                               
==3229==                                                                                                                                
==3229== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 55 from 1)                                                                
==3229== malloc/free: in use at exit: 22,308 bytes in 495 blocks.                                                                       
==3229== malloc/free: 907 allocs, 412 frees, 97,193 bytes allocated.                                                                    
==3229== For counts of detected errors, rerun with: -v                                                                                  
==3229== searching for pointers to 495 not-freed blocks.                                                                                
==3229== checked 382,024 bytes.                                                                                                         
==3229==                                                                                                                                
==3229== LEAK SUMMARY:                                                                                                                  
==3229==    definitely lost: 0 bytes in 0 blocks.                                                                                       
==3229==      possibly lost: 0 bytes in 0 blocks.                                                                                       
==3229==    still reachable: 22,308 bytes in 495 blocks.                                                                                
==3229==         suppressed: 0 bytes in 0 blocks.                                                                                       
==3229== Rerun with --leak-check=full to see details of leaked memory.                                                                  
==3206== Syscall param clone(parent_tidptr) contains uninitialised byte(s)                                                              
==3206==    at 0x4589DC8: clone (in /lib/libc-2.8.so)                                                                                   
==3206==    by 0x80A0FF3: ???                                                                                                           
==3206==                                                                                                                                
==3206== Syscall param clone(tlsinfo) contains uninitialised byte(s)                                                                    
==3206==    at 0x4589DC8: clone (in /lib/libc-2.8.so)                                                                                   
==3206==    by 0x80A0FF3: ???                                                                                                           
==3206==                                                                                                                                
==3206== Syscall param clone(child_tidptr) contains uninitialised byte(s)                                                               
==3206==    at 0x4589DC8: clone (in /lib/libc-2.8.so)                                                                                   
==3206==    by 0x80A0FF3: ???                                                                                                           
DEBUG: lxc_container.c: lxcContainerAvailable (clone call returned Cannot allocate memory, container support is not enabled)            
==3206==                                                                                                                                
==3206== Conditional jump or move depends on uninitialised value(s)                                                                     
==3206==    at 0x401867E: (within /lib/ld-2.8.so)                                                                                       
==3206==    by 0x4009249: (within /lib/ld-2.8.so)                                                                                       
==3206==                                                                                                                                
==3206== Conditional jump or move depends on uninitialised value(s)                                                                     
==3206==    at 0x401868D: (within /lib/ld-2.8.so)                                                                                       
==3206==    by 0x4009249: (within /lib/ld-2.8.so)                                                                                       
==3206==                                                                                                                                
==3206== Conditional jump or move depends on uninitialised value(s)                                                                     
==3206==    at 0x401869C: (within /lib/ld-2.8.so)                                                                                       
==3206==    by 0x4009249: (within /lib/ld-2.8.so)                                                                                       
==3206==                                                                                                                                
==3206== Conditional jump or move depends on uninitialised value(s)                                                                     
==3206==    at 0x40186AB: (within /lib/ld-2.8.so)                                                                                       
==3206==    by 0x4009249: (within /lib/ld-2.8.so)                                                                                       
==3206==
==3206== Conditional jump or move depends on uninitialised value(s)
==3206==    at 0x401881C: (within /lib/ld-2.8.so)
==3206==    by 0x4009429: (within /lib/ld-2.8.so)
==3206==
==3206== Conditional jump or move depends on uninitialised value(s)
==3206==    at 0x4018827: (within /lib/ld-2.8.so)
==3206==    by 0x4009429: (within /lib/ld-2.8.so)
==3206==
==3206== Conditional jump or move depends on uninitialised value(s)
==3206==    at 0x401881C: (within /lib/ld-2.8.so)
==3206==    by 0x400B869: (within /lib/ld-2.8.so)
==3206==
==3206== Conditional jump or move depends on uninitialised value(s)
==3206==    at 0x4018827: (within /lib/ld-2.8.so)
==3206==    by 0x400B869: (within /lib/ld-2.8.so)
==3206==
==3206== Invalid read of size 4
==3206==    at 0x4018813: (within /lib/ld-2.8.so)
==3206==    by 0x400BA64: (within /lib/ld-2.8.so)
==3206==  Address 0x46d2350 is 16 bytes inside a block of size 17 alloc'd
==3206==    at 0x402803E: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==3206==
==3206== Invalid read of size 4
==3206==    at 0x4018829: (within /lib/ld-2.8.so)
==3206==    by 0x400BA64: (within /lib/ld-2.8.so)
==3206==  Address 0x46d29ec is 20 bytes inside a block of size 21 alloc'd
==3206==    at 0x402803E: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
Segmentation fault
voyager emre # valgrind /usr/sbin/libvirtd 
==2971== Memcheck, a memory error detector.
==2971== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==2971== Using LibVEX rev 1854, a library for dynamic binary translation.
==2971== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.       
==2971== Using valgrind-3.3.1, a dynamic binary instrumentation framework.
==2971== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.  
==2971== For more details, rerun with: -v                                 
==2971==                                                                  
==2971== Warning: noted but unhandled ioctl 0x89a0 with no size/direction hints
==2971==    This could cause spurious value errors to appear.                  
==2971==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
libvir: error : internal error cannot execute binary 'dnsmasq': No such file or directory
==3057==                                                                                 
==3057== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 55 from 1)                 
==3057== malloc/free: in use at exit: 22,308 bytes in 495 blocks.                        
==3057== malloc/free: 907 allocs, 412 frees, 97,193 bytes allocated.                     
==3057== For counts of detected errors, rerun with: -v                                   
==3057== searching for pointers to 495 not-freed blocks.                                 
==3057== checked 382,008 bytes.                                                          
==3057==                                                                                 
==3057== LEAK SUMMARY:                                                                   
==3057==    definitely lost: 0 bytes in 0 blocks.                                        
==3057==      possibly lost: 0 bytes in 0 blocks.                                        
==3057==    still reachable: 22,308 bytes in 495 blocks.                                 
==3057==         suppressed: 0 bytes in 0 blocks.                                        
==3057== Rerun with --leak-check=full to see details of leaked memory.                   
==2971== Syscall param clone(parent_tidptr) contains uninitialised byte(s)               
==2971==    at 0x4589DC8: clone (in /lib/libc-2.8.so)                                    
==2971==    by 0x80A0FF3: ???                                                            
==2971==                                                                                 
==2971== Syscall param clone(tlsinfo) contains uninitialised byte(s)                     
==2971==    at 0x4589DC8: clone (in /lib/libc-2.8.so)                                    
==2971==    by 0x80A0FF3: ???                                                            
==2971==                                                                                 
==2971== Syscall param clone(child_tidptr) contains uninitialised byte(s)                
==2971==    at 0x4589DC8: clone (in /lib/libc-2.8.so)                                    
==2971==    by 0x80A0FF3: ???                                                            
==2971==                                                                                 
==2971== Conditional jump or move depends on uninitialised value(s)                      
==2971==    at 0x401867E: (within /lib/ld-2.8.so)                                        
==2971==    by 0x4009249: (within /lib/ld-2.8.so)                                        
==2971==                                                                                 
==2971== Conditional jump or move depends on uninitialised value(s)                      
==2971==    at 0x401868D: (within /lib/ld-2.8.so)                                        
==2971==    by 0x4009249: (within /lib/ld-2.8.so)                                        
==2971==                                                                                 
==2971== Conditional jump or move depends on uninitialised value(s)                      
==2971==    at 0x401869C: (within /lib/ld-2.8.so)                                        
==2971==    by 0x4009249: (within /lib/ld-2.8.so)                                        
==2971==                                                                                 
==2971== Conditional jump or move depends on uninitialised value(s)                      
==2971==    at 0x40186AB: (within /lib/ld-2.8.so)                                        
==2971==    by 0x4009249: (within /lib/ld-2.8.so)                                        
==2971==
==2971== Conditional jump or move depends on uninitialised value(s)
==2971==    at 0x401881C: (within /lib/ld-2.8.so)
==2971==    by 0x4009429: (within /lib/ld-2.8.so)
==2971==
==2971== Conditional jump or move depends on uninitialised value(s)
==2971==    at 0x4018827: (within /lib/ld-2.8.so)
==2971==    by 0x4009429: (within /lib/ld-2.8.so)
==2971==
==2971== Conditional jump or move depends on uninitialised value(s)
==2971==    at 0x401881C: (within /lib/ld-2.8.so)
==2971==    by 0x400B869: (within /lib/ld-2.8.so)
==2971==
==2971== Conditional jump or move depends on uninitialised value(s)
==2971==    at 0x4018827: (within /lib/ld-2.8.so)
==2971==    by 0x400B869: (within /lib/ld-2.8.so)
==2971==
==2971== Invalid read of size 4
==2971==    at 0x4018813: (within /lib/ld-2.8.so)
==2971==    by 0x400BA64: (within /lib/ld-2.8.so)
==2971==  Address 0x46d2350 is 16 bytes inside a block of size 17 alloc'd
==2971==    at 0x402803E: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==2971==
==2971== Invalid read of size 4
==2971==    at 0x4018829: (within /lib/ld-2.8.so)
==2971==    by 0x400BA64: (within /lib/ld-2.8.so)
==2971==  Address 0x46d29ec is 20 bytes inside a block of size 21 alloc'd
==2971==    at 0x402803E: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
Segmentation fault

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]