[libvirt] Segfault in libxl_osevent_occurred_timeout [Was: Re: [libvirt-users] About debugging of libvirt.]
cool dharma06
cooldharma06 at gmail.com
Fri Dec 20 08:54:59 UTC 2013
thanks to all and that is working with libvirt-1.2.0 and xen-4.3.1.
thanks to all people for their valuable suggestions.
regards,
cooldharma06.
On Thu, Dec 19, 2013 at 5:01 PM, Dario Faggioli
<dario.faggioli at citrix.com>wrote:
> [Moving this to libvir, libvir-users in Bcc. Also, added xen-devel]
>
> Jim,
>
> cooldharma06 reports having issues when destroying VMs with libvirt
> 1.2.0 and Xen 4.2.1 (is that the case, cooldharma06)?
>
> Look at the stack trace at the end of this message, or here:
> https://bugzilla.redhat.com/show_bug.cgi?id=1044838
>
> Ian, since it looks like events are involved... Any idea?
>
> cooldharma06, about the Xen 4.2.1, thing, is there a specific reason why
> you use it instead, of, e.g., 4.3.x ? Since you are building from source
> anyway, could you at least try Xen 4.3, to see whether the issue could
> be a bug in 4.2's libxl? I'm asking because, with both Xen 4.3 and the
> curret git tip, I don't have issues destroying domains with virsh.
>
> Thanks and Regards,
> Dario
>
> On gio, 2013-12-19 at 15:42 +0530, cool dharma06 wrote:
> > i did the debugging as you said. Kindly refer the following logs:
> >
> >
> > (gdb) c
> > Continuing.
> > thread apply all bt
> > [New Thread 0x7fc337c6b700 (LWP 29520)]
> >
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x0000000000000000 in ?? ()
> > (gdb) thread apply all bt
> >
> >
> > Thread 12 (Thread 0x7fc337c6b700 (LWP 29520)):
> > #0 0x00007fc33509f18d in read ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc32b1555c4 in read_all (fd=26, data=0x7fc3380e0a70,
> > data at entry=0x20, len=len at entry=16, nonblocking=nonblocking at entry=0) at
> > xs.c:365
> > #2 0x00007fc32b1556d8 in read_message (h=h at entry=0x7fc324013ee0,
> > nonblocking=nonblocking at entry=0) at xs.c:1071
> > #3 0x00007fc32b156005 in read_thread (arg=0x7fc324013ee0) at
> > xs.c:1137
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 11 (Thread 0x7fc330e7f700 (LWP 29170)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbd80,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0cb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db870) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 10 (Thread 0x7fc33067e700 (LWP 29171)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbd80,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0cb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db640) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 9 (Thread 0x7fc32fe7d700 (LWP 29172)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbd80,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0cb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db870) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 8 (Thread 0x7fc32f67c700 (LWP 29173)):
> > #0 0x00007fc33497cb9f in realloc ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #1 0x00007fc3349eec85 in __vasprintf_chk ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > ---Type <return> to continue, or q <return> to quit---
> > #2 0x00007fc3371aac0a in vasprintf (__ap=0x7fc32f67b6d0,
> > __fmt=0x7fc337391017 "%llu: %s : %s:%d : %s\n", __ptr=0x7fc32f67b830)
> > at /usr/include/x86_64-linux-gnu/bits/stdio2.h:199
> > #3 virVasprintfInternal (report=report at entry=false, domcode=0,
> > filename=0x0, funcname=0x0, linenr=0, strp=0x7fc32f67b830,
> > fmt=fmt at entry=0x7fc337391017 "%llu: %s : %s:%d : %s\n",
> > list=list at entry=0x7fc32f67b6d0) at util/virstring.c:337
> > #4 0x00007fc3371aad1b in virAsprintfInternal
> > (report=report at entry=false, domcode=domcode at entry=0,
> > filename=filename at entry=0x0, funcname=funcname at entry=0x0,
> > linenr=linenr at entry=0, strp=strp at entry=0x7fc32f67b830,
> > fmt=fmt at entry=0x7fc337391017 "%llu: %s : %s:%d : %s\n") at
> > util/virstring.c:358
> > #5 0x00007fc33718cb59 in virLogFormatString (str=<optimized out>,
> > priority=VIR_LOG_DEBUG, funcname=0x7fc337394734 "virObjectRef",
> > linenr=293, msg=0x7fc32f67b830)
> > at util/virlog.c:719
> > #6 virLogVMessage (source=VIR_LOG_FROM_TRACE, priority=VIR_LOG_DEBUG,
> > filename=0x7fc337394597 "util/virobject.c", linenr=293,
> > funcname=0x7fc337394734 "virObjectRef",
> > metadata=0x0, fmt=fmt at entry=0x7fc3373945f1 "OBJECT_REF: obj=%p",
> > vargs=vargs at entry=0x7fc32f67b8c0) at util/virlog.c:846
> > #7 0x00007fc33718cfc7 in virLogMessage
> > (source=source at entry=VIR_LOG_FROM_TRACE,
> > priority=priority at entry=VIR_LOG_DEBUG,
> > filename=filename at entry=0x7fc337394597 "util/virobject.c",
> > linenr=linenr at entry=293, funcname=funcname at entry=0x7fc337394734
> > "virObjectRef",
> > metadata=metadata at entry=0x0, fmt=fmt at entry=0x7fc3373945f1
> > "OBJECT_REF: obj=%p") at util/virlog.c:778
> > #8 0x00007fc33719c3d7 in virObjectRef (anyobj=0x7fc338102720) at
> > util/virobject.c:293
> > #9 0x00007fc32af1824f in libxlDriverConfigGet
> > (driver=driver at entry=0x7fc338108930) at libxl/libxl_conf.c:1124
> > #10 0x00007fc32af1a50e in libxlVmCleanup
> > (driver=driver at entry=0x7fc338108930, vm=vm at entry=0x7fc32400f530,
> > reason=reason at entry=VIR_DOMAIN_SHUTOFF_DESTROYED)
> > at libxl/libxl_driver.c:264
> > #11 0x00007fc32af20274 in libxlVmReap (driver=0x7fc338108930,
> > vm=0x7fc32400f530, reason=VIR_DOMAIN_SHUTOFF_DESTROYED) at
> > libxl/libxl_driver.c:338
> > #12 0x00007fc32af207de in libxlDomainDestroyFlags (dom=0x7fc3380e3770,
> > flags=<optimized out>) at libxl/libxl_driver.c:1507
> > #13 0x00007fc33723cd37 in virDomainDestroy
> > (domain=domain at entry=0x7fc3380e3770) at libvirt.c:2342
> > #14 0x00007fc337cb9ba4 in remoteDispatchDomainDestroy (args=<optimized
> > out>, rerr=0x7fc32f67bcd0, client=0x7fc32400ee60, server=<optimized
> > out>, msg=<optimized out>)
> > at remote_dispatch.h:3165
> > #15 remoteDispatchDomainDestroyHelper (server=<optimized out>,
> > client=0x7fc32400ee60, msg=<optimized out>, rerr=0x7fc32f67bcd0,
> > args=<optimized out>,
> > ret=<optimized out>) at remote_dispatch.h:3143
> > #16 0x00007fc3372b4d79 in virNetServerProgramDispatchCall
> > (msg=0x7fc324014070, client=0x7fc32400ee60, server=0x7fc3380dbba0,
> > prog=0x7fc324007750)
> > at rpc/virnetserverprogram.c:435
> > #17 virNetServerProgramDispatch (prog=0x7fc324007750,
> > server=server at entry=0x7fc3380dbba0, client=0x7fc32400ee60,
> > msg=0x7fc324014070) at rpc/virnetserverprogram.c:305
> > #18 0x00007fc3372aea78 in virNetServerProcessMsg (msg=<optimized out>,
> > prog=<optimized out>, client=<optimized out>, srv=0x7fc3380dbba0) at
> > rpc/virnetserver.c:165
> > #19 virNetServerHandleJob (jobOpaque=<optimized out>,
> > opaque=0x7fc3380dbba0) at rpc/virnetserver.c:186
> > #20 0x00007fc3371af02e in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db640) at util/virthreadpool.c:144
> > #21 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #22 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #23 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #24 0x0000000000000000 in ?? ()
> >
> >
> > Thread 7 (Thread 0x7fc32ee7b700 (LWP 29174)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbd80,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0cb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db870) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > ---Type <return> to continue, or q <return> to quit---
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 6 (Thread 0x7fc32e67a700 (LWP 29175)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbe18,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0eb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db640) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 5 (Thread 0x7fc32de79700 (LWP 29176)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbe18,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0eb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db870) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 4 (Thread 0x7fc32d678700 (LWP 29177)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbe18,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0eb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db640) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 3 (Thread 0x7fc32ce77700 (LWP 29178)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbe18,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0eb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db870) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > ---Type <return> to continue, or q <return> to quit---
> > Thread 2 (Thread 0x7fc32c676700 (LWP 29179)):
> > #0 0x00007fc33509c2d4 in pthread_cond_wait@@GLIBC_2.3.2 ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #1 0x00007fc3371ae9ea in virCondWait (c=c at entry=0x7fc3380dbe18,
> > m=m at entry=0x7fc3380dbd58) at util/virthreadpthread.c:117
> > #2 0x00007fc3371af0eb in virThreadPoolWorker
> > (opaque=opaque at entry=0x7fc3380db640) at util/virthreadpool.c:103
> > #3 0x00007fc3371ae686 in virThreadHelper (data=<optimized out>) at
> > util/virthreadpthread.c:161
> > #4 0x00007fc335097b50 in start_thread ()
> > from /lib/x86_64-linux-gnu/libpthread.so.0
> > #5 0x00007fc3349daa3d in clone ()
> > from /lib/x86_64-linux-gnu/libc.so.6
> > #6 0x0000000000000000 in ?? ()
> >
> >
> > Thread 1 (Thread 0x7fc337c4d7c0 (LWP 29169)):
> > #0 0x0000000000000000 in ?? ()
> > #1 0x00007fc32ace881e in libxl_osevent_occurred_timeout
> > (ctx=<optimized out>, for_libxl=0x7fc3380edf38) at libxl_event.c:1039
> > #2 0x00007fc32af199c7 in libxlDomainObjTimerCallback
> > (timer=<optimized out>, timer_info=0x7fc3380e3310) at
> > libxl/libxl_domain.c:216
> > #3 0x00007fc33717cf7d in virEventPollDispatchTimeouts () at
> > util/vireventpoll.c:451
> > #4 virEventPollRunOnce () at util/vireventpoll.c:644
> > #5 0x00007fc33717b70b in virEventRunDefaultImpl () at
> > util/virevent.c:274
> > #6 0x00007fc3372b0065 in virNetServerRun (srv=0x7fc3380dbba0) at
> > rpc/virnetserver.c:1112
> > #7 0x00007fc337c9851c in main (argc=<optimized out>, argv=<optimized
> > out>) at libvirtd.c:1513
> > (gdb)
> > (gdb)
> >
> >
> > ****
> >
> >
> > regards,
> > cooldharma06.
> >
> >
> > On Thu, Dec 19, 2013 at 3:30 PM, Daniel P. Berrange
> > <berrange at redhat.com> wrote:
> > On Thu, Dec 19, 2013 at 03:09:14PM +0530, cool dharma06 wrote:
> > > I am newbie to this gdb debugging. now i did the bt in the
> > debugging. In my
> > > side still it showing the error. Following are the
> > stacktrace which i got.
> > > Kindly refer the following logs for full information:
> > >
> > > ****
> > > [New Thread 0x7f8503695700 (LWP 20012)]
> > >
> > > Program received signal SIGSEGV, Segmentation fault.
> > > 0x0000000000000000 in ?? ()
> > > (gdb) bt
> > > #0 0x0000000000000000 in ?? ()
> > > #1 0x00007f84f671281e in libxl_osevent_occurred_timeout
> > (ctx=<optimized
> > > out>, for_libxl=0x7f8503bbbc38) at libxl_event.c:1039
> > > #2 0x00007f84f69439c7 in libxlDomainObjTimerCallback
> > (timer=<optimized
> > > out>, timer_info=0x7f8503bb5930) at libxl/libxl_domain.c:216
> > > #3 0x00007f8502ba6f7d in virEventPollDispatchTimeouts () at
> > > util/vireventpoll.c:451
> > > #4 virEventPollRunOnce () at util/vireventpoll.c:644
> > > #5 0x00007f8502ba570b in virEventRunDefaultImpl () at
> > util/virevent.c:274
> > > #6 0x00007f8502cda065 in virNetServerRun
> > (srv=0x7f8503ba6ba0) at
> > > rpc/virnetserver.c:1112
> > > #7 0x00007f85036c251c in main (argc=<optimized out>,
> > argv=<optimized out>)
> > > at libvirtd.c:1513
> > > (gdb)
> >
> >
> > In this case, that stack trace looks like it ought to be
> > sufficient
> > to identify the problem - a clear NULL pointer reference. In
> > general
> > though you want to capture a stack trace of all threads in a
> > program
> >
> > eg instead of 'bt', do 'thread apply all bt' next time.
> >
> > Regards,
> > Daniel
> > --
> > |: http://berrange.com -o-
> > http://www.flickr.com/photos/dberrange/ :|
> > |: http://libvirt.org -o-
> > http://virt-manager.org :|
> > |: http://autobuild.org -o-
> > http://search.cpan.org/~danberr/ :|
> > |: http://entangle-photo.org -o-
> > http://live.gnome.org/gtk-vnc :|
> >
> >
>
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20131220/76398a8f/attachment-0001.htm>
More information about the libvir-list
mailing list