[Libvir] "virsh list" command of libvirt consumes a lot of CPU in the domain-0

[Daniel Veillard]

On Tue, Mar 11, 2008 at 04:48:57PM +0100, jean-paul.pigache at bull.net wrote:
> 
>    Hi all,
>    I know that this is not a libvirt issue but this badly impacts libvirt
>    usage.
>    Is anyone aware of any status on this issue ? Daniel ?
>    Here is some history I could get from the libvirt mailing list :
>    * October 12, 2006 (Daniel Berrange).
>    I've  been  trying to track down just why talking to XenD is resulting
>    in so much CPU time being
>    comsumed  by both xend & xenstored. As a test case, I'm running 'virsh
>    dominfo demo' which results in
>    a  single  HTTP  request  to  Xend  to  fetch  domain  info,  eg  'GET
>    /xend/domains/demo'
>    Run  this  in  a tight loop & I'll see xenstored taking > 50% CPU, and
>    XenD taking another 11%

  yes this is a serious performance issue in xend.

[...]
>    single read in XenD. Now if I
>    monitor the status of 20 domains, once per second that's causing 40 MB
>    of writes & 40 MB of reads
>    every second which is utterly ridiculous & completely non scalable for
>    enterprise deployment :-(

  agreed

[...]
>    > Xen 3.0.3 has a serious performance bug
>    > (see
>    http://lists.xensource.com/archives/html/xen-devel/2006-10/msg00487.ht
>    ml)
>    > This bug is fixed in Xen 3.0.4
>    No  it isn't. The performance bug is actually at least x2 worse in Xen
>    3.0.4

I was told that xenstored had been rewritten for the Xen Enterprise version.
There is little I can do at that level honnestly, in libvirt we need the
xend access only to make the binding between the domain ID and the domain
name. All update informations are provided by the hypervisor, including
the ID list, something we could do is to try to cache the id <-> name 
domain association, this is a bit risky since there is no way libvirt can
learn that a binding has changed (either because of an xm rename command
or the domain was destroyed and a new domain created with same id).
If we keep the association in the daemon and use a timeout flush maybe
this could be worked out, but it's really a bad workaround, and the
proper thing would be to fix this long standing bug in xenstored, it's a bit
depressing that no progress had been made on the open source version.

Daniel

-- 
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard      | virtualization library  http://libvirt.org/
veillard at redhat.com  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine  http://rpmfind.net/