[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[libvirt] Xen 4.1 misc issues



Hi everyone

I'm using Ubuntu 12.04.3 LTS with the Xen hypervisor
installed (package version: 4.1.2-2ubuntu2.10),
and Libvirt v0.10.2.2 (that interacts with Xen through
the libxl driver).

I performed some adjustments to have the required
features:

 - Switch from Xend to libxl:
     I created upstart jobs for xenstored and xenconsoled
     so that necessary daemons are started earlier enough
     and Libvirt can contact the hypervisor through libxl.

 - Add support for Open vSwitch:
     I added the script 'vif-openvswitch', taken from
     the master branch of the GIT repository, to the
     /etc/xen/scripts directory in order to plug network
     interfaces to Open vSwitch switches.

I know that the support for libxl is experimental
in Xen 4.1, but I'm wondering if someone has a solution
to the following issues:

1) On domain destroy, the host configuration is not clean:
   if the flag 'force' is set to 0 some entries in the
   xenstored database are not removed (console, ...)
   if the flag 'force' is set to 1 (as it is done
   in Libvirt), the event 'online' associated to the
   network interface is not triggered and, thus, the
   vif is not removed from the Open vSwitch switch.
   To fix this issue, I temporarily added a call
   to vif-openvswitch into the xen-hotplug-cleanup script.

2) When a domain is started from Libvirt, the console
   source path does not appear in the output of the
   command 'virsh dumpxml instance-00000381'

3) Sometimes, a domU is not started at all. If I start
   a virtual machine from Libvirt through virsh or directly
   through xl, these tools report that the virtual machine
   is running but I see no messages coming from the console
   and the CPU time remains fixed.

I'm attaching some files that may be useful to understand
the problem:

 - libvirt.xml: domain specifications (Libvirt format)
 - domain.xl: domain specifications (Xen format)
 - libvirt_virsh_dumpxml.log: output of the command
   'virsh dumpxml instance-00000381'
 - xl_create.log: output of the command 'xl create -d domain.xl'
 - xl_info.log: output of the command 'xl info'
 - qemu-dm-instance-00000381.log: qemu-dm log
 - syslog_vif6_0.log: vif attach/detach logs from /var/log/syslog
 - xenstore-db-not-clean.log: xenstore db entries not removed
   after 'xl destroy'
 - libvirt_libxl.log: log from /var/log/libvirt/libxl.log
   after the execution of the command 'virsh create libvirt.xml'

Lastly, I want to add that the image I'm using for the tests
is Ubuntu Precise 12.04.3 LTS, taken from the URL:

http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img

and the kernel is taken from the URL:

http://cloud-images.ubuntu.com/precise/current/unpacked/precise-server-cloudimg-amd64-vmlinuz-virtual


Thanks

Roberto Sassu
<domain type="xen">
  <uuid>a65c782d-05fe-4c54-9911-8e7e4cb33244</uuid>
  <name>instance-00000381</name>
  <memory>524288</memory>
  <vcpu>1</vcpu>
  <os>
    <type>xen</type>
    <kernel>/var/lib/nova/instances/instance-00000381/kernel</kernel>
    <cmdline>ro root=/dev/xvda1</cmdline>
    <root>/dev/xvda1</root>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset="utc"/>
  <devices>
    <disk type="file" device="disk">
      <driver name="tap" type="raw" cache="none"/>
      <source file="/var/lib/nova/instances/instance-00000381/disk"/>
      <target bus="xen" dev="xvda"/>
    </disk>
    <interface type="bridge">
      <mac address="fa:16:3e:55:5d:e0"/>
      <script path="/etc/xen/scripts/vif-openvswitch"/>
      <source bridge="br-tvd-91970" bridgetype="openvswitch"/>
      <virtualport type="openvswitch">
        <parameters interfaceid="67d41c55-2849-4bc6-b138-c685ae9d3d5f"/>
      </virtualport>
    </interface>
    <console type="pty"/>
    <graphics type="vnc" autoport="yes" keymap="en-us" listen="node-110"/>
  </devices>
</domain>
name = "instance-00000381"
uuid = "a65c782d-05fe-4c54-9911-8e7e4cb33244"
maxmem = 512
memory = 512
vcpus = 1
kernel = "/var/lib/nova/instances/instance-00000381/kernel"
extra = "ro root=/dev/xvda1"
localtime = 0
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "destroy"
vfb = [ "type=vnc,vncunused=1,vnclisten=node-110,keymap=en-us" ]
disk = [ "tap:raw:/var/lib/nova/instances/instance-00000381/disk,xvda,w" ]
vif = [ "mac=fa:16:3e:55:5d:e0,bridge=br-tvd-91970,script=vif-bridge" ]

<domain type='xen' id='2'>
  <name>instance-00000381</name>
  <uuid>a65c782d-05fe-4c54-9911-8e7e4cb33244</uuid>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <os>
    <type arch='x86_64' machine='xenpv'>linux</type>
    <kernel>/var/lib/nova/instances/instance-00000381/kernel</kernel>
    <cmdline>ro root=/dev/xvda1</cmdline>
    <root>/dev/xvda1</root>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <disk type='file' device='disk'>
      <driver name='tap' type='raw' cache='none'/>
      <source file='/var/lib/nova/instances/instance-00000381/disk'/>
      <target dev='xvda' bus='xen'/>
    </disk>
    <interface type='bridge'>
      <mac address='fa:16:3e:55:5d:e0'/>
      <source bridge='br-tvd-91970' bridgetype='openvswitch'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='67d41c55-2849-4bc6-b138-c685ae9d3d5f'/>
      </virtualport>
      <script path='/etc/xen/scripts/vif-openvswitch'/>
    </interface>
    <console type='pty'>
      <target type='xen' port='0'/>
    </console>
    <input type='mouse' bus='xen'/>
    <graphics type='vnc' port='5900' autoport='yes' listen='node-110' keymap='en-us'>
      <listen type='address' address='node-110'/>
    </graphics>
    <video>
      <model type='xen' vram='4096' heads='1'/>
    </video>
    <memballoon model='xen'/>
  </devices>
</domain>

Parsing config file domain.xl
(domain
        (domid -1)
        (create_info)
        (hvm 0)
        (hap 1)
        (oos 1)
        (ssidref 0)
        (name instance-00000381)
        (uuid <unknown>)
        (cpupool Pool-0)
        (xsdata (null))
        (platformdata (null))
        (build_info)
        (max_vcpus 1)
        (tsc_mode 0)
        (max_memkb 524288)
        (target_memkb 524288)
        (nomigrate 0)
        (image
                (linux 0)
                        (kernel /var/lib/nova/instances/instance-00000381/kernel)
                        (cmdline ro root=/dev/xvda1)
                        (ramdisk (null))
                )
        )
        (device
                (tap
                        (backend_domid 0)
                        (frontend_domid 0)
                        (physpath /var/lib/nova/instances/instance-00000381/disk)
                        (phystype 2)
                        (virtpath xvda)
                        (unpluggable 0)
                        (readwrite 1)
                        (is_cdrom 0)
                )
        )
        (device
                (vif
                        (backend_domid 0)
                        (frontend_domid 0)
                        (devid 0)
                        (mtu 1492)
                        (model rtl8139)
                        (mac fa163e555de0)
                )
        )
        (device
                (vfb
                        (backend_domid 0)
                        (frontend_domid 0)
                        (devid 0)
                        (vnc 1)
                        (vnclisten node-110)
                        (vncdisplay 0)
                        (vncunused 1)
                        (keymap en-us)
                        (sdl 0)
                        (opengl 0)
                        (display (null))
                        (xauthority (null))
                )
        )
)
Daemon running with PID 3658
host                   : node-110
release                : 3.2.0-40-ima
version                : #64 SMP Tue Apr 9 14:21:15 CEST 2013
machine                : x86_64
nr_cpus                : 2
nr_nodes               : 1
cores_per_socket       : 2
threads_per_core       : 1
cpu_mhz                : 2128
hw_caps                : bfebfbff:20100800:00000000:00000940:0000e3bd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 2023
free_memory            : 151
free_cpus              : 0
xen_major              : 4
xen_minor              : 1
xen_extra              : .2
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline        : placeholder
cc_compiler            : gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) 
cc_compile_by          : roberto.sassu
cc_compile_domain      : polito.it
cc_compile_date        : Fri Jul 26 11:33:14 CEST 2013
xend_config_format     : 4
domid: 7
Warning: vlan 0 is not connected to host network
-videoram option does not work with cirrus vga device model. Videoram set to 4M.
char device redirected to /dev/pts/3
xs_read(): target get error. /local/domain/7/target.
xs_read(): vncpasswd get error. /vm/a65c782d-05fe-4c54-9911-8e7e4cb33244/vncpasswd.
Sep 11 13:42:08 node-110 logger: /etc/xen/scripts/vif-openvswitch: online type_if=vif XENBUS_PATH=backend/vif/6/0
Sep 11 13:42:08 node-110 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=30 -- --if-exists del-port vif6.0 -- add-port br-tvd-91970 vif6.0 -- set interface vif6.0 "external-ids:\"xen-vm-name\"=\"instance-00000381\"" -- set interface vif6.0 "external-ids:\"xen-vm-uuid\"=\"a65c782d-05fe-4c54-9911-8e7e4cb33244\"" -- set interface vif6.0 "external-ids:\"attached-mac\"=\"fa:16:3e:55:5d:e0\""
Sep 11 13:42:08 node-110 kernel: [  243.392972] device vif6.0 entered promiscuous mode
Sep 11 13:42:08 node-110 kernel: [  243.399111] ADDRCONF(NETDEV_UP): vif6.0: link is not ready
Sep 11 13:42:08 node-110 logger: /etc/xen/scripts/vif-openvswitch: Successful vif-openvswitch online for vif6.0.
Sep 11 13:42:08 node-110 logger: /etc/xen/scripts/vif-openvswitch: Writing backend/vif/6/0/hotplug-status connected to xenstore.
Sep 11 13:42:09 node-110 kernel: [  244.678444] ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready
Sep 11 13:42:12 node-110 kernel: [  247.923648] vif vif-6-0: 2 reading script
Sep 11 13:42:12 node-110 kernel: [  247.927140] device vif6.0 left promiscuous mode
Sep 11 13:42:12 node-110 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/qdisk/6/51712
Sep 11 13:42:12 node-110 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/vif/6/0
Sep 11 13:42:12 node-110 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/vfb/6/0
Sep 11 13:42:12 node-110 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/console/6/0
Sep 11 13:42:12 node-110 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/vkbd/6/0
Sep 11 13:42:13 node-110 logger: /etc/xen/scripts/vif-openvswitch: offline type_if=vif XENBUS_PATH=backend/vif/6/0
Sep 11 13:42:13 node-110 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=30 -- --if-exists del-port vif6.0
Sep 11 13:42:13 node-110 logger: /etc/xen/scripts/vif-openvswitch: ip link set vif6.0 down failed
Sep 11 13:42:13 node-110 logger: /etc/xen/scripts/vif-openvswitch: Successful vif-openvswitch offline for vif6.0.

tool = ""
 xenstored = ""
local = ""
 domain = ""
  0 = ""
   name = "Domain-0"
   memory = ""
    target = "1387864"
    static-max = "4294967292"
    freemem-slack = "153544"
   backend = ""
    qdisk = ""
     1 = ""
      51712 = ""
       frontend = "/local/domain/1/device/vbd/51712"
       params = "aio:/var/lib/nova/instances/instance-00000381/disk"
       frontend-id = "1"
       online = "0"
       removable = "0"
       bootable = "1"
       state = "5"
       dev = "xvda"
       type = "tap"
       mode = "w"
       feature-barrier = "1"
       info = "0"
       sector-size = "512"
       sectors = "4194304"
       hotplug-status = "connected"
    vfb = ""
     1 = ""
      0 = ""
       frontend = "/local/domain/1/device/vfb/0"
       frontend-id = "1"
       online = "0"
       state = "5"
       domain = "instance-00000381"
       vnc = "1"
       vnclisten = "node-110"
       vncdisplay = "0"
       vncunused = "1"
       sdl = "0"
       opengl = "0"
       feature-resize = "1"
       hotplug-status = "connected"
       request-update = "1"
    vkbd = ""
     1 = ""
      0 = ""
       frontend = "/local/domain/1/device/vkbd/0"
       frontend-id = "1"
       online = "0"
       state = "5"
       domain = "instance-00000381"
       feature-abs-pointer = "1"
       hotplug-status = "connected"
    console = ""
     1 = ""
      0 = ""
       frontend = "/local/domain/1/console"
       frontend-id = "1"
       online = "0"
       state = "5"
       domain = "instance-00000381"
       protocol = "vt100"
       hotplug-status = "connected"
   device-model = ""
vm = ""
 00000000-0000-0000-0000-000000000000 = ""
  memory = "1355"
domainbuilder: detail: xc_dom_allocate: cmdline="ro root=/dev/xvda1", features="(null)"
domainbuilder: detail: xc_dom_kernel_file: filename="/var/lib/nova/instances/instance-00000381/kernel"
domainbuilder: detail: xc_dom_malloc_filemap    : 4849 kB
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... 
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... 
domainbuilder: detail: xc_dom_malloc            : 18250 kB
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x4b35c9 -> 0x11d2aa0
domainbuilder: detail: loader probe OK
xc: detail: elf_parse_binary: phdr: paddr=0x1000000 memsz=0xad6000
xc: detail: elf_parse_binary: phdr: paddr=0x1c00000 memsz=0xe30e0
xc: detail: elf_parse_binary: phdr: paddr=0x1ce4000 memsz=0x143c0
xc: detail: elf_parse_binary: phdr: paddr=0x1cf9000 memsz=0x35f000
xc: detail: elf_parse_binary: memory: 0x1000000 -> 0x2058000
xc: detail: elf_xen_parse_note: GUEST_OS = "linux"
xc: detail: elf_xen_parse_note: GUEST_VERSION = "2.6"
xc: detail: elf_xen_parse_note: XEN_VERSION = "xen-3.0"
xc: detail: elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
xc: detail: elf_xen_parse_note: ENTRY = 0xffffffff81cf9200
xc: detail: elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
xc: detail: elf_xen_parse_note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
xc: detail: elf_xen_parse_note: PAE_MODE = "yes"
xc: detail: elf_xen_parse_note: LOADER = "generic"
xc: detail: elf_xen_parse_note: unknown xen elf note (0xd)
xc: detail: elf_xen_parse_note: SUSPEND_CANCEL = 0x1
xc: detail: elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
xc: detail: elf_xen_parse_note: PADDR_OFFSET = 0x0
xc: detail: elf_xen_addr_calc_check: addresses:
xc: detail:     virt_base        = 0xffffffff80000000
xc: detail:     elf_paddr_offset = 0x0
xc: detail:     virt_offset      = 0xffffffff80000000
xc: detail:     virt_kstart      = 0xffffffff81000000
xc: detail:     virt_kend        = 0xffffffff82058000
xc: detail:     virt_entry       = 0xffffffff81cf9200
xc: detail:     p2m_base         = 0xffffffffffffffff
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0xffffffff81000000 -> 0xffffffff82058000
domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x20000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc            : 1024 kB
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_alloc_segment:   kernel       : 0xffffffff81000000 -> 0xffffffff82058000  (pfn 0x1000 + 0x1058 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x1000+0x1058 at 0x7f42bafa8000
xc: detail: elf_load_binary: phdr 0 at 0x139924581548032 -> 0x139924592910336
xc: detail: elf_load_binary: phdr 1 at 0x139924594130944 -> 0x139924595060960
xc: detail: elf_load_binary: phdr 2 at 0x139924595064832 -> 0x139924595147712
xc: detail: elf_load_binary: phdr 3 at 0x139924595150848 -> 0x139924596039680
domainbuilder: detail: xc_dom_alloc_segment:   phys2mach    : 0xffffffff82058000 -> 0xffffffff82158000  (pfn 0x2058 + 0x100 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x2058+0x100 at 0x7f42ef6ba000
domainbuilder: detail: xc_dom_alloc_page   :   start info   : 0xffffffff82158000 (pfn 0x2158)
domainbuilder: detail: xc_dom_alloc_page   :   xenstore     : 0xffffffff82159000 (pfn 0x2159)
domainbuilder: detail: xc_dom_alloc_page   :   console      : 0xffffffff8215a000 (pfn 0x215a)
domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0xffff000000000000 -> 0xffffffffffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0xffffff8000000000 -> 0xffffffffffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0xffffffff80000000 -> 0xffffffffbfffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0xffffffff80000000 -> 0xffffffff823fffff, 18 table(s)
domainbuilder: detail: xc_dom_alloc_segment:   page tables  : 0xffffffff8215b000 -> 0xffffffff82170000  (pfn 0x215b + 0x15 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x215b+0x15 at 0x7f42ef800000
domainbuilder: detail: xc_dom_alloc_page   :   boot stack   : 0xffffffff82170000 (pfn 0x2170)
domainbuilder: detail: xc_dom_build_image  : virt_alloc_end : 0xffffffff82171000
domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0xffffffff82400000
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: arch_setup_bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x20000
domainbuilder: detail: clear_page: pfn 0x215a, mfn 0x4adad
domainbuilder: detail: clear_page: pfn 0x2159, mfn 0x4adae
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x2158+0x1 at 0x7f42ef7ff000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0xffffffff81001000 pfn=0x1001
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail:    allocated
domainbuilder: detail:       malloc             : 19381 kB
domainbuilder: detail:       anon mmap          : 0 bytes
domainbuilder: detail:    mapped
domainbuilder: detail:       file mmap          : 4849 kB
domainbuilder: detail:       domU mmap          : 17848 kB
domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0x4bb68
domainbuilder: detail: shared_info_x86_64: called
domainbuilder: detail: vcpu_x86_64: called
domainbuilder: detail: vcpu_x86_64: cr3: pfn 0x215b mfn 0x4adac
domainbuilder: detail: launch_vm: called, ctxt=0x7f42eb556300
domainbuilder: detail: xc_dom_release: called

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]