[libvirt] Cannot start domain w/xenner emulator

[Gerry Reno]

Gerry Reno wrote:
> Daniel P. Berrange wrote:
>> On Tue, Dec 15, 2009 at 12:02:50AM -0500, Gerry Reno wrote:
>>> # virsh start Ubuntu-domU-x86-1
>>> Connecting to uri: qemu:///system
>>> error: Failed to start domain Ubuntu-domU-x86-1
>>> error: internal error unable to start guest: libvir: error : cannot 
>>> execute binary /usr/bin/xenner: Permission denied
>>>
>>> # ls -l /usr/bin/xenner
>>> -rwxr-xr-x 1 root root 118936 2009-12-14 17:05 /usr/bin/xenner
>>>
>>> Sure looks executable. And there's no SELinux on the box.
>>>
>>> libvirt is 0.7.0
>>>
>>> What the..?
>>
>> Seriously bizarre ! What distro ? You might want to strace the 
>> libvirtd daemon
>>
>> strace -f -o trace.log $PID-OF-LIBVIRTD
>>
>> and then look in trace.log for EPERM / EACCESS on any syscall.
>>
>> Daniel
>
>
> The host is Ubuntu 9.10. And apparmor has been disabled but what's 
> strange is that it looks like the kernel is loading a profile and then 
> denying access and then removing the profile.
>
> # virsh start Ubuntu-domU-x86-1
> Connecting to uri: qemu:///system
> error: Failed to start domain Ubuntu-domU-x86-1
> error: internal error unable to start guest: libvir: error : cannot 
> execute binary /usr/bin/xenner: Permission denied
>
> # ls -l /usr/bin/xenner
> -rwxr-xr-x 1 root root 118936 2009-12-14 17:05 /usr/bin/xenner
>
> + apparmor_status
> apparmor module is loaded.
> 0 profiles are loaded.
> 0 profiles are in enforce mode.
> 0 profiles are in complain mode.
> 0 processes have profiles defined.
> 0 processes are in enforce mode :
> 0 processes are in complain mode.
> 0 processes are unconfined but have a profile defined.
>
>
> # /var/log/kern.log:
> Dec 15 11:28:35 grp-01-23-02 kernel: [213217.260223] type=1503 
> audit(1260894515.683:73): operation="exec" pid=16770 parent=16769 
> profile="libvirt-74367128-9bd6-3264-3833-f661c47b464e" 
> requested_mask="x::" denied_mask="x::" fsuid=0 ouid=0 
> name="/usr/bin/xenner"
> Dec 15 11:28:35 grp-01-23-02 kernel: [213217.305593] br0: port 
> 2(vnet0) entering disabled state
> Dec 15 11:28:35 grp-01-23-02 kernel: [213217.344700] device vnet0 left 
> promiscuous mode
> Dec 15 11:28:35 grp-01-23-02 kernel: [213217.344703] br0: port 
> 2(vnet0) entering disabled state
> Dec 15 11:28:35 grp-01-23-02 kernel: [213217.560461] type=1505 
> audit(1260894515.976:74): operation="profile_remove" pid=16772 
> name=libvirt-74367128-9bd6-3264-3833-f661c47b464e namespace=default
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.133717] type=1505 
> audit(1260894620.556:75): operation="profile_load" pid=16995 
> name=libvirt-74367128-9bd6-3264-3833-f661c47b464e
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.137277] device vnet0 
> entered promiscuous mode
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.138430] br0: port 
> 2(vnet0) entering learning state
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.139715] type=1503 
> audit(1260894620.556:76): operation="exec" pid=16999 parent=16998 
> profile="libvirt-74367128-9bd6-3264-3833-f661c47b464e" 
> requested_mask="x::" denied_mask="x::" fsuid=0 ouid=0 
> name="/usr/bin/xenner"
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.181318] br0: port 
> 2(vnet0) entering disabled state
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.220628] device vnet0 left 
> promiscuous mode
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.220631] br0: port 
> 2(vnet0) entering disabled state
> Dec 15 11:30:20 grp-01-23-02 kernel: [213322.359241] type=1505 
> audit(1260894620.773:77): operation="profile_remove" pid=17001 
> name=libvirt-74367128-9bd6-3264-3833-f661c47b464e namespace=default
>
>
> -Gerry
>
>
For the moment I got by the apparmor profile issue by completely 
removing the package.

The domU guest still does not start however:

+ virsh start Ubuntu-domU-x86-1
Connecting to uri: qemu:///system
error: Failed to start domain Ubuntu-domU-x86-1
error: internal error unable to start guest: [xenner,1] 
qemu_monitor_config: 
unix:/var/run/libvirt/qemu/Ubuntu-domU-x86-1.monitor,server,nowait
[xenner,1] qemu_serial_config: pty
[xenner,1] qemu_disk_config_blkbackd: 
file=/var/lib/libvirt/images/ubuntu.9-10.x86.img,if=scsi,index=0
[xenner,1] qemu_disk_config_blkbackd: if != xen, ignoring disk
[xenner,1] qemu_net_config_netbackd: nic,macaddr=54:52:07:2d:e1:81,vlan=0
[xenner,1] qemu_net_config_netbackd: 0: mac="54:52:07:2d:e1:81"
[xenner,1] qemu_net_config_netbackd: tap,fd=16,vlan=0
[xenner,1] qemu_net_config_netbackd: 0: if="vnet2"
started as: "/usr/bin/xenner" "-S" "-M" "pc" "-m" "2048" "-smp" "4" 
"-name" "Ubuntu-domU-x86-1" "-uuid" 
"74367128-9bd6-3264-3833-f661c47b464e" "-domid" "6" "-nographic" 
"-monitor" 
"unix:/var/run/libvirt/qemu/Ubuntu-domU-x86-1.monitor,server,nowait" 
"-boot" "c" "-kernel" 
"/home/greno/xen/domU/x86/ubuntu-9.10-karmic/kernel/boot/vmlinuz-2.6.31.6" 
"-append" "xencons=xvc console=tty1 console=xvc0" "-drive" 
"file=/var/lib/libvirt/images/ubuntu.9-10.x86.img,i

It cuts the command line off as you can see in the output but if I take 
that xenner command line from the log and run it manually then the domU 
guest starts fine. So is there a workaround to this qemu_monitor_config 
error?

-Gerry