This chapter will cover some basic troubleshooting tips and hints. The normal approach to troubleshooting, including techniques and tools, apply to a guest operating system using para-virtualized drivers. However there are some messages and tasks which are specific to guest operating systems using para-virtualized drivers.
The sections below provide methods you can use for identifying where the problems lie and how to fix them.
Verifying the para-virtualized drivers have successfuly loaded
One of the first tasks you will want to do is to verify that the drivers have actually been loaded into your system.
After the para-virtualized drivers have been installed and the guest has been rebooted you can verify that the drivers have loaded. First you should confirm the drivers have logged their loading into /var/log/messages
[root@rhel4u4]# grep -E "vif|vbd|xen" /var/log/messages xen_mem: Initialising balloon driver vif vif-0: 2 parsing device/vif/0/mac vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/21/76 vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/21/76 xen-vbd: registered block device major 202
You can also use the lsmod command to list the loaded para-virtualized drivers. It should output a list containing the xen_vnif, xen_vbd, xen_platform_pci and xen_balloon modules.
[root@rhel4u4]# lsmod|grep xen xen_vbd 19168 1 xen_vnif 28416 0 xen_balloon 15256 1 xen_vnif xen_platform_pci 98520 3 xen_vbd,xen_vnif,xen_balloon,[permanent]
Manually loading the para-virtualized drivers
If for some reason the para-virtalized drivers failed to load automatically during the boot process you can attempt to load them manually.
This will allow you to reconfigure network or storage entities or identify why they failed to load in the first place. The steps below should load the para-virtualized driver modules.
First, locate the para-virtualized driver modules on your system.
[root@rhel]# cd /lib/modules/`uname -r`/ [root@rhel]# find . -name 'xen-*.ko' -print
Take note of the location and load the modules manually. Substitute {LocationofPV-drivers} with the correct location you noted from the output of the commands above.
[root@rhel]# insmod \
/lib/modules/'uname -r'/{LocationofPV-drivers}/xen-platform-pci.ko
[root@rhel]# insmod \
/lib/modules/'uname -r'/{LocationofPV-drivers}/xen-balloon.ko
[root@rhel]# insmod \
/lib/modules/'uname -r'/{LocationofPV-drivers}/xen-vnif.ko
[root@rhel]# insmod \
/lib/modules/'uname -r'/{LocationofPV-drivers}/xen-vbd.ko
Identifying available storage and partitions
Verify the block driver is loaded and the devices and partitions are available to the guest. This can be done by executing "cat /proc/partitions" as seen below.
[root@rhel3]# cat /proc/partitions major minor #blocks name 202 16 104857600 xvdb 3 0 8175688 hda
Xen Virtual Ethernet not found by network tools
The networking tools cannot identify the 'Xen Virtual Ethernet' networking card inside the guest operation system you should execute cat /etc/modprobe.conf(in Red Hat Enterprise Linux 4 and Red Hat Enterprise Linux 5) or cat /etc/modules.conf(in Red Hat Enterprise Linux 3). The output should contain the line “alias eth0 xen-vnif” and a similar line for each additional interface. To fix this problem you will need to add the aliasing lines (for example, alias eth0 xen-vnif) for every para-virtualized interface for the guest.
The system still has limited throughput
If network throughput is still limited even after installing the para-virtualized drivers and you have confirmed they are loaded correctly (see Verifying the para-virtualized drivers have successfuly loaded). To fix this problem, remove the 'type=ioemu' part of 'vif=' line in your guest's configuration file.
What to do if the guest operating system has been booted with virt-manager or virsh
As mentioned in the installation notes, a guest operating system with network para-virtualized drivers installed must be started using the “# xm create GuestName” command. You can only use other methods for starting the guest in Red Hat Enterprise Linux 5.2.
If the guest operating system has been booted using the virt-manager(the GUI tool) or virsh(the command line application) interface the boot process will detect the “new” old Realtek card. This due to the fact libvirt, as the underlying API to virt-manager and virsh, will always add type=ioemu to the networking section followed by prompting the systems adminstrator to reconfigure networking inside the guest. It is recommend you interrupt the boot process (using virt-manager, virsh or xm) and to boot the guest using the xm command. In the event of the guest operating system has booted all the way to multi-user mode you will detect that there is no networking active as the backend and frontend drivers are not connected properly.
To fix this issue, shut down the guest and boot it using “xm create”. During the boot process kudzu (the hardware detection process) will detect the “old” Realtek card. Simply select “Remove Configuration” to delete the Realtek card from the guest operating system. The guest should continue to boot and configure the network interfaces correctly.
You can identify if your guest has been booted with virt-manager, virsh or “xm create” using the command “# xm list –long YourGuestName”
In the screenshot below you can see the entry “ioemu” highlighted in the “device vif” (networking) section. This would mean the guest was booted with virt-manager or virsh and networking is not configured correctly, that is, without the para-virtualized network driver.
In the screeshot below you can see there is no “type ioemu” entry in the “device vif” section so you can safely assume the guest has been booted with “xm create YourGuestName”. This means networking is configured to use the para-virtualized network driver.
Para-virtualized fail to load on a Red Hat Enterprise Linux 3 guest operating system
Red Hat Enterprise Linux 3 is uses processor architecture specific kernel RPMs and because of this the para-virtualized drivers may fail to load if the para-virtualized driver RPM does not match the installed kernel architecture.
When the para-virtualized driver modules are inserted, a long list of unresolved modules will be displayed. A shortened excerpt of the error can be seen below.
insmod xen-platform-pci.o Warning: kernel-module version mismatch xen-platform-pci.o was compiled for kernel version 2.4.21-52.EL while this kernel is version 2.4.21-50.EL xen-platform-pci.o: unresolved symbol __ioremap_R9eac042a xen-platform-pci.o: unresolved symbol flush_signals_R50973be2 xen-platform-pci.o: unresolved symbol pci_read_config_byte_R0e425a9e xen-platform-pci.o: unresolved symbol __get_free_pages_R9016dd82 [...]
The solution is to use the correct RPM package for your hardware architecture for the para-virtualized drivers.
A warning message is displayed while installing the para-virtualized drivers on Red Hat Enterprise Linux 3
Installing the para-virtualized drivers on a Red Hat Enterprise Linux 3 kernel prior to 2.4.21-52 may result in a warning message being displayed stating the modules have been compiled with a newer version than the running kernel.
This message, as seen below, can be safely ingnored.
Warning: kernel-module version mismatch
xen-platform-pci.o was compiled for kernel version 2.4.21-52.EL
while this kernel is version 2.4.21-50.EL
Warning: loading xen-platform-pci.o will taint the kernel: forced load
See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Module xen-platform-pci loaded, with warnings
The important part of the message above is the last line which should state the module has been loaded with warnings.