Issue #3 January 2005

Desktop and hardware configuration

by David Zeuthen

Introduction

Notwithstanding the fact that operating systems based on the Linux kernel sport hardware compatibility for a wide range of devices, having those device Just Work has often been missing. Traditionally, users have often had to search the Internet and drop down to the command line to make their hardware work; highly technical jargon like /dev/ttyUSB0 or /dev/sda1 have more often than not, served to confuse the user. To make the desktop usable for as many people as possible, we must let them use their hardware without making them jump through hoops to make their hardware work.

Included in Fedora Core 3 and in upcoming Red Hat Enterprise Linux releases, is a new layer called HAL (Hardware Abstraction Layer). It provides an easy way for applications to discover the hardware on the system.

How is HAL used in Fedora Core 3? :
In Fedora Core 3, HAL is used for discovering storage, networking, digital cameras, and printers.

In Fedora Core 3, HAL is used for discovering storage, networking, digital cameras, and printers.

Central management

Traditionally, desktop applications discovered hardware by talking directly to the operating system kernel (the kernel maintains the list of devices attached to the system). This is a tedious process and is not exact because sometimes the kernel doesn't know everything about a device. For example, some digital cameras and portable music players show up as just another hard disk in the user interface. Thus, not many user interfaces have been built for hardware discovery.

With HAL, all the interesting information about certain classes of hardware is easily accessible in a well-defined format. When a new device is added to the system, an asynchronous signal is broadcast on the system message bus detailing what kind of device was added. Any desktop application can easily connect to the message bus to discover hardware. In addition, system-level scripts can be run to configure the device.

What is a system message bus? :
Fedora Core 3 ships with D-BUS that is, among other things, used for providing a system wide message bus that enables applications to talk to one another.

As shown in Figure 1, “Architecture”, the HAL daemon communicates with lower layers via standard interfaces provided by linux-hotplug, udev, and the kernel, and is notified when devices are added or removed.

Architecture
Figure 1. Architecture

Internally, the HAL daemon maintains a list of device objects that contains well-defined key/value pairs describing what the object represents. Each device object is identified by an Unique Device Identifier, or UDI, which is unique across all device objects. The key/value pairs, called device properties, are typed and are defined in the HAL specification, so users of HAL know what values each property can assume. As an example, the device properties for a wired networking adapter are shown in Figure 2, “Device Manager”.

Device Manager
Figure 2. Device Manager

At a first glance, there appears to be a one-to-one correspondence between the device objects in the kernel and the device objects in the HAL daemon. One main difference, however, is that the device objects in the daemon contain a lot more information than what the kernel knows; the reason for this is that the daemon merges information from several other sources than just the kernel:

  1. For storage devices, the HAL daemon probes the file system type, something the kernel cannot, and should not, do, as it is an expensive operation.
  2. Wired networking devices are monitored by HAL to report link status and speed.
  3. XML Files, known as Device Information Files, are used for matching certain key/value pairs. On match, additional key/value pairs are merged in these files. For more information, refer to the section called “Device Information Files”.
Why is it expensive to probe for file systems? :
Any storage device, be it a hard disk, floppy disk, zip disk, memory card, or optical disc is exposed as a block device. While the kernel will split the block device into smaller block devices, one for each partition, it will not attempt to probe for a file system on each partition. Probing for a file system involves checking a number of magic signatures that can be spread out on the disk, and this can involve a lot of I/O traffic. HAL can detect more than 15 different file systems, including extracting the file system label or UUID if it exists.

Device Information Files

Device Information Files, are one of the most powerful features of HAL — it allows HAL to merge new or adjust existing properties of a device object. For example, there is no systematic way to determine from the hardware itself that a given storage device is a Compact Flash reader. However, if the storage device is USB-based one can look at the Vendor and Product identifiers of the hardware. This is achieved by the following device information file: 


<deviceinfo version="0.2">
  <device>
    <match key="@storage.physical_device:info.bus" string="usb">
      <match key="@storage.physical_device:usb.vendor_id" int="0x05dc">
        <match key="@storage.physical_device:usb.product_id" int="0x0002">
	  <merge key="storage.drive_type" type="string">compact_flash</merge>
        </match>
      </match>
    </match>
  </device>
</deviceinfo>

This file essentially says "if the physical device that this storage device stems from is a USB device with vendor_id 0x05dc and product_id 0x0002, then set the drive type to compact_flash." As shown in the top right corner of Figure 2, “Device Manager”, the GNOME desktop has an appropriate icon (the one with the CANON_DC label) that resembles Compact Flash media because it has support for querying HAL.

Device objects

Once the HAL daemon has collected all the information about a device, including merging information from device information files, it is ready to announce the presence of the new device object.

For a card reader, for example, one or more device objects are created; one for the storage device and one for each volume (which can be a file system). The lshal command from the hal package can be used to query the HAL daemon for devices — it returns a list of all devices objects that the HAL daemon knows about (output is truncated for brevity): 


Dumping 37 device(s) from the Global Device List:
-------------------------------------------------
udi = '/org/freedesktop/Hal/devices/pci_8086_3340'
  info.parent = '/org/freedesktop/Hal/devices/computer'  (string)
  info.udi = '/org/freedesktop/Hal/devices/pci_8086_3340'  (string)
  pci.device_protocol = 0  (0x0)  (int)
  pci.device_subclass = 0  (0x0)  (int)
  pci.device_class = 6  (0x6)  (int)
  info.vendor = 'Intel Corp.'  (string)
  info.product = '82855PM Processor to I/O Controller'  (string)
  pci.subsys_vendor = 'IBM'  (string)
  pci.product = '82855PM Processor to I/O Controller'  (string)
  pci.vendor = 'Intel Corp.'  (string)
  pci.subsys_product_id = 1321  (0x529)  (int)
  pci.subsys_vendor_id = 4116  (0x1014)  (int)
  pci.product_id = 13120  (0x3340)  (int)
  pci.vendor_id = 32902  (0x8086)  (int)
  pci.linux.sysfs_path = '/sys/devices/pci0000:00/0000:00:00.0'  (string)
  linux.sysfs_path_device = '/sys/devices/pci0000:00/0000:00:00.0'  (string)
  linux.sysfs_path = '/sys/devices/pci0000:00/0000:00:00.0'  (string)
  info.bus = 'pci'  (string)

udi = '/org/freedesktop/Hal/devices/pci_1002_4c57'
  info.udi = '/org/freedesktop/Hal/devices/pci_1002_4c57'  (string)
  pci.device_protocol = 0  (0x0)  (int)
  pci.device_subclass = 0  (0x0)  (int)

...

  linux.sysfs_path_device = '/sys/devices/pci0000:00/0000:00:1f.6'  (string)
  linux.sysfs_path = '/sys/devices/pci0000:00/0000:00:1f.6'  (string)
  info.bus = 'pci'  (string)

udi = '/org/freedesktop/Hal/devices/computer'
  storage.policy.default.mount_option.exec = true  (bool)
  storage.policy.default.mount_option.pamconsole = true  (bool)
  storage.policy.default.mount_option.noauto = true  (bool)
  storage.policy.default.managed_keyword.secondary = 'kudzu'  (string)
  storage.policy.default.managed_keyword.primary = 'managed'  (string)
  storage.policy.default.use_managed_keyword = true  (bool)
  storage.policy.default.mount_root = '/media'  (string)
  linux.is_selinux_enabled = true  (bool)
  kernel.machine = 'i686'  (string)
  kernel.version = '2.6.10-1.1063_FC4'  (string)
  kernel.name = 'Linux'  (string)
  info.udi = '/org/freedesktop/Hal/devices/computer'  (string)
  info.product = 'Computer'  (string)
  linux.sysfs_path_device = '(none)'  (string)
  info.bus = 'unknown'  (string)


Dumped 37 device(s) from the Global Device List
-----------------------------------------------

Some of the more interesting properties for the device object representing the storage device discussed above, are: 


block.device = '/dev/sda'  (string)
block.have_scanned = false  (bool)
block.is_volume = false  (bool)
block.major = 8  (0x8)  (int)
block.minor = 0  (0x0)  (int)
block.no_partitions = false  (bool)
block.storage_device = '/org/freedesktop/Hal/devices/block_8_0'  (string)
storage.bus = 'usb'  (string)
storage.drive_type = 'compact_flash'  (string)
storage.hotpluggable = true  (bool)
storage.media_check_enabled = true  (bool)
storage.model = 'READER'  (string)
storage.removable = true  (bool)
storage.requires_eject = false  (bool)
storage.vendor = 'LEXAR CF'  (string)
info.product = 'READER'  (string)
info.vendor = 'LEXAR CF'  (string)
info.udi = '/org/freedesktop/Hal/devices/block_8_0' (string)
info.capabilities = 'block storage'
storage.physical_device = '/org/freedesktop/Hal/devices/usb_usb_device_5dc_2_1_-1_noserial_0'

From these properties it is possible to infer the make and model of the storage device (storage.vendor and storage.model), and we can also see that this device can be attached and detached while the system is running (storage.hotpluggable) and that it uses removable media (storage.removable). We also know that media cannot be ejected from the drive (storage.requires_eject) and that it is connected to a USB bus (storage.bus). Finally, since the device uses removable media, HAL has enabled media detection for the drive which involves polling the drive for media every few seconds. The storage.drive_type property describes the kind of drive and, as evident, this is set to compact_flash. Under normal circumstances this would be set to disk, but we used the device information mentioned above to override it. Addressing details for accessing the device represented by this device object is in the block.* properties; notably the special device file can be obtained from the block.device property.

For management, the info.udi property contains the Unique Device Identifier for the device object, and info.product and info.vendor contain, as generic properties, the make and model of the device that the device object represents — in this case this is copied directly from the storage.vendor and storage.model properties. But, for example, for printers it is copied from printer.vendor and printer.model. The info.capabilities property is used to indicate the presence of block.* and storage.* properties: what the capabilities of the device objects are.

One can also see that the storage.physical_device property contains the Unique Device Identifier for the device object representing the physical device that is the storage device — in this case a USB Mass Storage interface for a USB device. In turn, the device object for this USB Mass Storage Interface received other well defined properties. Here are some of the interesting ones: 


info.bus = 'usb'  (string)
info.udi = '/org/freedesktop/Hal/devices/usb_usb_device_5dc_2_1_-1_noserial_0'  (string)
info.product = 'USB Mass Storage Interface'  (string)
usb.interface.subclass = 6  (0x6)  (int)
usb.interface.protocol = 50  (0x32)  (int)
usb.interface.number = 0  (0x0)  (int)
usb.interface.class = 8  (0x8)  (int)
usb.configuration_value = 1  (0x1)  (int)
usb.device_class = 0  (0x0)  (int)
usb.device_protocol = 0  (0x0)  (int)
usb.device_subclass = 0  (0x0)  (int)
usb.max_power = 100  (0x64)  (int)
usb.num_configurations = 1  (0x1)  (int)
usb.num_interfaces = 1  (0x1)  (int)
usb.device_revision_bcd = 1  (0x1)  (int)
usb.is_self_powered = false  (bool)
usb.can_wake_up = false  (bool)
usb.product_id = 2  (0x2)  (int)
usb.vendor_id = 1500  (0x5dc)  (int)
usb.vendor = 'Lexar Media, Inc.'  (string)
usb.product = 'CF READER   ?'  (string)
usb.bus_number = 3  (0x3)  (int)
usb.port_number = 2  (0x2)  (int)
usb.level_number = 1  (0x1)  (int)
usb.num_ports = 0  (0x0)  (int)
usb.speed_bcd = 4608  (0x1200)  (int)
usb.version_bcd = 256  (0x100)  (int)

All of the usb.* properties are also defined in the HAL specification. Now, returning to the card reader, if there is media in the device, then one or more device objects of capability volume will appear; some interesting properties for these are: 


block.device = '/dev/sda1'  (string)
block.have_scanned = false  (bool)
block.is_volume = true  (bool)
block.major = 8  (0x8)  (int)
block.minor = 1  (0x1)  (int)
block.no_partitions = false  (bool)
block.storage_device = '/org/freedesktop/Hal/devices/block_8_0'  (string)
volume.block_size = 512  (0x200)  (int)
volume.fstype = 'vfat'  (string)
volume.fsusage = 'filesystem'  (string)
volume.fsversion = 'FAT16'  (string)
volume.is_disc = false  (bool)
volume.is_mounted = true  (bool)
volume.is_partition = true  (bool)
volume.label = 'CANON_DC'  (string)
volume.mount_point = '/media/CANON_DC'  (string)
volume.num_blocks = 251632  (0x3d6f0)  (int)
volume.partition.msdos_part_table_type = 6  (0x6)  (int)
volume.partition.number = 1  (0x1)  (int)
volume.size = 128835584  (0x7ade000)  (uint64)
volume.uuid = '0A52-0269'  (string)
info.capabilities = 'block volume'  (string)
info.product = 'CANON_DC'  (string)
info.udi = '/org/freedesktop/Hal/devices/block_0A52-0269'  (string)

Here one can see that the volume.* properties contain a number of interesting facts about the device. First of all, it is a mountable file system (volume.fsusage) of type FAT (volume.fstype). It also tells the file system label (volume.label) and unique identifier (volume.uuid). Also, the fact that the volume stems from a partitioned disk is visible, including the partition type — in this case the volume is the first partition on a MSDOS partitioned disk and has type 0x06 (volume.partition.msdos_part_table_type). Again, the block.* properties describe how to address the device that the HAL device object represents. For example, one can see that the /dev/sda1 special device file can be used to access the file system: in other words, one can tell the kernel to mount /dev/sda1. Apart from mountable file systems, HAL understands a few other volumes without data such as optical discs that are either blank or contains only audio: 


block.no_partitions = true  (bool)
block.have_scanned = false  (bool)
block.is_volume = true  (bool)
block.device = '/dev/hdc'  (string)
block.major = 22  (0x16)  (int)
block.minor = 0  (0x0)  (int)
block.storage_device = '/org/freedesktop/Hal/devices/block_22_0'  (string)
volume.disc.is_rewritable = false  (bool)
volume.disc.is_appendable = false  (bool)
volume.disc.is_blank = true  (bool)
volume.disc.has_data = false  (bool)
volume.disc.has_audio = false  (bool)
volume.disc.type = 'cd_r'  (string)
volume.size = 2048  (0x800)  (uint64)
volume.block_size = 2048  (0x800)  (int)
volume.num_blocks = 4  (0x4)  (int)
volume.is_disc = true  (bool)
volume.is_mounted = false  (bool)
volume.mount_point = ''  (string)
volume.label = ''  (string)
volume.uuid = ''  (string)
volume.fsversion = ''  (string)
volume.fsusage = ''  (string)
volume.fstype = ''  (string)
info.udi = '/org/freedesktop/Hal/devices/block_22_0-0'  (string)
info.capabilities = 'block volume'  (string)

The volume.is_disc property is TRUE, so we can see that the device object represent an optical disc. We also see that the disc has no data nor audio and is, in fact, a blank CD-R disc.

Apart from the lshal command, the hal package included in Fedora Core 3 also contains the hal-get-property and hal-set-property commands which can be used in shell scripts. The hal-gnome package includes the hal-device-manager program (used to produce the screenshot shown in Figure 2, “Device Manager”) which gives a graphical user interface for querying the HAL daemon. This tool is useful for discovering what kind of information the HAL daemon exports - when properties on device objects are changing as well as when device objects are added or removed, the UI is updated in real time.

Callouts

Just prior to announcing the presence of a device object, operating system specific programs may be run to configure the base operating system. For example, for an unprivileged user to mount a file system, an entry must be added to the file system's table file /etc/fstab. In HAL, this is achieved through a callout — any program (or symlink to a program) in the /etc/hal/device.d/ directory is run whenever a device is added or removed. On Fedora Core 3, this applies to the fstab-sync program. The sole purpose of this program is to modify /etc/fstab when a mountable file system on a storage device is added or removed to the system. For the Compact Flash reader discussed earlier, fstab-sync adds the following entry: 


/dev/sda1 /media/CANON_DC vfat pamconsole,noatime,sync,fscontext=system_u:object_r:removable_t,exec,noauto,managed 0 0

Note the two new mount options pamconsole and managed. The former specifies that any (unprivileged) user sitting at the console may mount the file system. The latter, a no-op, specifies that this line was added by a program and not by the system administrator — hence the option managed is useful if the administrator has already manually added an entry since fstab-sync refuses to add an entry in that case.

In addition, device information files are used to determine if an entry should be added at all; this is useful because one has to be careful about automatically adding entries to /etc/fstab since it could mean data corruption. For the volume of our Compact Flash card, the following properties are merged: 


volume.policy.desired_mount_point = 'CANON_DC'  (string)
volume.policy.mount_filesystem = 'vfat'  (string)
volume.policy.mount_option.fscontext=system_u:object_r:removable_t = true  (bool)
volume.policy.mount_option.noatime = true  (bool)
volume.policy.mount_option.sync = true  (bool)
volume.policy.should_mount = true  (bool)

The full policy is spelled out in the file /usr/share/hal/fdi/90systempolicy/storage-policy.fdi on your Fedora Core 3 system, but basically the policy comes down to:

  1. Only allow mounting of USB, IDE, IEEE1394, SATA, and legacy floppy drives. For SCSI, only allow mounting of optical drives.
  2. For MSDOS-style partitioned media, only allow mounting if the partition table ID is in a white list.
  3. Only allow mounting if the storage device is hotpluggable or contains removable media.
  4. Use pamconsole, noauto, managed, and exec mount options per default. If SELinux is enabled, also use the fscontext=system_u:object_r:removable_t mount option. Finally, if the volume is smaller than 2GB use the noatime and sync mount options as well.
  5. If a volume has a label only in ASCII, use that label as the mount point.

It is possible to override this policy with device information files, for instance if the system administrator wants to force the mount point of a certain file system he would add the following device information file in the /usr/share/hal/fdi/95userpolicy/ directory. 


<device>
  <match key="block.is_volume" bool="true">
    <match key="volume.fsusage" string="filesystem">
      <match key="volume.uuid" string="4150-3F34">
        <merge key="volume.policy.desired_mount_point" type="string">my_location</merge>
      </match>
    </match>
  </match>
</device>

With this, fstab-sync would create the /media/my_location mount point and add this entry to the /etc/fstab file. This example matches on the filesystem UUID, but it is possible to match on other properties. For instance, some drives actually export a unique serial number that HAL exports as storage.serial. Another option is to match on the serial number of the USB device.

Notification for desktop applications

Once all callouts have completed, the HAL daemon broadcasts the presence of the device object on the system message bus. Included with Fedora Core 3 is the GNOME Volume Manager, a simple program that allows the user to customize actions for different removable media. Whenever HAL announces a new device (such as a mountable file system, an audio disc, or a digital camera), GNOME Volume Manager performs user customizable actions. Figure 3, “GNOME Volume Manager” shows the various actions that can be performed.

GNOME Volume Manager
Figure 3. GNOME Volume Manager

In addition to the GNOME Volume Manager, the GNOME Virtual File System uses HAL and its plethora of information to detect local storage devices — as a result the computer:/// location in the Nautilus file manager (that uses the GNOME VFS) looks like Figure 4, “GNOME VFS and Nautilus”.

GNOME VFS and Nautilus
Figure 4. GNOME VFS and Nautilus

Readers familiar with GNOME VFS will note that both the icons and labels describe the nature of the storage device more precisely. Instead of just the standard grey drive icon, icons now convey what kind of device is attached; instead of a cryptic name like cdrecorder, External CD-RW/DVD±RW Drive is used instead. For example, if the device is connected through a USB port, there is a USB emblem. If the device is a Compact Flash reader, the icon resembles a Compact Flash card.

From a development perspective, using HAL is easy since the API is exposed through D-BUS — connecting to the D-BUS system message bus to interact with the HAL service. Convenience libraries libhal and libhal-storage provides simple C libraries for interacting with HAL. In addition, D-BUS has bindings for a number of languages. Here's one example in Python: 


#!/usr/bin/python

import dbus

bus = dbus.Bus (dbus.Bus.TYPE_SYSTEM)
hal_service = bus.get_service ('org.freedesktop.Hal')
hal_manager = hal_service.get_object ('/org/freedesktop/Hal/Manager',
                                      'org.freedesktop.Hal.Manager')

volume_udi_list = hal_manager.FindDeviceByCapability ('volume')
for udi in volume_udi_list:
    volume = hal_service.get_object (udi, 'org.freedesktop.Hal.Device')
    device_file = volume.GetProperty ('block.device')
    fstype = volume.GetProperty ('volume.fstype')
    storage_udi = volume.GetProperty ('block.storage_device')
    storage = hal_service.get_object (storage_udi, 'org.freedesktop.Hal.Device')
    drive_type = storage.GetProperty ('storage.drive_type')
    print 'udi=%s device_file=%s fstype=%s drive_type=%s'%(udi, device_file, fstype, drive_type)

It's pretty straightforward — the HAL D-BUS service (org.freedesktop.Hal) exports an object in a well known (/org/freedesktop/Hal/Manager) location. This object implements a well known programmatic interface (org.freedesktop.Hal.Manager) that can be used to query for HAL device objects. The example program uses the FindDeviceByCapability method to query for all HAL device objects of capability volume. As discussed earlier, such device objects represent mountable file systems and optical discs. The method returns a list of UDIs, and the example program traverses these. All device objects implement another programmatic interface, org.freedesktop.Hal.Device. This interface is very simply. In a nutshell, it just provides the GetProperty() and SetProperty() methods to get and set device properties — for security, the latter method can only be invoked by the superuser. Since the HAL specification guarantees that all device objects of capable volumes export a certain set of properties, we can safely ask for those. Note that the program also ask for the drive_type of the storage device that the volume belongs to.

On a Fedora Core 3 system, the example program prints the following: 


udi=/org/freedesktop/Hal/devices/block_0A52-0269 device_file=/dev/sdb1 fstype=vfat drive_type=compact_flash
udi=/org/freedesktop/Hal/devices/block_419D-2DC5 device_file=/dev/sda fstype=vfat drive_type=floppy
udi=/org/freedesktop/Hal/devices/block_22_0-0 device_file=/dev/hdc fstype= drive_type=cdrom
udi=/org/freedesktop/Hal/devices/block_760a50a6-3185-47ee-b616-b4e7a1c4fc6d device_file=/dev/hda1 fstype=ext3 drive_type=disk
udi=/org/freedesktop/Hal/devices/block_3_2 device_file=/dev/hda2 fstype=LVM2_member drive_type=disk
udi=/org/freedesktop/Hal/devices/block_3_3 device_file=/dev/hda3 fstype=swap drive_type=disk
udi=/org/freedesktop/Hal/devices/block_3_4 device_file=/dev/hda4 fstype= drive_type=disk
udi=/org/freedesktop/Hal/devices/block_58a62f33-f935-4637-947e-77333decce19 device_file=/dev/hda5 fstype=ext3 drive_type=disk

Conclusion and future development

HAL provides a single API for discovering hardware attached to the system as well as extension points used for configuring the operating system to use a device. Key features of HAL include the ability to gather information about devices from several sources (the hardware itself and device information files), asynchronous notification through the system message bus, and a richly extensible set of data about each device.

An interesting feature that is currently under development in HAL is persistent properties that, in a nutshell, involve making sure that properties on a device object are retained when the device is plugged in again. This is more complicated than it sounds, because most devices lack serial numbers meaning that it is necessary to track other properties including the slot or port the device was attached to the last time.

With persistent properties, one can do a number of interesting things. For instance it will be relatively easy to build user interfaces for configuring devices. So, instead of having to write the device information file saying "this is a compact flash reader," the user can simply select this in the user interface (from a development point of view all it entails is invoking the SetProperty() method on the device object). Such a device configuration interface is also planned, which, long term, will include the ability for the user to generate a device information file and submit it to a public repository. Operating system vendors, like Red Hat, can then, from time to time, pull information from such a repository, verify the information, and include it with the operating system. This generates network effects, since the next user that plugs in a device of the same make need not to go into the interface and configure it.

Now, it gets a lot more interesting when one starts using HAL for more interesting things than just "this is a compact flash reader" since that only affects what icon you get on the desktop. As device properties of HAL are extensible, presumably one can merge information saying "this device requires the foobar42 driver" from a device information file. This can be used for a couple of things; first of all, when the kernel supports driver binding, the operating system can bind the correct driver to the specific device in question. Second, if the driver is not installed, the desktop may prompt the user to install the correct driver. With the recent developments in package management and the advent of several yum-enabled package repositories for Fedora Core, the end user experience from plugging in a random device until it's fully working, may be as simple as a few mouse clicks.

Look for more classes of devices that will "Just Work" in upcoming releases of Fedora Core and Red Hat Enterprise Linux.

Further reading

About the author

David Zeuthen is the maintainer for the HAL project hosted at freedesktop.org and works for Red Hat in the Desktop group. When he was younger, he spent a lot of time playing curling.