[libvirt-users] libvirt does not recognize all devices in iscsi and mpath pools in a predictable manner

[Guido Winkelmann]

Am Dienstag, 13. September 2011, 17:39:16 schrieben Sie:
> On Tue, Sep 13, 2011 at 05:17:54PM +0200, Guido Winkelmann wrote:
[...]
> > I chose <path>/dev/disk/by-id</path> over /dev/disk/by-path for the iscsi
> > pool because I need to be able to migrate running virtual machines to
> > other hosts, so the actual device paths for the disks need to be the
> > same on all hosts in the cluster.
> 
> /dev/disk/by-path should be the same across all hosts too, at least for
> iSCSI, but perhaps not FibreChannel - depending on the udev naming scheme.

Not in my case. The by-path device names contain the ip address of the iSCSI 
array, and in my case, two host machines access the array via different IP 
addresses...
Granted, that's only a test setup and the planned production setup won't have 
that problem, but still, there is no guarantee that the by-path are always the 
same across all hosts.

Also, if the used iSCSI array can be accessed via multiple different IP 
addresses, each volume will be listed multiple times under by-path.
 
[...]
> > Both LUNs do appear under both naming schemes under /dev/disk/by-id:
> > 
> > ls /dev/disk/by-id/
> > [...]
> > scsi-36782bcb0000859f3000007004e680190
> > [...]
> > scsi-36782bcb0000859f3000007294e6e73f2
> > scsi-36a4badb00b0f910012e0fccb07606fe6
> > [...]
> > wwn-0x6782bcb0000859f3000007004e680190
> > [...]
> > wwn-0x6782bcb0000859f3000007294e6e73f2
> > wwn-0x6a4badb00b0f910012e0fccb07606fe6
> > [...]
> > 
> > (snipped some irrelevant parts)
> 
> This is the problem. Our code is assuming that /dev/disk/by-id contains
> only 1 symlink per disk. For some reason your udev rules are creating
> multiple symlinks per disk, and hence breaking libvirt.

Well, I did not change anything in the udev rules. They're the defaults from 
Fedora 14...
 
> We iterate over the entries in /dev/disk/by-id, so we get them back in
> whatever order the filesystem feels like today. What we need todo is
> to read all the matches for that disk, sort the results, and then
> pick the first result.
> 
> Or, we might want to make it possible to specify a target path of
> 
>   /dev/disk/by-id/www-*
> 
> so that you can choose which naming scheme to use.

The last one sounds like a good idea to me. I'm still trying to figure out 
where the extra "3" comes from in front of the world-wide id in the scsi- 
naming scheming or in that of the multipathd...

One thing I've noticed in the meantime is that, if I know the device node 
exists on the host, I can just give it as the pathname when defining a new 
virtual machine, even if libvirt won't show that volume in vol-list, or won't 
show it under that name, and it'll just work.
While that solves my problem, it means I'm now working entirely around 
libvirt's storage subsystem. I might as well not even bother defining any 
pools any more... :-(

Regards,

        Guido

PS: I'm sorry, this mail was supposed to go the mailing list, not to Daniel 
personally...