[Linux-cluster] Re: [Iscsitarget-devel] Problem in clvmd and iscsi-target

[Nuno Fernandes]

> > Meanwhile i narrow it down to a iscsi-target problem. Doing strace i saw
> > it requires exclusive open. So solve it it created an sort of an
> > hack.. :)
>
> Ok, you cannot have local services and remote services accessing
> the same raw data sectors at the same time.
>
> I expressedly made blockio open the device exclusively to avoid
> just what you are trying to do.
Why? Did you have any problems in this scenario? Using blockio iscsi-target 
does not use any cache, right?


> If you really need access to the iSCSI target locally, then
> you will need to install an iSCSI initiator on the server and
> connect to the target and use the new disk volume that is
> created.
Ok.. and is the following scenario workable?

HP package cluster with MSA500 (basically 2 servers connected to a scsi shared 
storage)

both servers  export the same msa500 volume
on clients i use iscsi-initiator + multipath to support the failure of one 
server

In this scenario there are two iscsi-target server accessing the same data but 
on different machines. Basically it's the same thing as another process 
accessing the exported volume... Do you agree?

> > from hdb and using device-mapper i create a linear mapping to 2 devices:
> >
> > hdb-int
> > hdb-ext
> >
> > Next i put hdb-ext in ietd.conf and hdb-int in lvm.conf. iscsi-target
> > still opens in exclusive mode but it only opens hdb-ext device.
> > clvmd uses hdb-int that has no exclusive lock.
>
> You are heading down a path here that is fully unsupported and
> discouraged, if you loose your data it will be completely and
> totally because of this.

:(


> > Another way that we rejected is to put "gray" machine exporting the iscsi
> > volume to it self also and using that device in clvmd. This option also
> > worked but has less performance as all local contend has to be encoded to
> > iscsi and decoded in the same machine.
>
> iSCSI encoding/decoding is minimal and if you are using a loopback
> to connect should be as fast as the machine can do it. Do not try
> this with fileio unit types though or the page cache will deadlock
> between iSCSI target/initiator.

IMHO encoding and decoding it locally would be a performance penalty that 
would not be required.. I dind't used fileio exactly because it caches 
content.

> > The problem of iscsi-target opening the device excluse remais.
>
> And it will continue to remain.
>
> Why not try to explain what it is your are trying to accomplish
> and a valid solution or two can be given.
What i'm trying to do is the following:

2 servers connected to msa500 exporting a volume through iscsi-target
6 servers with iscsi-initiator + multipath accessing the volume

all 8 servers with clvmd accessing the volume so i can create LVs in any of 
the 8 servers.

The LVs are to be used by Xen virtual machines in any machine in the cluster

I didn't want to use iscsi-target + iscsi-initiator on the msa connected 
machines as it would be a performance penalty.

Thanks,
Nuno Fernandes