[Linux-cluster] gnbd and caching

[Markus Hochholdinger]

Hi,

Am Donnerstag, 26. Juli 2007 02:12 schrieb Benjamin Marzinski:
> On Tue, Jul 24, 2007 at 05:34:16PM +0200, Markus Hochholdinger wrote:
> > i'm running some standalone gnbd servers (option -c) and i'm wondering
> > where the (read-)caching of the gnbd happens and when/how the cache will
> > be emptied.
> > As far as i understand it, the caching happens on the gnbd client. Is
> > this correct?
> No. On the gnbd client side, the device works just like every other block
> device. When the device is closed, the cache is automatically flushed.

so nothing on the gnbd-client side inside gnbd makes any caching. This is 
good.


> > If so, when will the cache of the gnbd client be emptied? Will it be
> > emptied if no one accesses the device? Or will it be cached until the
> > device is reimported (removed and imported)?
> > Are there other possibilities to clear the read cache? Are there
> > possibilities to see how much data is cached (allocated memory of
> > gnbd_recvd)? Or have i to look for linux page cache?
> Like I said, the caching option has nothing to do with the client side.
> GNBD acts just like any other block device.
> The caching option does only one thing.  I determines whether or not the
> gnbd server daemon opens the exported block device with the O_DIRECT flag
> or not.

Ah, so this caching option is only relevant on the server. So if one gnbd 
client asks the server, it could serve data from the cache. If another 
gnbd-client asks the same server, it also could serve from the same cache. So 
as long as the exported block device will not change, my data would be 
konsistent.
And if i access the block device from the gnbd client, the normal linux 
caching mechanismen will take place.


> If the device is uncached, then the gnbd server will access the exported
> device with O_DIRECT, which bypasses the cache. Otherwise, the gnbd server
> will not use O_DIRECT, and use the page cache. If the exported device was
> on a SAN, and being used by exported by multiple gnbd servers (which would
> allow you to use dm-multipath for failover), you would want it uncached,
> because otherwise changes on other machines might be missed, if you pulled
> from cache.  Also, if you were say, exporting part of an LVM volume, which
> you were also using locally, you would want to be uncached. Otherwise would
> have one cache on top of the exported device, and another cache (for the
> same device) on top of the LVM volume.

I think i understand it now. This gnbd caching thing is all about block device 
caching on the gnbd server, not on the gnbd client.

If i access one exported block device from multiple gnbd clients with O_DIRECT 
my data will be konsistent.
This means, if one gnbd client reads (O_DIRECT) data from a cached gnbd 
server, this data will be cached on the gnbd server. Then another gnbd client 
writes data on the same position. If now the first gnbd client reads again 
the same position, it will get the data written from the second client, 
because the gnbd server serves both clients from the same cache.
Are am I correct?

Many thanks for your explanations.


-- 
greetings

eMHa
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20070726/e03970f7/attachment.sig>