[Linux-cluster] gnbd and caching

[Benjamin Marzinski]

On Tue, Jul 24, 2007 at 05:34:16PM +0200, Markus Hochholdinger wrote:
> Hi,
> 
> i'm running some standalone gnbd servers (option -c) and i'm wondering where 
> the (read-)caching of the gnbd happens and when/how the cache will be 
> emptied.
> 
> As far as i understand it, the caching happens on the gnbd client. Is this 
> correct?


No. On the gnbd client side, the device works just like every other block
device. When the device is closed, the cache is automatically flushed. 
 
> If so, when will the cache of the gnbd client be emptied? Will it be emptied 
> if no one accesses the device? Or will it be cached until the device is 
> reimported (removed and imported)?
> 
> Are there other possibilities to clear the read cache? Are there possibilities 
> to see how much data is cached (allocated memory of gnbd_recvd)? Or have i to 
> look for linux page cache?
> 

Like I said, the caching option has nothing to do with the client side. GNBD
acts just like any other block device.

The caching option does only one thing.  I determines whether or not the gnbd
server daemon opens the exported block device with the O_DIRECT flag or not.

If the device is uncached, then the gnbd server will access the exported device
with O_DIRECT, which bypasses the cache. Otherwise, the gnbd server will not use
O_DIRECT, and use the page cache. If the exported device was on a SAN, and
being used by exported by multiple gnbd servers (which would allow you to use
dm-multipath for failover), you would want it uncached, because otherwise
changes on other machines might be missed, if you pulled from cache.  Also, if
you were say, exporting part of an LVM volume, which you were also using
locally, you would want to be uncached. Otherwise would have one cache on top of
the exported device, and another cache (for the same device) on top of the LVM
volume. 

-Ben

> I'm using gnbd as network block devices for virtual hosts on other servers. 
> These virtual hosts can be moved between several hardware, but i want to be 
> sure no old (cached) data will be read by the virtual servers.
> 
> 
> -- 
> greetings
> 
> eMHa



> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster