[dm-devel] Reg. Persistent Metadata API

[Karthi]

Hi everyone,

Our group works on a device mapper target called dmDedup that performs
inline data deduplication. The dmDedup target internally uses the
Persistent Metadata API (located in linux/drivers/md/persistent-data)to
manage its meta-data.

In our system, there is a main writer thread (or several of them) and a
separate flusher thread. When a write request comes in, the writer thread
checks for the duplicates in the data and updates corresponding meta-data.
The flusher thread periodically (10 sec) flushes the meta-data to disk by
calling the commit function in the transaction manager of Persistant
Metadata API.

When we benchmark dmDedup target with a sequential unique data write
workload, we observed that write performance periodically drops to 0
ops/sec for 8 to 15 seconds. It happens because the flusher thread blocks
the writer thread. Our further analysis showed that, the writer thread is
blocked on the dm_bufio mutex lock, which is acquired by the flusher thread
on the dm_bufio_client  when the dirty buffers are being flushed as a part
of transaction. Though these dirty buffers are flushed asynchronously, the
function enters a loop with the lock acquired and exits only when it places
the request for all the dirty buffers. (drivers/md/dm-bufio.c -
dm_bufio_write_dirty_buffers()). The writer thread is blocked as it needs
this lock to lookup the metadata. The underlying I/O queue size is 128. The
flusher thread waits to place IO request for all dirty buffers (which is
much greater than 128) onto the queue. This way the writes becomes
synchronous during this entire period, thus blocking the other writer
thread.

We want a metadata flush which is not affected by the writer thread so
drastically. Specifically, we would like the flush to write out dirty
buffers, while the writer still can update metadata (using COW to preserve
consistency). Is it something that is possible with a current design of
persistent metadata subsystem? Our current understanding is that during the
commit all updates to the meta-data are blocked until all of the meta-data
is written out.

We appreciate any suggestions!

Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20130517/f1d8ab84/attachment.htm>