[dm-devel] Huge memory allocation

[Eli Malul]

I would like to accelerate read IO performance for user data which
resides on one machine (with low IO performance due to slow media) and
might be scattered on many different extents by mirroring these extents
to another machine with much better IO performance and direct read
requests to that machine (I already added a patch to the dm-raid1.c to
set preferred read capability).

Any suggestions would be appreciated...

-----Original Message-----
From: Zdenek Kabelac [mailto:zkabelac at redhat.com] 
Sent: Thursday, March 03, 2011 10:42 AM
To: undisclosed-recipients
Cc: Eli Malul; device-mapper development
Subject: Re: [dm-devel] Huge memory allocation

Dne 3.3.2011 09:05, Eli Malul napsal(a):
> I am expecting to have lots of (up to million) scattered extents
across
> some volumes which I am required to mirror.
> Since mirror mapped device with a table that large consume unbearable
> amount of memory (e.g. for 10,000 extents I saw about 6 Giga of memory
> allocated by the device-mapper) I am going to create two linear
devices
> which maps these extents and mirror them.
> 
> In addition, I am required to preserve the original extent's offsets
> since they are an existing user data used by DB applications.
> To achieve that I will create another linear device to simulate the
> original extent's offsets which shall be mapped to the created
mirrored
> device so, the client will continue to read and write the same
offsets.
> 



Aren't you trying to reinvent  dm-replicator target ?
(available as an extra kernel patch)

Maybe you should first describe exactly what are you trying to achieve.
I'd guess there would be better ways to achieve that goal.

Updating kernel tables is expensive operation - especially if you plan
to have
its size in the range of multiple megabytes - so it looks like wrong
plan...

Zdenek