[dm-devel] [Lsf-pc] [LSF/MM TOPIC] a few storage topics

[Andrea Arcangeli]

On Thu, Jan 19, 2012 at 10:46:37AM +0100, Jan Kara wrote:
> So to make this useful the logic won't be trivial. Thirdly, the benefit is
> questionable anyway (at least for most of realistic workloads) because
> flusher thread doesn't write the pages all that often - when there are not
> many pages, we write them out just once every couple of seconds, when we
> have lots of dirty pages we cycle through all of them so one page is not
> written that often.

If you mean migrate as in mm/migrate.c that's also not cheap, it will
page fault anybody accessing the page, it'll do the page copy, and
it'll IPI all cpus that had the mm on the TLB, it locks the page too
and does all sort of checks. But it's true it'll be CPU bound... while
I understand the current solution is I/O bound.

> 
> > Do you have a more detailed description of the workload? Is it theoretically
> > avoidable?
>   See https://lkml.org/lkml/2011/10/23/156. Using page migration or copyout
> would solve the problems of this guy.

Copying in the I/O layer should be better than page migration,
1) copying the page to a I/O kernel buffer won't involve expensive TLB
IPIs that migration requires, 2) copying the page to a I/O kernel
buffer won't cause page faults because of migration entries being set,
3) migration has to copy too so the cost on the memory bus is the
same.

So unless I'm missing something page migration and pte/tlb mangling (I
mean as in mm/migrate.c) is worse in every way than bounce buffering
at the I/O layer if you notice the page can be modified while it's
under I/O.