[libvirt] [PATCH RFC 0/8] Sparse streams

[Michal Privoznik]

** NOT TO BE MERGED UPSTREAM **

This is merely an RFC.

What's the problem?
We have APIs for transferring disk images from/to host. Problem is, disk images
can be sparse files. Our code is, however, unaware of that fact so if for
instance the disk is one big hole of 8GB all those bytes have to: a) be read b)
come through our event loop. This is obviously very inefficient way.

How to deal with it?
The way I propose (and this is actually what I like you to comment on) is to
invent set of new API. We need to make read from and write to a stream
sparseness aware. The new APIs are as follows:

  int virStreamSendOffset(virStreamPtr stream,
                          unsigned long long offset,
                          const char *data,
                          size_t nbytes);

  int virStreamRecvOffset(virStreamPtr stream,
                          unsigned long long *offset,
                          char *data,
                          size_t nbytes);

The SendOffset() is fairly simple. It is given an offset to write @data from so
it basically lseek() to @offset and write() data.
The RecvOffset() has slightly complicated design - it has to be aware of the
fact that @offset it is required to read from fall into a hole. If that's the
case it sets @offset to new location where data starts.

Are there other ways possible?
Sure! If you have any specific in mind I am happy to discuss it. For instance
one idea I've heard (from Martin) was instead of SendOffset() and RecvOffset()
we may just invent our variant of lseek().

What's left to be done?
Basically, I still don't have RPC implementation. But before I dive into that I
thought of sharing my approach with you - because it may turn out that a
different approach is going to be needed and thus my work would render useless.

Is there any bug attached?
You can bet! https://bugzilla.redhat.com/show_bug.cgi?id=1282859

Michal Privoznik (8):
  fdstream: Realign
  Introduce virStream{Recv,Send}Offset
  Introduce VIR_STREAM_SPARSE flag
  fdstream: Implement virStreamSendOffset
  fdstream: Implement virStreamRecvOffset
  fdstreamtest: switch from boolean array to @flags
  fdstreamtest: Test virStreamRecvOffset
  fdstreamtest: test virStreamSendOffset

 include/libvirt/libvirt-stream.h |   9 +++
 src/driver-stream.h              |  13 ++++
 src/fdstream.c                   | 150 +++++++++++++++++++++++++++++++++++----
 src/internal.h                   |  20 ++++++
 src/libvirt-stream.c             | 119 +++++++++++++++++++++++++++++++
 src/libvirt_public.syms          |   6 ++
 tests/fdstreamtest.c             | 116 ++++++++++++++++++++++++------
 7 files changed, 397 insertions(+), 36 deletions(-)

-- 
2.4.10