[libvirt] [PATCH v3 00/31] Implement sparse streams for libvirt

[Michal Privoznik]

v3 of:

https://www.redhat.com/archives/libvir-list/2017-April/msg00671.html

All the patches can be found on my github:

  https://github.com/zippy2/libvirt/tree/sparse_streams2

diff to v2:
 - renamed APIs from Skip & GetHoleSize to SendHole & RecvHole
 - switched from 'unsigned long long len' to 'long long len' (where len is size of a hole)
 - introduced @flags to public APIs for future extensibility
 - couple of coding style fixes
 - couple of fixes suggested by John in review of v2

As expressed earlier, a lot of these patches should have Reviewed-by tag as
John reviewed majority of them. But we don't have a clear agreement when to use
the tag, so I'm not putting it in just yet. However, will do before pushing.

Some patches were ACKed. However, changes described above changed them, so I'm
not sure ACK still stands.

Michal Privoznik (31):
  virfdstream: Use messages instead of pipe
  util: Introduce virFileInData
  Introduce virStreamRecvFlags
  Implement virStreamRecvFlags to some drivers
  Introduce virStreamSendHole
  Introduce virStreamRecvHole
  Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag
  Introduce virStreamSparseRecvAll
  Introduce virStreamSparseSendAll
  Introduce virStreamInData
  virNetClientStreamNew: Track origin stream
  Add new flag to daemonCreateClientStream and virNetClientStreamNew
  RPC: Introduce virNetStreamHole
  Introduce VIR_NET_STREAM_HOLE message type
  Teach wireshark plugin about VIR_NET_STREAM_HOLE
  daemon: Introduce virNetServerProgramSendStreamHole
  virnetclientstream: Introduce virNetClientStreamSendHole
  daemon: Implement VIR_NET_STREAM_HOLE handling
  virnetclientstream: Introduce virNetClientStreamHandleHole
  remote_driver: Implement virStreamSendHole
  virNetClientStreamRecvPacket: Introduce @flags argument
  Introduce virNetClientStreamRecvHole
  remote: Implement virStreamRecvHole
  virNetClientStream: Wire up VIR_NET_STREAM_HOLE
  remote_driver: Implement VIR_STREAM_RECV_STOP_AT_HOLE
  daemonStreamHandleRead: Wire up seekable stream
  fdstream: Implement sparse stream
  gendispatch: Introduce @sparseflag for our calls
  Introduce virStorageVol{Download,Upload}Flags
  virsh: Implement sparse stream to vol-download
  virsh: Implement sparse stream to vol-upload

 daemon/remote.c                      |   2 +-
 daemon/stream.c                      | 148 ++++++++-
 daemon/stream.h                      |   3 +-
 include/libvirt/libvirt-storage.h    |   9 +
 include/libvirt/libvirt-stream.h     | 115 ++++++-
 src/driver-stream.h                  |  25 ++
 src/esx/esx_stream.c                 |  16 +-
 src/libvirt-storage.c                |   4 +-
 src/libvirt-stream.c                 | 526 ++++++++++++++++++++++++++++++
 src/libvirt_internal.h               |   4 +
 src/libvirt_private.syms             |   2 +
 src/libvirt_public.syms              |   9 +
 src/libvirt_remote.syms              |   3 +
 src/remote/remote_driver.c           |  99 +++++-
 src/remote/remote_protocol.x         |   2 +
 src/rpc/gendispatch.pl               |  21 +-
 src/rpc/virnetclient.c               |   1 +
 src/rpc/virnetclientstream.c         | 238 +++++++++++++-
 src/rpc/virnetclientstream.h         |  18 +-
 src/rpc/virnetprotocol.x             |  17 +-
 src/rpc/virnetserverprogram.c        |  35 ++
 src/rpc/virnetserverprogram.h        |   8 +
 src/storage/storage_driver.c         |   4 +-
 src/storage/storage_util.c           |  10 +-
 src/util/virfdstream.c               | 609 +++++++++++++++++++++++++++++++----
 src/util/virfdstream.h               |   1 +
 src/util/virfile.c                   |  82 +++++
 src/util/virfile.h                   |   4 +
 src/virnetprotocol-structs           |   5 +
 tests/virfiletest.c                  | 203 ++++++++++++
 tools/virsh-util.c                   |  65 ++++
 tools/virsh-util.h                   |  29 ++
 tools/virsh-volume.c                 |  50 ++-
 tools/virsh.pod                      |   6 +-
 tools/wireshark/src/packet-libvirt.c |  52 +++
 tools/wireshark/src/packet-libvirt.h |   2 +
 36 files changed, 2301 insertions(+), 126 deletions(-)

-- 
2.13.0