[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[libvirt] [PATCH v2 00/13] Introduce a virtlogd daemon



This is a v2 of:

  https://www.redhat.com/archives/libvir-list/2015-November/msg00085.html

Currently we have stdout + stderr of QEMU guests setup to write
to a log file (eg /var/log/libvirt/qemu/$GUEST.log). This is nice
and simple, but it in fact opens the possibility of a malicious or
accidental denial of service, whereby QEMU can write logs of data
to stdio and thus fill up the host filesystem holding the log.

Although we have logrotate policy in place, this is only processed
once a day, so there is still a window where disk usage is not
constrained.

The only way to solve this is to not let QEMU directly write to
the log file, instead connect its stdio to a pipe and copy data
from the pipe to the real log file, performing file rollover
when it reaches a certain size.

If we do this, we need something to keep open the pipe for as long
as QEMU is running. This can't be libvirtd since we expect libvirtd
to be able to be stopped while QEMU is running. Thus we introduce a
new single-purpose daemon virtlogd whose job is exclusively to deal
with log file writing. This daemon has support for SIGUSR1 to tell
it to re-exec itself while keeping open the pipe to QEMU so it can
be safely upgraded while QEMU is running.

This series switches QEMU to use virtlogd by default, but in case
of problems we can revert back to the old direct file access by
setting 'stdio_handler = "file"' in /etc/libvirt/qemu.conf

This series is only the first step. The same problem exists when
character devices are told to use the file backend. There is a
further use case from OpenStack which that they want to allow
the use of both a TCP backend and a file backend at the same
time. The idea is that the serial port is to be used for an
interactive console, so needs the TCP backend support, but we
also want to be able to record all output on that serial port to
a file for logging purposes.

Thus, in a followup I will work on creating a new character
device backend "logd" that will use virtlogd to provide this
combined tcp+logfile facility for QEMU guests.

Changed in v2:

 - Expanded the virrotatingfile module to handle reading from
   files and querying position / seeking

 - Fixed rollover when no backups are enabled

 - Rollover early if we see a \n in the last 80 bytes

 - Totally refactor the QEMU code dealing with log files
   to remove all direct use of file handles and hide it
   in a qemuDomainLogContextPtr. This ensures we can fetch
   startup errors from virtlogd when needed

Daniel P. Berrange (13):
  util: add APIs for reading/writing from/to rotating files
  Import stripped down virtlockd code as basis of virtlogd
  logging: introduce log handling protocol
  logging: add client for virtlogd daemon
  qemu: remove writing to QEMU log file for rename operation
  qemu: unify code for reporting errors from QEMU log files
  qemu: introduce a qemuDomainLogContext object
  qemu: convert log file creation to use qemuDomainLogContextPtr
  qemu: change qemuDomainTaint APIs to accept qemuDomainLogContextPtr
  qemu: convert qemuLogOperation to take a qemuDomainLogContextPtr
  qemu: convert process stop/attach to use qemuDomainLogContextPtr
  qemu: convert monitor to use qemuDomainLogContextPtr indirectly
  qemu: add support for sending QEMU stdout/stderr to virtlogd

 .gitignore                         |    7 +
 cfg.mk                             |    6 +-
 include/libvirt/virterror.h        |    1 +
 libvirt.spec.in                    |   24 +-
 po/POTFILES.in                     |    6 +
 src/Makefile.am                    |  178 +++++-
 src/libvirt_private.syms           |   21 +
 src/logging/log_daemon.c           | 1207 ++++++++++++++++++++++++++++++++++++
 src/logging/log_daemon.h           |   45 ++
 src/logging/log_daemon_config.c    |  203 ++++++
 src/logging/log_daemon_config.h    |   50 ++
 src/logging/log_daemon_dispatch.c  |  143 +++++
 src/logging/log_daemon_dispatch.h  |   31 +
 src/logging/log_handler.c          |  524 ++++++++++++++++
 src/logging/log_handler.h          |   63 ++
 src/logging/log_manager.c          |  283 +++++++++
 src/logging/log_manager.h          |   61 ++
 src/logging/log_protocol.x         |  115 ++++
 src/logging/test_virtlogd.aug.in   |   12 +
 src/logging/virtlogd.aug           |   45 ++
 src/logging/virtlogd.conf          |   67 ++
 src/logging/virtlogd.init.in       |   94 +++
 src/logging/virtlogd.pod.in        |  162 +++++
 src/logging/virtlogd.service.in    |   17 +
 src/logging/virtlogd.socket.in     |    8 +
 src/logging/virtlogd.sysconf       |    3 +
 src/qemu/libvirtd_qemu.aug         |    1 +
 src/qemu/qemu.conf                 |   15 +
 src/qemu/qemu_conf.c               |   18 +
 src/qemu/qemu_conf.h               |    1 +
 src/qemu/qemu_domain.c             |  353 +++++++----
 src/qemu/qemu_domain.h             |   37 +-
 src/qemu/qemu_driver.c             |   38 +-
 src/qemu/qemu_migration.c          |    2 +-
 src/qemu/qemu_monitor.c            |   89 +--
 src/qemu/qemu_monitor.h            |    8 +-
 src/qemu/qemu_process.c            |  320 ++++------
 src/qemu/qemu_process.h            |    2 -
 src/qemu/test_libvirtd_qemu.aug.in |    1 +
 src/util/virerror.c                |    1 +
 src/util/virrotatingfile.c         |  608 ++++++++++++++++++
 src/util/virrotatingfile.h         |   62 ++
 tests/Makefile.am                  |    6 +
 tests/virrotatingfiletest.c        |  698 +++++++++++++++++++++
 44 files changed, 5180 insertions(+), 456 deletions(-)
 create mode 100644 src/logging/log_daemon.c
 create mode 100644 src/logging/log_daemon.h
 create mode 100644 src/logging/log_daemon_config.c
 create mode 100644 src/logging/log_daemon_config.h
 create mode 100644 src/logging/log_daemon_dispatch.c
 create mode 100644 src/logging/log_daemon_dispatch.h
 create mode 100644 src/logging/log_handler.c
 create mode 100644 src/logging/log_handler.h
 create mode 100644 src/logging/log_manager.c
 create mode 100644 src/logging/log_manager.h
 create mode 100644 src/logging/log_protocol.x
 create mode 100644 src/logging/test_virtlogd.aug.in
 create mode 100644 src/logging/virtlogd.aug
 create mode 100644 src/logging/virtlogd.conf
 create mode 100644 src/logging/virtlogd.init.in
 create mode 100644 src/logging/virtlogd.pod.in
 create mode 100644 src/logging/virtlogd.service.in
 create mode 100644 src/logging/virtlogd.socket.in
 create mode 100644 src/logging/virtlogd.sysconf
 create mode 100644 src/util/virrotatingfile.c
 create mode 100644 src/util/virrotatingfile.h
 create mode 100644 tests/virrotatingfiletest.c

-- 
2.5.0


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]