[libvirt] [RFC] Adding new filesystem 'proxy' to 9p

[Daniel P. Berrange]

On Wed, Sep 28, 2011 at 07:49:34PM +0530, M. Mohan Kumar wrote:
> Pass-through security model in QEMU 9p server needs root privilege to do few
> file operations (like chown, chmod to any mode/uid:gid).  There are two issues
> in pass-through security model
> 
> 1) TOCTTOU vulnerability: Following symbolic links in the server could
> provide access to files beyond 9p export path.
> 
> 2) When libvirt is configured to run qemu as non-root user (for example, if
> qemu is configured to run as normal user 'qemu'), running file operations on
> pass-through security model would fail because it needs root privileges.
> 
> To overcome above issues, following approach is suggested: A new filesytem
> type 'proxy' is introduced. Proxy FS uses chroot + socket combination for
> securing the vulnerability known with following symbolic links. Intention of 
> adding a new filesystem type is to allow qemu to run in non-root mode, but 
> doing privileged operations using socket IO.
> 
> A new binary (known as proxy helper) will be provided as part of qemu. Proxy
> helper will chroot into 9p export path and create a socket pair or a named
> socket based on the command line parameter. Qemu and proxy helper will
> communicate using this socket.
> 
> We need following changes in the libvirt code to accomodate new 'proxy' 
> filesystem type:
> If qemu 9p server is configured to use 'proxy' FS, libvirt will do
> * Create a socket pair
> * invoke proxy_helper binary with one of the socket id from the pair as
> command line parameters to it with root privilege
> * invoke qemu with one of socket id from the pair as paramter to qemu virtfs
> after dropping to the configured user privilege.
> ie, libvirt will invoke proxy_helper as:
>     proxy_helper -i <socket_fd_from_socket_pair> -p <9p-path-to-export>
> 
> and qemu will be invoked with following virtfs parameter:
>     -virtfs proxy,id=<id>,sock_fd=<socket_fd_from_socket_pair>
> 	,path=/tmp/,security_model=prox,mount_tag=v_pass

Interesting proposal. Explicitly comparing the security characteristics
of running QEMU as root, vs using the proxy helper

 * QEMU run as root

     - QEMU is root, with full capabilities
     - QEMU has read/write any file/dir, regardless of whether it is
       exported via 9p

 * QEMU run as non-root, with proxy_helper root

     - QEMU is non-root, with no capabilities
     - QEMU has write to only files with matching UID/GID
     - proxy_helper is root, with full capabilities
     - proxy_helper has read/write to any file/dir

Since QEMU can send arbitrary FS calls to the proxy_helper, the overall
security of the system clearly depends on the security of the proxy_helper
process.

If we assume that QEMU gets exploited, and that QEMU can find some flaw
in the proxy_helper that it can exploit, what damage can the proxy_helper
do ?  Clearly we want it to not be able to read/write any files other
than those exported, even when it becomes compromised, ideally also
without requiring SELinux/AppArmour to make it safe. If proxy_helper
is running as root with full capabilities, it can trivially escape
the chroot[1], so this isn't all that nice for overall system security.
The attacker simply needs to find a vulnerability in QEMU and in the
proxy_helper, instead of just in QEMU. We want a stronger guarantee
than that.

In order to be able to chown/chmod files etc, the proxy_helper is going
to require some elevated privileges, however, this does not actually
imply that proxy_helper needs to run as root. You don't want the proxy_helper
to be allowed todo various other things that root would be allowed, even
if it is inside a chroot. eg you don't want it being allowed to reboot,
mlock, change network setting, mount volumes, make device nodes, and
quite alot more.

Looking at the man page 'capabilities(7)', I'd venture to suggest that
the proxy_helper does not in fact need to run as the 'root' UID/GID at
all. All it needs is to have the CAP_CHOWN, CAP_DAC_OVERRIDE,
CAP_DAC_READ_SEARCH, CAP_FOWNER capabilities, at which point it could
still run as the 'qemu' user ID (or perhaps another 'qemuhelper' user
ID).

So in summary, I think you have a good idea to separate the file handling
code into a separate proxy_helper binary. It should not, however, be run
as root with all capabilities. It can start as root, and then drop to a
non-root user, only keeping the few capabilities it needs. Or in libvirt
case, libvirt can spawn it with the minimal set of capabilities directly.

eg

 * QEMU run as non-root, with proxy_helper partially privileged

     - QEMU is non-root, with no capabilities
     - QEMU has write to only files with matching UID/GID
     - proxy_helper is non-root, with selected capabilities
     - proxy_helper has read/write to any file/dir inside the
       chroot for the 9p export.


Regards,
Daniel

[1] http://www.bpfh.net/simes/computing/chroot-break.html
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|