[Libguestfs] [NBDKIT SECURITY] Denial of Service / Amplification Attack in nbdkit

Eric Blake eblake at redhat.com
Fri Sep 20 13:58:31 UTC 2019


On 9/12/19 12:41 PM, Richard W.M. Jones wrote:
> We have discovered a potential Denial of Service / Amplification Attack
> in nbdkit.

Unfortunately, our fix for this issue cause another potential Denial of
Service attack:

> 
> Lifecycle
> ---------
> 
> Reported: 2019-09-11  Fixed: 2019-09-11  Published: 2019-09-12
> 
> There is no CVE number assigned for this issue yet, but the bug is
> being categorized and processed by Red Hat's security team which may
> result in a CVE being published later.
> 

Reported: 2019-09-18  Fixed: 2019-09-19  Published: 2019-09-20

Also pending Red Hat security review for whether this deserves a CVE
(presumably either both issues, or neither, will have a CVE)

> Credit
> ------
> 
> Reported and patched by Richard W.M. Jones <rjones at redhat.com>.
> 
> Reviewed by Eric Blake <eblake at redhat.com> and
> Daniel Berrangé <berrange at redhat.com>.
> 
> Description
> -----------
> 
> nbdkit is a Network Block Device (NBD) server with multiple plugins.
> 
> An attacker, by connecting to nbdkit’s TCP port, is able to make the
> current plugin’s .open() callback run.  For some plugins this callback
> does a considerable amount of work, for example connecting to a remote
> machine (nbdkit-ssh-plugin) or launching a small VM
> (nbdkit-guestfs-plugin).  Thus for a small amount of work done by the
> attacker (opening many TCP connections but not sending any data), a
> large amount of work is done by nbdkit (eg. making many full ssh
> connections to a remote machine or launching hundreds of VMs).
> 
> A related problem is that if nbdkit is configured to use TLS for
> client authentication, even clients which are not authorized are able
> to attack nbdkit in this way because all they have to do is open a TCP
> connection.

The NBD specification allows a client to request details about an export
(NBD_OPT_INFO) prior to committing to use that export (NBD_OPT_GO).
However, the fix to defer a plugin's .open() callback to as late as
possible means that a compliant client handshake now triggers
back-to-back opens from the same connection rather than sharing a single
open, and the second one fails due to an assertion because the first
open did not have a matching .close() callback.  Most NBD clients do not
query info, therefore, a malicious client can use the assertion failure
by querying info to cause a denial of service to another client that
does not query info.

> 
> Mitigating this is that the current plugin is NOT selected by the
> attacker, but is part of the server configuration.  Therefore the
> effectiveness of the attack depends on the particular configuration.

Mitigating this is that if TLS is used to filter out untrusted guest,
and you can ensure that trusted guests do not query info, then the
assertion is unreachable.

> 
> There are two situations in which you will NOT be affected:
> 
>  * You are not exposing nbdkit over TCP (eg. using nbdkit -U), or the
>    TCP port is not accessible to untrusted attackers (eg. private LAN,
>    firewall rules).
> 
>  * The plugin you are using does not do much work in its .open()
>    callback.  For example the memory plugin does nothing in its
>    callback so is not vulnerable:
>    https://github.com/libguestfs/nbdkit/blob/afbe7f11098df256acf8eafa9d4ec1e3cfd32910/plugins/memory/memory.c#L110-L115
> 
> Test if nbdkit is a vulnerable version
> --------------------------------------
> 
> Run the following command (requires bash):
> 
> nbdkit -fv null --run 'sleep 1 >/dev/tcp/localhost/10809' 2>&1 |
>   grep -q 'open readonly' && echo vulnerable || echo patched

At this time, there are no known readily-available open source clients
which query info, thus no easy way to test for the second vulnerability
other than creating a custom client.

> 
> Workarounds
> -----------
> 
> It is recommended to upgrade to a fixed version of nbdkit (see next
> section).  However if this cannot be done, apply network filtering
> (eg. firewall, TCP wrappers, etc.) to ensure that untrusted clients
> cannot connect to nbdkit’s public TCP port.  Or use a Unix domain
> socket instead of a loopback TCP connection.
> 
> Fixes
> -----
> 
> This affects all versions of nbdkit.  A fix is available for 1.12,
> 1.14 and the current development branch.
> 
> * development branch (1.15)
>   https://github.com/libguestfs/nbdkit/commit/c05686f9577fa91b6a3a4d8c065954ca6fc3fd62
>   or use nbdkit >= 1.15.1 from
>   http://download.libguestfs.org/nbdkit/1.15-development/

Use nbdkit >= 1.15.2, containing
https://github.com/libguestfs/nbdkit/commit/a6b88b195a959b17524d1c8353fd425d4891dc5f

> 
> * stable branch 1.14
>   https://github.com/libguestfs/nbdkit/commit/e06cde00659ff97182173d0e33fff784041bcb4a
>   or use nbdkit >= 1.14.1 from
>   http://download.libguestfs.org/nbdkit/1.14-stable/

Use nbdkit >= 1.14.2, containing
https://github.com/libguestfs/nbdkit/commit/bf0d61883a2f02f4388ec10dc92d4c61c093679e

> 
> * stable branch 1.12
>   https://github.com/libguestfs/nbdkit/commit/22b30adb796bb6dca264a38598f80b8a234ff978
>   or use nbdkit >= 1.12.7 from
>   http://download.libguestfs.org/nbdkit/1.12-stable/

Use nbdkit >= 1.12.8, containing
https://github.com/libguestfs/nbdkit/commit/b2bc6683ea3cd1f6be694e8a681dfa411b7d15f3

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20190920/bcdb4beb/attachment.sig>


More information about the Libguestfs mailing list