[libvirt] question about rdma migration

[Michael R. Hines]

Is the QEMU process (after startup) actually running as the QEMU userid ?

/*
  * Michael R. Hines
  * Platform Engineer, DigitalOcean.
  */

On 02/19/2016 02:43 PM, Roy Shterman wrote:
> First off all thank you for your answer,
>
> I couldn't figured how to start virtual machine with increased MEMLOCK,
>
> tried to add into /etc/security/limits.d
>
> qemu            soft    memlock  3221225
> qemu            hard    memlock  3221225
>
> so max locked-in-memory will be 3G, but it didn't worked.
>
> still has MEMLOCK of 60kb per each VM.
>
> Maybe you can spot what I'm doing wrong?
>
> On Tue, Feb 9, 2016 at 5:16 PM, Michael R. Hines <michael at hinespot.com 
> <mailto:michael at hinespot.com>> wrote:
>
>     Hi Roy,
>
>     On 02/09/2016 03:57 AM, Roy Shterman wrote:
>
>         Hi,
>
>         I tried to understand the rdma-migration in qemu code and i
>         have two questions about it:
>
>         1. I'm working with qemu-kvm using libvirt and i'm getting
>
>         MEMLOCK    max locked-in-memory address space  65536 65536 bytes
>
>         in qemu process so I don't understand how can you use
>         rdma-pin-all with such low MEMLOCK.
>
>         I found a solution in libvirt to lock all vm memory in advance
>         and to enlarge MEMLOCK.
>         It uses memoryBacking locking and memory tuning hard_limit of
>         vm memory but I couldn't find a usage of this in
>         rdma-migration code.
>
>
>     You're absolutey right, the RDMA migration code itself doesn't set
>     this lock limit explicitly because there are system-wide
>     restrictions in both appArmour,
>     /etc/security, as well as SELINUX that restrict applications from
>     arbitrarily setting their maximum memory lock limits.
>
>     The other problem is CGROUPS: If someone sets a cgroup control for
>     maximum memory and forgets about that mlock() limits, then
>     there will be a conflict.
>
>     So, libvirt must have a policy to deal with all of these
>     possibilities, not just handle a special case for RDMA migration.
>
>     The only way "simple" way (without patching the problems above) to
>     apply a higher lock limit to QEMU is to set the ulimit for libvirt
>     (or for QEMU if starting QEMU manually) in your environment or the
>     command line with $ ulimit # before attempting the migration,
>     then the RDMA subsystem will be able to lock the memory successfully.
>
>     The other option is to use /etc/security/limits.conf and set the
>     option for a specific libvirt process user and make sure your
>     libvirt/qemu
>     are not running as root.
>
>     QEMU itself also has a "mlock" option built into the command line,
>     but it also suffers from the same problem --- you have to find
>     a way (currently) to increase the limit before using the option.
>
>         2. Do you have any comparison of IOPS and bandwidth between
>         TCP migration and rdma migration?
>
>     Yes, lots of comparisons.
>
>     http://wiki.qemu.org/Features/RDMALiveMigration
>     http://www.canturkisci.com/ETC/papers/IBMJRD2011/preprint.pdf
>
>
>         Regards,
>         Roy
>
>
>
>
>
>
> --
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20160219/69508014/attachment-0001.htm>