[vfio-users] Problem with vfio-pci device type and memory > 10G (or 8G)

[Alex Williamson]

On Tue, Dec 13, 2016 at 4:15 PM, Anjali Kulkarni <anjali at juniper.net> wrote:

> Thanks Alex, I think you got it, there is a lot of memory (251G), but it
> seems to be mostly all in use:
>
>     vmstat -s
>
>     263843488 K total memory
>
>     254973760 K used memory
>
>
> Everything looks used up?
>

The difference being just around 8.5G, so yeah, unless that used memory can
get swapped out you're not going to be able to start a VM with an assigned
device with the memory size you're looking for.  Without an assigned device
the VM memory is not pre-allocated by default, so the resident memory for
the VM might only be 100s of MB.  With device assignment, every page of VM
memory needs to be allocated and mapped through the IOMMU as a potential
DMA target for the assigned device.  There is no memory overcommit or
swapping of devices making use of device assignment.  You're running into
the Out-of-Memory killer, and it's never a pleasant thing.  Good luck,

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20161213/6453f821/attachment.htm>