[vfio-users] Problem with vfio-pci device type and memory > 10G (or 8G)

[Anjali Kulkarni]

Thanks Alex! Will try to reduce the memory used and check.

Anjali

From: Alex Williamson <alex.l.williamson at gmail.com<mailto:alex.l.williamson at gmail.com>>
Date: Tuesday, December 13, 2016 at 3:23 PM
To: Anjali Kulkarni <anjali at juniper.net<mailto:anjali at juniper.net>>
Cc: "vfio-users at redhat.com<mailto:vfio-users at redhat.com>" <vfio-users at redhat.com<mailto:vfio-users at redhat.com>>
Subject: Re: [vfio-users] Problem with vfio-pci device type and memory > 10G (or 8G)

On Tue, Dec 13, 2016 at 4:15 PM, Anjali Kulkarni <anjali at juniper.net<mailto:anjali at juniper.net>> wrote:
Thanks Alex, I think you got it, there is a lot of memory (251G), but it seems to be mostly all in use:

    vmstat -s

    263843488 K total memory

    254973760 K used memory


Everything looks used up?

The difference being just around 8.5G, so yeah, unless that used memory can get swapped out you're not going to be able to start a VM with an assigned device with the memory size you're looking for.  Without an assigned device the VM memory is not pre-allocated by default, so the resident memory for the VM might only be 100s of MB.  With device assignment, every page of VM memory needs to be allocated and mapped through the IOMMU as a potential DMA target for the assigned device.  There is no memory overcommit or swapping of devices making use of device assignment.  You're running into the Out-of-Memory killer, and it's never a pleasant thing.  Good luck,

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20161213/936997d8/attachment.htm>