[RFC] exposing 'nodedev assigned to domain' info to users

[Daniel Henrique Barboza]

On 1/6/21 7:09 AM, Daniel P. Berrangé wrote:
> On Tue, Jan 05, 2021 at 05:18:13PM -0300, Daniel Henrique Barboza wrote:
>> Hi,
>>
>> This is something I've been giving a thought after working in Gitlab issue
>> #72 and decided to run through the ML before hitting the code.
>>
>> We don't have an easy way to retrieve the domain that is using an specific
>> hostdev.  Let's say that I want to know which domain is using the PCI card
>> pci_0000_01_00_2. 'nodedev-dumpxml' will return the hardware/driver capabilities
>> of the device, such as IOMMU group, driver and so on, but will not inform
>> which domain is using the hostdev, if any. 'nodedev-list' will simply list
>> all nodedev names known to Libvirt, without outputting any other information.
>>
>> IIUC, the only existing way I can reliably tell whether a hostdev is being
>> used by domain, aside from having to register the information by myself
>> during domain definition of course, is via 'virsh dumpxml <domain>' each
>> existing running domain and matching the nodedev name with the source.address
>> element of the XML.
>>
>> When we consider SR-IOV devices that can have 28+ VFs each (and have lots of
>> fun caveats, like Github #72 showed us), the capability of hot plug/unplug
>> hostdevs freely, and lots of running domains, it is clear that we're putting a
>> considerable pressure in the upper layers (OVirt, or a poor human admin) to
>> keep track of the nodedevs each running domain is using. An info that we
>> already have internally and can just expose it.
>>
>> I have a few ideas to make this happen:
>>
>> 1 - upgrade 'nodedev-list' to add an extra 'assigned to' column
>>
>> This is the more straightforward way of exposing the info. A simple 'nodedev-list'
>> call can retrieve which domain is using which nodedev. To preserve the existing
>> usage we can add an "--show-assigned-domains" option to control whether we
>> will display this info.
> 
> That would mean nodedev-list has to fetch XML for every running guest
> and parse and extract it. That's not a scalable solution.
> 
>> 2 - add an '<assigned_to>' element in nodedev XML definition
>>
>> I'm not a fan of exposing this in this particular XML because we would mix
>> host/hw related attributes with domain info. But it would be easier to pull
>> this off comparing to (1), so I'm mentioning it for the record.
> 
> This is similar to what we do for  the nwfilter-binding and net-port XML
> where we have an <owner> element present.
> 
> The complication here is that right now we don't ever touch the nodedev
> driver when doing host device assignment, and so don't especially want
> to introduce a dependancy.

One possible alternative would be a new API that operates on hostdevs instead
of nodedevs. "hostdev-list" would list the devices assigned to any domain, as
opposed to "nodedev-list" that lists all nodedevs of the host. I'm not sure if this
differentiation between hostdev and nodedev (i.e. hostdev is a nodedev that is
assigned to a domain) would be clear enough to users though. We would need to
document it clearer in the docs.


Yet another alternative is a new API under "Device Commands". We already have
attach-device, detach-device and so on, might as well have a new "list-devices"
that does the deed. This fits with the "The following commands manipulate
devices associated to domains." claim that we make about this class of commands.


Thanks,

DHB


> 
> 
> Regards,
> Daniel
>