[libvirt-users] If it's possible for a third-party PCIe card to be shared by multiple containers

[WANG Cheng D]

Dear Daniel,
I found that lxc driver doesn't support hostdev with type of 'pci'.
As you mentioned before, I tried to expose the device node (the PCIe card) to the container, my XML is as follow:
<devices>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address bus='0x01' slot='0x00' function='0x0'/>
      </source>
    </hostdev>
</devices>
When I started the container, I got the error message:
Error: failed to start domain
Error: unsupported configuration: Unsupported hostdev type pci.
I want to confirm if the element hostdev of PCIe device is not supported by lxc driver currently.
Have a nice day,
Cheng Wang

-----Original Message-----
From: Daniel P. Berrange [mailto:berrange at redhat.com] 
Sent: 2014年1月16日 18:34
To: WANG Cheng D
Cc: libvirt-users at redhat.com
Subject: Re: [libvirt-users] If it's possible for a third-party PCIe card to be shared by multiple containers

On Thu, Jan 16, 2014 at 08:55:53AM +0000, WANG Cheng D wrote:
> Dear Daniel,
> 
> The thirty-party PCIe card is based on the Xilinx’ FPGA which is off the shelf, the main features are as follows:
> 1) x8 Gen3, 8Gb/s per lane/direction
> 2) MSI and legacy interrupt support
> 3) Scatter-gather packet DMA engine provide by Northwest Logic We hope 
> multiple Linux Containers to access the PCIe card in time division mode, for example, during slot 1, lxc1 read/write the PCIe card; during slot 2, lxc2 read/write the PCIe card.
> Taking reading operation (data flows from PCIe card to host memory) as example, in native mode, the PCIe driver should tell PCIe card the host memory address where to store the incoming data from PCIe card. With this address, the PCIe card will transmit the data to host memory itself.
> In container case, the problem is how application running in the container knows the expected host memory address.
> I wonder if we can have a process running in the which solely operates the PCIe card, all the containers will send/receive data to this process via socket.

Ok, so from the LXC driver POV I guess this card is accessed via some special device node with ioctls or something ? If so then you can just expose the device node to as many containers as you like at the same time without restrictions.

http://libvirt.org/formatdomain.html#elementsHostDevCaps

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|