[libvirt-users] [Qemu-devel] pci-assign fails with read error on config-space file

[Henning Schild]

Am Wed, 2 Nov 2016 09:54:16 +0000
schrieb "Daniel P. Berrange" <berrange at redhat.com>:

> On Fri, Oct 28, 2016 at 01:28:19PM +0200, Henning Schild wrote:
> > Hey,
> > 
> > i am running an unusual setup where i assign pci devices behind the
> > back of libvirt. I have two options to do that:
> > 1. a wrapper script for qemu that takes care of suid-root and
> > appends arguments for pci-assign
> > 2. virsh qemu-monitor-command ... 'device_add pci-assign...'
> > 
> > I know i should probably not be doing this, it is a workaround to
> > introduce fine-grained pci-assignment in an openstack setup, where
> > vendor and device id are not enough to pick the right device for a
> > vm.
> > 
> > In both cases qemu will crash with the following output:
> >   
> > > qemu: hardware error: pci read failed, ret = 0 errno = 22  
> > 
> > followed by the usual machine state dump. With strace i found it to
> > be a failing read on the config space file of my device.
> > /sys/bus/pci/devices/0000:xx:xx.x/config
> > A few reads out of that file succeeded, as well as accesses on
> > vendor etc.  
> 
> errno == 22, means EINVAL, so it feels unlikely to be a permissions
> problem unless the kernel or QEMU is reporting the wrong errno.
> 
> > Manually launching a qemu with the pci-assign works without a
> > problem, so i "blame" libvirt and the cgroup environment the qemu
> > ends up in.  
> 
> The 'config' file is a plain file, so not affected by cgroups - that
> only affects block devices.
> 
> When libvirt runs QEMU, it runs unprivileged qemu:qemu user/group,
> so perhaps it is a permissions thing, despite the fact that you're
> getting EINVAL, not EACCESS.

If the wrapper qemu decides to assign a PCI device it will use a
suid-root qemu to do so. So it is no EACCESS, as i said other reads
worked fine.

> It would be interesting to know just what part of the config space
> QEMU was trying to read I guess, to better understand why it might
> be failing

I should have said that before, it is a one byte read on offest 64. So
just behind the regular cfg-space.

regards,
Henning