[libvirt] QEMU/KVM support for non zero PCI domain

Tue Apr 16 15:23:29 UTC 2013

Hi Laine,
   Here are the information you have requested:

1) What is your Linux distro and version

RHEL 6.1 - 2.6.32-131.0.15.el6.x86_64

2) what version of qemu/kvm do you have

[root at uv2 qemu]# /usr/libexec/qemu-kvm -version
QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright (c) 2003-2008 Fabrice Bellard

3) what version of libvirt?

[root at uv2 qemu]# libvirtd --version
libvirtd (libvirt) 0.10.2

4) Does libvirt actually get to the point of running qemu or kvm command?

5) If the answer to (4) is "yes", can you send the qemu command that is
generated (it will be at the end of /var/log/libvirt/qemu/${guestname}.log)

LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name UV2-VM1-test1 -S -M rhel6.4.0 -enable-kvm -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid e2428c26-f06a-b03c-c8d5-c528f2e85596 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/UV2-VM1-test1.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/UV2-VM1-test1.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:51:5f:5f,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device pci-assign,host=02:00.4,id=hostdev0,configfd=24,bus=pci.0,addr=0x6 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

char device redirected to /dev/pts/2
get_real_device: /sys/bus/pci/devices/0000:02:00.4/resource: No such file or directory
qemu-kvm: -device pci-assign,host=02:00.4,id=hostdev0,configfd=24,bus=pci.0,addr=0x6: pci-assign: Error: Couldn't get real device (hostdev0)!
qemu-kvm: -device pci-assign,host=02:00.4,id=hostdev0,configfd=24,bus=pci.0,addr=0x6: Device 'pci-assign' could not be initialized
2013-04-15 19:38:31.454+0000: shutting down

********************

As you can see, I cannot run qemu command directly from libvirt. But if invoke qemu-kvm from command line:

[root at uv2 VM_test]# /usr/libexec/qemu-kvm -smp 2 -hda rhel_62.img -m 8192 -redir tcp:52110::22 -device pci-assign,host=02:00.0 -net nic -net user
get_real_device: /sys/bus/pci/devices/0000:02:00.0/config: No such file or directory
qemu-kvm: -device pci-assign,host=02:00.0: pci-assign: Error: Couldn't get real device ((null))!
qemu-kvm: -device pci-assign,host=02:00.0: Device 'pci-assign' could not be initialize

[root at uv2 VM_test]# /usr/libexec/qemu-kvm -smp 2 -hda rhel_62.img -m 8192 -redir tcp:52110::22 -device pci-assign,host=0001:02:00.0 -net nic -net user
qemu-kvm: -device pci-assign,host=0001:02:00.0: Property 'pci-assign.host' doesn't take value '0001:02:00.0'

[root at uv2 VM_test]# /usr/libexec/qemu-kvm --version
QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright (c) 2003-2008 Fabrice Bellard

*********
Installing version 1.4.0 of qemu:
[root at uv2 VM_test]# /usr/local/bin/qemu-system-x86_64 -smp 2 -hda rhel_62.img -m 8192 -redir tcp:52110::22 -device pci-assign,host=02:00.0 -net nic -net user
qemu-system-x86_64: -device pci-assign,host=02:00.0: pci-assign: error: requires KVM support
qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'kvm-pci-assign' could not be initialized
[root at uv2 VM_test]# /usr/local/bin/qemu-system-x86_64 --version
QEMU emulator version 1.4.0, Copyright (c) 2003-2008 Fabrice Bellard

[root at uv2 VM_test]# /usr/local/bin/qemu-system-x86_64 -smp 2 -hda rhel_62.img -m 8192 -redir tcp:52110::22 -device pci-assign,host=0001:02:00.0 -net nic -net user
qemu-system-x86_64: -device pci-assign,host=0001:02:00.0: pci-assign: error: requires KVM support
qemu-system-x86_64: -device pci-assign,host=0001:02:00.0: Device 'kvm-pci-assign' could not be initialized

Any idea, how I could get support for 0001 pci domain?

Thanks,
     Charles

----- Original Message -----
From: "Laine Stump" <laine at laine.org>
To: libvir-list at redhat.com
Sent: Friday, April 12, 2013 8:48:24 PM
Subject: Re: [libvirt] QEMU/KVM support for non zero PCI domain

On 04/12/2013 05:00 PM, Charles Peri wrote:
> Hi,
>   I am trying to run Virt-Manager, using KVM on a SGIUV100 system with RHEL 6.1. I am trying to enable PCI pass-thorugh for the Mellanox, but the PCI domain for the device is 0001. 

Interesting! I knew it existed, but I've never seen hardware that
actually had multiple domains.

>
> 0001:01:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
> 0001:02:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)
>
> When I try to add the device, I get an error stating that "Can't assign device inside non- zero PCI segment, as this KVM module doesn't support it". Is there a version of KVM module that supports non zero PCI segment?

Huh. I would have figured it would be stopped by libvirt first, as
libvirt's PCI address parsing currently rejects anything with domain or
bus that isn't 0. That doesn't sound like a libvirt error message
though, which implies that, at least for host-side PCI addresses,
libvirt is doing the right thing, and the error is coming back from kvm.

> Sorry, if this is not the right email list to ask this question, I checked on the KVM IRC, but haven't got any answer.

A few questions:

1) What is your Linux distro and version
2) what version of qemu/kvm do you have
3) what version of libvirt?
4) Does libvirt actually get to the point of running qemu or kvm command?
5) If the answer to (4) is "yes", can you send the qemu command that is
generated (it will be at the end of /var/log/libvirt/qemu/${guestname}.log)

--
libvir-list mailing list
libvir-list at redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list