[vfio-users] VFIO hard instant host reboot after VM start with Qualcomm Atheros AR9580

Mon Aug 8 23:43:45 UTC 2016

Hi,

whenever I try to start a KVM VM with 'Qualcomm Atheros AR9580 Wireless
Network Adapter' assigned it will immediately reboot the host. I've tried
already a couple of things:

1. Played with module parameters.

   vfio_pci nointxmask=1
   vfio_iommu_type1vfio_iommu_type1 disable_hugepages=1

2. Tried kernel 4.4, 4.6 and 4.8rc1.r2.g574673c2

3. Tried all different PCIe slots in the system. It is a dual socket
   system with 6 PCIe slots, where 3 are connected to CPU1 and 3 to
   CPU2.

I don't have any clue what the root cause could be. I have tried another
Wireless card which works fine with VFIO passthrough. It was some old
D-Link PCIe model but with different chipset.

Any help would be much appreciated to narrow down the problem.

The system itself has the following configuration:

- Supermicro H8DG6/H8DGi
- 2x Opteron 6366 HE
- IOMMU is enabled in BIOS.
- VFIO uses the following devices:
  02:10.0-02:10.7
  02:11.0-02:11.5
  05:00.0 (not working)
  41:00.0
  42:00.0

lspci output:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 Northbridge only dual slot (2x16) PCI-e GFX Hydra part (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port D)
00:0c.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890S PCI Express bridge for GPP2 port 1
00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port B)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0 USB OHCI1 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0 USB OHCI1 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 3d)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
00:1a.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:1a.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:1a.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:1a.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:1a.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:1a.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
00:1b.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:1b.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:1b.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:1b.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:1b.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:1b.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
01:04.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
02:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.5 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:10.7 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:11.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:11.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:11.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:11.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:11.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
02:11.5 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
04:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
05:00.0 Network controller: Qualcomm Atheros AR9580 Wireless Network Adapter (rev 01)
40:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 Northbridge only dual slot (2x16) PCI-e GFX Hydra part (rev 02)
40:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
40:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port D)
40:0b.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (NB-SB link)
41:00.0 Multimedia controller: Digital Devices GmbH Octopus DVB Adapter
42:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 03)

--Maik