large number of hot plug disks

Daniel P. Berrangé berrange at redhat.com
Tue Sep 8 12:46:07 UTC 2020


On Tue, Sep 08, 2020 at 02:38:17PM +0200, Peter Krempa wrote:
> On Tue, Sep 08, 2020 at 08:19:21 -0400, Alexander Wels wrote:
> > Hi,
> > 
> > I am working on a project, and one of the requirements is that I can
> > hotplug a large number of disks. Think several dozens to potentially 100+.
> > The guest OS will be some linux flavor so it should have no problem with a
> > large number of disks.
> > 
> > During my experimentation I quickly ran into the "No available PCI slots"
> > limit. I found https://libvirt.org/pci-hotplug.html explaining to me why
> > the limit is there. Now I could add a bunch of pcie-root-ports ahead of
> > time, but there is likely a limit to that as well. So before I go down some
> > crazy paths, I figured I would ask on here if there is a better plan than
> > adding a bunch of root ports for adding a large number of hotplugged disks?
> 
> I strongly suggest using a virtio-scsi controller and plugging disks
> into it. You can have a fairly large number of LUNs per controller, so
> that will severely decrease the amount of required PCI slots. You can
> obviously add more controllers if you need even more LUNs.

A single virtio-scsi controller can support ~4 million LUNs. IOW you'll
face a wall of other problems before you get anywhere near needing a
second virtio-scsi controller due to LUN exhaustion :-)

Only real reason for multiple virtio-scsi controllers that I see is
if you want to setup different host CPU affinity for each controller
for I/O thread performance tuning

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|




More information about the libvir-list mailing list