<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="#0563C1" vlink="#954F72"> <div class="WordSection1"> Hi<o:p></o:p> I would like to ask for comments and propose a possible new libvirt feature.<o:p></o:p> <o:p> </o:p> Problem statement:<o:p></o:p> At present when you boot a vm via libvirt it is possible to pin vm vCPUs to host CPUs to improve performance in the guest under certain conditions.<o:p></o:p> If hostdev interrupts are not pined to a specific core/cpuset suboptimal processing of irqs may occur reducing the performance of both guest and host.<o:p></o:p> I would like to propose extending libvirt to automatically pin interrupts for hostdev devices if they are present in the guest.<o:p></o:p> <o:p> </o:p> By affinitizing interrupts, cache line sharing between the specified interrupt and guest can be achieved. <o:p></o:p> If CPU affinity for the guest, as set by the cpuset parameter, <o:p></o:p> is intelligently chosen to place the guest on the same numa node as the hostdev, cross socket traffic can be mitigated. <o:p></o:p> As a result, latency which would be introduce if the interrupt processing was scheduled to a non-local numa node CPU can be reduced via interrupt pinning. <o:p></o:p> <o:p> </o:p> Proposed change:<o:p></o:p> <![if !supportLists]>· <![endif]>util/virpci and util/virhostdev will be extended to retrieve IRQ and msi_interupt information from sysfs.<o:p></o:p> <![if !supportLists]>· <![endif]>util/virinterupt will be created<o:p></o:p> <![if !supportLists]>· <![endif]>util/virinterupt will implement managing interrupt affinity via /proc/irq/x/smp_affinity<o:p></o:p> <![if !supportLists]>· <![endif]>qemuProcessInitCpuAffinity will be extended to conditionally affinitize hostdev interrupts to vm vCpus when hostdevs are present in the vm definition.<o:p></o:p> Alternative implementation:<o:p></o:p> <o:p> </o:p> In addition to the above changes the hostdev element could be extended to include a cpuset attribute:<o:p></o:p> <![if !supportLists]>· <![endif]>if the cpuset is auto: the interrupts would be pinned to the same cpuset as the vms vCPU<o:p></o:p> <![if !supportLists]>· <![endif]>if a cpuset is specified: the interrupts would be affinitized as per the set cpuset<o:p></o:p> <![if !supportLists]>· <![endif]>if the cpuset is native: the interrupts will not be pinned and the current behaviour will not be changed.<o:p></o:p> <![if !supportLists]>· <![endif]>If the cpuset attribute is not present either the behaviour of auto or native could be used as a default.<o:p></o:p> <![if !supportLists]>o <![endif]>Using auto as the default would allow transparent use of the feature.<o:p></o:p> <![if !supportLists]>o <![endif]>Using native as the default would allow no changes to any existing deployments unless the feature is requested.<o:p></o:p> <o:p></o:p> Any feedback is welcomed.<o:p></o:p> Regards<o:p></o:p> Sean.<o:p></o:p> </div> -------------------------------------------------------------- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. </body> </html>