[libvirt] [V4 RESEND PATCH] Expose resource control capabilites on cache bank

Eli Qiao qiaoliyong at gmail.com
Mon Jun 5 08:27:01 UTC 2017


> >  
> > This could be wrong on new platform in Intel’s SKX CPU after check with platform guys.
> >  
> > The cbm_mask is “7ff” (11 bits) on SKX. I will refine this by counting the bits.
> >  
> > We can virFileReadValueString() then convert it to unsigned int, then count the bits of ‘1’.
> >  
> > thought ? or a new utils function in virfile is required?
>  
> I missed this patch, sorry. I was thinking about changing it to bit
> counting, but I thought it will always be divisible by 4. I will send a
> trivial update to this, but it would be nice to have test cases for it.
> Could you get some of that data from such a machine n the meantime? If
> not, I can just coy what we have and change the mask.
>  
I don’t have a SKX machine by hand, I ask from other guys, the only different is  

cat /sys/fs/resctrl/info/L3/cbm_mask

7ff

it’s 11 bits, which can not divided by 4.  :( I don’t understand why the hardware designed as this.

////////////////////////// below are the /sys/fs/resctrl contents on SKX /////////////////////////

# cd /sys/fs/resctrl

# mkdir p1

# tree /sys/fs/resctrl/

/sys/fs/resctrl/

├── cpus

├── info

│   └── L3

│       ├── cbm_mask

│       ├── min_cbm_bits

│       └── num_closids

├── p1

│   ├── cpus

│   ├── schemata

│   └── tasks

├── schemata

└── tasks

  

# cat /sys/fs/resctrl/info/L3/cbm_mask

7ff

  

# cat /sys/fs/resctrl/info/L3/min_cbm_bits

1

  

# cat /sys/fs/resctrl/info/L3/num_closids

16

  

# cat /sys/fs/resctrl/cpus

00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff

  

# cat /sys/fs/resctrl/schemata

L3:0=7ff;1=7ff
  
>  
> > I am not sure min_cbm_bits should be used.
> >  
> > Besides, on old platform (haswell), the min_cbm_bits is 2, but we still can specify a cbm like
> > “0x7” ) 3bit while do the cache allocation.
> >  
>  
>  
> Yes, you can. min_cbm_bits is not a granularity. it is the minimum
> number of consecutive bits that you need to have when allocating. We
> should also add the granularity there. I'll fix that as well then.
>  

Great,
for “haswell" platform, which is old and has bad support for CAT,  
 https://github.com/torvalds/linux/blob/master/arch/x86/kernel/cpu/intel_rdt.c#L166

I think we can deal with this in libvirt as a special case ?
  

  
>  
> > Later we need this on doing cache allocation(instead of read these value from the host again), we will
> > need
> > 1. total cache size
> > 2. what the cache size of 1 (CBM) bit stand for, so that we can calculate how many bits we want.
> >  
> > Maybe we need to add another filed like ’step’ (to indicate 1 bits stand for) ?
> >  
> > > > control->scope = scope;
> > > >  
> > > > @@ -1732,10 +1743,10 @@ virCapabilitiesInitCaches(virCapsPtr caps)
> > > > VIR_CACHE_TYPE_BOTH) < 0)
> > > > goto cleanup;
> > > > } else if (typeret == 1) {
> > > > - if ((virCapabilitiesGetCacheControl(bank,
> > > > - VIR_CACHE_TYPE_CODE) < 0) ||
> > > > - (virCapabilitiesGetCacheControl(bank,
> > > > - VIR_CACHE_TYPE_DATA) < 0))
> > > > + if (virCapabilitiesGetCacheControl(bank,
> > > > + VIR_CACHE_TYPE_CODE) < 0 ||
> > > > + virCapabilitiesGetCacheControl(bank,
> > > > + VIR_CACHE_TYPE_DATA) < 0)
> > > > goto cleanup;
> > > > }
> > > >  
> > > > diff --git i/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml w/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > > > index c9f460d8a227..49aa0b98ca88 100644
> > > > --- i/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > > > +++ w/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > > > @@ -42,12 +42,12 @@
> > > > </topology>
> > > > <cache>
> > > > <bank id='0' level='3' type='both' size='15360' unit='KiB' cpus='0-5'>
> > > > - <control min='768' unit='KiB' scope='code' max_allocation='8'/>
> > > > - <control min='768' unit='KiB' scope='data' max_allocation='8'/>
> > > > + <control min='768' unit='KiB' type='code' maxAllocs='8'/>
> > > > + <control min='768' unit='KiB' type='data' maxAllocs='8'/>
> > > > </bank>
> > > > <bank id='1' level='3' type='both' size='15360' unit='KiB' cpus='6-11'>
> > > > - <control min='768' unit='KiB' scope='code' max_allocation='8'/>
> > > > - <control min='768' unit='KiB' scope='data' max_allocation='8'/>
> > > > + <control min='768' unit='KiB' type='code' maxAllocs='8'/>
> > > > + <control min='768' unit='KiB' type='data' maxAllocs='8'/>
> > > > </bank>
> > > > </cache>
> > > > </host>
> > > > diff --git i/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml w/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml
> > > > index 04a5eb895727..cb78b4ab788d 100644
> > > > --- i/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml
> > > > +++ w/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml
> > > > @@ -42,10 +42,10 @@
> > > > </topology>
> > > > <cache>
> > > > <bank id='0' level='3' type='both' size='15360' unit='KiB' cpus='0-5'>
> > > > - <control min='768' unit='KiB' scope='both' max_allocation='4'/>
> > > > + <control min='1536' unit='KiB' type='both' maxAllocs='4'/>
> > > > </bank>
> > > > <bank id='1' level='3' type='both' size='15360' unit='KiB' cpus='6-11'>
> > > > - <control min='768' unit='KiB' scope='both' max_allocation='4'/>
> > > > + <control min='1536' unit='KiB' type='both' maxAllocs='4'/>
> > > >  
> > >  
> >  
> >  
> >  
> > > > </bank>
> > > > </cache>
> > > > </host>
> > > > --
> > > > Martin
> > > >  
> > >  
> >  
> >  
>  
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20170605/e11ceaf3/attachment-0001.htm>


More information about the libvir-list mailing list