[libvirt] [PATCH resend V10 03/12] Resctrl: Add new xml element to support cache tune

[Martin Kletzander]

On Wed, Mar 15, 2017 at 05:48:09PM -0300, Marcelo Tosatti wrote:
>On Wed, Mar 15, 2017 at 08:49:57PM +0100, Martin Kletzander wrote:
>> On Wed, Mar 15, 2017 at 02:11:23PM -0300, Marcelo Tosatti wrote:
>> >On Wed, Mar 15, 2017 at 03:59:31PM +0100, Martin Kletzander wrote:
>> >>On Wed, Mar 15, 2017 at 02:23:26PM +0000, Daniel P. Berrange wrote:
>> >>>On Wed, Mar 15, 2017 at 03:11:26PM +0100, Martin Kletzander wrote:
>> >>>>On Mon, Mar 06, 2017 at 06:06:32PM +0800, Eli Qiao wrote:
>> >>>>> This patch adds new xml element to support cache tune as:
>> >>>>>
>> >>>>> <cputune>
>> >>>>>  ...
>> >>>>>  <cachetune id='1' host_id='0' type='l3' size='2816' unit='KiB'
>> >>>>
>> >>>>Again, this was already discussed, probably, I just can't find the
>> >>>>source of it.  But host_id actually already selects what cache is
>> >>>>supposed to be used, so instead of type='l3' we only need scope='both'
>> >>>>(or data/instruction) there.  Or am I missing something?  What I'm
>> >>>>concerned about is that the host_id is mostly stable on one host (when
>> >>>>the previously mentioned problems are fixed), but it will make no sense
>> >>>>when the VM is migrated to another one.
>> >
>> >This is the same conditions as pinning a vcpu to a pcpu.
>> >
>> >So yes, it might be that you want to migrate to a host where
>> >a different "host ID" number is used (which might or might not
>> >be a different socket).
>> >
>> >>  Unfortunately, the only
>> >>>>solution I can think of is using multiple keys to precisely describe the
>> >>>>bank we want (e.g. host's cpu id, cache level and scope), but that seems
>> >>>>very unclean.
>> >>>
>> >>>I tend to view use of this cachetune setting as being similar to
>> >>>using host CPU passthrough - you're intentionally trading off
>> >>>migratability of your  guest to get a perf boost.
>> >>>
>> >>>Even without the host_id bit, this is still non-portable, as you
>> >>>might be requesting separate regions for code + data, but the
>> >>>target host of migration may only support shared regions.
>> >
>> >Then migration should fail.
>> >
>>
>> Yes, it will automatically fail when you can't do the same allocations,
>> that's easy.  The difference with host_id is that you cannot foresee
>> whether keeping the host_id makes sense on the destination as well.
>
>But on the destination, the user can use a different vcpu<->pcpu
>configuration, right?
>
>That is, you do not enforce that if the source has say vcpu0 <-> pcpu2,
>the destination must have vcpu0 <-> pcpu2 (the configuration for that
>part is different).
>

Yes, you can change it by supplying different destination XML to the
migration API.

>So, the same thing can be applied to cache configuration.
>

Yes, it can.  However it is way easier to do when you select the
particular cache bank by socket/pcpu, level and scope, then just
host_id.

>> >>Sure, but those are things we can check during migration.  I'd be OK
>> >>with disabling migration (or making it --unsafe) when migrating with
>> >>cachetune.
>> >
>> >Migration support is required (for NFV usecase they want to migrate
>> >VMs around).
>> >
>> >
>>
>> So we at least need a check for the fact that either the caches are the
>> somehow same or destination XML is supplied.
>>
>> The other option would be what I hinted above, that is to change
>> host_id= to something like level, scope, and pcpu(s)/socket(s).
>
>host_id maps to L3 socket via:
>
>commit d57e3ab7e34c51a8badeea1b500bfb738d0af66e
>Author: Fenghua Yu <fenghua.yu at intel.com>
>Date:   Sat Oct 22 06:19:50 2016 -0700
>
>    x86/intel_cacheinfo: Enable cache id in cache info
>
>    Cache id is retrieved from APIC ID and CPUID leaf 4 on x86.
>
>    For more details please see the section on "Cache ID Extraction
>    Parameters" in "Intel 64 Architecture Processor Topology
>Enumeration".
>
>    Also the documentation of the CPUID instruction in the "Intel 64 and
>    IA-32 Architectures Software Developer's Manual"
>
>So on the destination the user has to configure host_id= of the PCPU
>which the guest is pinned to.
>

Oh, OK, I missed that it is actually taken from the host.  that's
because the code generates host_id numbers by using and incrementing
global variable, so it does not correspond to the one kernel knows
about.  Is cache/index*/id the one from APIC?

>> That is
>> fairly easy change from the code point of view, but I'm trying to think
>> more about the user experience.
>>
>> Martin
>
>
>--
>libvir-list mailing list
>libvir-list at redhat.com
>https://www.redhat.com/mailman/listinfo/libvir-list
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Digital signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20170317/fb8a6298/attachment-0001.sig>