[libvirt] [PATCH 1/7] Implement public API for virDomainIOThreadsSet

John Ferlan jferlan at redhat.com
Wed Mar 25 23:59:57 UTC 2015


>>
>> This seems ok for the use case where we just give a domain a few
>> iothreads and all the disks will share them.
>>
>> But if you go for one iothread per disk, creating holes in iothread
>> indexes could make sense - we allow unplugging disks from the middle.
>>
>> I can't think of a case where hotplugging a thread with id=6 would be
>> useful if there are only 3 threads.
>>
>> Maybe there should be separate APIs for adding and deleting threads?
> 
> I have to agree here. Separate API for adding and removing (or perhaps
> just a parameter to switch?) specific threads is the way to go. Why?
> 
> Well this function looks like it was copied from virDomainSetVcpus which
> I'm currently planning to replace with specific vcpu hotplug api.
> 
> The reasons there are the same as Jan pointed out just translated into
> the context of vCPUS. At the point where there is no other mapping or
> relationship of belonging the API is fine. This was true for vCPUs in
> the pre-NUMA era. With NUMA, (or disks IO executed by iothreads in this
> case) you have a relationship where  a thread (or vCPU) belongs to a dis
> (or NUMA node) and you need to be able to select them individually.
> 

Obviously at this point these changes won't make 1.2.14... Would have
been nice to have all IOThreads changes in one release, but better to
get things right.

I think the following is close, but I'm still stuck on how to handle
deleting a thread in the middle. Compared to the vCPU algorithm - if you
remove a vCPU you're not given the choice of which one to remove. The
last one is removed.

The problem is how to or whether to represent/save the thread_id array
values in the event of a "hole" in the sequence. Sure we could
create/save a bitmap of 'iothreads' count entries in use, but is there a
'max' value for that? Technically there's no restriction to the number
of IOThreads (theoretically there is ;-))

Here's some rough thoughts for an Add and Del API :

 * virDomainAddIOThread:
 * @domain: a domain object
 * @count: the number of IOThread's to add the domain
 * @flags: bitwise-OR of virDomainModificationImpact

 * virDomainDelIOThread:
 * @domain: a domain object
 * @iothread_id: the specific IOThread ID value to delete
 * @flags: bitwise-OR of virDomainModificationImpact

A "live" domain has 'n' IOThreads in an iothreads array[].  We "add" to
or "refill" the array and adjust the count of iothreads. The refill
option would be if we delete in the middle of the array.

NB: Deletion will always fail if a disk is assigned to a thread, so
discount that. It would be nice if deletion only allowed deletion off
the end of the array, but that's not guaranteed, so per

When "del"eting entries, things get tricky, especially in the middle. At
the end it's easy... The middle just presents problems which I need to
think about or get some feedback...


Thus... start with 4 threads, live domain:
 - Add 'n' threads
    - Search live iothreads array for empty entries to refill
      - If an entry is found, refill it, reduce 'n' keep going
      - If none or we reach array size,
        add 'n' to the live and config count

 - Del Thr#4
    - The live iothreads array entry is zeroed
    - The live & config number of threads reduced to 3

 - Del Thr#1 (or 2 or 3)
   - The live iothreads array entry is zeroed
   - The live and config iothreads entry count remains at 4

For an inactive domain with 4 threads
  - Add 'n' threads
    - Just adjust iothreads array count
  - Del 'Thr#4'
    - Adjust count to 3
  - Del 'Thr#1' (or 2 or 3)
    - Fail

    OR

    - If subsequent threads don't have disks assigned, then threads from
'#' to end are removed.  e.g, Del 3 would then remove 3 & 4... Del 2
would remove 2, 3, & 4.

    Tough to document, but enforcing that the config always can create
'live' threads of some numbered count without having some sort of hole.


Other bright ideas welcome -

Tks -

John




More information about the libvir-list mailing list