qemu modularization of qemu-5.1 vs libvirt domcapabilities cache?

[Jim Fehlig]

On 8/20/20 2:07 PM, Jim Fehlig wrote:
> On 8/20/20 12:01 PM, Daniel P. Berrangé wrote:
>> On Thu, Aug 20, 2020 at 11:42:45AM -0600, Jim Fehlig wrote:
>>> On 8/20/20 10:54 AM, Mark Mielke wrote:
>>>> On Thu, Aug 20, 2020 at 11:48 AM Christian Ehrhardt
>>>> <christian.ehrhardt at canonical.com
>>>> <mailto:christian.ehrhardt at canonical.com>> wrote:
>>>>
>>>>      On Thu, Aug 20, 2020 at 5:15 PM Mark Mielke <mark.mielke at gmail.com
>>>>      <mailto:mark.mielke at gmail.com>> wrote:
>>>>       > On Thu, Aug 20, 2020 at 8:55 AM Christian Ehrhardt
>>>>      <christian.ehrhardt at canonical.com 
>>>> <mailto:christian.ehrhardt at canonical.com>>
>>>>      wrote:
>>>>       >> This was meant for upgrades, but if libvirt would define a known 
>>>> path in
>>>>       >> there like /var/run/qemu/last_packaging_change the packages could 
>>>> easily
>>>>       >> touch it on any install/remove/update as Daniel suggested and 
>>>> libvirt could
>>>>       >> check this path like it does with the date of the qemu binary already.
>>>>       > Earlier in this thread - I think one or two of us had asked about the
>>>>      timestamp on the directory that contains the modules.
>>>>       > I'm wondering if a "last_packaging_change" provides any value over and
>>>>      above the timestamp of the directory itself? Wouldn't the directory be 
>>>> best
>>>>      - as it would work automatically for both distro packaging as well as 
>>>> custom
>>>>      installs?
>>>>      Sure, if
>>>>      - "list of files in module dir"
>>>>      - "stat dates of files in module dir"
>>>>      would be checked by libvirt that would also be a no-op for packaging
>>>>      and thereby land faster.
>>>>
>>>>
>>>> Why is the list of files and stat dates needed? Any change done by RPM
>>>> to add or remove a file from the directory, should cause the mtime for
>>>> the directory to be updated. It doesn't really matter what the change is
>>>> - it only matters that the change is detected.
>>>>
>>>> The case for needing to check the files *in* the directory, would be a
>>>> concern about the modules keeping the same name, but being updated in
>>>> place. I'm doubtful that this ever occurs for real, as it would cause
>>>> breakage for running programs. Existing running programs would mmap() or
>>>> similar the binaries into memory, and cannot be updated in place.
>>>> Instead, the inode remains alive, and a new file is created with the
>>>> same name, to replace the old file, and once all file descriptors are
>>>> closed - the inode is deleted.
>>>>
>>>> So, unless there is a hierarchical directory structure - I believe
>>>> checking the modules directory timestamp is near 100% accuracy for
>>>> whether modules were added, removed, or updated using "safe" deployment
>>>> practices either by RPM or "make install".
>>>
>>> I wrote a small test program that checks mtime (also tried ctime) and it
>>> only changes on the directory when files are added and deleted. There is no
>>> change to either if a file in the directory has been updated. It would be
>>> really convenient to check only the directory to see if its' contents have
>>> changed but I'm not aware of how to do that other than with something like
>>> inotify. Suggestions welcomed.
>>
>> IIUC, Mark's point is that an RPM update won't replace the file in-placec.
>> It will write out a new temporary file and then rename over the top of the
>> old file, which should trigger an update on the directory mtime.
> 
> Ah, right. I wasn't considering the behavior of package mangers. My simple test 
> of appending to an existing file doesn't simulate that. Indeed checking mtime on 
> the directory should work.
> 
> I can work on a patch, but one last question is the location of the directory? 
> It looks like qemu searches all over the place for modules: QEMU_MODULE_DIR env 
> var, CONFIG_QEMU_MODDIR, directory up from the executable location, directory of 
> executable location, var/run/qemu. I suppose downstreams will go with 
> CONFIG_QEMU_MODDIR for module location. Should libvirt have a config option to 
> specify the path, so downstream can coordinate that between the packages?

I've sent a patch that includes a 'qemu_moddir' configure option

https://www.redhat.com/archives/libvir-list/2020-August/msg00688.html

Regards,
Jim