qemu modularization of qemu-5.1 vs libvirt domcapabilities cache?

[Jim Fehlig]

On 8/20/20 12:01 PM, Daniel P. Berrangé wrote:
> On Thu, Aug 20, 2020 at 11:42:45AM -0600, Jim Fehlig wrote:
>> On 8/20/20 10:54 AM, Mark Mielke wrote:
>>> On Thu, Aug 20, 2020 at 11:48 AM Christian Ehrhardt
>>> <christian.ehrhardt at canonical.com
>>> <mailto:christian.ehrhardt at canonical.com>> wrote:
>>>
>>>      On Thu, Aug 20, 2020 at 5:15 PM Mark Mielke <mark.mielke at gmail.com
>>>      <mailto:mark.mielke at gmail.com>> wrote:
>>>       > On Thu, Aug 20, 2020 at 8:55 AM Christian Ehrhardt
>>>      <christian.ehrhardt at canonical.com <mailto:christian.ehrhardt at canonical.com>>
>>>      wrote:
>>>       >> This was meant for upgrades, but if libvirt would define a known path in
>>>       >> there like /var/run/qemu/last_packaging_change the packages could easily
>>>       >> touch it on any install/remove/update as Daniel suggested and libvirt could
>>>       >> check this path like it does with the date of the qemu binary already.
>>>       > Earlier in this thread - I think one or two of us had asked about the
>>>      timestamp on the directory that contains the modules.
>>>       > I'm wondering if a "last_packaging_change" provides any value over and
>>>      above the timestamp of the directory itself? Wouldn't the directory be best
>>>      - as it would work automatically for both distro packaging as well as custom
>>>      installs?
>>>      Sure, if
>>>      - "list of files in module dir"
>>>      - "stat dates of files in module dir"
>>>      would be checked by libvirt that would also be a no-op for packaging
>>>      and thereby land faster.
>>>
>>>
>>> Why is the list of files and stat dates needed? Any change done by RPM
>>> to add or remove a file from the directory, should cause the mtime for
>>> the directory to be updated. It doesn't really matter what the change is
>>> - it only matters that the change is detected.
>>>
>>> The case for needing to check the files *in* the directory, would be a
>>> concern about the modules keeping the same name, but being updated in
>>> place. I'm doubtful that this ever occurs for real, as it would cause
>>> breakage for running programs. Existing running programs would mmap() or
>>> similar the binaries into memory, and cannot be updated in place.
>>> Instead, the inode remains alive, and a new file is created with the
>>> same name, to replace the old file, and once all file descriptors are
>>> closed - the inode is deleted.
>>>
>>> So, unless there is a hierarchical directory structure - I believe
>>> checking the modules directory timestamp is near 100% accuracy for
>>> whether modules were added, removed, or updated using "safe" deployment
>>> practices either by RPM or "make install".
>>
>> I wrote a small test program that checks mtime (also tried ctime) and it
>> only changes on the directory when files are added and deleted. There is no
>> change to either if a file in the directory has been updated. It would be
>> really convenient to check only the directory to see if its' contents have
>> changed but I'm not aware of how to do that other than with something like
>> inotify. Suggestions welcomed.
> 
> IIUC, Mark's point is that an RPM update won't replace the file in-placec.
> It will write out a new temporary file and then rename over the top of the
> old file, which should trigger an update on the directory mtime.

Ah, right. I wasn't considering the behavior of package mangers. My simple test 
of appending to an existing file doesn't simulate that. Indeed checking mtime on 
the directory should work.

I can work on a patch, but one last question is the location of the directory? 
It looks like qemu searches all over the place for modules: QEMU_MODULE_DIR env 
var, CONFIG_QEMU_MODDIR, directory up from the executable location, directory of 
executable location, var/run/qemu. I suppose downstreams will go with 
CONFIG_QEMU_MODDIR for module location. Should libvirt have a config option to 
specify the path, so downstream can coordinate that between the packages?

Regards,
Jim