NVDIMM sizes and DIMM hot plug

[Martin Kletzander]

On Thu, Jun 18, 2020 at 11:22:46AM +0200, Milan Zamazal wrote:
>Peter Krempa <pkrempa at redhat.com> writes:
>
>> On Tue, Jun 16, 2020 at 12:54:29 +0200, Milan Zamazal wrote:
>>> Hi,
>>>
>>
>>> I've found out that NVDIMM size and label size matter for regular
>>> (non-NV) DIMM hot plug.  If the NVDIMM is not aligned correctly, the
>>> guest OS will not accept the hot plugged memory and will complain with
>>> messages such as
>>>
>>>   Block size [0x8000000] unaligned hotplug range: start 0x225000000, size 0x10000000
>>>
>>> The start address above is also reported within <memory> element of the
>>> hot plugged memory in the domain XML:
>>>
>>>   <address type='dimm' slot='1' base='0x225000000'/>
>>>
>>> Apparently, in order to make memory hot plug working in the guest OS,
>>> the inserted memory must be aligned to the platform memory alignment
>>> (128 MB on x86_64).
>>>
>>> I'd like to clarify, how libvirt makes the DIMM address above.  How is
>>
>> If the address isn't provided in the device XML of the attached device,
>> libvirt attaches the device without any address at all and then
>> refreshes the address from qemu in 'qemuDomainUpdateMemoryDeviceInfo'.
>
>OK, I can look into the QEMU source code, but I'd still like to have
>some official confirmation, especially regarding possible pitfalls or
>future changes.  We can't risk data loss.
>
>>> the NVDIMM memory range determined?  According to my experiments, it
>>> seems the NVDIMM specified <size> is taken, NVDIMM <label> size is
>>> subtracted from it and the resulting value is reduced to the nearest
>>> multiple of NVDIMM <alignsize>.  Is this observation correct?  Is it
>>> guaranteed to be stable in future versions?  I need to determine the
>>> right NVDIMM size to make the subsequent memory modules correctly
>>> aligned and then I can't change the NVDIMM size, to not damage data
>>> stored in the NVDIMM.
>>
>>
>> Unfortunatelly I didn't implement NVDIMM support so I don't know the
>> intricacies. I've cc'd Martin Kletzander who did that part.
>
>Martin, do you know how the QEMU part is supposed to work?  I haven't
>received any response on the QEMU list, do you know who could I ask
>directly?
>

Sorry, no idea, it was actually Michal (Cc'd) who implemented it if I'm not
mistaken.

>>> Additionally, when adjusting maxMemory due to NVDIMM presence, should I
>>> increase it by the specified NVDIMM <size> or a different value?
>>>
>>> Thank you,
>>> Milan
>>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20200619/3895c087/attachment.sig>