[libvirt] Designing NVDIMM & memory-backend-file

[Daniel P. Berrange]

On Mon, Jul 04, 2016 at 11:21:20AM +0200, Michal Privoznik wrote:
> On 01.07.2016 17:06, Daniel P. Berrange wrote:
> > On Fri, Jul 01, 2016 at 04:35:18PM +0200, Michal Privoznik wrote:
> >> Dear list,
> >>
> >> I'd like to fix the following bug [1]. Long story short, the only way
> >> how to have a domain use memory-backend-file object is to configure
> >> hugepages. Either for whole domain under <memoryBacking/> element, or in
> >> <memory/> device:
> >>
> >>   <memoryBacking>
> >>     <hugepages>
> >>       <page size='2048' unit='KiB' nodeset='1'/>
> >>     </hugepages>
> >>   </memoryBacking>
> >>
> >> -object memory-backend-file,id=ram-node1,prealloc=yes,\
> >> mem-path=/dev/hugepages2M/libvirt/qemu,size=1073741824,host-nodes=0-3,\
> >> policy=bind \
> >> -numa node,nodeid=1,cpus=1,memdev=ram-node1 \
> >>
> >>
> >>     <memory model='dimm'>
> >>       <source>
> >>         <nodemask>1-3</nodemask>
> >>         <pagesize unit='KiB'>2048</pagesize>
> >>       </source>
> >>       <target>
> >>         <size unit='KiB'>524287</size>
> >>         <node>0</node>
> >>       </target>
> >>     </memory>
> >>
> >> -object memory-backend-file,id=memdimm1,prealloc=yes,\
> >> mem-path=/dev/hugepages2M/libvirt/qemu,size=536870912,host-nodes=1-3,policy=bind
> >> \
> >> -device pc-dimm,node=0,memdev=memdimm1,id=dimm1 \
> >>
> >> Now, there's a request in the BZ to allow applications to use the
> >> memory-backend-file but let them use different backend, well different
> >> path for backing the memory, e.g. shm which is usually mounted at
> >> /dev/shm. And I'd like to consult my idea before I dig deep in the patches.
> >>
> >> My idea is to extend <memory/> device we have, more precisely <source/>
> >> element to allow something like this:
> >>
> >>     <memory model='dimm'>
> >>       <source>
> >>         <path>/dev/shm</path>
> >>       </source>
> >>       <target/>
> >>     </memory>
> >>
> >> This way we can allow users to pass an arbitrary path to the
> >> memory-backend-file. Also the amount of code needing change would be
> >> fairly small O:-)
> > 
> > I'm not really a fan of exposing file paths for shm in the XML. This
> > filesystem exposure of shm is a linux specific concept which other
> > UNIX with shm support don't do afaik. This is the same reasn why
> > we don't expose the actual hugepages path in the XML, just let the
> > user request hugepages and libvirt figures out the linux-specific
> > way to enable it.
> 
> Okay, fair enough. And I can see it working with /dev/shm. But for
> NVDIMM we certainly have to let users specify path to the DIMM module.
> Therefore I suggested what I've suggested. But maybe I'm joining two
> separate problems together.
> 
> So for the /dev/shm part there seems to be some movement. But for the
> NVDIMM module - any ideas how to express that one?

Ok, i guess it is sensible to view the NVDIMM as just another type of
block device, so from that POV it is reasonable to allow user to specify
a path to the backing store for NVDIMM, in the same way we do with <disk/>
sources. So we just need to make clear the distinction between NVDIMM
config, and general ability to use volatile shared memory backing with
regular DIMMs.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|