<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Feb 25, 2020 at 6:29 PM Daniel P. Berrangé <<a href="mailto:berrange@redhat.com">berrange@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, Feb 19, 2020 at 02:39:22AM +0000, Shaju Abraham wrote:<br>
> <br>
> <br>
> On 2/11/20, 7:06 PM, "Daniel P. Berrangé" <<a href="mailto:berrange@redhat.com" target="_blank">berrange@redhat.com</a>> wrote:<br>
> <br>
> On Tue, Feb 11, 2020 at 10:05:53AM +0100, Martin Kletzander wrote:<br>
> > > On Wed, Feb 05, 2020 at 05:32:50PM +0000, Daniel P. Berrangé wrote:<br>
> > > > On Mon, Feb 03, 2020 at 12:43:32PM +0000, Daniel P. Berrangé wrote:<br>
> > > > > From: Shaju Abraham <<a href="mailto:shaju.abraham@nutanix.com" target="_blank">shaju.abraham@nutanix.com</a>><br>
> > > > > <br>
> > > > There are various config paths that a VM uses. The monitor paths and<br>
> > > > > other lib paths are examples. These paths are tied to the VM name or<br>
> > > > > UUID. The local migration breaks the assumption that there will be only<br>
> > > > > one VM with a unique UUID and name. During local migrations there can be<br>
> > > > > multiple VMs with same name and UUID in the same host. Append the<br>
> > > > > domain-id field to the path so that there is no duplication of path<br>
> > > > names.<br>
> > > > <br>
> > > >This is the really critical problem with localhost migration.<br>
> > > ><br>
> > > >Appending the domain-id looks "simple" but this is a significant<br>
> > > >behavioural / functional change for applications, and I don't think<br>
> > > >it can fully solve the problem either.<br>
> > > > <br>
> > > >This is changing thue paths used in various places where libvirt<br>
> > > >internally generates unique paths (eg QMP socket, huge page or<br>
> > > >file based memory paths, and defaults used for auto-filling<br>
> > > >device paths (<channel> if not specified).<br>
> > > ><br>
> > > >Some of these paths are functionally important to external<br>
> > > >applications and cannot be changed in this way. eg stuff<br>
> > > >integrating with QEMU can be expecting certain memory backing<br>
> > > >file paths, or certain <channel> paths & is liable to break<br>
> > > >if we change the naming convention.<br>
> > > ><br>
> > > >For sake of argument, lets assume we can changing the naming<br>
> > > >convention without breaking anything...<br>
> > > ><br>
> > ><br>
> > >This was already done in (I would say) most places as they use<br>
> > >virDomainDefGetShortName() to get a short, unique name of a directory -- it uses<br>
> > >the domain ID as a prefix.<br>
> > ><br>
> > > > This only applies to paths libvirt generates at VM startup.<br>
> > > ><br>
> > > >There are plenty of configuration elements in the guest XML<br>
> > > >that are end user / mgmt app defined, and reference paths in<br>
> > > >the host OS.<br>
> > > ><br>
> > > >For example <graphics>, <serial>, <console>, <channel>,<br>
> > > >all support UNIX domain sockets and TCP sockets. A UNIX<br>
> > > >domain socket cannot be listened on by multiple VMs<br>
> > > >at once. If the UNIX socket is in client mode, we cannot<br>
> > > >assume the thing QEMU is connecting to allows multiple<br>
> > > >concurrent connections. eg 2 QEMU's could have their<br>
> > > ><serial> connected together over a UNIX socket pair.<br>
> > > >Similarly if automatic TCP port assignment is not used<br>
> > > >we cannot have multiple QEMU's listening on the same<br>
> > > >host.<br>
> > > ><br>
> > > >One answer is to say that localhost migration is just<br>
> > > >not supported for such VMs, but I don't find that very<br>
> > > >convincing because the UNIX domain socket configs<br>
> > > >affected are in common use.<br>
> > > ><br>
> > ><br>
> > >I would be okay with saying that these either need to be changed in a provided<br>
> > >destination XML or the migration will probably break. I do not think it is<br>
> > >unreasonable to say that if users are trying to shoot themselves, we should not<br>
> > >spend a ridiculous time just so we can prevent that. Otherwise we will get to<br>
> > >the same point as we are now -- there might be a case where a local migration<br>
> > >would work, but users cannot execute it even if they were very cautious and went<br>
> > >through all the things that can prevent it from the technical point of view,<br>
> > >libvirt will still disallow that.<br>
> <br>
> >If there are clashing resources, we can't rely on QEMU reporting an<br>
> >error. For example with a UNIX domain socket, the first thing QEMU<br>
> >does is unlink(/socket/path), which will blow away the UNIX domain<br>
> >socket belonging to the original QEMU. As a result if migration<br>
> >fails, and we rollback, the original QEMU will be broken.<br>
> <br>
> By appending the id field to the path, we are effectively nullifying this particular<br>
> concern. Each qemu instance will get its own unique path and monitor. If a migration<br>
> fails, we can roll back.<br>
<br>
No, you've not nullified the problem. This only helps the cases where<br>
libvirt generates the path. This is only a subset of affected cases.<br>
Just one example:<br>
<br>
<graphics type='vnc' socket='/some/path/QEMUGuest1-vnc.sock'><br>
<br>
there are many other parts of the domain XML that accept UNIX socket<br>
paths where the mgmt app picks the socket path. Nothing is validating<br>
this to prevent clashes between the src+dst QEMU on the same host,<br>
meaning on migration rollback, src QEMU is broken.<br>
<br></blockquote><div><div>It is true that I have not covered all the use cases. I would like to know if the approach</div><div>using the symlink is acceptable. In that case we can have the same design for externally</div><div>generated paths as well. Siting your example, if management application provides a path like</div><span class="gmail-im"><div><br></div><div><graphics type='vnc' socket='/some/path/QEMUGuest1-vnc.sock'></div><div><br></div></span><div>We can always consider the path "'/some/path/QEMUGuest1-vnc.sock'" as a symlink to the</div><div>internally generated path generated appending the ID field. <br></div><div><br></div><div>/some/path/QEMUGuest1-vnc.sock--->/some/path-ID/QEMUGuest1-vnc.sock.</div><div><br></div><div>The management application will always refer by the path it has provided</div><div>and will be valid even during migration. At the end of migration the symlink will be changed</div><div>to point to the destination QEMU.</div><div><br></div><div>Regards</div><font color="#888888"><div>Shaju</div></font> </div></div></div>