<div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div>[Adding Tomas Golembiovsky]</div><div dir="ltr"><br></div><div dir="ltr">On Wed, Sep 26, 2018 at 12:11 PM Richard W.M. Jones <<a href="mailto:rjones@redhat.com">rjones@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Rather than jumping to a solution, can you explain what the problem<br>
is that you're trying to solve?<br>
<br>
You need to do <X>, you tried virt-v2v, it doesn't do <X>, etc.<br></blockquote><div><br></div><div><div>Well, that's mainly IMS related challenges. We're working on OpenStack output</div><div>support and migration throttling and this implies changes to virt-v2v-wrapper.</div><div>This is then the opportunity to think about virt-v2v-wrapper maintenance and</div><div>feature set. It has been created in the first place to simplify interaction</div><div>with virt-v2v from ManageIQ.</div><div><br></div><div>The first challenge we faced is the interaction with virt-v2v. It's highly</div><div>versatile and proposes a lot of options for input and output. The downside of</div><div>it is that over time is becomes more and more difficult to know them all. And</div><div>all the error messages are made for human beings, not machines, so providing</div><div>feedback through a launcher, such as virt-v2v-wrapper, is difficult.</div><div><br></div><div>The second challenge was monitoring the virt-v2v process liveliness and</div><div>progress. For liveliness, the virt-v2v-wrapper stores the PID and checks that</div><div>it's still present and when absent checks its return code for success (0) or</div><div>failure (!0), and any other launcher could do the same. For progress, the only</div><div>way to know what happens is to run virt-v2v in debug mode (-v -x) and parse the</div><div>(very extensive) output. Virt-v2v-wrapper does it for us in IMS, but it is</div><div>merely a workaround. I'd expect a conversion tool to provide a comprehensive</div><div>progress, such as "I'm converting VM 'my_vm' and more specifically disk X/Y</div><div>(XX%). Total conversion progress is XX%". Of course, I'd also expect a machine</div><div>readable output (JSON, CSV, YAML…). Debug mode ensures we have all the data in</div><div>case of failure, so I don't say remove it, but simply add specialized outputs.</div><div><br></div><div>The third challenge was to clean up in case of virt-v2v failure. For example,</div><div>when it fails converting a disk to RHV, it doesn't clean the finished and</div><div>unfinished disks. Virt-v2v-wrapper was initially written by RHV team (Tomas)</div><div>for RHV migrations, so it sounded fair(ish). But, extending the outputs to</div><div>OpenStack, we'll have to deal with leftovers in OpenStack too. Maybe a cleanup</div><div>on failure option would be a good idea, with a default to false to not break</div><div>existing behaviour.</div><div><br></div><div>The fourth challenge is to limit the resources allocated to virt-v2v during</div><div>conversion, because concurrent conversions may have a huge impact on conversion</div><div>host performance. In the case of an oVirt host, this can impact the virtual</div><div>machines that run on it. This is not covered yet by the wrapper, but</div><div>implementation will likely be based on Linux cgroups and tc.</div><div><br></div><div>The wrapper also adds an interesting feature: both virt-v2v and virt-v2v-wrapper</div><div>run daemonized and we can asynchronously poll the progress. This is really key</div><div>for IMS (and maybe for others): this allows us to start as many conversions in</div><div>parallel as needed and monitor them. Currently, the Python code forks and</div><div>detaches itself, after providing the paths to the state file. In the discussion</div><div>about cgroups, it was mentioned that systemd units could be used, and it echoes</div><div>with the daemonization, as systemd-run allows running processes under systemd</div><div>and in their own slice, on which cgroups limits can be set.</div><div><br></div><div>About the evolution of virt-v2v-wrapper that I'm going to describe, let me state</div><div>that this is my personal view and it endorses only myself.</div><div><br></div><div>I would like to see the machine-to-machine interaction, logging and cleanup in</div><div>virt-v2v itself because it is valuable to everyone, not only IMS.</div><div><br></div><div>I would also like to convert virt-v2v-wrapper to a conversion API and Scheduler</div><div>service. The idea is that it would provide an as-a-Service endpoint for</div><div>conversions, that would allow creation of conversion jobs (POST), fetching of</div><div>the status (GET), cancelation of a conversion (DELETE) and changing of the</div><div>limits (PATCH). In the background, a basic scheduler would simply ensure that</div><div>all the jobs are running. Each virt-v2v process would be run as a systemd unit</div><div>(journald could capture the debug output), so that it is independent from</div><div>the API and Scheduler processes.</div><div><br></div><div>I know that I can propose patches for changes to virt-v2v, or at least file</div><div>RFEs in Bugzilla (my developer skills and programing languages breadth are</div><div>limited). For the evolved wrapper, my main concern is its housing and</div><div>maintenance. It doesn't work only for oVirt, so having its lifecycle tied to</div><div>oVirt doesn't seem relevant in the long term. In fact, it can be for any</div><div>virt-v2v output, so my personal opinion is that it should live in the virt-v2v</div><div>ecosystem and follow it's lifecycle. As for its maintenance, we still have to</div><div>figure out who will be responsible for it, i.e. who will be able to dedicate</div><div>time to it.</div></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Rich.<br>
<br>
-- <br>
Richard Jones, Virtualization Group, Red Hat <a href="http://people.redhat.com/~rjones" rel="noreferrer" target="_blank">http://people.redhat.com/~rjones</a><br>
Read my programming and virtualization blog: <a href="http://rwmj.wordpress.com" rel="noreferrer" target="_blank">http://rwmj.wordpress.com</a><br>
virt-top is 'top' for virtual machines. Tiny program with many<br>
powerful monitoring features, net stats, disk stats, logging, etc.<br>
<a href="http://people.redhat.com/~rjones/virt-top" rel="noreferrer" target="_blank">http://people.redhat.com/~rjones/virt-top</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><p style="margin:0px;padding:0px"><b style="font-family:arial,helvetica,sans-serif;font-size:small"><font color="#cc0000"><span style="margin:0px;padding:0px">Fabien</span> <span style="margin:0px;padding:0px">Dupont</span></font></b><br></p><p style="color:rgb(0,0,0);margin:0px;padding:0px"></p><p style="font-size:small;color:rgb(0,0,0);margin:0px;padding:0px"><span style="font-family:arial,helvetica,sans-serif;font-size:x-small">PRINCIPAL SOFTWARE ENGINEER</span><font size="1" face="arial, helvetica, sans-serif"><br style="margin:0px;padding:0px"></font></p><p style="font-size:small;margin:0px;padding:0px"><font face="arial, helvetica, sans-serif" size="1" color="#000000">Red Hat - Solutions Engineering</font></p><p style="margin:0px;padding:0px"><font size="1" face="arial, helvetica, sans-serif"><span style="margin:0px;padding:0px"><span style="margin:0px;padding:0px"><a href="mailto:fabien@redhat.com" target="_blank"><font color="#0b5394">fabien@redhat.com</font></a></span><font color="#000000"> </font></span><span style="margin:0px;padding:0px"><font color="#000000">M: </font><a href="tel:+33662784971" style="margin:0px;padding:0px" target="_blank"><font color="#0b5394">+33 (0) 662 784 971</font></a></span></font></p><p style="margin:0px;padding:0px"><span style="margin:0px;padding:0px"><font size="1" face="arial, helvetica, sans-serif"><a href="http://redhat.com" style="color:rgb(0,0,0)" target="_blank"><img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="96" height="30"></a><font color="#000000"> </font><span style="margin:0px;padding:0px"><font color="#cc0000"><b>TRIED. TESTED. TRUSTED.</b></font></span></font></span></p><p style="color:rgb(0,0,0);margin:0px;padding:0px"><font size="1" face="arial, helvetica, sans-serif"><span style="margin:0px;padding:0px"></span></font></p><div style="color:rgb(0,0,0);margin:0px;padding:0px"><div style="margin:0px;padding:0px"><font size="1" face="arial, helvetica, sans-serif">Twitter: <a href="https://twitter.com/redhatway" target="_blank">@redhatway</a> | Instagram: <a href="https://www.instagram.com/redhatinc/" target="_blank">@redhatinc</a> | Snapchat: @redhatsnaps</font></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>