<div dir="ltr"><div>1.</div><div><br></div><div>2017-02-02 13:23:59,568 job L0356 INFO | vt.setup.keep_guest_running False</div><div><br></div><div>2.</div><div><br></div><div>We call vm.create( ... params ....) line ~ 170 - 180 on old VM object. This is our mistake.</div><div><br></div><div><br></div><div>For example</div><div>----------------</div><div><br></div><div>VM object from previous test already has options:</div><div><br></div><div><div> self.spice_options = {}</div></div><div><br></div><div>Go to : qemu_vm.py Line: ~~ 2028</div><div><br></div><div><div> for skey in spice_keys:</div><div> value = params.get(skey, None)</div><div> if value:</div><div> logging.warn("Add: %s, %s", skey, value)</div><div> self.spice_options[skey] = value <-------- If next test doesn't define Spice params than params from previous test remain. We do not flush self.spice_options.</div></div><div><br></div><div>We do not flush all old VM.xxxxxxxx members. And sometimes, they are taken from previous tests.</div><div><div><br></div><div>As a result VM sometimes gets wrongs cmdline.<br></div></div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 2, 2017 at 1:56 PM, Lukáš Doktor <span dir="ltr"><<a href="mailto:ldoktor@redhat.com" target="_blank">ldoktor@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello Andrei,<br>
<br>
first, can you please confirm you are using the `keep_guest_running` to minimize the environment differences.<br>
<br>
Then to your reproducer, I'm not sure how to trigger it. I use a modified `boot` test where I run the pre-process twice with modified params. This way I get your "Old vm is destroyed, but, it is still present in env." message, but this message only means the instance is reused. It does not mean it is used to boot the machine. The important part is that `start_vm` is set to `True` which means that around line `173` the old `params` are replaced with the new fresh ones so at least in my understanding it should work properly. Anyway mistakes happen so would you please give me a simple reproducer or at least more info about where this does not work.<br>
<br>
Regards,<br>
Lukáš<br>
<br>
<br>
Dne 2.2.2017 v 12:53 Andrei Stepanov napsal(a):<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_549163404730263420gmail-">
Lukáš Hi!<br>
<br>
I added next debug code:<br>
<br>
diff --git a/virttest/env_process.py b/virttest/env_process.py<br>
index d05976e..64c78ac 100644<br>
--- a/virttest/env_process.py<br>
+++ b/virttest/env_process.py<br>
@@ -162,6 +162,12 @@ def preprocess_vm(test, params, env, name):<br>
vm.devices = None<br>
start_vm = True<br>
old_vm.destroy(gracefully=gra<wbr>cefully_kill)<br>
+ for key1 in env.keys():<br>
+ vm1 = env[key1]<br>
+ if not isinstance(vm1, virt_vm.BaseVM):<br>
+ continue<br></span>
+ if <a href="http://vm1.name" rel="noreferrer" target="_blank">vm1.name</a> <<a href="http://vm1.name" rel="noreferrer" target="_blank">http://vm1.name</a>> == <a href="http://old_vm.name" rel="noreferrer" target="_blank">old_vm.name</a><br>
<<a href="http://old_vm.name" rel="noreferrer" target="_blank">http://old_vm.name</a>>:<span class="gmail-m_549163404730263420gmail-"><br>
+ logging.warn("Old vm is destroyed, but, it<br>
is still present in env.")<br>
update_virtnet = True<br>
<br>
if start_vm:<br>
<br>
<br>
<br>
Than logs have message: "Old vm is destroyed, but, it is still present<br>
in env."<br>
<br>
So, VM was destroyed, VM object is still in env.<br>
<br>
Let's go to line 690 in the same file:<br>
<br></span><span class="gmail-m_549163404730263420gmail-">
if <a href="http://vm.name" rel="noreferrer" target="_blank">vm.name</a> <<a href="http://vm.name" rel="noreferrer" target="_blank">http://vm.name</a>> not in requested_vms:<br>
<br></span><span class="gmail-m_549163404730263420gmail-">
VM for next test has the same name.<br>
<br>
As a result: next test uses VM object from previous test. VM is started<br>
using params from previous test.<br>
<br>
And this behavior is serious bug.<br>
<br>
<br>
On Wed, Feb 1, 2017 at 3:05 PM, Lukáš Doktor <<a href="mailto:ldoktor@redhat.com" target="_blank">ldoktor@redhat.com</a><br></span><div><div class="gmail-m_549163404730263420gmail-h5">
<mailto:<a href="mailto:ldoktor@redhat.com" target="_blank">ldoktor@redhat.com</a>>> wrote:<br>
<br>
Hello Andrei,<br>
<br>
if this happens than there is something really wrong because Avocado<br>
should re-create the VM for 2 reasons:<br>
<br>
1) by default VMs are not shared between tests (can be enabled in<br>
cfg by setting `keep_guest_running = True` in `vt.setup` section)<br>
2) when the params of the existing VM and the current params are<br>
different, it's recreated.<br>
<br>
The (2) is checked in `virttest.env_process` on line `159` where it<br>
executes `vm.needs_restart`. The implementation of this function is<br>
defined mainly in `virttest.virt_vm` and unless overridden it uses<br>
the `virttest.virt_vm.make_create_<wbr>command` to compare the original<br>
and the new command line to create the VM. If they are the same it<br>
reuses the VM (when (1) is enabled), otherwise it destroys the old<br>
VM and creates a new one.<br>
<br>
The question is how different your machines are. The<br>
`make_create_command` does not compares the extra dynamic stuff like<br>
migration. More info about this can be found in<br>
`virttest.qemu_vm.create()` function (if using qemu as a backend).<br>
<br>
Would you please (publicly or in private) share more details I might<br>
be able to identify why the machine is not being re-created.<br>
<br>
Regards,<br>
Lukáš<br>
<br>
Dne 31.1.2017 v 18:15 Andrei Stepanov napsal(a):<br>
<br>
Hi.<br>
<br>
It seems that avocado-vt has a serious bug.<br>
<br>
Test case: run a few avocado-vt tests in a bunch. For example<br>
two tests.<br>
Test1 starts just right after Test2.<br>
<br>
Test1.<br>
Test2.<br>
<br>
Test1 & Test2 use the same name for VM in cartesian configs:<br>
vms = guest<br>
<br>
Other options for VM() objects are different, for example port<br>
VNC port,<br>
some device config, etc....<br>
<br>
The problem is that: KVM from Test2 uses VM() object that was<br>
created<br>
for Test1.<br>
<br>
For Test2:<br>
virttest/env_process.py:<br>
<br>
def preprocess_vm(test, params, env, name):<br>
<br>
vm = env.get_vm(name) <--- returns VM that was created<br>
for Test1.<br>
create_vm == False<br>
<br>
It can be fixed by:<br>
<br>
diff --git a/virttest/env_process.py b/virttest/env_process.py<br>
index d05976e..7c08df4 100644<br>
--- a/virttest/env_process.py<br>
+++ b/virttest/env_process.py<br>
@@ -687,9 +687,8 @@ def preprocess(test, params, env):<br>
vm = env[key]<br>
if not isinstance(vm, virt_vm.BaseVM):<br>
continue<br></div></div>
- if <a href="http://vm.name" rel="noreferrer" target="_blank">vm.name</a> <<a href="http://vm.name" rel="noreferrer" target="_blank">http://vm.name</a>> <<a href="http://vm.name" rel="noreferrer" target="_blank">http://vm.name</a>> not in<span class="gmail-m_549163404730263420gmail-"><br>
requested_vms:<br>
- vm.destroy()<br>
- del env[key]<br>
+ vm.destroy()<br>
+ del env[key]<br>
<br>
if (params.get("auto_cpu_model") == "yes" and<br>
vm_type == "qemu"):<br>
<br>
<br>
Could you please confirm that bug exists in real?<br>
<br>
<br>
<br>
</span></blockquote>
<br>
</blockquote></div><br></div></div>