[Avocado-devel] Broken sessions.

[Andrei Stepanov]

It also affects on next code:

runner = remote.RemoteRunner(session=ssn)
vdagentd = service.Factory.create_specific_service("spice-vdagentd",
                                                       run=runner.run)


Which uses a session to manage services on a VM.

There is no pause. It instantly tears off a session.
As I know, SSH is based on TCP/IP, it is very stable even for very
unreliable networks.
So, I think it is something wrong in aexpect implementation.


2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|   File
"/mnt/tests/spice/qe-tests/avocado-vt/virttest/staging/service.py",
line 854, in create_specific_service
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|     helper =
Factory.FactoryHelper(run)
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|   File
"/mnt/tests/spice/qe-tests/avocado-vt/virttest/staging/service.py",
line 720, in __init__
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|     result = run("true")
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|   File
"/mnt/tests/spice/qe-tests/avocado-vt/virttest/remote.py", line 1226,
in run
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|
(self.stderr_pipe, self.stderr_pipe))
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|   File
"/usr/lib/python2.7/site-packages/aexpect/client.py", line 1059, in
cmd_output
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|     raise
ShellProcessTerminatedError(cmd, e.status, o)
2016-10-21 12:15:37,839 stacktrace       L0044 ERROR|
ShellProcessTerminatedError: Shell process terminated while waiting
for command to complete: 'cat /var/tmp/tmpgIBcbY;rm -f
/var/tmp/tmpgIBcbY'    (status: 255,    output: 'Write failed: Broken
pipe\n')
2016-10-21 12:15:37,839 stacktrace       L0045 ERROR|
2016-10-21 12:15:37,840 test             L0572 ERROR| Traceback (most
recent call last):

2016-10-21 12:15:37,840 test             L0572 ERROR|   File
"/mnt/tests/spice/qe-tests/avocado-vt/avocado_vt/test.py", line 434,
in _run_avocado
    raise test_exception

2016-10-21 12:15:37,840 test             L0572 ERROR|
ShellProcessTerminatedError: Shell process terminated while waiting
for command to complete: 'cat /var/tmp/tmpgIBcbY;rm -f
/var/tmp/tmpgIBcbY'    (status: 255,    output: 'Write failed: Broken
pipe\n')



On Thu, Oct 20, 2016 at 1:33 PM, Andrei Stepanov <astepano at redhat.com>
wrote:

> Hello
>
> I need to find a solution to one problem.
> Sometimes, ssh sessions to vms are terminated.
> Different sessions, at different places.
> I cannot find any explanation for this.
> Today in the morning I got ~20 times of this error.
> After a while it just disappeared.
> I just was running the test again and again until it is successful.
> The same VM, the same host, same test, same env. Errors are like:
>
> 2016-10-20 12:52:36,913 stacktrace       L0044 ERROR|     print_func)
> 2016-10-20 12:52:36,913 stacktrace       L0044 ERROR|   File
> "/usr/lib/python2.7/site-packages/aexpect/client.py", line 1136, in
> cmd_status_output
> 2016-10-20 12:52:36,913 stacktrace       L0044 ERROR|     o =
> self.cmd_output(cmd, timeout, internal_timeout, print_func, safe)
> 2016-10-20 12:52:36,913 stacktrace       L0044 ERROR|   File
> "/usr/lib/python2.7/site-packages/aexpect/client.py", line 1059, in
> cmd_output
> 2016-10-20 12:52:36,914 stacktrace       L0044 ERROR|     raise
> ShellProcessTerminatedError(cmd, e.status, o)
> 2016-10-20 12:52:36,914 stacktrace       L0044 ERROR|
> ShellProcessTerminatedError: Shell process terminated while waiting for
> command to complete: 'export DISPLAY=:0.0'    (status: 255,    output:
> 'Write failed: Broken pipe\n')
>
>
> OR
>
>
> 2016-10-20 12:44:09,573 stacktrace       L0044 ERROR| ShellStatusError:
> Could not get exit status of command: "ss -x src '*X11-unix*' | grep -q -s
> X11"    (output: '')
> 2016-10-20 12:44:09,573 stacktrace       L0045 ERROR|
> 2016-10-20 12:44:09,574 test             L0572 ERROR| Traceback (most
> recent call last):
> 2016-10-20 12:44:09,574 test             L0572 ERROR|   File
> "/mnt/tests/spice/qe-tests/avocado-vt/avocado_vt/test.py", line 434, in
> _run_avocado
>     raise test_exception
> 2016-10-20 12:44:09,574 test             L0572 ERROR| ShellStatusError:
> Could not get exit status of command: "ss -x src '*X11-unix*' | grep -q -s
> X11"    (output: '')
> 2016-10-20 12:44:09,574 test             L0589 ERROR| ERROR
> 1-rv_connect_fail.172936.req__ss__password__on -> ShellStatusError: Could
> not get exit status of command: "ss -x src '*X11-unix*' | grep -q -s X11"
>  (output: '')
>
>
>
> OR
>
>
> 2016-10-20 12:20:05,707 stacktrace       L0044 ERROR|   File
> "/usr/lib/python2.7/site-packages/aexpect/client.py", line 1059, in
> cmd_output
> 2016-10-20 12:20:05,707 stacktrace       L0044 ERROR|     raise
> ShellProcessTerminatedError(cmd, e.status, o)
> 2016-10-20 12:20:05,707 stacktrace       L0044 ERROR|
> ShellProcessTerminatedError: Shell process terminated while waiting for
> command to complete: 'true 1>/var/tmp/tmp1SwO8Q 2>/var/tmp/tmpVN1k3u'
>  (status: 255,    output: 'Write failed: Broken pipe\n')
> 2016-10-20 12:20:05,707 stacktrace       L0045 ERROR|
> 2016-10-20 12:20:05,708 test             L0572 ERROR| Traceback (most
> recent call last):
> 2016-10-20 12:20:05,708 test             L0572 ERROR|   File
> "/mnt/tests/spice/qe-tests/avocado-vt/avocado_vt/test.py", line 434, in
> _run_avocado
>     raise test_exception
> 2016-10-20 12:20:05,708 test             L0572 ERROR|
> ShellProcessTerminatedError: Shell process terminated while waiting for
> command to complete: 'true 1>/var/tmp/tmp1SwO8Q 2>/var/tmp/tmpVN1k3u'
>  (status: 255,    output: 'Write failed: Broken pipe\n')
>
>
> I tried to stop vm and login from host to VM manually - always successful.
> Next 20 iterations of this test are successful.
>
> Sometimes it is really annoying.
>
> Any suggestions ?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20161021/daf29858/attachment.htm>