[Avocado-devel] setting timeout in mux variants
Lukáš Doktor
ldoktor at redhat.com
Thu Jan 2 15:49:58 UTC 2020
Dne 29. 06. 19 v 17:53 Brian J. Murrell napsal(a):
> In a small scale test:
>
> from avocado import Test
> from time import sleep
>
> class MyTest(Test):
> def setUp(self):
> print "setUp(), timeout=%s" % self.timeout
>
> def test(self):
> print "test()"
> sleep(9)
>
> With the following yaml file:
>
> timeout: 10
> tests: !mux
> one:
> timeout: 5
> two:
> timeout: 7
> three:
>
> things work as expected:
>
> $ avocado run --mux-yaml foo.yaml -- foo.py
> JOB ID : 1388d2ac1d6c752007456f4a5bf18c0bf066d8ff
> JOB LOG : /home/brian/daos/daos/avocado/job-results/job-2019-06-29T11.49-1388d2a/job.log
> (1/3) foo.py:MyTest.test;one-a087: INTERRUPTED (5.10 s)
> (2/3) foo.py:MyTest.test;two-0ee9: INTERRUPTED (7.12 s)
> (3/3) foo.py:MyTest.test;three-6700: PASS (9.10 s)
> RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 2 | CANCEL 0
> JOB TIME : 22.04 s
> JOB HTML : /home/brian/daos/daos/avocado/job-results/job-2019-06-29T11.49-1388d2a/results.html
>
> When I try to scale that multiple-timeout use-case up as such:
>
> hosts:
> test_machines:
> - boro-A
> - boro-B
> - boro-C
> - boro-D
> - boro-E
> - boro-F
> - boro-G
> - boro-H
> # Note that subtests below can set their own timeout so this
> # should be a general average of all tests not including outliers
> # (I'm looking at you "rebuild tests")
> timeout: 600
> server_config:
> name: daos_server
> daos_tests:
> num_clients:
> num_clients: 1
> num_replicas:
> num_replicas: 1
> Tests: !mux
> test_r_0-24:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="0-24"
> timeout: 1500
> test_r_25:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="25"
> test_r_26:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="26"
> test_r_27:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="27"
> test_r_28:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="28"
> test_r_29:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="29"
> test_r_30:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="30"
> test_r_31:
> daos_test: r
> test_name: rebuild tests
> args: -s3 -u subtests="31"
> test_d:
> daos_test: d
> test_name: DAOS degraded-mode tests
> test_m:
> daos_test: m
> test_name: Management tests
> test_p:
> daos_test: p
> test_name: Pool tests
> test_c:
> daos_test: c
> test_name: DAOS container tests
> test_e:
> daos_test: e
> test_name: DAOS epoch tests
> test_i:
> daos_test: i
> test_name: IO test
> test_A:
> daos_test: A
> test_name: DAOS Array tests
> test_C:
> daos_test: C
> test_name: DAOS capability tests
> test_o:
> daos_test: o
> test_name: Epoch recovery tests
> test_R:
> daos_test: R
> test_name: DAOS MD replication tests
> test_O:
> daos_test: O
> test_name: OID Allocator tests
> timeout: 900
>
> I get the following error:
>
> 2019-06-29 13:22:07,152 output L0655 DEBUG| Process Process-1:
> 2019-06-29 13:22:07,152 output L0655 DEBUG| Traceback (most recent call last):
> 2019-06-29 13:22:07,152 output L0655 DEBUG| File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
> 2019-06-29 13:22:07,152 output L0655 DEBUG| self.run()
> 2019-06-29 13:22:07,152 output L0655 DEBUG| File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
> 2019-06-29 13:22:07,152 output L0655 DEBUG| self._target(*self._args, **self._kwargs)
> 2019-06-29 13:22:07,152 output L0655 DEBUG| File "/usr/lib/python2.7/site-packages/avocado/core/runner.py", line 327, in _run_test
> 2019-06-29 13:22:07,153 output L0655 DEBUG| instance = loader.load_test(test_factory)
> 2019-06-29 13:22:07,153 output L0655 DEBUG| File "/usr/lib/python2.7/site-packages/avocado/core/loader.py", line 338, in load_test
> 2019-06-29 13:22:07,153 output L0655 DEBUG| test_instance = test_class(**test_parameters)
> 2019-06-29 13:22:07,153 output L0655 DEBUG| File "/var/lib/jenkins/jenkins-1/docker_1/workspace/daos-stack_daos_PR-697 at 6/src/tests/ftest/util/apricot/apricot/test.py", line 150, in __init__
> 2019-06-29 13:22:07,154 output L0655 DEBUG| super(TestWithServers, self).__init__(*args, **kwargs)
> 2019-06-29 13:22:07,154 output L0655 DEBUG| File "/var/lib/jenkins/jenkins-1/docker_1/workspace/daos-stack_daos_PR-697 at 6/src/tests/ftest/util/apricot/apricot/test.py", line 57, in __init__
> 2019-06-29 13:22:07,154 output L0655 DEBUG| super(Test, self).__init__(*args, **kwargs)
> 2019-06-29 13:22:07,154 output L0655 DEBUG| File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 238, in __init__
> 2019-06-29 13:22:07,155 output L0655 DEBUG| self.timeout = self.params.get("timeout", default=default_timeout)
> 2019-06-29 13:22:07,155 output L0655 DEBUG| File "/usr/lib/python2.7/site-packages/avocado/core/varianter.py", line 186, in get
> 2019-06-29 13:22:07,155 output L0655 DEBUG| value = self._get(key, path, default)
> 2019-06-29 13:22:07,155 output L0655 DEBUG| File "/usr/lib/python2.7/site-packages/avocado/core/varianter.py", line 205, in _get
> 2019-06-29 13:22:07,155 output L0655 DEBUG| return param.get_or_die(path, key)
> 2019-06-29 13:22:07,155 output L0655 DEBUG| File "/usr/lib/python2.7/site-packages/avocado/core/varianter.py", line 300, in get_or_die
> 2019-06-29 13:22:07,155 output L0655 DEBUG| for _ in ret]))
> 2019-06-29 13:22:07,156 output L0655 DEBUG| ValueError: Multiple leaves contain the key 'timeout'; ['/run=>600', '/run=>600', '/run=>600', '/run=>600', '/run/daos_tests/Tests/test_r_0-24=>1500']
>
> I'm wondering why the first/small-scale example of this above works but
> the larger/real-world one does not.
>
> Any ideas?
>
> Cheers,
> b.
>
Hello Brian,
this is because the simple example contains only a single leaf with the default "timeout" keyword, so the test parameters contain only a single timeout value. In the complex example the test receives a multiple leaves and there are 2 values comming from "/run" and "/run/daos_tests/Tests/test_r_0-24" and Avocado can't decide which value to use. To fix the issue you'd have to move the default timeout value to "/run/daos_tests/Tests" node.
Another solution (unfortunately not really usable for "timeout" keyword) is to use `params.get("variable", path="/daos_tests/Tests/*)`. It won't work for `timeout`, because this one is handled in `test.py`, but you can use this approach (and it's how it was expected) to partition your params.
Additionally to demonstrate the issue, let's add additional node to the simple example:
timeout: 10
tests: !mux
one:
timeout: 5
two:
timeout: 7
three:
bbb:
ccc: 1
now the first test gets: ["/tests/one", "/bbb"] nodes, second one ["/tests/two", "/bbb"] and the third ["/tests/three", "/bbb"] nodes and `params.get("timeout")` will fail on the first two tests while it should work well on the third, because the timeout variable is the same. You can see the nodes that are given to tests by `avocado variants -m $FILE` as well as their values by `avocado variants -m $FILE -c` (note there is also the `-t` option that shows nice representation of the node tree).
Regards,
Lukáš
PS: I'm sorry the reply took so long, I was on another project when this message arrived and noticed it only now when processing old emails after Christmas.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20200102/a4466aa5/attachment.sig>
More information about the Avocado-devel
mailing list