[Freeipa-devel] [PATCH 016, 024, 025] First part of the replica promotion tests + testplan
Martin Basti
mbasti at redhat.com
Tue Mar 22 14:03:09 UTC 2016
On 22.03.2016 14:30, Oleg Fayans wrote:
>
> On 03/22/2016 02:06 PM, Martin Basti wrote:
>>
>> On 21.03.2016 15:54, Oleg Fayans wrote:
>>> Hi Lukas, Martin,
>>>
>>> Looks I've implemented the approach proposed by Martin. The issue seems
>>> to have gone (see the external_ca_out for external_ca test output).
>>> Would like you to take a look and tell me what'd you think.
>>>
>>>
>>> On 03/17/2016 08:37 PM, Lukas Slebodnik wrote:
>>>> On (17/03/16 16:00), Oleg Fayans wrote:
>>>>> Hi Lukas,
>>>>>
>>>>> On 03/17/2016 11:28 AM, Lukas Slebodnik wrote:
>>>>>> On (10/03/16 23:09), Oleg Fayans wrote:
>>>>>>> Hi Martin,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 03/08/2016 08:18 PM, Martin Basti wrote:
>>>>>>>> On 08.03.2016 18:24, Martin Basti wrote:
>>>>>>>>> On 08.03.2016 12:38, Oleg Fayans wrote:
>>>>>>>>>> The patches were rebased against the current master
>>>>>>>>>>
>>>>>>>>>> On 03/04/2016 05:33 PM, Martin Basti wrote:
>>>>>>>>>>> * old messages have been removed *
>>>>>>>>>>>>>>> 1)
>>>>>>>>>>>>>>> this method is unused please remove it
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> def test_kra_install_master(self):
>>>>>>>>>>>> Well, in fact it is used twice: in both domain levels, so I'd
>>>>>>>>>>>> better
>>>>>>>>>>>> keep it:
>>>>>>>>>>>>
>>>>>>>>>>>> -bash-4.3$ ipa-run-tests
>>>>>>>>>>>> test_integration/test_replica_promotion.py
>>>>>>>>>>>> --collect-only
>>>>>>>>>>>> ====================================================================================
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> test session starts
>>>>>>>>>>>> =====================================================================================
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
>>>>>>>>>>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
>>>>>>>>>>>> pytest.ini
>>>>>>>>>>>> plugins: sourceorder, multihost
>>>>>>>>>>>> collected 8 items
>>>>>>>>>>>> <Module 'test_integration/test_replica_promotion.py'>
>>>>>>>>>>>> <Class 'TestReplicaPromotionLevel0'>
>>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>>> <Function 'test_kra_install_master'>
>>>>>>>>>>>> <Function 'test_promotion_disabled'>
>>>>>>>>>>>> <Function 'test_backup_restore'>
>>>>>>>>>>>> <Class 'TestKRAInstall'>
>>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>>> <Function 'test_kra_install_without_replica_file'>
>>>>>>>>>>>> <Class 'TestCAInstall'>
>>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>>> <Function 'test_ca_install_without_replica_file'>
>>>>>>>>>>>> <Class 'TestReplicaPromotionLevel1'>
>>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>>> <Function 'test_kra_install_master'>
>>>>>>>>>>>> <Function 'test_replica_prepare_disabled'>
>>>>>>>>>>>> <Class 'TestReplicaManageCommands'>
>>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>>> <Function 'test_replica_manage_commands'>
>>>>>>>>>>> aah my bad, I forgot that pytest executes it when it begins
>>>>>>>>>>> with test_*
>>>>>>>>>>> even in parent class
>>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>>> Why are these there? I do not see any usage
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> from env_config import get_global_config
>>>>>>>>>>>>>>> config = get_global_config()
>>>>>>>>>>>> Removed
>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3) nitpick
>>>>>>>>>>>>>>> + num_clients = 0
>>>>>>>>>>>>>>> this is set by default
>>>>>>>>>>>> Removed
>>>>>>>>>>>>
>>>>>>>>>>>>>>> otherwise LGTM
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Results of testing tomorrow.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Martin^2
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I applied all patches including workarounds, but test failed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion.TestReplicaPromotionLevel0
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51] RUN
>>>>>>>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>>>>>>>> 'Secret123',
>>>>>>>>>>>>>> '--setup-ca', '--ip-address', '192.168.144.102',
>>>>>>>>>>>>>> '/root/ipatests/replica-info.gpg']
>>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51]
>>>>>>>>>>>>>> The host
>>>>>>>>>>>>>> replica1.ipa.test already exists on the master server.
>>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51]
>>>>>>>>>>>>>> You should
>>>>>>>>>>>>>> remove it before proceeding:
>>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51] % ipa
>>>>>>>>>>>>>> host-del replica1.ipa.test
>>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51]
>>>>>>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>>>>>>>> ipa-replica-install command failed. See
>>>>>>>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51] Exit
>>>>>>>>>>>>>> code: 3
>>>>>>>>>>>>>> FAILED
>>>>>>>>>>>> this is exactly the error that happens when a workaround for
>>>>>>>>>>>> 5627
>>>>>>>>>>>> is not
>>>>>>>>>>>> applied. I have re-run the tests with all the patches and
>>>>>>>>>>>> everything
>>>>>>>>>>>> passed. Could you please double-check, whether patch 0027 was
>>>>>>>>>>>> applied
>>>>>>>>>>>> correctly?
>>>>>>>>>>>>
>>>>>>>>>>>> bash-4.3$ ipa-run-tests
>>>>>>>>>>>> test_integration/test_replica_promotion.py
>>>>>>>>>>>> --pdb
>>>>>>>>>>>> ====================================================================================
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> test session starts
>>>>>>>>>>>> =====================================================================================
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
>>>>>>>>>>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
>>>>>>>>>>>> pytest.ini
>>>>>>>>>>>> plugins: sourceorder, multihost
>>>>>>>>>>>> collected 8 items
>>>>>>>>>>>>
>>>>>>>>>>>> test_integration/test_replica_promotion.py ........
>>>>>>>>>>>>
>>>>>>>>>>>> ================================================================================
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 8 passed in 7561.93 seconds
>>>>>>>>>>>> =================================================================================
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> I will
>>>>>>>>>>>
>>>>>>>>>>>>> And it needs ticket, otherwise it will not be in 4-3 branch.
>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5723
>>>>>>>>> NACK
>>>>>>>>>
>>>>>>>>> 1)
>>>>>>>>> ipatests.test_integration.test_replica_promotion.TestReplicaPromotionLevel0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]
>>>>>>>>>
>>>>>>>>> RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>>> 'Secret123', '--setup-ca', '--ip-address', '192.168.200.103', '-r',
>>>>>>>>> 'IPA.TEST']
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] RUN
>>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
>>>>>>>>> '--setup-ca', '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] IPA
>>>>>>>>> client is
>>>>>>>>> already configured on this system, ignoring the --domain, --server,
>>>>>>>>> --realm, --hostname, --password and --keytab options.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] Your
>>>>>>>>> system
>>>>>>>>> may be partly configured.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] Run
>>>>>>>>> /usr/sbin/ipa-server-install --uninstall to clean up.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65]
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65]
>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR You must
>>>>>>>>> provide a file generated by ipa-replica-prepare to create a replica
>>>>>>>>> when the domain is at level 0.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65]
>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>>> ipa-replica-install command failed. See
>>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] Exit
>>>>>>>>> code: 1
>>>>>>>>> FAILED
>>>>>>>>>
>>>>>>>>> 2)
>>>>>>>>> ipatests.test_integration.test_replica_promotion.TestKRAInstall
>>>>>>>>>
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]
>>>>>>>>>
>>>>>>>>> RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>>> 'Secret123', '--setup-ca', '--ip-address', '192.168.200.103', '-r',
>>>>>>>>> 'IPA.TEST']
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] RUN
>>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
>>>>>>>>> '--setup-ca', '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] IPA
>>>>>>>>> client is
>>>>>>>>> already configured on this system, ignoring the --domain, --server,
>>>>>>>>> --realm, --hostname, --password and --keytab options.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Your
>>>>>>>>> system
>>>>>>>>> may be partly configured.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Run
>>>>>>>>> /usr/sbin/ipa-server-install --uninstall to clean up.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR You must
>>>>>>>>> provide a file generated by ipa-replica-prepare to create a replica
>>>>>>>>> when the domain is at level 0.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>>> ipa-replica-install command failed. See
>>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Exit
>>>>>>>>> code: 1
>>>>>>>>> FAILED
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 3)
>>>>>>>>> ipatests.test_integration.test_replica_promotion.TestCAInstall
>>>>>>>>>
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]
>>>>>>>>>
>>>>>>>>> RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>>> 'Secret123', '--setup-dns', '--forwarder', '10.34.78.1',
>>>>>>>>> '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] RUN
>>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
>>>>>>>>> '--setup-dns', '--forwarder', '10.34.78.1', '--ip-address',
>>>>>>>>> '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] IPA
>>>>>>>>> client is
>>>>>>>>> already configured on this system, ignoring the --domain, --server,
>>>>>>>>> --realm, --hostname, --password and --keytab options.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Your
>>>>>>>>> system
>>>>>>>>> may be partly configured.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Run
>>>>>>>>> /usr/sbin/ipa-server-install --uninstall to clean up.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR You must
>>>>>>>>> provide a file generated by ipa-replica-prepare to create a replica
>>>>>>>>> when the domain is at level 0.
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>>> ipa-replica-install command failed. See
>>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Exit
>>>>>>>>> code: 1
>>>>>>>>> FAILED
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Reason:
>>>>>>>>>
>>>>>>>>> def install_replica(master, replica, setup_ca=True,
>>>>>>>>> setup_dns=False,
>>>>>>>>> - setup_kra=False, extra_args=()):
>>>>>>>>> + setup_kra=False, extra_args=(),
>>>>>>>>> domain_level=None):
>>>>>>>>> + if domain_level is None:
>>>>>>>>> + domain_level = domainlevel(master)
>>>>>>> Oops. Sorry, must have been a result of inaccurate rebase. Fixed
>>>>>>> The successful run is attached
>>>>>>>
>>>>>>>>> - if domainlevel(master) == DOMAIN_LEVEL_0:
>>>>>>>>> + if domainlevel == DOMAIN_LEVEL_0:
>>>>>>>>>
>>>>>>>>> domain_level variable is actually unused
>>>>>>>>>
>>>>>>>> Also with your patch that removes host_prepare, I see many following
>>>>>>>> errors in test debug output
>>>>>>>>
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd8] -bash:
>>>>>>>> line 1:
>>>>>>>> cd: /root/ipatests: No such file or directory
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd8] -bash:
>>>>>>>> line 2:
>>>>>>>> /root/ipatests/env.sh: No such file or directory
>>>>>>>>
>>>>>>>>
>>>>>>>> I do not see these errors in current tests, so the patch does not
>>>>>>>> seem
>>>>>>>> right to me.
>>>>>>> http://jenkins.idm.lab.eng.brq.redhat.com:8080/job/freeipa-integration-f23master-customized_ds_config_install-domlevel-1/23/consoleFull
>>>>>>>
>>>>>>>
>>>>>>> I've been experimenting a lot to find get rid of it but was
>>>>>>> unsuccessful. Seemingly it does not affect the tests.
>>>>>>>
>>>>>> Oleg it looks like you broke something
>>>>>> in the test_integration/test_external_ca.py
>>>>>>
>>>>>> _______________________ TestExternalCA.test_external_ca
>>>>>> ________________________
>>>>>>
>>>>>> self = <ipatests.test_integration.test_external_ca.TestExternalCA
>>>>>> object at 0x7ff560a51750>
>>>>>>
>>>>>> def test_external_ca(self):
>>>>>> # Step 1 of ipa-server-install
>>>>>> self.master.run_command([
>>>>>> 'ipa-server-install', '-U',
>>>>>> '-a', self.master.config.admin_password,
>>>>>> '-p', self.master.config.dirman_password,
>>>>>> '--setup-dns', '--no-forwarders',
>>>>>> '-n', self.master.domain.name,
>>>>>> '-r', self.master.domain.realm,
>>>>>> '--domain-level=%i' % self.master.config.domain_level,
>>>>>> '--external-ca'
>>>>>> ])
>>>>>> nss_db = os.path.join(self.master.config.test_dir,
>>>>>> 'testdb')
>>>>>> external_cert_file = os.path.join(nss_db, 'ipa.crt')
>>>>>> external_ca_file = os.path.join(nss_db, 'ca.crt')
>>>>>> noisefile = os.path.join(self.master.config.test_dir,
>>>>>> 'noise.txt')
>>>>>> pwdfile = os.path.join(self.master.config.test_dir,
>>>>>> 'pwdfile.txt')
>>>>>> # Create noise and password files for NSS database
>>>>>>> self.master.run_command('date | sha256sum > %s' % noisefile)
>>>>>> test_integration/test_external_ca.py:49:
>>>>>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>>>>> _ _ _ _ _ _
>>>>>> ../pytest_multihost/host.py:246: in run_command
>>>>>> command.wait(raiseonerr=raiseonerr)
>>>>>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>>>>> _ _ _ _ _ _
>>>>>>
>>>>>> self = <pytest_multihost.transport.SSHCommand object at
>>>>>> 0x7ff5609eb610>
>>>>>> raiseonerr = True
>>>>>>
>>>>>> def wait(self, raiseonerr=True):
>>>>>> """Wait for the remote process to exit
>>>>>> Raises an excption if the exit code is not 0,
>>>>>> unless raiseonerr is
>>>>>> true.
>>>>>> """
>>>>>> if self._done:
>>>>>> return self.returncode
>>>>>> self._end_process()
>>>>>> self._done = True
>>>>>> if raiseonerr and self.returncode:
>>>>>> self.log.error('Exit code: %s', self.returncode)
>>>>>>> raise subprocess.CalledProcessError(self.returncode,
>>>>>>> self.argv)
>>>>>> E CalledProcessError: Command 'date | sha256sum >
>>>>>> /root/ipatests/noise.txt' returned non-zero exit status 1
>>>>>>
>>>>>> ../pytest_multihost/transport.py:159: CalledProcessError
>>>>>> ---------------------------- Captured stdout setup
>>>>>> -----------------------------
>>>>>> <ipatests.test_integration.config.Config object at 0x7ff560a51cd0>
>>>>>> ========================== 1 failed in 32.25 seconds
>>>>>> ===========================
>>>>>>
>>>>>>
>>>>>>
>>>>>> The command "date | sha256sum > /root/ipatests/noise.txt"
>>>>>>
>>>>>> The only explanation is that the directory /root/ipatests/ does not
>>>>>> exist.
>>>>> Obviously this is caused by my patch N 0025
>>>>> freeipa-ofayans-0025-Removed-a-constantly-failing-call-to-prepare_host.patch
>>>>>
>>>>>
>>>>> Without it replica_promotion tests would fail. The only compromise I
>>>>> see
>>>>> is to remove this line in unapply_fixes function in
>>>>> ipatests/test_integration/tasks.py:
>>>>>
>>>>> host.run_command(['rm', '-rvf', host.config.test_dir])
>>>>>
>>>>> Now the biggest question is: why do we keep removing the config folder
>>>>> anyway? All the files (backups of /etc/hosts, etc) that ever get there
>>>>> get overwritten each time anyway so it's quite safe to just keep the
>>>>> folder throughout the whole test execution.
>>>>>
>>>> I do not know why it was removed. IMHO it shoudl be enough
>>>> to remove just content of this directory.
>>>>
>>>> But it will bee godd to fix it ASAP
>>>> and unblock test test_integration/test_external_ca.py
>>>> I hope we want to have green test in 4.3 branch.
>>>> Please also open a ticket so it can be backported to
>>>> stable branch (if needed)
>>>>
>>>> LS
>>>>
>> NACK
>>
>> 0)
>> it is not refactoring, it is bugfixing because it is broken
>>
>> 1)
>> Originally prepare hosts were called for every installation where tasks
>> install_server, install_replica, install_client were used. Now hosts are
>> prepared only in base install class, so all test cases where install
>> classmethod is overriden will have no test directory prepared. Does have
>> py.test any method that is called for each test class before install?
> Well, we can revert my changes from patch-0025 that actually removed
> prepare_host call from a mh fixture in
> ipatests/pytest_plugins/integration.py
> Then we probably do not need most of the changes from this patch except
> the try-except section in prepare_host method itself.
OK
>
>> 2)
>> I dont like new option in unapply fixes, it breaks symetry, if
>> prepare_host is adding directory, then reverse operation should remove
>> directory, IMO unapply_fixes is reverse operation for prepare_host and
>> it should not have any option that changes behavior.
> Agreed. Will remove it.
>
More information about the Freeipa-devel
mailing list