[Freeipa-devel] [PATCH 016, 024, 025] First part of the replica promotion tests + testplan
Oleg Fayans
ofayans at redhat.com
Tue Mar 22 13:30:08 UTC 2016
On 03/22/2016 02:06 PM, Martin Basti wrote:
>
>
> On 21.03.2016 15:54, Oleg Fayans wrote:
>> Hi Lukas, Martin,
>>
>> Looks I've implemented the approach proposed by Martin. The issue seems
>> to have gone (see the external_ca_out for external_ca test output).
>> Would like you to take a look and tell me what'd you think.
>>
>>
>> On 03/17/2016 08:37 PM, Lukas Slebodnik wrote:
>>> On (17/03/16 16:00), Oleg Fayans wrote:
>>>> Hi Lukas,
>>>>
>>>> On 03/17/2016 11:28 AM, Lukas Slebodnik wrote:
>>>>> On (10/03/16 23:09), Oleg Fayans wrote:
>>>>>> Hi Martin,
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 03/08/2016 08:18 PM, Martin Basti wrote:
>>>>>>>
>>>>>>> On 08.03.2016 18:24, Martin Basti wrote:
>>>>>>>>
>>>>>>>> On 08.03.2016 12:38, Oleg Fayans wrote:
>>>>>>>>> The patches were rebased against the current master
>>>>>>>>>
>>>>>>>>> On 03/04/2016 05:33 PM, Martin Basti wrote:
>>>>>>>>>> * old messages have been removed *
>>>>>>>>>>>>>> 1)
>>>>>>>>>>>>>> this method is unused please remove it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> def test_kra_install_master(self):
>>>>>>>>>>> Well, in fact it is used twice: in both domain levels, so I'd
>>>>>>>>>>> better
>>>>>>>>>>> keep it:
>>>>>>>>>>>
>>>>>>>>>>> -bash-4.3$ ipa-run-tests
>>>>>>>>>>> test_integration/test_replica_promotion.py
>>>>>>>>>>> --collect-only
>>>>>>>>>>> ====================================================================================
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> test session starts
>>>>>>>>>>> =====================================================================================
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
>>>>>>>>>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
>>>>>>>>>>> pytest.ini
>>>>>>>>>>> plugins: sourceorder, multihost
>>>>>>>>>>> collected 8 items
>>>>>>>>>>> <Module 'test_integration/test_replica_promotion.py'>
>>>>>>>>>>> <Class 'TestReplicaPromotionLevel0'>
>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>> <Function 'test_kra_install_master'>
>>>>>>>>>>> <Function 'test_promotion_disabled'>
>>>>>>>>>>> <Function 'test_backup_restore'>
>>>>>>>>>>> <Class 'TestKRAInstall'>
>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>> <Function 'test_kra_install_without_replica_file'>
>>>>>>>>>>> <Class 'TestCAInstall'>
>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>> <Function 'test_ca_install_without_replica_file'>
>>>>>>>>>>> <Class 'TestReplicaPromotionLevel1'>
>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>> <Function 'test_kra_install_master'>
>>>>>>>>>>> <Function 'test_replica_prepare_disabled'>
>>>>>>>>>>> <Class 'TestReplicaManageCommands'>
>>>>>>>>>>> <Instance '()'>
>>>>>>>>>>> <Function 'test_replica_manage_commands'>
>>>>>>>>>> aah my bad, I forgot that pytest executes it when it begins
>>>>>>>>>> with test_*
>>>>>>>>>> even in parent class
>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>> Why are these there? I do not see any usage
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> from env_config import get_global_config
>>>>>>>>>>>>>> config = get_global_config()
>>>>>>>>>>> Removed
>>>>>>>>>>>
>>>>>>>>>>>>>> 3) nitpick
>>>>>>>>>>>>>> + num_clients = 0
>>>>>>>>>>>>>> this is set by default
>>>>>>>>>>> Removed
>>>>>>>>>>>
>>>>>>>>>>>>>> otherwise LGTM
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Results of testing tomorrow.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Martin^2
>>>>>>>>>>>>>>
>>>>>>>>>>>>> I applied all patches including workarounds, but test failed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> ipatests.test_integration.test_replica_promotion.TestReplicaPromotionLevel0
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51] RUN
>>>>>>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>>>>>>> 'Secret123',
>>>>>>>>>>>>> '--setup-ca', '--ip-address', '192.168.144.102',
>>>>>>>>>>>>> '/root/ipatests/replica-info.gpg']
>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51]
>>>>>>>>>>>>> The host
>>>>>>>>>>>>> replica1.ipa.test already exists on the master server.
>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51]
>>>>>>>>>>>>> You should
>>>>>>>>>>>>> remove it before proceeding:
>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51] % ipa
>>>>>>>>>>>>> host-del replica1.ipa.test
>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51]
>>>>>>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>>>>>>> ipa-replica-install command failed. See
>>>>>>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd51] Exit
>>>>>>>>>>>>> code: 3
>>>>>>>>>>>>> FAILED
>>>>>>>>>>> this is exactly the error that happens when a workaround for
>>>>>>>>>>> 5627
>>>>>>>>>>> is not
>>>>>>>>>>> applied. I have re-run the tests with all the patches and
>>>>>>>>>>> everything
>>>>>>>>>>> passed. Could you please double-check, whether patch 0027 was
>>>>>>>>>>> applied
>>>>>>>>>>> correctly?
>>>>>>>>>>>
>>>>>>>>>>> bash-4.3$ ipa-run-tests
>>>>>>>>>>> test_integration/test_replica_promotion.py
>>>>>>>>>>> --pdb
>>>>>>>>>>> ====================================================================================
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> test session starts
>>>>>>>>>>> =====================================================================================
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> platform linux2 -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
>>>>>>>>>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile:
>>>>>>>>>>> pytest.ini
>>>>>>>>>>> plugins: sourceorder, multihost
>>>>>>>>>>> collected 8 items
>>>>>>>>>>>
>>>>>>>>>>> test_integration/test_replica_promotion.py ........
>>>>>>>>>>>
>>>>>>>>>>> ================================================================================
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 8 passed in 7561.93 seconds
>>>>>>>>>>> =================================================================================
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> I will
>>>>>>>>>>
>>>>>>>>>>>> And it needs ticket, otherwise it will not be in 4-3 branch.
>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/5723
>>>>>>>> NACK
>>>>>>>>
>>>>>>>> 1)
>>>>>>>> ipatests.test_integration.test_replica_promotion.TestReplicaPromotionLevel0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]
>>>>>>>>
>>>>>>>> RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>> 'Secret123', '--setup-ca', '--ip-address', '192.168.200.103', '-r',
>>>>>>>> 'IPA.TEST']
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] RUN
>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
>>>>>>>> '--setup-ca', '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] IPA
>>>>>>>> client is
>>>>>>>> already configured on this system, ignoring the --domain, --server,
>>>>>>>> --realm, --hostname, --password and --keytab options.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] Your
>>>>>>>> system
>>>>>>>> may be partly configured.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] Run
>>>>>>>> /usr/sbin/ipa-server-install --uninstall to clean up.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65]
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR You must
>>>>>>>> provide a file generated by ipa-replica-prepare to create a replica
>>>>>>>> when the domain is at level 0.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>> ipa-replica-install command failed. See
>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd65] Exit
>>>>>>>> code: 1
>>>>>>>> FAILED
>>>>>>>>
>>>>>>>> 2)
>>>>>>>> ipatests.test_integration.test_replica_promotion.TestKRAInstall
>>>>>>>>
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]
>>>>>>>>
>>>>>>>> RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>> 'Secret123', '--setup-ca', '--ip-address', '192.168.200.103', '-r',
>>>>>>>> 'IPA.TEST']
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] RUN
>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
>>>>>>>> '--setup-ca', '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] IPA
>>>>>>>> client is
>>>>>>>> already configured on this system, ignoring the --domain, --server,
>>>>>>>> --realm, --hostname, --password and --keytab options.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Your
>>>>>>>> system
>>>>>>>> may be partly configured.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Run
>>>>>>>> /usr/sbin/ipa-server-install --uninstall to clean up.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR You must
>>>>>>>> provide a file generated by ipa-replica-prepare to create a replica
>>>>>>>> when the domain is at level 0.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>> ipa-replica-install command failed. See
>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Exit
>>>>>>>> code: 1
>>>>>>>> FAILED
>>>>>>>>
>>>>>>>>
>>>>>>>> 3)
>>>>>>>> ipatests.test_integration.test_replica_promotion.TestCAInstall
>>>>>>>>
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.ParamikoTransport]
>>>>>>>>
>>>>>>>> RUN ['ipa-replica-install', '-U', '-p', 'Secret123', '-w',
>>>>>>>> 'Secret123', '--setup-dns', '--forwarder', '10.34.78.1',
>>>>>>>> '--ip-address', '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] RUN
>>>>>>>> ['ipa-replica-install', '-U', '-p', 'Secret123', '-w', 'Secret123',
>>>>>>>> '--setup-dns', '--forwarder', '10.34.78.1', '--ip-address',
>>>>>>>> '192.168.200.103', '-r', 'IPA.TEST']
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] IPA
>>>>>>>> client is
>>>>>>>> already configured on this system, ignoring the --domain, --server,
>>>>>>>> --realm, --hostname, --password and --keytab options.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Your
>>>>>>>> system
>>>>>>>> may be partly configured.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Run
>>>>>>>> /usr/sbin/ipa-server-install --uninstall to clean up.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR You must
>>>>>>>> provide a file generated by ipa-replica-prepare to create a replica
>>>>>>>> when the domain is at level 0.
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22]
>>>>>>>> ipa.ipapython.install.cli.install_tool(Replica): ERROR The
>>>>>>>> ipa-replica-install command failed. See
>>>>>>>> /var/log/ipareplica-install.log for more information
>>>>>>>> [ipa.ipatests.test_integration.host.Host.replica2.cmd22] Exit
>>>>>>>> code: 1
>>>>>>>> FAILED
>>>>>>>>
>>>>>>>>
>>>>>>>> Reason:
>>>>>>>>
>>>>>>>> def install_replica(master, replica, setup_ca=True,
>>>>>>>> setup_dns=False,
>>>>>>>> - setup_kra=False, extra_args=()):
>>>>>>>> + setup_kra=False, extra_args=(),
>>>>>>>> domain_level=None):
>>>>>>>> + if domain_level is None:
>>>>>>>> + domain_level = domainlevel(master)
>>>>>> Oops. Sorry, must have been a result of inaccurate rebase. Fixed
>>>>>> The successful run is attached
>>>>>>
>>>>>>>> - if domainlevel(master) == DOMAIN_LEVEL_0:
>>>>>>>> + if domainlevel == DOMAIN_LEVEL_0:
>>>>>>>>
>>>>>>>> domain_level variable is actually unused
>>>>>>>>
>>>>>>> Also with your patch that removes host_prepare, I see many following
>>>>>>> errors in test debug output
>>>>>>>
>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd8] -bash:
>>>>>>> line 1:
>>>>>>> cd: /root/ipatests: No such file or directory
>>>>>>> [ipa.ipatests.test_integration.host.Host.replica1.cmd8] -bash:
>>>>>>> line 2:
>>>>>>> /root/ipatests/env.sh: No such file or directory
>>>>>>>
>>>>>>>
>>>>>>> I do not see these errors in current tests, so the patch does not
>>>>>>> seem
>>>>>>> right to me.
>>>>>> http://jenkins.idm.lab.eng.brq.redhat.com:8080/job/freeipa-integration-f23master-customized_ds_config_install-domlevel-1/23/consoleFull
>>>>>>
>>>>>>
>>>>>> I've been experimenting a lot to find get rid of it but was
>>>>>> unsuccessful. Seemingly it does not affect the tests.
>>>>>>
>>>>> Oleg it looks like you broke something
>>>>> in the test_integration/test_external_ca.py
>>>>>
>>>>> _______________________ TestExternalCA.test_external_ca
>>>>> ________________________
>>>>>
>>>>> self = <ipatests.test_integration.test_external_ca.TestExternalCA
>>>>> object at 0x7ff560a51750>
>>>>>
>>>>> def test_external_ca(self):
>>>>> # Step 1 of ipa-server-install
>>>>> self.master.run_command([
>>>>> 'ipa-server-install', '-U',
>>>>> '-a', self.master.config.admin_password,
>>>>> '-p', self.master.config.dirman_password,
>>>>> '--setup-dns', '--no-forwarders',
>>>>> '-n', self.master.domain.name,
>>>>> '-r', self.master.domain.realm,
>>>>> '--domain-level=%i' % self.master.config.domain_level,
>>>>> '--external-ca'
>>>>> ])
>>>>> nss_db = os.path.join(self.master.config.test_dir,
>>>>> 'testdb')
>>>>> external_cert_file = os.path.join(nss_db, 'ipa.crt')
>>>>> external_ca_file = os.path.join(nss_db, 'ca.crt')
>>>>> noisefile = os.path.join(self.master.config.test_dir,
>>>>> 'noise.txt')
>>>>> pwdfile = os.path.join(self.master.config.test_dir,
>>>>> 'pwdfile.txt')
>>>>> # Create noise and password files for NSS database
>>>>>> self.master.run_command('date | sha256sum > %s' % noisefile)
>>>>> test_integration/test_external_ca.py:49:
>>>>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>>>> _ _ _ _ _ _
>>>>> ../pytest_multihost/host.py:246: in run_command
>>>>> command.wait(raiseonerr=raiseonerr)
>>>>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>>>> _ _ _ _ _ _
>>>>>
>>>>> self = <pytest_multihost.transport.SSHCommand object at
>>>>> 0x7ff5609eb610>
>>>>> raiseonerr = True
>>>>>
>>>>> def wait(self, raiseonerr=True):
>>>>> """Wait for the remote process to exit
>>>>> Raises an excption if the exit code is not 0,
>>>>> unless raiseonerr is
>>>>> true.
>>>>> """
>>>>> if self._done:
>>>>> return self.returncode
>>>>> self._end_process()
>>>>> self._done = True
>>>>> if raiseonerr and self.returncode:
>>>>> self.log.error('Exit code: %s', self.returncode)
>>>>>> raise subprocess.CalledProcessError(self.returncode,
>>>>>> self.argv)
>>>>> E CalledProcessError: Command 'date | sha256sum >
>>>>> /root/ipatests/noise.txt' returned non-zero exit status 1
>>>>>
>>>>> ../pytest_multihost/transport.py:159: CalledProcessError
>>>>> ---------------------------- Captured stdout setup
>>>>> -----------------------------
>>>>> <ipatests.test_integration.config.Config object at 0x7ff560a51cd0>
>>>>> ========================== 1 failed in 32.25 seconds
>>>>> ===========================
>>>>>
>>>>>
>>>>>
>>>>> The command "date | sha256sum > /root/ipatests/noise.txt"
>>>>>
>>>>> The only explanation is that the directory /root/ipatests/ does not
>>>>> exist.
>>>> Obviously this is caused by my patch N 0025
>>>> freeipa-ofayans-0025-Removed-a-constantly-failing-call-to-prepare_host.patch
>>>>
>>>>
>>>> Without it replica_promotion tests would fail. The only compromise I
>>>> see
>>>> is to remove this line in unapply_fixes function in
>>>> ipatests/test_integration/tasks.py:
>>>>
>>>> host.run_command(['rm', '-rvf', host.config.test_dir])
>>>>
>>>> Now the biggest question is: why do we keep removing the config folder
>>>> anyway? All the files (backups of /etc/hosts, etc) that ever get there
>>>> get overwritten each time anyway so it's quite safe to just keep the
>>>> folder throughout the whole test execution.
>>>>
>>> I do not know why it was removed. IMHO it shoudl be enough
>>> to remove just content of this directory.
>>>
>>> But it will bee godd to fix it ASAP
>>> and unblock test test_integration/test_external_ca.py
>>> I hope we want to have green test in 4.3 branch.
>>> Please also open a ticket so it can be backported to
>>> stable branch (if needed)
>>>
>>> LS
>>>
>
> NACK
>
> 0)
> it is not refactoring, it is bugfixing because it is broken
>
> 1)
> Originally prepare hosts were called for every installation where tasks
> install_server, install_replica, install_client were used. Now hosts are
> prepared only in base install class, so all test cases where install
> classmethod is overriden will have no test directory prepared. Does have
> py.test any method that is called for each test class before install?
Well, we can revert my changes from patch-0025 that actually removed
prepare_host call from a mh fixture in
ipatests/pytest_plugins/integration.py
Then we probably do not need most of the changes from this patch except
the try-except section in prepare_host method itself.
>
> 2)
> I dont like new option in unapply fixes, it breaks symetry, if
> prepare_host is adding directory, then reverse operation should remove
> directory, IMO unapply_fixes is reverse operation for prepare_host and
> it should not have any option that changes behavior.
Agreed. Will remove it.
--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.
More information about the Freeipa-devel
mailing list