[Freeipa-devel] [IPAQE][REVIEW-REQUEST][TEST PLAN] Installation tests

[Martin Basti]

On 02.12.2015 09:52, Oleg Fayans wrote:
> Hi all,
>
> I've updated the test plan according to your feedback. The tests for 
> 'ca' suffix would be added to Topology Plugin testplan during this week.
>
>
>
> On 11/27/2015 04:37 PM, Martin Basti wrote:
>>
>>
>> On 27.11.2015 15:05, Martin Basti wrote:
>>>
>>>
>>> On 26.11.2015 14:39, Petr Vobornik wrote:
>>>> On 11/23/2015 06:51 PM, Oleg Fayans wrote:
>>>>> Hi all,
>>>>>
>>>>> Here is a draft of the Replica Promotion test plan
>>>>> http://www.freeipa.org/page/V4/Replica_Promotion/Test_plan
>>>>>
>>>>>
>>>>
>>>> == Test case: Unprivileged users are not allowed to enroll and
>>>> promote clients ==
>>>> User credentials are passed there through -p $principal and -w
>>>> $password options. It is correct atm because it is required for
>>>> connection check. But end goal of replica promotion is the avoid it.
>>>> See https://fedorahosted.org/freeipa/ticket/5497 and
>>>> https://fedorahosted.org/freeipa/ticket/5498 for more information.
>>>>
>>>> == Missing test cases ==
>>>> 1. ipa-replica-install works on CA-less master with with both domain
>>>> levels
>>>> 2. ipa-server-install works with --setup-dns option with both domain
>>>> levels
>>>> 3. ipa-server-install works with externally signed CA cert with both
>>>> domain levels
>>>> 4. ipa-replica-install with options(and their combination):
>>>> --setup-ca --setup-dns --setup-kra works with both domain levels
>>>>
>>>> Note: Not sure if #2 and #3 belongs here, but should be tested. Maybe
>>>> tests for domain level 0 already exist.
>>>
>>> Many of them are already part of some tests.
>>>
>>> I wanted to crate install tests in one place with all combination of
>>> options, but I did not have time for it yet.
>>> I have a proof of concept test plan I can share it if you want.
>>>
>>> Martin
>>>
>> Proof of concept test plan:
>>
>> Master:
>> * ipa-server-install
>> * ipa-server-install --setup-dns
>> * caless ipa-server-install
>> * ipa-server-install && ipa-dns-install
>> * ipa-server-install && ipa-dns-install --dnssec-master
>> * caless ipa-server-install && ipa-dns-install
>> * caless ipa-server-install && ipa-ca-install
>> * caless ipa-server-install && ipa-ca-install && ipa-kra-install
>> * ipa-server-install && ipa-kra-install
>>
>> Replica:
>> ipa-replica-install
>> ipa-replica-install --setup-dns
>> ipa-replica-install --setup-ca
>> ipa-replica-install --setup-dns --setup-ca
>> ipa-replica-install --setup-ca --setup-kra
>> ipa-replica-install --setup-dns --setup-ca --setup-kra
>> ipa-replica-install && ipa-dns-install
>> ipa-replica-install && ipa-dns-install --dnssec-master
>> ipa-replica-install && ipa-ca-install
>> ipa-replica-install && ipa-ca-install && ipa-kra-install
>> ipa-replica-install && ipa-dns-install && ipa-ca-install && 
>> ipa-kra-install
>> caless ipa-replica-install && ipa-ca-install
>> caless ipa-replica-install --setup-dns
>> caless ipa-replica-install && ipa-dns-install
>>
>> These are basic tests, other specialized test like "install KRA first on
>> replica then on master" should be in separate test suite IMO
>>
>> I'm not sure if caless install test should be covered in basic install
>> tests.
>> Same for DNSSEC master (this is fuly covered in dnssec tests)
>>
>
Hello,
as I wrote above, the install tests should be in *separated* test plan, 
and should cover only ability to be installed, not any other functional 
tests, we have functional tests separated (vault, dnssec, ...)

Also is need to cover multiple variation of options, not just install it 
with all options (--setup-dns, --setup-ca, --setup-kra)

Your:
Test case: ipa-server-install works with all supported options under 
domain level 0
Test case: ipa-server-install works with all supported options under 
domain level 1

Should not be part of replica promotion testing.

Also I do not think that you should test DNSSEC in these tests, DNSSEC 
test are almost fully covered in DNSSEC integration test, it is just 
enough to run it on both domain levels.

If you disagree to have separate "Installation test suite" please write 
the reason why it should be a part of replica pro motion test plan.

Test case: ipa-ca install and ipa-kra-install work on master and replica 
under domain level 0
In this test, to be able install CA on master server, master has to be 
installed as CA-less. Shouldn't be there: expecting failure as result?

In case we will create basic install test we may exclude testing 
ipa-ca-install and ipa-kra-install from this testsuite.
Also I do not see reason why DNSSEC should be tested there.

Shouldn't be ipa-kra-install and uninstall a separate test case?

Martin