[Freeipa-devel] IPA Server upgrade 4.2 design

[Martin Basti]

On 03/03/15 16:47, Martin Kosek wrote:
> On 03/03/2015 04:43 PM, Martin Basti wrote:
>> On 03/03/15 16:11, Simo Sorce wrote:
>>> On Tue, 2015-03-03 at 10:04 -0500, Rob Crittenden wrote:
>>>> Martin Basti wrote:
>>>>> On 03/03/15 15:33, Martin Kosek wrote:
>>>>>> On 03/03/2015 03:16 PM, Simo Sorce wrote:
>>>>>>> On Mon, 2015-03-02 at 18:54 +0100, Martin Basti wrote:
>>>>>>>> On 02/03/15 18:28, Martin Kosek wrote:
>>>>>>>>> On 03/02/2015 06:12 PM, Martin Basti wrote:
>>>>>>>>>> On 02/03/15 15:43, Rob Crittenden wrote:
>>>>>>>>>>> Martin Basti wrote:
>>>>>>>>> ...
>>>>>>>>>>> But you haven't explained any case why LDAPI would fail. If LDAPI
>>>>>>>>>>> fails
>>>>>>>>>>> then you've got more serious problems that I'm not sure binding
>>>>>>>>>>> as DM is
>>>>>>>>>>> going to solve.
>>>>>>>>>>>
>>>>>>>>>>> The only case where DM would be handy IMHO is either some worst case
>>>>>>>>>>> scenario upgrade where 389-ds is up but not binding to LDAPI or
>>>>>>>>>>> if you
>>>>>>>>>>> want to allow running LDAP updates as non-root.
>>>>>>>>>> I don't know cases when LDAPI would failed, except the case LDAPI is
>>>>>>>>>> misconfigured by user, or disabled by user.
>>>>>>>>> Wasn't LDAPI needed for the DM password less upgrade so that
>>>>>>>>> upgrader could
>>>>>>>>> simply bind as root with EXTERNAL auth?
>>>>>>>> We can do upgrade in both way, using LDAPI or using DM password,
>>>>>>>> preferred is LDAPI.
>>>>>>>> Question is, what is the use case for using DM password instead of
>>>>>>>> LDAPI
>>>>>>>> during upgrade.
>>>>>>> There is no use case for using the DM password.
>>>>>> +1, so we will only use LDAPI and ditch DM password options and
>>>>>> querying that
>>>>>> we now have with ipa-ldap-updater?
>>>>>>
>>>>>>>>>> It is not big effort to keep both DM binding and LDAPI in code.  A
>>>>>>>>>> user can
>>>>>>>>>> always found som unexpected use case for LDAP update with DM
>>>>>>>>>> password.
>>>>>>>>>>
>>>>>>>>>>>>> On ipactl, would it be overkill if there is a tty to prompt the
>>>>>>>>>>>>> user to
>>>>>>>>>>>>> upgrade? In a non-container world it might be surprising to
>>>>>>>>>>>>> have an
>>>>>>>>>>>>> upgrade happen esp since upgrades take a while.
>>>>>>>>>>>> In non-container enviroment, we can still use upgrade during RPM
>>>>>>>>>>>> transaction.
>>>>>>>>>>>>
>>>>>>>>>>>> So you suggest  not to do update automaticaly, just write Error
>>>>>>>>>>>> the IPA
>>>>>>>>>>>> upgrade is required?
>>>>>>>>>>> People do all sorts of strange things. Installing the packages with
>>>>>>>>>>> --no-script isn't in the range of impossible. A prompt, and I'm not
>>>>>>>>>>> saying it's a great idea, is 2 lines of code.
>>>>>>>>>>>
>>>>>>>>>>> I guess it just makes me nervous.
>>>>>>>>>> So lets summarize this:
>>>>>>>>>> * DO upgrade if possible during RPM transaction
>>>>>>>>> Umm, I thought we want to get rid of running upgrade during RPM
>>>>>>>>> transaction. It
>>>>>>>>> is extremely difficult to debug upgrade stuck during RPM
>>>>>>>>> transaction, it also
>>>>>>>>> makes RPM upgrade run longer than needed. It also makes admins
>>>>>>>>> nervous when
>>>>>>>>> their rpm upgrade is suddenly waiting right before the end. I even
>>>>>>>>> see the
>>>>>>>>> fingers slowly reaching to CTRL+C combo... (You can see the
>>>>>>>>> consequences)
>>>>>>>> People are used to have IPA upgraded and ready after RPM upgrade.
>>>>>>>> They may be shocked if IPA services will be in shutdown state after RPM
>>>>>>>> transaction.
>>>>>>> This is true, stopping IPA and requiring manual intervention is not ok.
>>>>>> What is the plan then? Keep upgrades done during RPM transaction? Note
>>>>>> that RPM
>>>>>> transaction was already stuck several times because IPA, or rather DS,
>>>>>> deadlocked.
>>>>>>
>>>>>> We also need to address https://fedorahosted.org/freeipa/ticket/3849. The
>>>>>> original plan was to do the upgrade during ipactl start, this would
>>>>>> fix this
>>>>>> ticket. Alternatively, should we remove the upgrade from RPM scriptlet
>>>>>> and only
>>>>>> call asynchronous "systemctl restart ipa.service" that would trigger the
>>>>>> upgrade in separate process and log results in ipa.service?
>>>>>>
>>>>>> Martin
>>>>> The plan is do upgrade during RPM transaction if possible. If not then
>>>>> ipactl start, will show warning for user to do manual upgrade (Rob
>>>>> wanted it in this way, not doing auto upgrade by ipactl)
>>>> Only if there is a tty which means no asking during package update,
>>>> which I thought was the idea. I just think it is rather unexpected to
>>>> update a package during a restart.
>>>>
>>>>> So the fedup case is: RPM upgrade failed, ipactl start will detect
>>>>> version mismatch, show error and prompt user to run ipa-server-upgrade
>>>> I'm beginning to have my own doubts about version, recognizing that
>>>> there isn't exactly another obvious solution. Running the updates every
>>>> time ipactl is run isn't great. The updates are not fast by any stretch,
>>>> 29s on one VM, and we need to log whenever an update is done. My
>>>> ipaupgrade log is 48M from 20 updates. How many times does one run
>>>> ipactl restart when diagnosing a problem?
>>>>
>>>> My biggest concern with version is who keeps count and where? This is
>>>> particularly problematic in packaged servers where changes are made
>>>> without rebasing (Fedora and RHEL). Somewhere the version would need to
>>>> be bumped with each release? Or only when updates are added? Or only
>>>> when someone remembers? It just seems fragile and prone to human error
>>>> unless you have some automatic version incrementor that takes this into
>>>> consideration.
>>>>
>>>> If fallible version or slow updates are the only option then I'd have to
>>>> go with slow updates if only to avoid a lot of support issues. And I
>>>> really hate the idea of updates during service restart.
>> I do not like idea of auto upgrading during (re)start as well, it is not
>> expected by user.
>> At least, upgrade takes 5-10 minutes, user may not to know the upgrade is
>> happening.
>>
>> We have 3 use cases:
>> * We can run update during RPM transaction --> just run upgrade as IPA do now
> Ok. We can add more better logging or text output, as Petr2 suggested.
>
>> * We can not run update during transaction(fedup, --no-script) --> user should
>> run ipa-server-upgrade
>> * Containers --> user should run ipa-server-upgrade
> Right. And error out in ipactl in these cases, so that user does not run
> container/VM with data/bits version mismatch.
>
> If we are all set, please update the design.
>
> Martin
I can't update design yet, this depends, if we will use just RPM vendor 
versions, or we will use Honza's proposal with update configuration 
every restart using sysupgrade.state (except LDAP data, there we still 
need version), we need to choose first.

-- 
Martin Basti