[Freeipa-devel] IPA Server upgrade 4.2 design

Tue Mar 3 08:36:13 UTC 2015

On 3.3.2015 09:33, Jan Cholasta wrote:
> Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
>> On 03/03/15 07:31, Jan Cholasta wrote:
>>> Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
>>>> On 02/03/15 13:12, Jan Cholasta wrote:
>>>>> Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
>>>>>> On 03/02/2015 07:49 AM, Jan Cholasta wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
>>>>>>>> Hello all,
>>>>>>>>
>>>>>>>> please read the design page, any objections/suggestions appreciated
>>>>>>>> http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
>>>>>>>>
>>>>>>>
>>>>>>> 1)
>>>>>>>
>>>>>>> "
>>>>>>> * Merge server update commands into the one command
>>>>>>> (ipa-server-upgrade)
>>>>>>> "
>>>>>>>
>>>>>>> So there is "ipa-server-install" to install the server,
>>>>>>> "ipa-server-install
>>>>>>> --uninstall" to uninstall the server and "ipa-server-upgrade" to
>>>>>>> upgrade the
>>>>>>> server. Maybe we should bring some consistency here and have one of:
>>>>>>>
>>>>>>>   a) "ipa-server-install [--install]", "ipa-server-install
>>>>>>> --uninstall",
>>>>>>> "ipa-server-install --upgrade"
>>>>>>>
>>>>>>>   b) "ipa-server-install [install]", "ipa-server-install uninstall",
>>>>>>> "ipa-server-install upgrade"
>>>>>>>
>>>>>>>   c) "ipa-server-install", "ipa-server-uninstall",
>>>>>>> "ipa-server-upgrade"
>>>>>>
>>>>>> Long term, I think we want C. Besides other advantages, it will let
>>>>>> us have
>>>>>> independent sets of options, based on what you want to do.
>>>>>>
>>>>>>> 2)
>>>>>>>
>>>>>>> "
>>>>>>> * Prevent to run IPA service, if code version and configuration
>>>>>>> version does
>>>>>>> not match
>>>>>>>    * ipactl should execute ipa-server-upgrade if needed
>>>>>>> "
>>>>>>>
>>>>>>> There should be no configuration version, configuration update
>>>>>>> should be run
>>>>>>> always. It's fast and hence does not need to be optimized like data
>>>>>>> update by
>>>>>>> using a monolithic version number, which brings more than a few
>>>>>>> problems on its
>>>>>>> own.
>>>>>>
>>>>>> I do not agree in this section. Why would you like to run it always,
>>>>>> even if it
>>>>>> was fast? No run is still faster than fast run.
>>>>>
>>>>> In the ideal case the installer would be idempotent and upgrade would
>>>>> be re-running the installer and we should aim to do just that. We kind
>>>>> of do that already, but there is a lot of code duplication in
>>>>> installers and ipa-upgradeconfig (I would like to fix that when
>>>>> refactoring installers). IMO it's better to always make 100% sure the
>>>>> configuration is correct rather than to save a second or two.
>>>> I doesn't like this idea, if user wants to fix something, the one should
>>>> use --skip-version-check option, and the IPA upgrade will be executed.
>>>
>>> Well, what I don't like is dealing with meaningless version numbers.
>>> They are causing us grief in API versioning and I don't see why it
>>> would be any different here.
>> However you must keep the version because of schema and data upgrade, so
>> why not to execute update as one batch instead of doing config upgrade
>> all the time, and then data upgrade only if required.
> 
> Because there is no exact mapping between version number and what features are
> actually available. A state file is tons better than a single version number.
> 
>>
>> Some configuration upgrades, like adding new DNS related services,
>> requires new schema, how we can handle this?
> 
> This does not sound right. Could you be more specific?
> 
>> Running schema upgrade every time?
>>>
>>>> What if a service changes in a way, the IPA configuration will not work?
>>>
>>> Then it's a bug and needs to be fixed, like any other bug. IIRC there
>>> was only one or two occurences of such bug in the past 3 years (I
>>> remember sshd_config), so I don't think you have a strong case here.
>> Ok
>>>
>>>> The user will need to change it manually, but after each restart,
>>>> upgrade will change the value back into IPA required configuration which
>>>> will not work.
>>>
>>> Says who? It's our code, we can do whatever we want, it doesn't have
>>> to be dumb like this.
>>>
>>>> Yes, we have upgrade state file, but then the comparing of one value is
>>>> faster then checking each state if was executed.
>>>
>>> How faster is that, like, few milliseconds? Are you seriously
>>> considering this the right optimization in a process that is
>>> magnitudes slower?
>> Ok the speed is not so important, but I still do not like the idea of
>> executing the code which is not needed to be executed, because I know
>> the version is the same as was before last restart, so nothing changed.
> 
> Weren't "clever" optimizations like this what got us into this whole
> refactoring bussiness in the first place?

I very much agree with Honza. We should always start with something
stupidly-simply and enhance it later, when it is clear if it is really necessary.

Do not over-engineer it from the very beginning.

Petr^2 Spacek

>>>> My personal opinion is, application should not try to fix itself every
>>>> restart.
>>>
>>> One could say that application should not try to upgrade itself every
>>> restart, but here we are, doing it anyway...
>> I want to do upgrade only if needed not every restart.
> 
> If there is nothing to upgrade, nothing will be upgraded. The effect is the same.
> 
>>>
>>>>>
>>>>>> In the past, I do not recall
>>>>>> ipa-upgradeconfig as being really fast, especially certmonger/Dogtag
>>>>>> related
>>>>>> updates were really slow thank to service restarts, etc.
>>>>>
>>>>> Correct, but I was talking about configuration file updates, not
>>>>> (re)starts, which have to always be done in ipactl anyway.
>>>>>
>>>>>>
>>>>>>> 3)
>>>>>>>
>>>>>>> "
>>>>>>> * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater
>>>>>>> "
>>>>>>>
>>>>>>> Even without arguments? Is ipactl now the only right place to
>>>>>>> trigger manual
>>>>>>> update?
>>>> Sorry, I will add more details there, if this is not clear.
>>>> ipa-upgrateconfig will be removed
>>>> ipa-ldap-updater will not be able to do overall update, you will need to
>>>> specify options and update file.
>>>>>>>
>>>>>>> 4)
>>>>>>>
>>>>>>> "
>>>>>>> Plugins are called from update files, using new directive
>>>>>>> update-plugin:<plugin-name>
>>>>>>> "
>>>>>>>
>>>>>>> Why "update-plugin" and not just "plugin"? Do you expect other kinds
>>>>>>> of plugins
>>>>>>> to be called from update files in the future? (I certainly don't.)
>>>>>>
>>>>>> I have no strong feelings on this one, but IMO it is always better to
>>>>>> have some
>>>>>> "plan B" if we choose to indeed implement some yet unforeseen plugin
>>>>>> based
>>>>>> capability...
>>>>>
>>>>> I doubt that will happen, but if it does, we can always add
>>>>> "plan-b-plugin" directive.
>>>> I do not insist on "update-plugin", I just wanted to be more specific
>>>> which type of plugin is expected there.
>>>
>>> Well, the names of the files end with .update and they are located in
>>> /usr/share/ipa/updates, I think that's enough hints as to what type of
>>> plugin is expected.
>>>
>> okay
>>>>>
>>>>>>
>>>>>>> 5)
>>>>>>>
>>>>>>> "
>>>>>>> New class UpdatePlugin is used for all update plugins.
>>>>>>> "
>>>>>>>
>>>>>>> Just reuse the existing Updater class, no need to reinvent the wheel.
>>>>>>>
>>>>>>> 6)
>>>>>>>
>>>>>>> I wonder why configuration update is done after data update and not
>>>>>>> before. I
>>>>>>> know it's been like that for a long time, but it seems kind of
>>>>>>> unnatural to me,
>>>>>>> especially now when schema update is separate from data update.
>>>>>>> (Rob?)
>>>> We need schema update first, but I haven't found any services which need
>>>> to have updated data (I might be wrong)
>>>>>>>
>>>>>>> 7)
>>>>>>>
>>>>>>> "
>>>>>>> keep --test option and fix the plugins which do not respect the
>>>>>>> option
>>>>>>> "
>>>>>>>
>>>>>>> Just a note, I believe this ticket is related:
>>>>>>> <https://fedorahosted.org/freeipa/ticket/3448>.
>>>>>>>
>>>>>>>
>>>>>>> Good work overall!
>>>>>>>
>>>>>>> Honza