[Freeipa-devel] IPA Server upgrade 4.2 design

Tue Mar 3 09:55:57 UTC 2015

Dne 3.3.2015 v 09:55 Martin Basti napsal(a):
> On 03/03/15 09:33, Jan Cholasta wrote:
>> Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
>>> On 03/03/15 07:31, Jan Cholasta wrote:
>>>> Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
>>>>> On 02/03/15 13:12, Jan Cholasta wrote:
>>>>>> Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
>>>>>>> On 03/02/2015 07:49 AM, Jan Cholasta wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
>>>>>>>>> Hello all,
>>>>>>>>>
>>>>>>>>> please read the design page, any objections/suggestions
>>>>>>>>> appreciated
>>>>>>>>> http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
>>>>>>>>>
>>>>>>>>
>>>>>>>> 1)
>>>>>>>>
>>>>>>>> "
>>>>>>>> * Merge server update commands into the one command
>>>>>>>> (ipa-server-upgrade)
>>>>>>>> "
>>>>>>>>
>>>>>>>> So there is "ipa-server-install" to install the server,
>>>>>>>> "ipa-server-install
>>>>>>>> --uninstall" to uninstall the server and "ipa-server-upgrade" to
>>>>>>>> upgrade the
>>>>>>>> server. Maybe we should bring some consistency here and have one
>>>>>>>> of:
>>>>>>>>
>>>>>>>>   a) "ipa-server-install [--install]", "ipa-server-install
>>>>>>>> --uninstall",
>>>>>>>> "ipa-server-install --upgrade"
>>>>>>>>
>>>>>>>>   b) "ipa-server-install [install]", "ipa-server-install
>>>>>>>> uninstall",
>>>>>>>> "ipa-server-install upgrade"
>>>>>>>>
>>>>>>>>   c) "ipa-server-install", "ipa-server-uninstall",
>>>>>>>> "ipa-server-upgrade"
>>>>>>>
>>>>>>> Long term, I think we want C. Besides other advantages, it will let
>>>>>>> us have
>>>>>>> independent sets of options, based on what you want to do.
>>>>>>>
>>>>>>>> 2)
>>>>>>>>
>>>>>>>> "
>>>>>>>> * Prevent to run IPA service, if code version and configuration
>>>>>>>> version does
>>>>>>>> not match
>>>>>>>>    * ipactl should execute ipa-server-upgrade if needed
>>>>>>>> "
>>>>>>>>
>>>>>>>> There should be no configuration version, configuration update
>>>>>>>> should be run
>>>>>>>> always. It's fast and hence does not need to be optimized like data
>>>>>>>> update by
>>>>>>>> using a monolithic version number, which brings more than a few
>>>>>>>> problems on its
>>>>>>>> own.
>>>>>>>
>>>>>>> I do not agree in this section. Why would you like to run it always,
>>>>>>> even if it
>>>>>>> was fast? No run is still faster than fast run.
>>>>>>
>>>>>> In the ideal case the installer would be idempotent and upgrade would
>>>>>> be re-running the installer and we should aim to do just that. We
>>>>>> kind
>>>>>> of do that already, but there is a lot of code duplication in
>>>>>> installers and ipa-upgradeconfig (I would like to fix that when
>>>>>> refactoring installers). IMO it's better to always make 100% sure the
>>>>>> configuration is correct rather than to save a second or two.
>>>>> I doesn't like this idea, if user wants to fix something, the one
>>>>> should
>>>>> use --skip-version-check option, and the IPA upgrade will be executed.
>>>>
>>>> Well, what I don't like is dealing with meaningless version numbers.
>>>> They are causing us grief in API versioning and I don't see why it
>>>> would be any different here.
>>> However you must keep the version because of schema and data upgrade, so
>>> why not to execute update as one batch instead of doing config upgrade
>>> all the time, and then data upgrade only if required.
>>
>> Because there is no exact mapping between version number and what
>> features are actually available. A state file is tons better than a
>> single version number.
>>
>>>
>>> Some configuration upgrades, like adding new DNS related services,
>>> requires new schema, how we can handle this?
>>
>> This does not sound right. Could you be more specific?
> at least ipa-dnskeysyncd service, requires updated schema for keys
> metadata.
> This service is mandratory for DNS, so it is newly configured during
> upgrade.
> Now it works because schema update is the first, so during configuration
> upgrade we have actual schema.

Right, but what's your point? We are not discussing order of updates 
here, I'm perfectly fine with schema updates being done before 
configuration updates.

>>
>>> Running schema upgrade every time?
>>>>
>>>>> What if a service changes in a way, the IPA configuration will not
>>>>> work?
>>>>
>>>> Then it's a bug and needs to be fixed, like any other bug. IIRC there
>>>> was only one or two occurences of such bug in the past 3 years (I
>>>> remember sshd_config), so I don't think you have a strong case here.
>>> Ok
>>>>
>>>>> The user will need to change it manually, but after each restart,
>>>>> upgrade will change the value back into IPA required configuration
>>>>> which
>>>>> will not work.
>>>>
>>>> Says who? It's our code, we can do whatever we want, it doesn't have
>>>> to be dumb like this.
>>>>
>>>>> Yes, we have upgrade state file, but then the comparing of one
>>>>> value is
>>>>> faster then checking each state if was executed.
>>>>
>>>> How faster is that, like, few milliseconds? Are you seriously
>>>> considering this the right optimization in a process that is
>>>> magnitudes slower?
>>> Ok the speed is not so important, but I still do not like the idea of
>>> executing the code which is not needed to be executed, because I know
>>> the version is the same as was before last restart, so nothing changed.
>>
>> Weren't "clever" optimizations like this what got us into this whole
>> refactoring bussiness in the first place?
> The "clever" optimizations worked in past, but IPA grown and now
> contains constraints/requirements which nobody expected.

Yes, then why do we need another one, especially so when it does not 
provide any significant speed-up?

> What if we
> will need some update which needs to execute time-consuming system check
> during every upgrade in future?

Then we deal with the optimization in the future, instead of doing it 
prematurely now.

> User can always run the upgrade manually, with --skip-version-check, and
> then configuration plugins will decide if the upgrade is needed.
>
>>
>>>>
>>>>>
>>>>> My personal opinion is, application should not try to fix itself every
>>>>> restart.
>>>>
>>>> One could say that application should not try to upgrade itself every
>>>> restart, but here we are, doing it anyway...
>>> I want to do upgrade only if needed not every restart.
>>
>> If there is nothing to upgrade, nothing will be upgraded. The effect
>> is the same.
>>
>>>>
>>>>>>
>>>>>>> In the past, I do not recall
>>>>>>> ipa-upgradeconfig as being really fast, especially certmonger/Dogtag
>>>>>>> related
>>>>>>> updates were really slow thank to service restarts, etc.
>>>>>>
>>>>>> Correct, but I was talking about configuration file updates, not
>>>>>> (re)starts, which have to always be done in ipactl anyway.
>>>>>>
>>>>>>>
>>>>>>>> 3)
>>>>>>>>
>>>>>>>> "
>>>>>>>> * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater
>>>>>>>> "
>>>>>>>>
>>>>>>>> Even without arguments? Is ipactl now the only right place to
>>>>>>>> trigger manual
>>>>>>>> update?
>>>>> Sorry, I will add more details there, if this is not clear.
>>>>> ipa-upgrateconfig will be removed
>>>>> ipa-ldap-updater will not be able to do overall update, you will
>>>>> need to
>>>>> specify options and update file.
>>>>>>>>
>>>>>>>> 4)
>>>>>>>>
>>>>>>>> "
>>>>>>>> Plugins are called from update files, using new directive
>>>>>>>> update-plugin:<plugin-name>
>>>>>>>> "
>>>>>>>>
>>>>>>>> Why "update-plugin" and not just "plugin"? Do you expect other
>>>>>>>> kinds
>>>>>>>> of plugins
>>>>>>>> to be called from update files in the future? (I certainly don't.)
>>>>>>>
>>>>>>> I have no strong feelings on this one, but IMO it is always
>>>>>>> better to
>>>>>>> have some
>>>>>>> "plan B" if we choose to indeed implement some yet unforeseen plugin
>>>>>>> based
>>>>>>> capability...
>>>>>>
>>>>>> I doubt that will happen, but if it does, we can always add
>>>>>> "plan-b-plugin" directive.
>>>>> I do not insist on "update-plugin", I just wanted to be more specific
>>>>> which type of plugin is expected there.
>>>>
>>>> Well, the names of the files end with .update and they are located in
>>>> /usr/share/ipa/updates, I think that's enough hints as to what type of
>>>> plugin is expected.
>>>>
>>> okay
>>>>>>
>>>>>>>
>>>>>>>> 5)
>>>>>>>>
>>>>>>>> "
>>>>>>>> New class UpdatePlugin is used for all update plugins.
>>>>>>>> "
>>>>>>>>
>>>>>>>> Just reuse the existing Updater class, no need to reinvent the
>>>>>>>> wheel.
>>>>>>>>
>>>>>>>> 6)
>>>>>>>>
>>>>>>>> I wonder why configuration update is done after data update and not
>>>>>>>> before. I
>>>>>>>> know it's been like that for a long time, but it seems kind of
>>>>>>>> unnatural to me,
>>>>>>>> especially now when schema update is separate from data update.
>>>>>>>> (Rob?)
>>>>> We need schema update first, but I haven't found any services which
>>>>> need
>>>>> to have updated data (I might be wrong)
>>>>>>>>
>>>>>>>> 7)
>>>>>>>>
>>>>>>>> "
>>>>>>>> keep --test option and fix the plugins which do not respect the
>>>>>>>> option
>>>>>>>> "
>>>>>>>>
>>>>>>>> Just a note, I believe this ticket is related:
>>>>>>>> <https://fedorahosted.org/freeipa/ticket/3448>.
>>>>>>>>
>>>>>>>>
>>>>>>>> Good work overall!
>>>>>>>>
>>>>>>>> Honza
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>

-- 
Jan Cholasta