[Freeipa-devel] IPA Server upgrade 4.2 design

Tue Mar 3 08:55:52 UTC 2015

On 03/03/15 09:33, Jan Cholasta wrote:
> Dne 3.3.2015 v 09:06 Martin Basti napsal(a):
>> On 03/03/15 07:31, Jan Cholasta wrote:
>>> Dne 2.3.2015 v 13:51 Martin Basti napsal(a):
>>>> On 02/03/15 13:12, Jan Cholasta wrote:
>>>>> Dne 2.3.2015 v 12:23 Martin Kosek napsal(a):
>>>>>> On 03/02/2015 07:49 AM, Jan Cholasta wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Dne 24.2.2015 v 19:10 Martin Basti napsal(a):
>>>>>>>> Hello all,
>>>>>>>>
>>>>>>>> please read the design page, any objections/suggestions 
>>>>>>>> appreciated
>>>>>>>> http://www.freeipa.org/page/V4/Server_Upgrade_Refactoring
>>>>>>>>
>>>>>>>
>>>>>>> 1)
>>>>>>>
>>>>>>> "
>>>>>>> * Merge server update commands into the one command
>>>>>>> (ipa-server-upgrade)
>>>>>>> "
>>>>>>>
>>>>>>> So there is "ipa-server-install" to install the server,
>>>>>>> "ipa-server-install
>>>>>>> --uninstall" to uninstall the server and "ipa-server-upgrade" to
>>>>>>> upgrade the
>>>>>>> server. Maybe we should bring some consistency here and have one 
>>>>>>> of:
>>>>>>>
>>>>>>>   a) "ipa-server-install [--install]", "ipa-server-install
>>>>>>> --uninstall",
>>>>>>> "ipa-server-install --upgrade"
>>>>>>>
>>>>>>>   b) "ipa-server-install [install]", "ipa-server-install 
>>>>>>> uninstall",
>>>>>>> "ipa-server-install upgrade"
>>>>>>>
>>>>>>>   c) "ipa-server-install", "ipa-server-uninstall",
>>>>>>> "ipa-server-upgrade"
>>>>>>
>>>>>> Long term, I think we want C. Besides other advantages, it will let
>>>>>> us have
>>>>>> independent sets of options, based on what you want to do.
>>>>>>
>>>>>>> 2)
>>>>>>>
>>>>>>> "
>>>>>>> * Prevent to run IPA service, if code version and configuration
>>>>>>> version does
>>>>>>> not match
>>>>>>>    * ipactl should execute ipa-server-upgrade if needed
>>>>>>> "
>>>>>>>
>>>>>>> There should be no configuration version, configuration update
>>>>>>> should be run
>>>>>>> always. It's fast and hence does not need to be optimized like data
>>>>>>> update by
>>>>>>> using a monolithic version number, which brings more than a few
>>>>>>> problems on its
>>>>>>> own.
>>>>>>
>>>>>> I do not agree in this section. Why would you like to run it always,
>>>>>> even if it
>>>>>> was fast? No run is still faster than fast run.
>>>>>
>>>>> In the ideal case the installer would be idempotent and upgrade would
>>>>> be re-running the installer and we should aim to do just that. We 
>>>>> kind
>>>>> of do that already, but there is a lot of code duplication in
>>>>> installers and ipa-upgradeconfig (I would like to fix that when
>>>>> refactoring installers). IMO it's better to always make 100% sure the
>>>>> configuration is correct rather than to save a second or two.
>>>> I doesn't like this idea, if user wants to fix something, the one 
>>>> should
>>>> use --skip-version-check option, and the IPA upgrade will be executed.
>>>
>>> Well, what I don't like is dealing with meaningless version numbers.
>>> They are causing us grief in API versioning and I don't see why it
>>> would be any different here.
>> However you must keep the version because of schema and data upgrade, so
>> why not to execute update as one batch instead of doing config upgrade
>> all the time, and then data upgrade only if required.
>
> Because there is no exact mapping between version number and what 
> features are actually available. A state file is tons better than a 
> single version number.
>
>>
>> Some configuration upgrades, like adding new DNS related services,
>> requires new schema, how we can handle this?
>
> This does not sound right. Could you be more specific?
at least ipa-dnskeysyncd service, requires updated schema for keys metadata.
This service is mandratory for DNS, so it is newly configured during 
upgrade.
Now it works because schema update is the first, so during configuration 
upgrade we have actual schema.
>
>> Running schema upgrade every time?
>>>
>>>> What if a service changes in a way, the IPA configuration will not 
>>>> work?
>>>
>>> Then it's a bug and needs to be fixed, like any other bug. IIRC there
>>> was only one or two occurences of such bug in the past 3 years (I
>>> remember sshd_config), so I don't think you have a strong case here.
>> Ok
>>>
>>>> The user will need to change it manually, but after each restart,
>>>> upgrade will change the value back into IPA required configuration 
>>>> which
>>>> will not work.
>>>
>>> Says who? It's our code, we can do whatever we want, it doesn't have
>>> to be dumb like this.
>>>
>>>> Yes, we have upgrade state file, but then the comparing of one 
>>>> value is
>>>> faster then checking each state if was executed.
>>>
>>> How faster is that, like, few milliseconds? Are you seriously
>>> considering this the right optimization in a process that is
>>> magnitudes slower?
>> Ok the speed is not so important, but I still do not like the idea of
>> executing the code which is not needed to be executed, because I know
>> the version is the same as was before last restart, so nothing changed.
>
> Weren't "clever" optimizations like this what got us into this whole 
> refactoring bussiness in the first place?
The "clever" optimizations worked in past, but IPA grown and now 
contains constraints/requirements which nobody expected.  What if we 
will need some update which needs to execute time-consuming system check 
during every upgrade in future?
User can always run the upgrade manually, with --skip-version-check, and 
then configuration plugins will decide if the upgrade is needed.

>
>>>
>>>>
>>>> My personal opinion is, application should not try to fix itself every
>>>> restart.
>>>
>>> One could say that application should not try to upgrade itself every
>>> restart, but here we are, doing it anyway...
>> I want to do upgrade only if needed not every restart.
>
> If there is nothing to upgrade, nothing will be upgraded. The effect 
> is the same.
>
>>>
>>>>>
>>>>>> In the past, I do not recall
>>>>>> ipa-upgradeconfig as being really fast, especially certmonger/Dogtag
>>>>>> related
>>>>>> updates were really slow thank to service restarts, etc.
>>>>>
>>>>> Correct, but I was talking about configuration file updates, not
>>>>> (re)starts, which have to always be done in ipactl anyway.
>>>>>
>>>>>>
>>>>>>> 3)
>>>>>>>
>>>>>>> "
>>>>>>> * Prevent user to use ipa-upgradeconfig and ipa-ldap-updater
>>>>>>> "
>>>>>>>
>>>>>>> Even without arguments? Is ipactl now the only right place to
>>>>>>> trigger manual
>>>>>>> update?
>>>> Sorry, I will add more details there, if this is not clear.
>>>> ipa-upgrateconfig will be removed
>>>> ipa-ldap-updater will not be able to do overall update, you will 
>>>> need to
>>>> specify options and update file.
>>>>>>>
>>>>>>> 4)
>>>>>>>
>>>>>>> "
>>>>>>> Plugins are called from update files, using new directive
>>>>>>> update-plugin:<plugin-name>
>>>>>>> "
>>>>>>>
>>>>>>> Why "update-plugin" and not just "plugin"? Do you expect other 
>>>>>>> kinds
>>>>>>> of plugins
>>>>>>> to be called from update files in the future? (I certainly don't.)
>>>>>>
>>>>>> I have no strong feelings on this one, but IMO it is always 
>>>>>> better to
>>>>>> have some
>>>>>> "plan B" if we choose to indeed implement some yet unforeseen plugin
>>>>>> based
>>>>>> capability...
>>>>>
>>>>> I doubt that will happen, but if it does, we can always add
>>>>> "plan-b-plugin" directive.
>>>> I do not insist on "update-plugin", I just wanted to be more specific
>>>> which type of plugin is expected there.
>>>
>>> Well, the names of the files end with .update and they are located in
>>> /usr/share/ipa/updates, I think that's enough hints as to what type of
>>> plugin is expected.
>>>
>> okay
>>>>>
>>>>>>
>>>>>>> 5)
>>>>>>>
>>>>>>> "
>>>>>>> New class UpdatePlugin is used for all update plugins.
>>>>>>> "
>>>>>>>
>>>>>>> Just reuse the existing Updater class, no need to reinvent the 
>>>>>>> wheel.
>>>>>>>
>>>>>>> 6)
>>>>>>>
>>>>>>> I wonder why configuration update is done after data update and not
>>>>>>> before. I
>>>>>>> know it's been like that for a long time, but it seems kind of
>>>>>>> unnatural to me,
>>>>>>> especially now when schema update is separate from data update.
>>>>>>> (Rob?)
>>>> We need schema update first, but I haven't found any services which 
>>>> need
>>>> to have updated data (I might be wrong)
>>>>>>>
>>>>>>> 7)
>>>>>>>
>>>>>>> "
>>>>>>> keep --test option and fix the plugins which do not respect the
>>>>>>> option
>>>>>>> "
>>>>>>>
>>>>>>> Just a note, I believe this ticket is related:
>>>>>>> <https://fedorahosted.org/freeipa/ticket/3448>.
>>>>>>>
>>>>>>>
>>>>>>> Good work overall!
>>>>>>>
>>>>>>> Honza
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


-- 
Martin Basti