[Freeipa-devel] Designing better API compatibility

[Simo Sorce]

On Fri, 2015-03-20 at 14:13 +0100, Martin Kosek wrote:
> Hi guys,
> 
> I would like to resurrect the discussion we had during DevConf.cz time, about 
> API compatibility in the FreeIPA server.
> 
> So right now, we maintain the backward compatibility, old clients can talk to 
> newer servers. Forward compatibility is not maintained. Unfortunately, this is 
> not very helpful in real deployments, where the server will often be some 
> RHEL/CentOS system and the client may be the newest Fedora - with newer API 
> than the server. This is the toughest part we need to solve.
> 
> There 3 main areas we wanted to attack with respect to compatibility:
> 
> 1) API publishing and/or API browser
> This is mostly documentation/interactive browser to see the supported API of 
> the server. It should not be difficult, it would just consume the metadata 
> already generated by the server.
> 
> Ticket: https://fedorahosted.org/freeipa/ticket/3129
> 
> 2) Forward compatibility of the direct API consumers
> Until now, to keep newer clients working against older server, we are using the 
> following trick in the ipa-client-install:
> 
> https://git.fedorahosted.org/cgit/freeipa.git/tree/ipa-client/ipa-install/ipa-client-install#n1649
> 
> It mostly works, one just needs to know the minimal version that needs to be 
> supported. It would be more user friendly, however, if this check is done on 
> the server automatically, without user having to research it. This applies both 
> for ipalib python lib consumer and for direct JSON-RPC consumers.
> 
> Ticket: https://fedorahosted.org/freeipa/ticket/4739
> 
> 3) Forward compatibility of the "ipa" client tool
> There are different approaches how to fix this, the generally accepted idea was 
> to implement very thin client, which would download and cache metadata from the 
> server on the client and generate the CLI from it. We would only need to have 
> separate client only plugins, basically implementing interactive_promt_callback 
> from existing server side plugins.
> 
> Tickets: https://fedorahosted.org/freeipa/ticket/4739, 
> https://fedorahosted.org/freeipa/ticket/4768
> 
> 
> Now, question is what we can do in 4.2. I do not think we can manage to rewrite 
> "ipa" command in the thin client, but we should do at least some portion of 1) 
> and 3).
> 
> I could not decipher that from our Devconf.cz notes. To me, the simplest way of 
> fixing forward compatibility seems to be following steps (besides not making 
> API backwards incompatible - i.e. what we do already):
> 
> - keep sending API version from client to server
> - server should not refuse newer API versions
> - only raise error when an unknown option or unknown command is used

This is not sufficient, older 3.3 and 4.x servers can't be changed and
we MUST be compatible with those.
Basically the plan MUST work with already released servers, this is a
constraint that cannot be releaxed, please work within this limitations.

> When plugins change the behavior, they should check for client version and base 
> it's action on it (sort of the capabilities we already have). This is the 
> simple way, it would work well with the global API number and thin client also.

Long term we need to provide versioned APIs and it is better if the
client falls back to known good old APIs, because the server certainly
can't fall forward.

> So this is the simple version. Simo, Nathaniel (and others), I know you 
> proposed versions the commands themselves, but I am now not sure how exactly 
> you wanted to do it. What exactly would it mean for the typical extension of 
> our API - adding new parameter and how user command extensions should be 
> treated (command parameter added).

I'll let Nathaniel weight in on the technical side as he had better
ideas IIRC.

>From my POV the client needs to find out what calls the server can
support and use appropriate ones.
At the same time newer servers must support older call versions so that
older clients can still work.

Simo.