[Freeipa-devel] Handling certificates in JSON/XML-RPC

[Adam Young]

On 10/14/2011 09:28 AM, John Dennis wrote:
> [ I had a private email exchange with Rob concerning ticket 1201, 
> we've had a long standing issue with how certificates are exchanged 
> because in LDAP they are binary values. I told Rob I had a proof of 
> concept working and Rob sent me a code snippet illustrating an earlier 
> attempt he had made to fix the problem that didn't pan out. I wrote 
> this reply as a response to Rob to explain my approach but it occurred 
> to me this is a design issue which should be in open discussion thus 
> this reply is to the devel list so others can review it and be in the 
> loop. Apologies if you missed some earlier context. ]
>
> -----------------------------------------------------------------
>
> Thanks for the code snippet. I think the problem you might have run 
> into is that you were doing the conversion at the point xmlrpc 
> parameters are marshaled, that doesn't cover the json case (or other 
> locations).
>
> The way I'm planning on tackling this is to do the pem conversion at 
> the moment it's read from LDAP. We already have a mechanism for that 
> and you helped point me towards it when you mentioned wehandle some 
> attributes specially based on it's LDAP syntax.
>
> We "decorate" the  find_entries() method in ldap2 with the 
> @decode_retval() decorator. What this does is examine the values 
> returned from the LDAP query and converts them to our preferred types. 
> I stumbled upon this while doing the DN work, I didn't know it 
> existed, it's not all that visible when reading the code this 
> conversion is occurring. It calls into something called an Encoder 
> object (in encoder.py) which is a parameterized mechanism setup in 
> ldap2. In summary what it does is use a table ( _SYNTAX_MAPPING) to 
> lookup the attributes syntax and as long as it's not in the table it 
> gets promoted to unicode, otherwise it's left as a str.
>
> With a few tweaks this mechanism could be a bit more flexible. Rather 
> than just using the existence of the attribute in the table as the 
> decision on whether to call a conversion routine on it (e.g. encoding 
> to unicode) the value in the table could be a function pointer to a 
> conversion routine. Thus the logic would become "if the attribute is 
> in the table, call the converter on the value, otherwise fallback to 
> the default conversion (which happens to be UTF-8 <-> unicode".
>
> In short what will happen is the whenever a 'usercertificate' is 
> returned from LDAP (DER format) it will be converted to PEM format as 
> a unicode object (i.e. indicating it's text data) in the result.
>
> I believe this is fine because most of the places we use certificates 
> internally already pass PEM formatted versions and call into 
> x509.normalize_certificate() which checks for PEM vs. DER encoding.
>
> This then completes another aspect of the "always use PEM formatting" 
> for certs code cleanup.
>
> The only thing left to do is modify the javascript code to stop 
> looking for usercerticate attributes as base64.
>
> The command line tools which save a cert to a file perform a binary to 
> PEM conversion at the point they write the file. At first blush you 
> might think that logic should be removed since now certs will be 
> returned in PEM format, hence no conversion is necessary. But I think 
> I'm going to leave that logic in place, here's why: The code calls 
> into x509.normalize_certificate() thus it can accept the cert either 
> as binary or PEM. The command line might be connected to a server 
> without the new modification, thus if we leave it as it currently is 
> the command line will work with either an old or new server response. 
> The javascipt code in the web UI is tied to the server so those 
> changes will occur in lockstep.
>

I wonder if we need to keep the Binary format for some use cases.  I 
know that there are cases where the PKI code needs to expose 
certificates to the browser as straight binary.  But that would not go 
through the XML or JSON RPCs.  Keep in mind how to do the conversion if 
it becomes necessary and we can talk then.