[Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 10/20/2016 03:52 PM, Ben Lipton wrote:
> On 10/17/2016 02:16 AM, Jan Cholasta wrote:
>> On 13.10.2016 17:23, Ben Lipton wrote:
>>> Thank you, this was a really helpful clarification of your point.
>>> Comments below. Once again, I'm sorry I missed the email for so long.
>>>
>>> Ben
>>>
>>> On 09/05/2016 06:52 AM, Jan Cholasta wrote:
>>>> On 27.8.2016 22:40, Ben Lipton wrote:
>>>>> On 08/25/2016 04:11 PM, Rob Crittenden wrote:
>>>>>> Ben Lipton wrote:
>>>>>>> On 08/23/2016 03:54 AM, Jan Cholasta wrote:
>>>>>>>> On 8.8.2016 22:23, Ben Lipton wrote:
>>>>>>>>> On 07/25/2016 07:45 AM, Jan Cholasta wrote:
>>>>>>>>>> On 25.7.2016 13:11, Alexander Bokovoy wrote:
>>>>>>>>>>> On Mon, 25 Jul 2016, Jan Cholasta wrote:
>>>>>>>>>>>> On 20.7.2016 16:05, Ben Lipton wrote:
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks very much for the feedback! Some responses below; I 
>>>>>>>>>>>>> hope
>>>>>>>>>>>>> you'll
>>>>>>>>>>>>> let me know what you think of my reasoning.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 07/20/2016 04:20 AM, Jan Cholasta wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 17.6.2016 00:06, Ben Lipton wrote:
>>>>>>>>>>>>>>> On 06/14/2016 08:27 AM, Ben Lipton wrote:
>>>>>>>>>>>>>>>> Hello all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have written up a design proposal for making certificate
>>>>>>>>>>>>>>>> requests
>>>>>>>>>>>>>>>> easier to generate when using alternate certificate 
>>>>>>>>>>>>>>>> profiles:
>>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The use case for this is described in
>>>>>>>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/4899. I will be
>>>>>>>>>>>>>>>> working on
>>>>>>>>>>>>>>>> implementing this design over the next couple of months.
>>>>>>>>>>>>>>>> If you
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> the time and interest, please take a look and share any
>>>>>>>>>>>>>>>> comments or
>>>>>>>>>>>>>>>> concerns that you have.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ben
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just a quick update to say that I've created a new document
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> covers
>>>>>>>>>>>>>>> the proposed schema additions in a more descriptive way 
>>>>>>>>>>>>>>> (with
>>>>>>>>>>>>>>> diagrams!)
>>>>>>>>>>>>>>> I'm very new to developing with LDAP, so some more 
>>>>>>>>>>>>>>> experienced
>>>>>>>>>>>>>>> eyes on
>>>>>>>>>>>>>>> the proposal would be very helpful, even if you don't have
>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>> absorb the full design. Please take a look at
>>>>>>>>>>>>>>> http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> if you have a chance.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I finally had a chance to take a look at this, here are some
>>>>>>>>>>>>>> comments:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) I don't like how transformation rules are tied to a
>>>>>>>>>>>>>> particular
>>>>>>>>>>>>>> helper and have to be duplicated for each of them. They
>>>>>>>>>>>>>> should be
>>>>>>>>>>>>>> generic and work with any helper, as helpers are just an
>>>>>>>>>>>>>> implementation detail and their resulting data is the same.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In fact, I think I would prefer if the CSR was generated 
>>>>>>>>>>>>>> using
>>>>>>>>>>>>>> python-cryptography's CertificateSigningRequestBuilder [1]
>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>> than
>>>>>>>>>>>>>> openssl or certutil or any other command line tool.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> There are lots of tools that users might want to use to 
>>>>>>>>>>>>> manage
>>>>>>>>>>>>> their
>>>>>>>>>>>>> private keys, so I don't know if we can assume that whatever
>>>>>>>>>>>>> library we
>>>>>>>>>>>>> prefer will actually be able to access the private key to 
>>>>>>>>>>>>> sign a
>>>>>>>>>>>>> CSR,
>>>>>>>>>>>>> which is why I thought it would be useful to support more 
>>>>>>>>>>>>> than
>>>>>>>>>>>>> one.
>>>>>>>>>>>>
>>>>>>>>>>>> python-cryptography has the notion of backends, which allow 
>>>>>>>>>>>> it to
>>>>>>>>>>>> support multiple crypto implementations. Upstream it currently
>>>>>>>>>>>> supports only OpenSSL [2], but some work has been done on 
>>>>>>>>>>>> PKCS#11
>>>>>>>>>>>> backend [3], which provides support for HSMs and soft-tokens
>>>>>>>>>>>> (like
>>>>>>>>>>>> NSS
>>>>>>>>>>>> databases).
>>>>>>>>>>>>
>>>>>>>>>>>> Alternatively, for NSS databases (and other "simple" 
>>>>>>>>>>>> cases), you
>>>>>>>>>>>> can
>>>>>>>>>>>> generate the private key with python-cryptography using the
>>>>>>>>>>>> default
>>>>>>>>>>>> backend, export it to a file and import the file to the target
>>>>>>>>>>>> database, so you don't actually need the PKCS#11 backend for
>>>>>>>>>>>> them.
>>>>>>>>>>>>
>>>>>>>>>>>> So, the only thing that's currently lacking is HSM support, 
>>>>>>>>>>>> but
>>>>>>>>>>>> given
>>>>>>>>>>>> that we don't support HSMs in IPA nor in certmonger, I don't
>>>>>>>>>>>> think
>>>>>>>>>>>> it's an issue for now.
>>>>>>>>>>>>
>>>>>>>>>>>>> The
>>>>>>>>>>>>> purpose of the mapping rule is to tie together the
>>>>>>>>>>>>> transformation
>>>>>>>>>>>>> rules
>>>>>>>>>>>>> that produce the same data into an object that's
>>>>>>>>>>>>> implementation-agnostic, so that profiles referencing those
>>>>>>>>>>>>> rules
>>>>>>>>>>>>> are
>>>>>>>>>>>>> automatically compatible with all the helper options.
>>>>>>>>>>>>
>>>>>>>>>>>> They are implementation-agnostic, as long as you consider
>>>>>>>>>>>> `openssl`
>>>>>>>>>>>> and `certutil` the only implementations :-) But I don't think
>>>>>>>>>>>> this
>>>>>>>>>>>> solution scales well to other possible implementations.
>>>>>>>>>>>>
>>>>>>>>>>>> Anyway, my main grudge is that the transformation rules 
>>>>>>>>>>>> shouldn't
>>>>>>>>>>>> really be stored on and processed by the server. The server
>>>>>>>>>>>> should
>>>>>>>>>>>> know the *what* (mapping rules), but not the *how*
>>>>>>>>>>>> (transformation
>>>>>>>>>>>> rules). The *how* is an implementation detail and does not
>>>>>>>>>>>> change in
>>>>>>>>>>>> time, so there's no benefit in handling it on the server. It
>>>>>>>>>>>> should be
>>>>>>>>>>>> handled exclusively on the client, which I believe would also
>>>>>>>>>>>> make
>>>>>>>>>>>> the
>>>>>>>>>>>> whole thing more robust (it would not be possible for a bug on
>>>>>>>>>>>> the
>>>>>>>>>>>> server to break all the clients).
>>>>>>>>>>> This is a good point. However, for the scope of Ben's project
>>>>>>>>>>> can we
>>>>>>>>>>> limit it by openssl and certutil support? Otherwise Ben
>>>>>>>>>>> wouldn't be
>>>>>>>>>>> able
>>>>>>>>>>> to complete the project in time.
>>>>>>>>>>
>>>>>>>>>> I'm fine with that, but I don't think it's up to me :-)
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>> This is turning out to be a common (and, I think, reasonable)
>>>>>>>>>>>>> reaction
>>>>>>>>>>>>> to the proposal. It is rather complex, and I worry that it
>>>>>>>>>>>>> will be
>>>>>>>>>>>>> difficult to configure. On the other hand, there is some 
>>>>>>>>>>>>> hidden
>>>>>>>>>>>>> complexity to enabling a simpler config format, as well. 
>>>>>>>>>>>>> One of
>>>>>>>>>>>>> the
>>>>>>>>>>>>> goals of the project as it was presented to me was to 
>>>>>>>>>>>>> allow the
>>>>>>>>>>>>> creation
>>>>>>>>>>>>> of profiles that add certificate extensions *that FreeIPA
>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>> yet
>>>>>>>>>>>>> know about*. With the current proposal, one only has to add a
>>>>>>>>>>>>> rule
>>>>>>>>>>>>> generating text that the helper will understand.
>>>>>>>>>>>>
>>>>>>>>>>>> ... which will be possible only as long as the helper
>>>>>>>>>>>> understands the
>>>>>>>>>>>> extension. Which it might not, thus the current proposal works
>>>>>>>>>>>> only
>>>>>>>>>>>> for *some* extensions that FreeIPA doesn't yet support.
>>>>>>>>>>> We can go ad infinitum here but with any helper implementation,
>>>>>>>>>>> be it
>>>>>>>>>>> python-cryptography or anything else, you will need to have a
>>>>>>>>>>> support
>>>>>>>>>>> there as well.
>>>>>>>>>>
>>>>>>>>>> My point was that the current proposal is not any better than my
>>>>>>>>>> proposal in this regard, as neither of them allows one to use an
>>>>>>>>>> arbitrary extension.
>>>>>>>>>>
>>>>>>>>>>> The idea with unknown extensions was to allow mapping
>>>>>>>>>>> their acceptance to a specific relationship between IPA objects
>>>>>>>>>>> (optionally) and an input from the CSR. A simplest example 
>>>>>>>>>>> would
>>>>>>>>>>> be an
>>>>>>>>>>> identity rule that would copy an ASN.1 encoded content from the
>>>>>>>>>>> CSR to
>>>>>>>>>>> the certificate.
>>>>>>>>>>>
>>>>>>>>>>> That's on the mapping side, not on the CSR generation side, 
>>>>>>>>>>> but it
>>>>>>>>>>> would
>>>>>>>>>>> go similarly for the CSR if you would be able to enter 
>>>>>>>>>>> unknown but
>>>>>>>>>>> otherwise correct ASN.1 stream. There is no difference at which
>>>>>>>>>>> helper
>>>>>>>>>>> type we are talking about because all of them support inserting
>>>>>>>>>>> ASN.1
>>>>>>>>>>> content.
>>>>>>>>>>>
>>>>>>>>>>>>> With your suggestion,
>>>>>>>>>>>>> if there's a mapping between "san_directoryname" and the
>>>>>>>>>>>>> corresponding
>>>>>>>>>>>>> API calls or configuration lines, we need some way for 
>>>>>>>>>>>>> users to
>>>>>>>>>>>>> augment
>>>>>>>>>>>>> that mapping without changing the code. If there's no
>>>>>>>>>>>>> mapping, and
>>>>>>>>>>>>> it's
>>>>>>>>>>>>> just done with text processing, we need enough in the config
>>>>>>>>>>>>> format to
>>>>>>>>>>>>> be able to generate fairly complex structures:
>>>>>>>>>>>>>
>>>>>>>>>>>>> builder =
>>>>>>>>>>>>> builder.subject_name(x509.Name(u'CN=user,O=EXAMPLE.COM'))
>>>>>>>>>>>>> builder =
>>>>>>>>>>>>> builder.add_extension(x509.SubjectAlternativeName([x509.RFC822Name(u'user at example.com'), 
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> x509.DirectoryName(x509.Name(u'CN=user,O=EXAMPLE.COM'))]),
>>>>>>>>>>>>> False)
>>>>>>>>>>>>>
>>>>>>>>>>>>> and we need to do it without it being equivalent to calling
>>>>>>>>>>>>> eval() on
>>>>>>>>>>>>> the config attributes. I'm not sure how to achieve this 
>>>>>>>>>>>>> (is it
>>>>>>>>>>>>> safe to
>>>>>>>>>>>>> call getattr(x509, extensiontype)(value) where 
>>>>>>>>>>>>> extensiontype and
>>>>>>>>>>>>> value
>>>>>>>>>>>>> are user-specified?) and it definitely would have to be tied
>>>>>>>>>>>>> to a
>>>>>>>>>>>>> particular library/tool.
>>>>>>>>>>>>
>>>>>>>>>>>> As I pointed out above, this needs to be figured out for the
>>>>>>>>>>>> generic
>>>>>>>>>>>> case for both the current proposal and my suggestion.
>>>>>>>>> I have a proof of concept[1] for using openssl-based rules to 
>>>>>>>>> add a
>>>>>>>>> subject alt name extension without using openssl's knowledge 
>>>>>>>>> of that
>>>>>>>>> extension. It's not extremely pretty, and it took some trial and
>>>>>>>>> error,
>>>>>>>>> but no code changes. So, I think this actually is a difference
>>>>>>>>> between
>>>>>>>>> the two proposals.
>>>>>>>>
>>>>>>>> With the obvious catch being that it works only with OpenSSL, 
>>>>>>>> which
>>>>>>>> might not work for everyone, e.g. when using HSMs or 
>>>>>>>> SmartCards, due
>>>>>>>> to a limited PKCS#11 support in OpenSSL.
>>>>>>>
>>>>>>> Very true. Even certutil's equivalent feature (--extGeneric) 
>>>>>>> doesn't
>>>>>>> seem like it would work very well in this context, as you are 
>>>>>>> supposed
>>>>>>> to pass in an already-encoded extension, so text-based templating
>>>>>>> wouldn't be able to do much.
>>>>>>
>>>>>> Yeah, I struggled with this myself. I ended up writing a pyasn1 
>>>>>> script
>>>>>> to generate the extension I needed, wrote that to a file, and passed
>>>>>> it to certutil using:
>>>>>>
>>>>>> --extGeneric 2.5.29.17:not-critical:/path/to/msupn.der
>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Next we have the easy case, extensions that we as FreeIPA 
>>>>>>>>> developers
>>>>>>>>> know are important and build support for. For these, the two
>>>>>>>>> proposals
>>>>>>>>> work equivalently well, but yours is simpler to configure because
>>>>>>>>> the
>>>>>>>>> knowledge of how to make a san_rfc822name is built into the 
>>>>>>>>> library
>>>>>>>>> instead of being stored on the server as a set of rules.
>>>>>>>>>
>>>>>>>>> Finally, we have the case of extensions that are known to the
>>>>>>>>> helper,
>>>>>>>>> but not to FreeIPA. In the existing proposal, new rules can be
>>>>>>>>> written
>>>>>>>>> to support these extensions under a particular helper. Further,
>>>>>>>>> those
>>>>>>>>> rules can be used by reference in many profiles, reducing
>>>>>>>>> duplication of
>>>>>>>>> effort/data/errors.
>>>>>>>>>
>>>>>>>>> As I understand it, the main objections in this thread are that
>>>>>>>>> transformation rules are implementation (i.e. helper) specific 
>>>>>>>>> data
>>>>>>>>> stored in the IPA server, and that the system has several 
>>>>>>>>> levels of
>>>>>>>>> schema when it could just embed rules in the profile. But without
>>>>>>>>> helper-specific rules, administrators could not take advantage of
>>>>>>>>> the
>>>>>>>>> additional extensions supported by the helper they are using.
>>>>>>>>
>>>>>>>> There is *no* advantage in forcing the user to choose between 
>>>>>>>> helpers
>>>>>>>> which differ only in the set of limitations on the CSR they are 
>>>>>>>> able
>>>>>>>> to produce. The user should specify a) where the private key is
>>>>>>>> located and b) what profile to use, and that's it, it should just
>>>>>>>> work.
>>>>>>> Ok, this is a good point about usability. The user creating the CSR
>>>>>>> shouldn't have to care about helpers, and I agree that the 
>>>>>>> current way
>>>>>>> they are exposed is clunky. I do think that an administrator 
>>>>>>> creating
>>>>>>> custom rules might want to take advantage of a helper, so they
>>>>>>> wouldn't
>>>>>>> need to understand the ASN.1 representation of their chosen
>>>>>>> certificate
>>>>>>> extension. Of course, the desired extension might not be 
>>>>>>> supported by
>>>>>>> the helper either. Since I don't know what specific extensions 
>>>>>>> people
>>>>>>> will want to use this for, I don't know how to balance the better
>>>>>>> administrator experience of adding extensions via a helper with the
>>>>>>> limited extension support.
>>>>>>>
>>>>>>> The original reason we arrived at the concept of "helpers" was to
>>>>>>> support different ways of getting at private keys, but perhaps this
>>>>>>> should not be the concern of the CSR data generator. In your 
>>>>>>> opinion,
>>>>>>> would it be sufficient to support just one key format (PKCS#12? 
>>>>>>> PEM?)
>>>>>>> and let the user deal with putting those keys into whatever
>>>>>>> formats/databases they need? If that's ok, maybe we can stop having
>>>>>>> *multiple* helpers, but if we want to replace helpers entirely I'm
>>>>>>> still
>>>>>>> not certain what to replace them with.
>>>>>>
>>>>>> I'd just add an option to specify the output format, e.g PEM, NSS,
>>>>>> Java keystore, PKCS#12, whatever. You can probably get away with the
>>>>>> first two for starters. Different output format is going to mean
>>>>>> different options but that is probably not a big deal.
>>>>>
>>>>> My point was that if we want to get rid of all the helpers but 
>>>>> one, or
>>>>> replace helpers with something else entirely like somehow templating
>>>>> ASN1 structures directly, it will get harder to support all those
>>>>> formats (or even both of the first two). For example, if we drop
>>>>> certutil as a helper, how will we sign CSRs with keys stored in NSS
>>>>> databases?
>>>>
>>>> 1. get the public part of the key from the NSS database
>>>> 2. construct a CertificationRequestInfo [1] from the template and the
>>>> public key
>>>> 3. sign the CertificationRequestInfo with NSS using the private key to
>>>> get a CSR
>>>>
>>>> This is purely client side, will work with any crypto library (just
>>>> substitute NSS for something else) and, if done right, using very
>>>> little code.
>>>
>>> Ok, I like this. If an encoded CertificationRequestInfo is something we
>>> can expect to be compatible with any reasonable library (it sounds like
>>> it should be) then the library can be used client-side to do the
>>> key-storage-specific parts. I'm going to try writing this data ->
>>> encoded CertificationRequestInfo -> CSR flow to make sure it works as
>>> well as it sounds. If it does, it will also be useful for the code I'm
>>> working on right now to connect certmonger with the current version of
>>> the CSR autogeneration tool.
>>
>> Note that this will most probably require calling C functions. You 
>> might want to look into python-cffi.

For now I just went ahead and implemented it in C, for simplicity. So 
far it only does the data + SubjectPublicKeyInfo -> 
CertificationRequestInfo conversion (data in the openssl config file 
format), but I'm convinced that both openssl and NSS should be able to 
sign this to turn it into a CSR. I'm also pretty sure you were right 
that calling C functions is required - none of the python libraries seem 
to have bindings for the functions that manipulate these objects. You 
can see the prototype here: 
https://github.com/LiptonB/freeipa-prototypes/blob/master/build_requestinfo.c
>>
>>>>
>>>>>>
>>>>>> Remember that the private key will be at rest for some period of 
>>>>>> time
>>>>>> while the CSR is being approved. The key needs to be protected at 
>>>>>> that
>>>>>> time.
>>>>>>
>>>>>> rob
>>>>>>
>>>>>>>>
>>>>>>>>> And
>>>>>>>>> without the separation of profiles from mapping rules in the 
>>>>>>>>> schema,
>>>>>>>>> rules would need to be copy+pasted among profiles, and grouping
>>>>>>>>> rules
>>>>>>>>> with the same effect under different helpers would be much
>>>>>>>>> uglier. We
>>>>>>>>> can and should discuss whether these are the right tradeoffs, but
>>>>>>>>> this
>>>>>>>>> is where those decisions came from.
>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> OTOH, I think we could use GSER encoding of the extension 
>>>>>>>>>>>> value:
>>>>>>>>>>>>
>>>>>>>>>>>>    { rfc822Name:"user at example.com",
>>>>>>>>>>>> directoryName:rdnSequence:"CN=user,O=EXAMPLE.COM" }
>>>>>>>>>>> GSER is not really used widely and does not have standardized
>>>>>>>>>>> encoding
>>>>>>>>>>> rules beyond its own definition. If you want to allow
>>>>>>>>>>> transformation
>>>>>>>>>>> rules in GSER that mention existing content in IPA objects, you
>>>>>>>>>>> would
>>>>>>>>>>> need to deal with templating anyway. At this point it becomes
>>>>>>>>>>> irrelevant
>>>>>>>>>>> what you are templating, though.
>>>>>>>>>>
>>>>>>>>>> True, but the goal here is not to avoid templating, but 
>>>>>>>>>> rather to
>>>>>>>>>> avoid implementation-specific bits on the server, and GSER is 
>>>>>>>>>> the
>>>>>>>>>> only
>>>>>>>>>> thing that is textual, implementation-neutral and, as a bonus,
>>>>>>>>>> standardized.
>>>>>>>>>>
>>>>>>>>> As I said elsewhere, we could use GSER as a textual output format
>>>>>>>>> instead of openssl or certutil, but it still needs its own
>>>>>>>>> "helper" to
>>>>>>>>> build the CSR, and unlike the other options, it seems like we 
>>>>>>>>> might
>>>>>>>>> need
>>>>>>>>> to implement that helper. I'm not sure it's fair to call it
>>>>>>>>> implementation-neutral if no implementation exists yet :)
>>>>>>>>
>>>>>>>> Right. Like I said, using GSER was just a quick idea off the top
>>>>>>>> of my
>>>>>>>> head. I would actually rather use some sort of data structure
>>>>>>>> templating rather than textual templating on top of any kind of
>>>>>>>> textual representation of said data structures. I don't know if 
>>>>>>>> there
>>>>>>>> is such a thing, though.
>>>>>>>
>>>>>>> This sounds interesting, can you give an example of what this might
>>>>>>> look
>>>>>>> like?
>>>>
>>>> It would be something like XSLT, but for ASN.1 rather than XML.
>>>>
>>>>>>>
>>>>>>> I learned that there's also an XML encoding for ASN.1, XER, but 
>>>>>>> that's
>>>>>>> still a textual representation and we'd have to insert the data
>>>>>>> textually.
>>>>
>>>> Well, yes and no. While it's true that it's still a textual
>>>> representation, what really makes a difference is that for XML, there
>>>> is a templating mechanism which understands the structure of the data
>>>> (XLST, as mentioned above).
>>>>
>>>> Unforutantely, XER has the same shortcoming as GSER: to be able to
>>>> convert it to DER, you need to know the ASN.1 definition of the data
>>>> structure. If we used XER+XSLT, we would also have to provide means of
>>>> adding custom ASN.1 definitions and run them through ASN.1 compiler to
>>>> convert between XER and DER.
>>>
>>> This is a little disappointing, but it makes sense. I don't think I
>>> realized that we'll need to compile the ASN.1 data definitions for any
>>> extensions we want to use in a cert. That limitation didn't come up 
>>> when
>>> we were only talking about extensions that were supported by the helper
>>> utility. But providing the ASN.1 spec for unusual extensions an admin
>>> wants to use in their certs is probably a reasonable expectation.
>>
>> Yes, that's what I think as well. It could be a simple IPA object 
>> with name, description, extension OID and the ASN.1 definition.
>>
>>>>
>>>>>>> It doesn't seem to be supported by any python libraries,
>>>>>>> either, but it does look like it's supported by the asn1 compiler
>>>>>>> in the
>>>>>>> IPA source distribution.I could imagine an implementation that 
>>>>>>> builds
>>>>>>> an XML representation of the CSR via python templating, then 
>>>>>>> makes a
>>>>>>> signed CSR out of it in C. I'm a little concerned about it 
>>>>>>> because it
>>>>>>> would have to implement the whole CSR structure from scratch, 
>>>>>>> but is
>>>>>>> this a prototype that you'd be interested in seeing?
>>>>
>>>> I can imagine something like this might work:
>>>>
>>>> 1. (client) generate a key pair
>>>> 2. (client) get SubjectPublicKeyInfo [2] for the public key
>>>> 3. (client) encode the SubjectPublicKeyInfo as XER using asn1c and
>>>> python-cffi in API mode [3]
>>>> 4. (client) call server to construct CertificationRequestInfo for
>>>> specified subject from a specified template and the 
>>>> SubjectPublicKeyInfo
>>>> 5. (server) get the subject's LDAP entry
>>>> 6. (server) create a XML document which contains the subject's LDAP
>>>> attributes and the SubjectPublicKeyInfo
>>>> 7. (server) use XSLT to transform the XML document to
>>>> CertificationRequestInfo using the specified template
>>>> 8. (server) return the CertificationRequestInfo to the client
>>>> 9. (client) convert the CertificationRequestInfo from XER to DER using
>>>> asn1c and python-cffi in API mode
>>>> 10. (client) sign the CertificationRequestInfo using the private key
>>>> to get a CSR
>>>>
>>>> It would be better if the XER-DER conversion was done on the server,
>>>> but I don't think that compiling and running code on the fly on the
>>>> server is a particularly good idea. Apparently there is a ASN.1
>>>> compiler available for PyASN1 [4], maybe that could be used instead,
>>>> but we would have to write a XER codec for PyASN1 ourselves (which
>>>> shouldn't be too hard IMO).
>>>
>>> Yeah, running programs compiled from arbitrary ASN.1 seems like a risk.
>>> Maybe a little better because the ASN.1 is provided by an 
>>> administrator,
>>> but we'd still be depending a lot on the security of the generated 
>>> code.
>>> On the other hand, if we compile on the client, the CSR generation
>>> feature is limited to platforms where asn1c can be installed. I wish I
>>> could think of a way to do the compilation once when the profile is
>>> created, but run it on the client. That seems like asking for
>>> compatibility problems, though...
>>
>> It seems you missed the most important thing in the above paragraph 
>> :-) - that is asn1ate, the PyASN1-based compiler. The nice thing 
>> about it is that it compiles the ASN.1 definition into a PyASN1 type 
>> object, which means you can compile the definition and use it to 
>> (un)parse data in the same Python program. If we used it, we could 
>> JIT-compile the ASN.1 definitions on the server, without the security 
>> risk of executing native code and without the compatibility issues of 
>> compilation on the client.
>
> What do you see as the risks of compiling native code with asn1c and 
> executing it that are not present when generating python code with 
> asn1ate and loading it? I would think that, native or not, we're 
> depending on the ASN.1 compiler to generate secure code from any ASN.1 
> definition the admin might give it. Even a parser like libtasn1 that 
> interprets the structure on the fly rather than generating executable 
> code could do something dangerous when given poorly-constructed input. 
> I don't mean to create a false equivalence, but are the interpreted 
> options really safer than the native code?
>>
>> I did a little research since my last email, andt doesn't seem to 
>> have there is also another library which allows you to compile and 
>> use ASN.1 definitions on the fly - libtasn1 [5]. Compared to asn1ate, 
>> it seems to be pretty stable (asn1ate is currently in alpha) and is 
>> written in C, so it makes it possible to use the 
>> administrator-defined extensions outside of IPA (specifically, it 
>> could be useful for certificate matching and mapping [6] in SSSD).
>
> Good find. That seems quite useful for being able to interact with 
> ASN.1 defined on the fly. I wonder how hard it would be to connect it 
> to pyasn1 to get more flexible ASN.1 decoding within python. Still 
> doesn't help with XER encoding/decoding, but I suppose that's a SMOP :)
>>
>>>
>>>>
>>>>>>>
>>>>> On further investigation, it turns out the version of
>>>>> python-cryptography in F24 includes a feature allowing arbitrary
>>>>> extensions to be added by adding an UnrecognizedExtension to the
>>>>> CertificateSigningRequestBuilder. This makes me feel somewhat better
>>>>> both about python-cryptography as a tool for this task and about the
>>>>> solution I just proposed. But I still don't have a clear idea that
>>>>> answers 1) how to make templates that we can turn into encoded
>>>>> extensions, and 2) how to deal with all the desired key formats.
>>>>
>>>> I hope the above clarifies these a little bit.
>>>>
>>>> [1] <https://tools.ietf.org/html/rfc2986#section-4.1>
>>>> [2] <https://tools.ietf.org/html/rfc5280#section-4.1.2.7>
>>>> [3] 
>>>> <https://cffi.readthedocs.io/en/latest/overview.html#abi-versus-api>
>>>> [4] <https://github.com/kimgr/asn1ate>
>>
>> [5] <https://www.gnu.org/software/libtasn1/>
>> [6] 
>> <https://www.redhat.com/archives/freeipa-devel/2016-October/msg00090.html>
>>
>