From ftweedal at redhat.com Tue Jul 1 02:42:59 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 1 Jul 2014 12:42:59 +1000 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B1BA49.1090809@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> Message-ID: <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> On Mon, Jun 30, 2014 at 02:28:09PM -0500, Endi Sukma Dewata wrote: > Hi, I have some comments about the design: > http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage > > I think all system/default profiles should remain file-based and all custom > profiles should be LDAP-based. It will make a clean separation: system > profiles are owned by us (Dogtag developers), custom profiles are owned by > the admin. > > The system profiles will be read-only. This way we will be able to update > the system profiles without writing any upgrade scripts because the files > will be updated automatically by RPM. Just one requirement, all server > instances must be upgraded to the same version. > > If the admin wants to change a system profile, they can clone it into a > custom profile and make the changes there. The custom profiles cannot have > the same names as the system profiles, so there's won't be any > conflict/confusion, and no need to support a "restore" command. In general > we won't need to write upgrade scripts for custom profiles except if we > change the LDAP schema. > This was my original proposal, with the exception of making system/default profiles read-only and cloning them to make modifications. I think this is a good modification to that proposal. Ade was in favour of moving to LDAP only, but perhaps this variation makes using a mix of LDAP and file-based profiles more palatable? It would be good to hear from more people on this part of the design - to see if there's a clear winner. > About the schema itself, I suppose we want to have something that resembles > the actual Profile data structure (see ProfileData Java class). There should > be an LDAP attribute for each single-valued Java attribute (e.g. name, > description, enabled, visible). This way the profile is more manageable and > can be queried based on these attributes. For collection attributes (e.g. > inputs, outputs, policySets) we can use child LDAP entries to represent > them. > > About the names, right now we use "profile" to refer to both cert profile in > CA and token profile in TPS. We probably should use separate terms to > distinguish them (e.g. CertProfile and TokenProfile). This applies to > Java/Python class names, REST interface, CLI, LDAP schema, etc. > Good point. I've s/profile/certProfile/ in the schema. > So the LDAP entries for a profile may look like this: > > dn: cn=MyUserCert,ou=Profiles,ou=CA,{suffix} > objectClass: certProfile > cn: MyUserCert > displayName: Manual User Dual-Use Certificate Enrollment > description: This certificate profile is for enrolling user certificates. > visible: true > enabled: true > enabledBy: admin > > dn: ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,{suffix} > objectClass: organizationalUnit > ou: Inputs > > dn: cn=i1,ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,o=pki-tomcat > objectClass: pkiCertProfileInput > cn: i1 > classId: keyGenInputImpl > Introducing this schema (and the more fine-grained functionality it would support) could happen later. I see the benefits but I think it's better to just get the profiles in LDAP as soon as possible, then look at how we can break them down to enable richer features. Also, if we retain file-based profiles as you suggest, whatever queries LDAP makes easier, we would still need to be able to do those same queries across the file-based profiles. So while file-based profiles remain, a finer-grained LDAP profile schema doesn't buy us anything in terms of reducing complexity or the amount of code to write. > About the REST interface & CLI, since this will be the primary way to edit > profiles, we might want to have more granular commands to modify parts of > the profile. Right now with ca-profile-mod command you need to send the > entire profile in a file. It would be nice to be able to specify some > parameters to change certain attributes only, or use separate commands to > manage the inputs/outputs. > > We'll also need an interface to find existing cert records that use a > certain profile and bulk modify them to use a different profile. This will > be useful when you create a clone to change the system profile. > Again, I agree with the benefits here, but feel that this may be out of scope for the initial changeover to LDAP profiles, given how much we need that for FreeIPA. > I'm not sure if we should support 10.2 -> 10.3 cloning. When we release 10.3 > the 10.2 will still be fairly new so it might be reasonable to require all > clones to be upgraded. It will reduce the amount of testing requirement too. > I'll incorporate this comment (and all your others) in the design proposal; you should see the updated document soon. Cheers, Fraser > -- > Endi S. Dewata > > On 6/18/2014 2:44 AM, Fraser Tweedale wrote: > >Hi all, > > > >A requirement from the FreeIPA side is the ability to add and > >customise CA profiles. Dogtag's current profile creation behaviour > >writes the new profile to the filesystem beside the standard > >profiles (as well as making the appropriate update to the registry, > >etc.) > > > >There does not seem to be a mechanism to distribute new/modified > >profiles to replicas - though perhaps I have missed something. > > > >Because this behaviour is required, unless I have overlooked > >something or there is a better way (in which case please shout out), > >I think it makes sense to begin a design proposal for an LDAP-based > >profile store. > > > >Finally, a brief mention of some tickets related to profile storage > >that could be good to tackle simultaneously should the proposed > >change go ahead: > > > >- https://fedorahosted.org/pki/ticket/778 > >- https://fedorahosted.org/freeipa/ticket/4002 > > > >_______________________________________________ > >Pki-devel mailing list > >Pki-devel at redhat.com > >https://www.redhat.com/mailman/listinfo/pki-devel > > > From alee at redhat.com Tue Jul 1 12:27:06 2014 From: alee at redhat.com (Ade Lee) Date: Tue, 01 Jul 2014 20:27:06 +0800 Subject: [Pki-devel] [PATCH] 225 fixes to resolve build issues on rawhide Message-ID: <1404217626.5622.3.camel@localhost.localdomain> Changes to fix rawhide build - Removed dependency on removed internal junit class - moved cmake reference to junit4.jar to junit.jar - Disambiguate a couple of references In the interest of getting a build going on rawhide and because these changes are pretty uncontroversial, I have pushed these to master. Please speak up if there are any objections. Ade -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-vakwetu-0225-Changes-to-fix-rawhide-build.patch Type: text/x-patch Size: 3778 bytes Desc: not available URL: From alee at redhat.com Tue Jul 1 12:52:46 2014 From: alee at redhat.com (Ade Lee) Date: Tue, 01 Jul 2014 20:52:46 +0800 Subject: [Pki-devel] [PATCH] 98-3 Fixes for comments on patch 98-2 In-Reply-To: <20140630032959.GL2417@dhcp-40-8.bne.redhat.com> References: <1403186323.2669.3.camel@akoneru.redhat.com> <20140620020243.GP11349@dhcp-40-8.bne.redhat.com> <1403645545.3075.3.camel@akoneru.redhat.com> <20140625064907.GC3122@dhcp-40-8.bne.redhat.com> <1403714870.6457.35.camel@localhost.localdomain> <1403888893.6624.1.camel@akoneru.redhat.com> <20140630032959.GL2417@dhcp-40-8.bne.redhat.com> Message-ID: <1404219166.5622.5.camel@localhost.localdomain> On Mon, 2014-06-30 at 13:29 +1000, Fraser Tweedale wrote: > On Fri, Jun 27, 2014 at 01:08:13PM -0400, Abhishek Koneru wrote: > > Please review the patch with fixes for Ade's comments on patch 98-2. > > Addressed all the comments except 8 and 10. which will be done > > separately. > > > > Patch does seem to address those issues and __main__ still works > nicely (including removing the added profile at end now - thanks). > > Switching to serialising the profile as JSON now means that the > upload-xml-profile codepath is no longer exercised by __main__. Not > a blocker I think, but if it's easy to write out a profile as XML so > that codepath is exercised, that would be a good idea. ACK. Please open a ticket to add the missing XML tests/functions. We can triage accordingly. I would suggest a milestone of 10.3. Ade > > ACK modulo preceding comment. > > > > Please apply patch 97 before applying 98-3 > > --Abhishek > > > > On Thu, 2014-06-26 at 00:47 +0800, Ade Lee wrote: > > > In profile.py: > > > > > > 1. Usually, we set @author and so on after the copyright notice. > > > > > > 2. In PolicyDefault, you have methods to add/remove/get attributes and > > > parameters, as in: > > > def add_attribute(self, profile_attribute): > > > self.policy_attributes.append(profile_attribute) > > > > > > Change profile_attribute to policy_attribute etc. You should indicate in > > > a docstring that the expected type for the policy_attribute etc. is a > > > ProfileAttribute. > > > > > > 3. I'm not a big fan of your _send_profile_request() and > > > _send_request_in_file() methods in that they mix create and update > > > operations. That might make sense if the operations had a lot of common > > > code and were doing the same HTTP operation, but otherwise is just > > > confusing. > > > > > > A better way to split this up might be to define: > > > > > > def _send_profile_create(self, profile_data) > > > > > > def _send_profile_modify(self, profile_id, profile_data) > > > > > > @staticmethod > > > def _get_profile_data_from_file(path_to_file, file_format) > > > > > > and then call these as needed. > > > > > > So for example, create_profile_using_file_input(self, path_to_file, > > > file_format) -> which I would rename to create_profile_from_file(..) -> > > > would call: > > > profile_data = _get_profile_data_from_file(path_to_file, file_format) > > > _send_profile_create(profile_data) > > > > > > 4. In the test where you add a duplicate profile, you should check for > > > the specific exception thrown -- is there a specific exception? > > > > > > 5. Change the profileID in the test to MySampleProfile instead of > > > MySampleCert. > > > > > > 6. In the test, we are testing file input using original.xml and > > > modified.xml. Where do these files come from? It would be better if > > > these files were generated as part of the previous tests - which means > > > you need a function to save files to xml/json. You should then test xml > > > and json. > > > > > > 7. The policy stuff gets confusing fast, so the example really helps. > > > We need more function docstrings though that specify when inputs/outputs > > > the functions have - and in particular the class of each input. > > > > > > 8. We also need more api documentation on the wiki as we have done in > > > the past. This is not part of this patch of course. > > > > > > 9. There are Pycharm errors about unused variables. Most of these will > > > be addressed by the refactoring I mention in point 3 above. > > > > > > 10. profile.py - which is all new code generates probably close to 200 > > > new messages in pylint. We really should fix these. To keep things > > > clear, you should do this in a separate patch. > > > > > > > > > > > > On Wed, 2014-06-25 at 16:49 +1000, Fraser Tweedale wrote: > > > > On Tue, Jun 24, 2014 at 05:32:25PM -0400, Abhishek Koneru wrote: > > > > > Please review the patch with fixes for comments on patch 98. > > > > > Issues addressed: > > > > > 1. Added a try catch block for issue 1(as noted below) caught by > > > > > ftweedal(nice catch!). > > > > > 2. Made the data_format mandatory for file_input methods. > > > > > 3. Added code to create a profile with PolicyOutput and PolicySets. > > > > > > > > > > -- Abhishek > > > > > Please apply patch 97 before applying this patch. > > > > > > > > > > > > > ACK > > > > > > > > > On Fri, 2014-06-20 at 12:02 +1000, Fraser Tweedale wrote: > > > > > > On Thu, Jun 19, 2014 at 09:58:43AM -0400, Abhishek Koneru wrote: > > > > > > > Please review the attached patch which adds methods that allow users to > > > > > > > pass the profile data in a file. > > > > > > > > > > > > > > Also attached two sample xml files, original.xml and modified.xml. > > > > > > > Place them in /tmp before running the profile.py module. > > > > > > > > > > > > > > Thanks, > > > > > > > Abhishek > > > > > > > > > > > > > > > > > > > Hi Abhishek, > > > > > > > > > > > > Patch applied and __main__ example run. Seems to do what it says on > > > > > > the tin; the usual "haven't used it in anger" caveat ^_^. Review > > > > > > comments inline below. > > > > > > > > > > > > Fraser > > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > > def modify_profile(self, profile_data): > > > > > > > """ > > > > > > > Modify an existing profile. > > > > > > > """ > > > > > > > - if profile_data is None: > > > > > > > - raise ValueError("No ProfileData specified") > > > > > > > + return self._send_profile_request(profile_data, 'modify') > > > > > > > + > > > > > > > + def _send_request_in_file(self, path_to_file, data_format, operation): > > > > > > > + > > > > > > > + if path_to_file is None: > > > > > > > + raise ValueError("File path must be specified.") > > > > > > > + > > > > > > > + if data_format not in ['xml', 'json']: > > > > > > > + raise ValueError("Unsupported data type: " + str(data_format)) > > > > > > > + > > > > > > > + if operation not in ['create', 'modify']: > > > > > > > + raise ValueError("Invalid operation specified: " + str(operation)) > > > > > > > + > > > > > > > + data = None > > > > > > > + with open(path_to_file) as input_file: > > > > > > > + data = input_file.read() > > > > > > > + > > > > > > > + if data_format == 'xml': > > > > > > > + self.headers['Content-type'] = 'application/xml' > > > > > > > > > > > > 1) > > > > > > > > > > > > Overwriting self.headers['Content-type'] is problematic. For > > > > > > example, if the data cannot be parsed or lacks an 'id' key, an > > > > > > exception will be raised and the ProfileClient will be stuck with > > > > > > the wrong Content-Type header. > > > > > > > > > > > > Possible solutions: > > > > > > > > > > > > - use try/finally or a context manager to ensure the header gets > > > > > > reset to 'application/json' even if an exception is raised. > > > > > > - Modify the _put and _post methods to include an optional argument > > > > > > to override the content-type header. To avoid special-cases too > > > > > > many things, this arg could even be a dict that can be merged with > > > > > > the default headers, e.g.: > > > > > > > > > > > > def _post(self, url, payload=None, headers=None): > > > > > > self.account_client.login() > > > > > > headers = dict(self.headers, **(headers or {})) > > > > > > r = self.connection.post(url, payload, headers, query_params) > > > > > > ... > > > > > > > > > > > > Then callers can supply custom headers for a single request > > > > > > without potentially affecting other requests. > > > > > > > > > > > > > + > > > > > > > + r = None > > > > > > > + # Sending the data to the server. > > > > > > > + if operation == 'create': > > > > > > > + r = self._post(self.profiles_url, data) > > > > > > > + else: > > > > > > > + profile_id = None > > > > > > > + if data_format == 'xml': > > > > > > > + profile_id = etree.fromstring(data).get('id') > > > > > > > + else: > > > > > > > + profile_id = json.loads(data)['id'] > > > > > > > + > > > > > > > + if profile_id is None: > > > > > > > + raise ValueError('Profile Id is not specified.') > > > > > > > + url = self.profiles_url + '/' + profile_id > > > > > > > + r = self._put(url, data) > > > > > > > + > > > > > > > + # Reset the Content-type header to json(As used by other methods). > > > > > > > + if data_format == 'xml': > > > > > > > + self.headers['Content-type'] = 'application/json' > > > > > > > > > > > > > > - url = self.profiles_url + '/' + str(profile_data.profile_id) > > > > > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > > > - sort_keys=True) > > > > > > > - r = self._put(url, profile_object) > > > > > > > return Profile.from_json(r.json()) > > > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > > + def create_profile_using_file_input(self, path_to_file, data_format="xml"): > > > > > > > + """ > > > > > > > + Create a new profile from a profile object stored in a file. > > > > > > > + Acceptable data formats - json, xml. > > > > > > > + """ > > > > > > > + return self._send_request_in_file(path_to_file, data_format, 'create') > > > > > > > + > > > > > > > > > > > > 2) > > > > > > > > > > > > Default ``data_format="xml"`` makes it too easy for people to misuse > > > > > > the API, e.g. a path to a JSON file, but no ``data_format`` kwarg > > > > > > given, resulting in default "xml" being used. I suggest either > > > > > > making it a compulsory argument, or making its default ``None`` and, > > > > > > if no explicit ``data_format`` arg given, using the file extension. > > > > > > > > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > > + def modify_profile_using_file_input(self, path_to_file, data_format="xml"): > > > > > > > + """ > > > > > > > + Modify a profile from a profile object stored in a file. > > > > > > > + Acceptable data formats - json, xml. > > > > > > > + """ > > > > > > > + return self._send_request_in_file(path_to_file, data_format, 'modify') > > > > > > > + > > > > > > > + @pki.handle_exceptions() > > > > > > > def delete_profile(self, profile_id): > > > > > > > """ > > > > > > > Delete a profile. > > > > > > > @@ -1185,6 +1253,23 @@ def main(): > > > > > > > # pylint: disable-msg=W0703 > > > > > > > except Exception as e: > > > > > > > print str(e) > > > > > > > + print > > > > > > > + > > > > > > > + # Creating a profile from file > > > > > > > + print('Creating a profile using file input.') > > > > > > > + print('------------------------------------') > > > > > > > + original = profile_client.create_profile_using_file_input( > > > > > > > + '/tmp/original.xml') > > > > > > > + print(original) > > > > > > > + print > > > > > > > + > > > > > > > + # Modifying a profile from file > > > > > > > + print('Modifying a profile using file input.') > > > > > > > + print('------------------------------------') > > > > > > > + modified = profile_client.modify_profile_using_file_input( > > > > > > > + '/tmp/modified.xml') > > > > > > > + print(modified) > > > > > > > + print > > > > > > > > > > > > 3) > > > > > > > > > > > > Nit-pick: could you delete the created profile here so that the > > > > > > example can be run multiple times? > > > > > > > > > > > > > > From 3a031e463e6fb0b5658ad9e1bc37a018c99f32de Mon Sep 17 00:00:00 2001 > > > > > From: Abhishek Koneru > > > > > Date: Thu, 19 Jun 2014 00:10:13 -0400 > > > > > Subject: [PATCH] Added methods for providing file input for profile request. > > > > > > > > > > Added new methods to allow user to provide file input to perform > > > > > operations like create profile/modify profile. > > > > > The supported file formats a re xml and json. > > > > > --- > > > > > base/common/python/pki/__init__.py | 4 +- > > > > > base/common/python/pki/profile.py | 265 ++++++++++++++++++++++++++++++++++--- > > > > > 2 files changed, 252 insertions(+), 17 deletions(-) > > > > > > > > > > diff --git a/base/common/python/pki/__init__.py b/base/common/python/pki/__init__.py > > > > > index e9b726cf763785b4a700ef314ff27774b13aba40..891d6ea6364b037f132ff3754b73b372c638b0f7 100644 > > > > > --- a/base/common/python/pki/__init__.py > > > > > +++ b/base/common/python/pki/__init__.py > > > > > @@ -168,7 +168,7 @@ class PKIException(Exception, ResourceMessage): > > > > > ret = cls(json_value['Message'], json_value['Code'], > > > > > json_value['ClassName']) > > > > > for attr in json_value['Attributes']['Attribute']: > > > > > - print(str(attr)) > > > > > + print str(attr) > > > > > ret.add_attribute(attr["name"], attr["value"]) > > > > > return ret > > > > > > > > > > @@ -299,7 +299,7 @@ class PropertyFile(object): > > > > > def show(self): > > > > > """ Show contents of property file.""" > > > > > for line in self.lines: > > > > > - print(line) > > > > > + print line > > > > > > > > > > def insert_line(self, index, line): > > > > > """ Insert line in property file """ > > > > > diff --git a/base/common/python/pki/profile.py b/base/common/python/pki/profile.py > > > > > index 4f08ee5ba9527855f63b2785c32f9aaddc8d1289..2d4ce5ace11bc6417032fb89ecf6b18707b6fca0 100644 > > > > > --- a/base/common/python/pki/profile.py > > > > > +++ b/base/common/python/pki/profile.py > > > > > @@ -1,12 +1,27 @@ > > > > > #!/usr/bin/python > > > > > """ > > > > > -Created on May 13,, 2014 > > > > > + at author: Abhishek Koneru > > > > > > > > > > - at author: akoneru > > > > > + This program is free software; you can redistribute it and/or modify > > > > > + it under the terms of the GNU General Public License as published by > > > > > + the Free Software Foundation; version 2 of the License. > > > > > + > > > > > + This program is distributed in the hope that it will be useful, > > > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > > > > + GNU General Public License for more details. > > > > > + > > > > > + You should have received a copy of the GNU General Public License along > > > > > + with this program; if not, write to the Free Software Foundation, Inc., > > > > > + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. > > > > > + > > > > > + Copyright (C) 2014 Red Hat, Inc. > > > > > + All rights reserved. > > > > > """ > > > > > > > > > > import json > > > > > import types > > > > > +from lxml import etree > > > > > > > > > > import pki > > > > > import pki.client as client > > > > > @@ -433,6 +448,38 @@ class PolicyDefault(object): > > > > > def policy_params(self, value): > > > > > setattr(self, 'params', value) > > > > > > > > > > + def add_attribute(self, profile_attribute): > > > > > + self.policy_attributes.append(profile_attribute) > > > > > + > > > > > + def remove_attribute(self, profile_attribute_name): > > > > > + for attr in self.policy_attributes: > > > > > + if attr.name == profile_attribute_name: > > > > > + self.policy_attributes.remove(attr) > > > > > + break > > > > > + > > > > > + def get_attribute(self, profile_attribute_name): > > > > > + for attr in self.policy_attributes: > > > > > + if attr.name == profile_attribute_name: > > > > > + return attr > > > > > + > > > > > + return None > > > > > + > > > > > + def add_parameter(self, profile_parameter): > > > > > + self.policy_params.append(profile_parameter) > > > > > + > > > > > + def remove_parameter(self, profile_parameter_name): > > > > > + for param in self.policy_params: > > > > > + if param.name == profile_parameter_name: > > > > > + self.policy_params.remove(param) > > > > > + break > > > > > + > > > > > + def get_parameter(self, profile_parameter_name): > > > > > + for param in self.policy_params: > > > > > + if param.name == profile_parameter_name: > > > > > + return param > > > > > + > > > > > + return None > > > > > + > > > > > @classmethod > > > > > def from_json(cls, json_value): > > > > > policy_def = cls() > > > > > @@ -531,6 +578,22 @@ class PolicyConstraint(object): > > > > > def policy_constraint_values(self, value): > > > > > setattr(self, 'constraint', value) > > > > > > > > > > + def add_constraint_value(self, policy_constraint_value): > > > > > + self.policy_constraint_values.append(policy_constraint_value) > > > > > + > > > > > + def remove_attribute(self, policy_constraint_value_name): > > > > > + for attr in self.policy_constraint_values: > > > > > + if attr.name == policy_constraint_value_name: > > > > > + self.policy_constraint_values.remove(attr) > > > > > + break > > > > > + > > > > > + def get_attribute(self, policy_constraint_value_name): > > > > > + for constraint in self.policy_constraint_values: > > > > > + if constraint.name == policy_constraint_value_name: > > > > > + return constraint > > > > > + > > > > > + return None > > > > > + > > > > > @classmethod > > > > > def from_json(cls, json_value): > > > > > policy_constraint = cls() > > > > > @@ -995,31 +1058,99 @@ class ProfileClient(object): > > > > > """ > > > > > return self._modify_profile_state(profile_id, 'disable') > > > > > > > > > > + def _send_profile_request(self, profile_data, operation): > > > > > + > > > > > + if profile_data is None: > > > > > + raise ValueError("No ProfileData specified") > > > > > + > > > > > + if operation not in ['create', 'modify']: > > > > > + raise ValueError("Invalid operation specified: " + str(operation)) > > > > > + > > > > > + profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > + sort_keys=True) > > > > > + r = None > > > > > + if operation == 'create': > > > > > + r = self._post(self.profiles_url, profile_object) > > > > > + else: > > > > > + if profile_data.profile_id is None: > > > > > + raise ValueError("Profile Id is not specified.") > > > > > + url = self.profiles_url + '/' + str(profile_data.profile_id) > > > > > + r = self._put(url, profile_object) > > > > > + > > > > > + return Profile.from_json(r.json()) > > > > > + > > > > > + @pki.handle_exceptions() > > > > > def create_profile(self, profile_data): > > > > > """ > > > > > Create a new profile for the given ProfileData object. > > > > > """ > > > > > - if profile_data is None: > > > > > - raise ValueError("No ProfileData specified") > > > > > - > > > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > - sort_keys=True) > > > > > - r = self._post(self.profiles_url, profile_object) > > > > > - return Profile.from_json(r.json()) > > > > > + return self._send_profile_request(profile_data, 'create') > > > > > > > > > > + @pki.handle_exceptions() > > > > > def modify_profile(self, profile_data): > > > > > """ > > > > > Modify an existing profile. > > > > > """ > > > > > - if profile_data is None: > > > > > - raise ValueError("No ProfileData specified") > > > > > + return self._send_profile_request(profile_data, 'modify') > > > > > + > > > > > + def _send_request_in_file(self, path_to_file, data_format, operation): > > > > > + > > > > > + if path_to_file is None: > > > > > + raise ValueError("File path must be specified.") > > > > > + > > > > > + if data_format not in ['xml', 'json']: > > > > > + raise ValueError("Unsupported data type: " + str(data_format)) > > > > > + > > > > > + if operation not in ['create', 'modify']: > > > > > + raise ValueError("Invalid operation specified: " + str(operation)) > > > > > + > > > > > + data = None > > > > > + with open(path_to_file) as input_file: > > > > > + data = input_file.read() > > > > > + > > > > > + if data_format == 'xml': > > > > > + self.headers['Content-type'] = 'application/xml' > > > > > + > > > > > + r = None > > > > > + try: > > > > > + # Sending the data to the server. > > > > > + if operation == 'create': > > > > > + r = self._post(self.profiles_url, data) > > > > > + else: > > > > > + profile_id = None > > > > > + if data_format == 'xml': > > > > > + profile_id = etree.fromstring(data).get('id') > > > > > + else: > > > > > + profile_id = json.loads(data)['id'] > > > > > + > > > > > + if profile_id is None: > > > > > + raise ValueError('Profile Id is not specified.') > > > > > + url = self.profiles_url + '/' + profile_id > > > > > + r = self._put(url, data) > > > > > + finally: > > > > > + # Reset the Content-type header to json(As used by other methods). > > > > > + if data_format == 'xml': > > > > > + self.headers['Content-type'] = 'application/json' > > > > > > > > > > - url = self.profiles_url + '/' + str(profile_data.profile_id) > > > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > - sort_keys=True) > > > > > - r = self._put(url, profile_object) > > > > > return Profile.from_json(r.json()) > > > > > > > > > > + @pki.handle_exceptions() > > > > > + def create_profile_using_file_input(self, path_to_file, data_format): > > > > > + """ > > > > > + Create a new profile from a profile object stored in a file. > > > > > + Acceptable data formats - json, xml. > > > > > + """ > > > > > + return self._send_request_in_file(path_to_file, data_format, 'create') > > > > > + > > > > > + @pki.handle_exceptions() > > > > > + def modify_profile_using_file_input(self, path_to_file, data_format): > > > > > + """ > > > > > + Modify a profile from a profile object stored in a file. > > > > > + Acceptable data formats - json, xml. > > > > > + """ > > > > > + return self._send_request_in_file(path_to_file, data_format, 'modify') > > > > > + > > > > > + @pki.handle_exceptions() > > > > > def delete_profile(self, profile_id): > > > > > """ > > > > > Delete a profile. > > > > > @@ -1108,6 +1239,7 @@ def main(): > > > > > renewal=False, xml_output=False, > > > > > authorization_acl="") > > > > > > > > > > + # Adding a profile input > > > > > profile_input = ProfileInput("i1", "subjectNameInputImpl") > > > > > profile_input.add_attribute(ProfileAttribute("sn_uid")) > > > > > profile_input.add_attribute(ProfileAttribute("sn_e")) > > > > > @@ -1121,6 +1253,92 @@ def main(): > > > > > > > > > > profile_data.add_input(profile_input) > > > > > > > > > > + # Adding a profile output > > > > > + profile_output = ProfileOutput("o1", name="Certificate Output", > > > > > + class_id="certOutputImpl") > > > > > + profile_output.add_attribute(ProfileAttribute("pretty_cert")) > > > > > + profile_output.add_attribute(ProfileAttribute("b64_cert")) > > > > > + > > > > > + profile_data.add_output(profile_output) > > > > > + > > > > > + # Create a Policy set with a list of profile policies > > > > > + policy_list = [] > > > > > + > > > > > + # Creating profile policy > > > > > + policy_default = PolicyDefault("Subject Name Default", > > > > > + "userSubjectNameDefaultImpl", > > > > > + "This default populates a User-Supplied " > > > > > + "Certificate Subject Name to the request.") > > > > > + > > > > > + attr_descriptor = Descriptor(syntax="string", description="Subject Name") > > > > > + policy_attribute = ProfileAttribute("name", descriptor=attr_descriptor) > > > > > + policy_default.add_attribute(policy_attribute) > > > > > + > > > > > + policy_constraint = PolicyConstraint("Subject Name Constraint", > > > > > + "This constraint accepts the subject " > > > > > + "name that matches UID=.*", > > > > > + "subjectNameConstraintImpl") > > > > > + constraint_descriptor = Descriptor(syntax="string", > > > > > + description="Subject Name Pattern") > > > > > + policy_constraint_value = PolicyConstraintValue("pattern", > > > > > + "UID=.*", > > > > > + constraint_descriptor) > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > + > > > > > + policy_list.append(ProfilePolicy("1", policy_default, policy_constraint)) > > > > > + > > > > > + # Creating another profile policy > > > > > + # Defining the policy default > > > > > + policy_default = PolicyDefault("Validity Default", "validityDefaultImpl", > > > > > + "This default populates a Certificate " > > > > > + "Validity to the request. The default " > > > > > + "values are Range=180 in days") > > > > > + attr_descriptor = Descriptor(syntax="string", description="Not Before") > > > > > + policy_attribute = ProfileAttribute("notBefore", descriptor=attr_descriptor) > > > > > + policy_default.add_attribute(policy_attribute) > > > > > + > > > > > + attr_descriptor = Descriptor(syntax="string", description="Not After") > > > > > + policy_attribute = ProfileAttribute("notAfter", descriptor=attr_descriptor) > > > > > + policy_default.add_attribute(policy_attribute) > > > > > + > > > > > + profile_param = ProfileParameter("range", 180) > > > > > + profile_param2 = ProfileParameter("startTime", 0) > > > > > + policy_default.add_parameter(profile_param) > > > > > + policy_default.add_parameter(profile_param2) > > > > > + > > > > > + #Defining the policy constraint > > > > > + policy_constraint = PolicyConstraint("Validity Constraint", > > > > > + "This constraint rejects the validity " > > > > > + "that is not between 365 days.", > > > > > + "validityConstraintImpl") > > > > > + constraint_descriptor = Descriptor(syntax="integer", > > > > > + description="Validity Range (in days)", > > > > > + default_value=365) > > > > > + policy_constraint_value = PolicyConstraintValue("range", 365, > > > > > + constraint_descriptor) > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > + > > > > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > > > > + description="Check Not Before against" > > > > > + " current time") > > > > > + policy_constraint_value = PolicyConstraintValue("notBeforeCheck", False, > > > > > + constraint_descriptor) > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > + > > > > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > > > > + description="Check Not After against" > > > > > + " Not Before") > > > > > + policy_constraint_value = PolicyConstraintValue("notAfterCheck", False, > > > > > + constraint_descriptor) > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > + > > > > > + policy_list.append(ProfilePolicy("2", policy_default, policy_constraint)) > > > > > + > > > > > + policy_set = PolicySet("userCertSet", policy_list) > > > > > + > > > > > + profile_data.add_policy_set(policy_set) > > > > > + > > > > > + # Create a new profile > > > > > created_profile = profile_client.create_profile(profile_data) > > > > > print(created_profile) > > > > > print > > > > > @@ -1185,6 +1403,23 @@ def main(): > > > > > # pylint: disable-msg=W0703 > > > > > except Exception as e: > > > > > print str(e) > > > > > + print > > > > > + > > > > > + # Creating a profile from file > > > > > + print('Creating a profile using file input.') > > > > > + print('------------------------------------') > > > > > + original = profile_client.create_profile_using_file_input( > > > > > + '/tmp/original.xml', 'xml') > > > > > + print(original) > > > > > + print > > > > > + > > > > > + # Modifying a profile from file > > > > > + print('Modifying a profile using file input.') > > > > > + print('------------------------------------') > > > > > + modified = profile_client.modify_profile_using_file_input( > > > > > + '/tmp/modified.xml', 'xml') > > > > > + print(modified) > > > > > + print > > > > > > > > > > > > > > > if __name__ == "__main__": > > > > > -- > > > > > 1.8.5.3 > > > > > > > > > > > > > _______________________________________________ > > > > Pki-devel mailing list > > > > Pki-devel at redhat.com > > > > https://www.redhat.com/mailman/listinfo/pki-devel > > > > > > > > > > > From 6f0acdb884fa8306be8d3478a22becb301cb6d79 Mon Sep 17 00:00:00 2001 > > From: Abhishek Koneru > > Date: Thu, 19 Jun 2014 00:10:13 -0400 > > Subject: [PATCH] Added methods for providing file input for profile request. > > > > Added new methods to allow user to provide file input to perform > > operations like create profile/modify profile. > > The supported file formats a re xml and json. > > --- > > base/common/python/pki/__init__.py | 7 +- > > base/common/python/pki/profile.py | 636 +++++++++++++++++++++++++++++++------ > > 2 files changed, 541 insertions(+), 102 deletions(-) > > > > diff --git a/base/common/python/pki/__init__.py b/base/common/python/pki/__init__.py > > index e9b726cf763785b4a700ef314ff27774b13aba40..b610a9f8d9dc13dcdaf1e7cc920a204d0277d3f3 100644 > > --- a/base/common/python/pki/__init__.py > > +++ b/base/common/python/pki/__init__.py > > @@ -168,7 +168,7 @@ class PKIException(Exception, ResourceMessage): > > ret = cls(json_value['Message'], json_value['Code'], > > json_value['ClassName']) > > for attr in json_value['Attributes']['Attribute']: > > - print(str(attr)) > > + print str(attr) > > ret.add_attribute(attr["name"], attr["value"]) > > return ret > > > > @@ -299,7 +299,7 @@ class PropertyFile(object): > > def show(self): > > """ Show contents of property file.""" > > for line in self.lines: > > - print(line) > > + print line > > > > def insert_line(self, index, line): > > """ Insert line in property file """ > > @@ -403,6 +403,9 @@ class Link: > > > > @classmethod > > def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > link = cls() > > for attr in attr_list: > > setattr(link, attr, attr_list[attr]) > > diff --git a/base/common/python/pki/profile.py b/base/common/python/pki/profile.py > > index 4f08ee5ba9527855f63b2785c32f9aaddc8d1289..f8caf10e0a89773f3287899c68a9ce60da5cee0e 100644 > > --- a/base/common/python/pki/profile.py > > +++ b/base/common/python/pki/profile.py > > @@ -1,13 +1,29 @@ > > #!/usr/bin/python > > """ > > -Created on May 13,, 2014 > > + This program is free software; you can redistribute it and/or modify > > + it under the terms of the GNU General Public License as published by > > + the Free Software Foundation; version 2 of the License. > > > > - at author: akoneru > > + This program is distributed in the hope that it will be useful, > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > + GNU General Public License for more details. > > + > > + You should have received a copy of the GNU General Public License along > > + with this program; if not, write to the Free Software Foundation, Inc., > > + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. > > + > > + Copyright (C) 2014 Red Hat, Inc. > > + All rights reserved. > > + > > + @author: Abhishek Koneru > > """ > > > > import json > > +import os > > import types > > > > + > > import pki > > import pki.client as client > > import pki.account as account > > @@ -36,6 +52,9 @@ class ProfileDataInfo(object): > > > > @classmethod > > def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > profile_data_info = cls() > > profile_data_info.profile_id = attr_list['profileId'] > > profile_data_info.profile_name = attr_list['profileName'] > > @@ -59,9 +78,9 @@ class ProfileDataInfoCollection(object): > > return iter(self.profile_data_list) > > > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > ret = cls() > > - profile_data_infos = json_value['entries'] > > + profile_data_infos = attr_list['entries'] > > if not isinstance(profile_data_infos, types.ListType): > > ret.profile_data_list.append( > > ProfileDataInfo.from_json(profile_data_infos)) > > @@ -70,7 +89,7 @@ class ProfileDataInfoCollection(object): > > ret.profile_data_list.append( > > ProfileDataInfo.from_json(profile_info)) > > > > - links = json_value['Link'] > > + links = attr_list['Link'] > > if not isinstance(links, types.ListType): > > ret.links.append(pki.Link.from_json(links)) > > else: > > @@ -128,6 +147,9 @@ class Descriptor(object): > > > > @classmethod > > def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > descriptor = cls() > > for attr in attr_list: > > setattr(descriptor, attr, attr_list[attr]) > > @@ -163,6 +185,9 @@ class ProfileAttribute(object): > > > > @classmethod > > def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > attribute = cls() > > attribute.name = attr_list['name'] > > if 'Value' in attr_list: > > @@ -241,15 +266,28 @@ class ProfileInput(object): > > setattr(self, 'ConfigAttribute', value) > > > > def add_attribute(self, profile_attribute): > > + """ > > + Add a ProfileAttribute object to the attributes list. > > + """ > > + if not isinstance(profile_attribute, ProfileAttribute): > > + raise ValueError("Object passed is not a ProfileAttribute.") > > self.attributes.append(profile_attribute) > > > > def remove_attribute(self, profile_attribute_name): > > + """ > > + Remove a ProfileAttribute object with the given name from the attributes > > + list. > > + """ > > for attr in self.attributes: > > if attr.name == profile_attribute_name: > > self.attributes.remove(attr) > > break > > > > def get_attribute(self, profile_attribute_name): > > + """ > > + Returns a ProfileAttribute object for the given name. > > + None, if no match. > > + """ > > for attr in self.attributes: > > if attr.name == profile_attribute_name: > > return attr > > @@ -257,31 +295,46 @@ class ProfileInput(object): > > return None > > > > def add_config_attribute(self, profile_attribute): > > - self.attributes.append(profile_attribute) > > + """ > > + Add a ProfileAttribute object to the config_attributes list. > > + """ > > + if not isinstance(profile_attribute, ProfileAttribute): > > + raise ValueError("Object passed is not a ProfileAttribute.") > > + self.config_attributes.append(profile_attribute) > > > > def remove_config_attribute(self, config_attribute_name): > > + """ > > + Remove a ProfileAttribute object with the given name from the > > + config_attributes list. > > + """ > > for attr in self.config_attributes: > > if attr.name == config_attribute_name: > > - self.attributes.remove(attr) > > + self.config_attributes.remove(attr) > > break > > > > def get_config_attribute(self, config_attribute_name): > > - for attr in self.attributes: > > + """ > > + Returns a ProfileAttribute object with the given name. > > + None, if there is no match in the config_attributes list. > > + """ > > + for attr in self.config_attributes: > > if attr.name == config_attribute_name: > > return attr > > > > return None > > > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > profile_input = cls() > > - profile_input.profile_input_id = json_value['id'] > > - profile_input.class_id = json_value['ClassID'] > > - profile_input.name = json_value['Name'] > > - if 'Text' in json_value: > > - profile_input.text = json_value['Text'] > > + profile_input.profile_input_id = attr_list['id'] > > + profile_input.class_id = attr_list['ClassID'] > > + profile_input.name = attr_list['Name'] > > + if 'Text' in attr_list: > > + profile_input.text = attr_list['Text'] > > > > - attributes = json_value['Attribute'] > > + attributes = attr_list['Attribute'] > > if not isinstance(attributes, types.ListType): > > profile_input.attributes.append( > > ProfileAttribute.from_json(attributes)) > > @@ -290,7 +343,7 @@ class ProfileInput(object): > > profile_input.attributes.append( > > ProfileAttribute.from_json(profile_info)) > > > > - config_attributes = json_value['ConfigAttribute'] > > + config_attributes = attr_list['ConfigAttribute'] > > if not isinstance(config_attributes, types.ListType): > > profile_input.config_attributes.append( > > ProfileAttribute.from_json(config_attributes)) > > @@ -334,15 +387,28 @@ class ProfileOutput(object): > > setattr(self, 'classId', value) > > > > def add_attribute(self, profile_attribute): > > + """ > > + Add a ProfileAttribute object to the attributes list. > > + """ > > + if not isinstance(profile_attribute, ProfileAttribute): > > + raise ValueError("Object passed is not a ProfileAttribute.") > > self.attributes.append(profile_attribute) > > > > def remove_attribute(self, profile_attribute_name): > > + """ > > + Remove a ProfileAttribute object with the given name from the attributes > > + list. > > + """ > > for attr in self.attributes: > > if attr.name == profile_attribute_name: > > self.attributes.remove(attr) > > break > > > > def get_attribute(self, profile_attribute_name): > > + """ > > + Returns a ProfileAttribute object for the given name. > > + None, if no match. > > + """ > > for attr in self.attributes: > > if attr.name == profile_attribute_name: > > return attr > > @@ -350,14 +416,17 @@ class ProfileOutput(object): > > return None > > > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > profile_output = cls() > > - profile_output.profile_output_id = json_value['id'] > > - profile_output.name = json_value['name'] > > - if 'text' in json_value: > > - profile_output.text = json_value['text'] > > - profile_output.class_id = json_value['classId'] > > - attributes = json_value['attributes'] > > + profile_output.profile_output_id = attr_list['id'] > > + profile_output.name = attr_list['name'] > > + if 'text' in attr_list: > > + profile_output.text = attr_list['text'] > > + profile_output.class_id = attr_list['classId'] > > + attributes = attr_list['attributes'] > > if not isinstance(attributes, types.ListType): > > profile_output.attributes.append( > > ProfileAttribute.from_json(attributes)) > > @@ -375,6 +444,9 @@ class ProfileParameter(object): > > > > @classmethod > > def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > param = cls() > > for attr in attr_list: > > setattr(param, attr, attr_list[attr]) > > @@ -433,17 +505,76 @@ class PolicyDefault(object): > > def policy_params(self, value): > > setattr(self, 'params', value) > > > > + def add_attribute(self, policy_attribute): > > + """ > > + Add a policy attribute to the attribute list. > > + @param policy_attribute - A ProfileAttribute object > > + """ > > + if not isinstance(policy_attribute, ProfileAttribute): > > + raise ValueError("Object passed is not a ProfileAttribute.") > > + self.policy_attributes.append(policy_attribute) > > + > > + def remove_attribute(self, policy_attribute_name): > > + """ > > + Remove a policy attribute with the given name from the attributes list. > > + """ > > + for attr in self.policy_attributes: > > + if attr.name == policy_attribute_name: > > + self.policy_attributes.remove(attr) > > + break > > + > > + def get_attribute(self, policy_attribute_name): > > + """ > > + Fetch the policy attribute with the given name from the attributes list. > > + """ > > + for attr in self.policy_attributes: > > + if attr.name == policy_attribute_name: > > + return attr > > + > > + return None > > + > > + def add_parameter(self, policy_parameter): > > + """ > > + Add a profile parameter to the parameters list. > > + @param policy_parameter - A ProfileParameter object. > > + """ > > + if not isinstance(policy_parameter, ProfileParameter): > > + raise ValueError("Object passed is not a ProfileParameter.") > > + self.policy_params.append(policy_parameter) > > + > > + def remove_parameter(self, profile_parameter_name): > > + """ > > + Remove a profile parameter with the given name from the parameters list. > > + """ > > + for param in self.policy_params: > > + if param.name == profile_parameter_name: > > + self.policy_params.remove(param) > > + break > > + > > + def get_parameter(self, profile_parameter_name): > > + """ > > + Fetch a profile parameter with the given name from the parameters list. > > + """ > > + for param in self.policy_params: > > + if param.name == profile_parameter_name: > > + return param > > + > > + return None > > + > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > policy_def = cls() > > - if 'id' in json_value: > > - policy_def.name = json_value['id'] > > - if 'classId' in json_value: > > - policy_def.class_id = json_value['classId'] > > - if 'description' in json_value: > > - policy_def.description = json_value['description'] > > - if 'policyAttribute' in json_value: > > - attributes = json_value['policyAttribute'] > > + if 'id' in attr_list: > > + policy_def.name = attr_list['id'] > > + if 'classId' in attr_list: > > + policy_def.class_id = attr_list['classId'] > > + if 'description' in attr_list: > > + policy_def.description = attr_list['description'] > > + if 'policyAttribute' in attr_list: > > + attributes = attr_list['policyAttribute'] > > if not isinstance(attributes, types.ListType): > > policy_def.policy_attributes.append( > > ProfileAttribute.from_json(attributes)) > > @@ -452,8 +583,8 @@ class PolicyDefault(object): > > policy_def.policy_attributes.append( > > ProfileAttribute.from_json(attr)) > > > > - if 'params' in json_value: > > - params = json_value['params'] > > + if 'params' in attr_list: > > + params = attr_list['params'] > > if not isinstance(params, types.ListType): > > policy_def.policy_params.append( > > ProfileParameter.from_json(params)) > > @@ -480,13 +611,16 @@ class PolicyConstraintValue(object): > > setattr(self, 'id', value) > > > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > ret = cls() > > > > - ret.name = json_value['id'] > > - ret.value = json_value['value'] > > - if 'descriptor' in json_value: > > - ret.descriptor = Descriptor.from_json(json_value['descriptor']) > > + ret.name = attr_list['id'] > > + ret.value = attr_list['value'] > > + if 'descriptor' in attr_list: > > + ret.descriptor = Descriptor.from_json(attr_list['descriptor']) > > > > return ret > > > > @@ -531,17 +665,49 @@ class PolicyConstraint(object): > > def policy_constraint_values(self, value): > > setattr(self, 'constraint', value) > > > > + def add_constraint_value(self, policy_constraint_value): > > + """ > > + Add a ProfileConstraintValue to the policy_constraint_values list. > > + """ > > + if not isinstance(policy_constraint_value, PolicyConstraintValue): > > + raise ValueError("Object passed not of type PolicyConstraintValue") > > + self.policy_constraint_values.append(policy_constraint_value) > > + > > + def remove_constraint_value(self, policy_constraint_value_name): > > + """ > > + Removes a PolicyConstraintValue with the given name form the > > + policy_constraint_values list. > > + """ > > + for attr in self.policy_constraint_values: > > + if attr.name == policy_constraint_value_name: > > + self.policy_constraint_values.remove(attr) > > + break > > + > > + def get_constraint_value(self, policy_constraint_value_name): > > + """ > > + Returns a PolicyConstraintValue object with the given name. > > + None, if there is no match. > > + """ > > + for constraint in self.policy_constraint_values: > > + if constraint.name == policy_constraint_value_name: > > + return constraint > > + > > + return None > > + > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > policy_constraint = cls() > > - if 'id' in json_value: > > - policy_constraint.name = json_value['id'] > > - if 'description' in json_value: > > - policy_constraint.description = json_value['description'] > > - if 'classId' in json_value: > > - policy_constraint.class_id = json_value['classId'] > > - if 'constraint' in json_value: > > - constraints = json_value['constraint'] > > + if 'id' in attr_list: > > + policy_constraint.name = attr_list['id'] > > + if 'description' in attr_list: > > + policy_constraint.description = attr_list['description'] > > + if 'classId' in attr_list: > > + policy_constraint.class_id = attr_list['classId'] > > + if 'constraint' in attr_list: > > + constraints = attr_list['constraint'] > > if not isinstance(constraints, types.ListType): > > policy_constraint.policy_constraint_values.append( > > PolicyConstraintValue.from_json(constraints)) > > @@ -592,9 +758,12 @@ class ProfilePolicy(object): > > setattr(self, 'constraint', value) > > > > @classmethod > > - def from_json(cls, json_value): > > - return cls(json_value['id'], PolicyDefault.from_json(json_value['def']), > > - PolicyConstraint.from_json(json_value['constraint'])) > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > + return cls(attr_list['id'], PolicyDefault.from_json(attr_list['def']), > > + PolicyConstraint.from_json(attr_list['constraint'])) > > > > > > class ProfilePolicySet(object): > > @@ -607,6 +776,9 @@ class ProfilePolicySet(object): > > > > @classmethod > > def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > policy_set = cls() > > > > policies = attr_list['policies'] > > @@ -649,19 +821,40 @@ class PolicySet(object): > > setattr(self, 'value', value) > > > > def add_policy(self, profile_policy): > > + """ > > + Add a ProfilePolicy object to the policy_list > > + """ > > + if not isinstance(profile_policy, ProfilePolicy): > > + raise ValueError("Object passed is not a ProfilePolicy.") > > self.policy_list.append(profile_policy) > > > > def remove_policy(self, policy_id): > > + """ > > + Removes a ProfilePolicy with the given ID from the PolicySet. > > + """ > > for policy in self.policy_list: > > if policy.policy_id == policy_id: > > - self.policy_list.pop(policy) > > + self.policy_list.remove(policy) > > + break > > + > > + def get_policy(self, policy_id): > > + """ > > + Returns a ProfilePolicy object with the given profile id. > > + """ > > + for policy in self.policy_list: > > + if policy.policy_id == policy_id: > > + return policy > > + return None > > > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > policy_set = cls() > > > > - policy_set.name = json_value['id'] > > - policies = json_value['value'] > > + policy_set.name = attr_list['id'] > > + policies = attr_list['value'] > > if not isinstance(policies, types.ListType): > > policy_set.policy_list.append(ProfilePolicy.from_json(policies)) > > else: > > @@ -694,17 +887,39 @@ class PolicySetList(object): > > setattr(self, 'PolicySet', value) > > > > def add_policy_set(self, policy_set): > > + """ > > + Add a PolicySet object to the policy_sets list. > > + """ > > + if not isinstance(policy_set, PolicySet): > > + raise ValueError("Object passed is not a PolicySet.") > > self.policy_sets.append(policy_set) > > > > def remove_policy_set(self, policy_set_name): > > + """ > > + Remove a PolicySet object with the given name from the policy_sets list. > > + """ > > for policy_set in self.policy_sets: > > if policy_set.name == policy_set_name: > > - self.policy_sets.pop(policy_set) > > + self.policy_sets.remove(policy_set) > > + break > > + > > + def get_policy_set(self, policy_set_name): > > + """ > > + Fetch the PolicySet object for the given name. > > + Returns None, if not found. > > + """ > > + for policy_set in self.policy_sets: > > + if policy_set.name == policy_set_name: > > + return policy_set > > + return None > > > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > + if attr_list is None: > > + return None > > + > > policy_set_list = cls() > > - policy_sets = json_value['PolicySet'] > > + policy_sets = attr_list['PolicySet'] > > if not isinstance(policy_sets, types.ListType): > > policy_set_list.policy_sets.append(PolicySet.from_json(policy_sets)) > > else: > > @@ -824,51 +1039,101 @@ class Profile(object): > > setattr(self, 'PolicySets', value) > > > > def add_input(self, profile_input): > > + """ > > + Add a ProfileInput object to the inputs list of the Profile. > > + """ > > + if not isinstance(profile_input, ProfileInput): > > + raise ValueError("Object passed is not a PolicyInput.") > > if profile_input is None: > > raise ValueError("No ProfileInput object provided.") > > self.inputs.append(profile_input) > > > > def remove_input(self, profile_input_id): > > + """ > > + Remove a ProfileInput from the inputs list of the Profile. > > + """ > > for profile_input in self.inputs: > > if profile_input_id == profile_input.profile_input_id: > > - self.inputs.pop(profile_input) > > + self.inputs.remove(profile_input) > > + break > > + > > + def get_input(self, profile_input_id): > > + """ > > + Fetches a ProfileInput with the given ProfileInput id. > > + Returns None, if there is no matching input. > > + """ > > + for profile_input in self.inputs: > > + if profile_input_id == profile_input.profile_input_id: > > + return profile_input > > + return None > > > > def add_output(self, profile_output): > > + """ > > + Add a ProfileOutput object to the outputs list of the Profile. > > + """ > > + if not isinstance(profile_output, ProfileOutput): > > + raise ValueError("Object passed is not a PolicyOutput.") > > if profile_output is None: > > raise ValueError("No ProfileOutput object provided.") > > self.outputs.append(profile_output) > > > > def remove_output(self, profile_output_id): > > + """ > > + Remove a ProfileOutput from the outputs list of the Profile. > > + """ > > for profile_output in self.outputs: > > if profile_output_id == profile_output.profile_output_id: > > - self.inputs.pop(profile_output) > > + self.inputs.remove(profile_output) > > + > > + def get_output(self, profile_output_id): > > + """ > > + Fetches a ProfileOutput with the given ProfileOutput id. > > + Returns None, if there is no matching output. > > + """ > > + for profile_input in self.inputs: > > + if profile_output_id == profile_input.profile_input_id: > > + return profile_input > > + return None > > > > def add_policy_set(self, policy_set): > > + """ > > + Add a PolicySet object to the policy_sets list of the Profile. > > + """ > > if policy_set is None: > > raise ValueError("No PolicySet object provided.") > > self.policy_set_list.add_policy_set(policy_set) > > > > def remove_policy_set(self, policy_set_name): > > + """ > > + Remove a PolicySet from the policy_sets list of the Profile. > > + """ > > self.policy_set_list.remove_policy_set(policy_set_name) > > > > + def get_policy_set(self, policy_set_name): > > + """ > > + Fetches a ProfileInput with the given ProfileInput id. > > + Returns None, if there is no matching input. > > + """ > > + return self.policy_set_list.get_policy_set(policy_set_name) > > + > > @classmethod > > - def from_json(cls, json_value): > > + def from_json(cls, attr_list): > > profile_data = cls() > > - profile_data.profile_id = json_value['id'] > > - profile_data.class_id = json_value['classId'] > > - profile_data.name = json_value['name'] > > - profile_data.description = json_value['description'] > > - profile_data.enabled = json_value['enabled'] > > - profile_data.visible = json_value['visible'] > > - if 'enabledBy' in json_value: > > - profile_data.enabled_by = json_value['enabledBy'] > > - if 'authenticatorId' in json_value: > > - profile_data.authenticator_id = json_value['authenticatorId'] > > - profile_data.authorization_acl = json_value['authzAcl'] > > - profile_data.renewal = json_value['renewal'] > > - profile_data.xml_output = json_value['xmlOutput'] > > + profile_data.profile_id = attr_list['id'] > > + profile_data.class_id = attr_list['classId'] > > + profile_data.name = attr_list['name'] > > + profile_data.description = attr_list['description'] > > + profile_data.enabled = attr_list['enabled'] > > + profile_data.visible = attr_list['visible'] > > + if 'enabledBy' in attr_list: > > + profile_data.enabled_by = attr_list['enabledBy'] > > + if 'authenticatorId' in attr_list: > > + profile_data.authenticator_id = attr_list['authenticatorId'] > > + profile_data.authorization_acl = attr_list['authzAcl'] > > + profile_data.renewal = attr_list['renewal'] > > + profile_data.xml_output = attr_list['xmlOutput'] > > > > - profile_inputs = json_value['Input'] > > + profile_inputs = attr_list['Input'] > > if not isinstance(profile_inputs, types.ListType): > > profile_data.inputs.append(ProfileInput.from_json(profile_inputs)) > > else: > > @@ -876,7 +1141,7 @@ class Profile(object): > > profile_data.inputs.append( > > ProfileInput.from_json(profile_input)) > > > > - profile_outputs = json_value['Output'] > > + profile_outputs = attr_list['Output'] > > if not isinstance(profile_outputs, types.ListType): > > profile_data.outputs.append( > > ProfileOutput.from_json(profile_outputs)) > > @@ -886,9 +1151,9 @@ class Profile(object): > > ProfileOutput.from_json(profile_output)) > > > > profile_data.policy_set_list = \ > > - PolicySetList.from_json(json_value['PolicySets']) > > + PolicySetList.from_json(attr_list['PolicySets']) > > > > - profile_data.link = pki.Link.from_json(json_value['link']) > > + profile_data.link = pki.Link.from_json(attr_list['link']) > > > > return profile_data > > > > @@ -904,6 +1169,20 @@ class Profile(object): > > } > > return str(attributes) > > > > + @staticmethod > > + def get_profile_data_from_file(path_to_file): > > + """ > > + Reads the file for the serialized Profile object. > > + Currently supports only data format in json. > > + """ > > + if path_to_file is None: > > + raise ValueError("File path must be specified.") > > + with open(path_to_file) as input_file: > > + data = input_file.read() > > + if data is not None: > > + return Profile.from_json(json.loads(data)) > > + return None > > + > > > > class ProfileClient(object): > > """ > > @@ -995,34 +1274,67 @@ class ProfileClient(object): > > """ > > return self._modify_profile_state(profile_id, 'disable') > > > > - def create_profile(self, profile_data): > > - """ > > - Create a new profile for the given ProfileData object. > > - """ > > + def _send_profile_create(self, profile_data): > > + > > if profile_data is None: > > raise ValueError("No ProfileData specified") > > > > profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > sort_keys=True) > > + > > r = self._post(self.profiles_url, profile_object) > > + > > return Profile.from_json(r.json()) > > > > - def modify_profile(self, profile_data): > > - """ > > - Modify an existing profile. > > - """ > > + def _send_profile_modify(self, profile_data): > > if profile_data is None: > > raise ValueError("No ProfileData specified") > > > > + profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > + sort_keys=True) > > + if profile_data.profile_id is None: > > + raise ValueError("Profile Id is not specified.") > > url = self.profiles_url + '/' + str(profile_data.profile_id) > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > - sort_keys=True) > > r = self._put(url, profile_object) > > + > > return Profile.from_json(r.json()) > > > > + @pki.handle_exceptions() > > + def create_profile(self, profile_data): > > + """ > > + Create a new profile for the given Profile object. > > + """ > > + return self._send_profile_create(profile_data) > > + > > + @pki.handle_exceptions() > > + def modify_profile(self, profile_data): > > + """ > > + Modify an existing profile with the given Profile object. > > + """ > > + return self._send_profile_modify(profile_data) > > + > > + def create_profile_from_file(self, path_to_file): > > + """ > > + Reads the file for the serialized Profile object. > > + Performs the profile create operation. > > + Currently supports only data format in json. > > + """ > > + profile_data = Profile.get_profile_data_from_file(path_to_file) > > + return self._send_profile_create(profile_data) > > + > > + def modify_profile_from_file(self, path_to_file): > > + """ > > + Reads the file for the serialized Profile object. > > + Performs the profile modify operation. > > + Currently supports only data format in json. > > + """ > > + profile_data = Profile.get_profile_data_from_file(path_to_file) > > + return self._send_profile_modify(profile_data) > > + > > + @pki.handle_exceptions() > > def delete_profile(self, profile_id): > > """ > > - Delete a profile. > > + Delete a profile with the given Profile Id. > > """ > > if profile_id is None: > > raise ValueError("Profile Id must be specified.") > > @@ -1058,6 +1370,11 @@ def main(): > > #Initialize the ProfileClient class > > profile_client = ProfileClient(connection) > > > > + # Folder to store the files generated during test > > + file_path = '/tmp/profile_client_test/' > > + if not os.path.exists(file_path): > > + os.makedirs(file_path) > > + > > #Fetching a list of profiles > > profile_data_infos = profile_client.list_profiles() > > print('List of profiles:') > > @@ -1102,12 +1419,14 @@ def main(): > > print('-----------------------') > > > > profile_data = Profile(name="My Sample User Cert Enrollment", > > - profile_id="MySampleCert", class_id="caEnrollImpl", > > + profile_id="MySampleProfile", > > + class_id="caEnrollImpl", > > description="Example User Cert Enroll Impl", > > enabled_by='admin', enabled=False, visible=False, > > renewal=False, xml_output=False, > > authorization_acl="") > > > > + # Adding a profile input > > profile_input = ProfileInput("i1", "subjectNameInputImpl") > > profile_input.add_attribute(ProfileAttribute("sn_uid")) > > profile_input.add_attribute(ProfileAttribute("sn_e")) > > @@ -1121,6 +1440,97 @@ def main(): > > > > profile_data.add_input(profile_input) > > > > + # Adding a profile output > > + profile_output = ProfileOutput("o1", name="Certificate Output", > > + class_id="certOutputImpl") > > + profile_output.add_attribute(ProfileAttribute("pretty_cert")) > > + profile_output.add_attribute(ProfileAttribute("b64_cert")) > > + > > + profile_data.add_output(profile_output) > > + > > + # Create a Policy set with a list of profile policies > > + policy_list = [] > > + > > + # Creating profile policy > > + policy_default = PolicyDefault("Subject Name Default", > > + "userSubjectNameDefaultImpl", > > + "This default populates a User-Supplied " > > + "Certificate Subject Name to the request.") > > + > > + attr_descriptor = Descriptor(syntax="string", description="Subject Name") > > + policy_attribute = ProfileAttribute("name", descriptor=attr_descriptor) > > + policy_default.add_attribute(policy_attribute) > > + > > + policy_constraint = PolicyConstraint("Subject Name Constraint", > > + "This constraint accepts the subject " > > + "name that matches UID=.*", > > + "subjectNameConstraintImpl") > > + constraint_descriptor = Descriptor(syntax="string", > > + description="Subject Name Pattern") > > + policy_constraint_value = PolicyConstraintValue("pattern", > > + "UID=.*", > > + constraint_descriptor) > > + policy_constraint.add_constraint_value(policy_constraint_value) > > + > > + policy_list.append(ProfilePolicy("1", policy_default, policy_constraint)) > > + > > + # Creating another profile policy > > + # Defining the policy default > > + policy_default = PolicyDefault("Validity Default", "validityDefaultImpl", > > + "This default populates a Certificate " > > + "Validity to the request. The default " > > + "values are Range=180 in days") > > + attr_descriptor = Descriptor(syntax="string", description="Not Before") > > + policy_attribute = ProfileAttribute("notBefore", descriptor=attr_descriptor) > > + policy_default.add_attribute(policy_attribute) > > + > > + attr_descriptor = Descriptor(syntax="string", description="Not After") > > + policy_attribute = ProfileAttribute("notAfter", descriptor=attr_descriptor) > > + policy_default.add_attribute(policy_attribute) > > + > > + profile_param = ProfileParameter("range", 180) > > + profile_param2 = ProfileParameter("startTime", 0) > > + policy_default.add_parameter(profile_param) > > + policy_default.add_parameter(profile_param2) > > + > > + #Defining the policy constraint > > + policy_constraint = PolicyConstraint("Validity Constraint", > > + "This constraint rejects the validity " > > + "that is not between 365 days.", > > + "validityConstraintImpl") > > + constraint_descriptor = Descriptor(syntax="integer", > > + description="Validity Range (in days)", > > + default_value=365) > > + policy_constraint_value = PolicyConstraintValue("range", 365, > > + constraint_descriptor) > > + policy_constraint.add_constraint_value(policy_constraint_value) > > + > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > + description="Check Not Before against" > > + " current time") > > + policy_constraint_value = PolicyConstraintValue("notBeforeCheck", False, > > + constraint_descriptor) > > + policy_constraint.add_constraint_value(policy_constraint_value) > > + > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > + description="Check Not After against" > > + " Not Before") > > + policy_constraint_value = PolicyConstraintValue("notAfterCheck", False, > > + constraint_descriptor) > > + policy_constraint.add_constraint_value(policy_constraint_value) > > + > > + policy_list.append(ProfilePolicy("2", policy_default, policy_constraint)) > > + > > + policy_set = PolicySet("userCertSet", policy_list) > > + > > + profile_data.add_policy_set(policy_set) > > + > > + # Write the profile data object to a file for testing a file input > > + with open(file_path+'/original.json', 'w') as output_file: > > + output_file.write(json.dumps(profile_data, > > + cls=encoder.CustomTypeEncoder, > > + sort_keys=True, indent=4)) > > + # Create a new profile > > created_profile = profile_client.create_profile(profile_data) > > print(created_profile) > > print > > @@ -1131,7 +1541,7 @@ def main(): > > > > try: > > profile_data = Profile(name="My Sample User Cert Enrollment", > > - profile_id="MySampleCert", > > + profile_id="MySampleProfile", > > class_id="caEnrollImpl", > > description="Example User Cert Enroll Impl", > > enabled_by='admin', enabled=False, visible=False, > > @@ -1151,40 +1561,66 @@ def main(): > > profile_data.add_input(profile_input) > > profile_client.create_profile(profile_data) > > # pylint: disable-msg=W0703 > > - except Exception as e: > > - print str(e) > > + except pki.BadRequestException as e: > > + print 'MySampleProfile ' + str(e) > > print > > > > # Modify the above created profile > > - print('Modifying the profile MySampleCert.') > > + print('Modifying the profile MySampleProfile.') > > print('-----------------------------------') > > > > - fetch = profile_client.get_profile('MySampleCert') > > + fetch = profile_client.get_profile('MySampleProfile') > > profile_input2 = ProfileInput("i2", "keyGenInputImpl") > > profile_input2.add_attribute(ProfileAttribute("cert_request_type")) > > profile_input2.add_attribute(ProfileAttribute("cert_request")) > > fetch.add_input(profile_input2) > > > > fetch.name += " (Modified)" > > + with open(file_path+'modified.json', 'w') as output_file: > > + output_file.write(json.dumps(fetch, cls=encoder.CustomTypeEncoder, > > + sort_keys=True, indent=4)) > > modified_profile = profile_client.modify_profile(fetch) > > print(modified_profile) > > print > > > > # Delete a profile > > - print ("Deleting the profile MySampleCert.") > > + print ("Deleting the profile MySampleProfile.") > > print ("----------------------------------") > > - profile_client.delete_profile('MySampleCert') > > - print ("Deleted profile MySampleCert.") > > + profile_client.delete_profile('MySampleProfile') > > + print ("Deleted profile MySampleProfile.") > > print > > > > # Testing deletion of a profile > > print('Test profile deletion.') > > print('----------------------') > > try: > > - profile_client.get_profile('MySampleCert') > > + profile_client.get_profile('MySampleProfile') > > # pylint: disable-msg=W0703 > > - except Exception as e: > > + except pki.ProfileNotFoundException as e: > > print str(e) > > + print > > + > > + # Creating a profile from file > > + print('Creating a profile using file input.') > > + print('------------------------------------') > > + original = profile_client.create_profile_from_file( > > + file_path + 'original.json') > > + print(original) > > + print > > + > > + # Modifying a profile from file > > + print('Modifying a profile using file input.') > > + print('------------------------------------') > > + modified = profile_client.modify_profile_from_file( > > + file_path + 'modified.json') > > + print(modified) > > + print > > + > > + # Test clean up > > + profile_client.delete_profile('MySampleProfile') > > + os.remove(file_path+'original.json') > > + os.remove(file_path+'modified.json') > > + os.removedirs(file_path) > > > > > > if __name__ == "__main__": > > -- > > 1.8.5.3 > > > From akoneru at redhat.com Tue Jul 1 14:53:46 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Tue, 01 Jul 2014 10:53:46 -0400 Subject: [Pki-devel] [PATCH] 98-3 Fixes for comments on patch 98-2 In-Reply-To: <1404219166.5622.5.camel@localhost.localdomain> References: <1403186323.2669.3.camel@akoneru.redhat.com> <20140620020243.GP11349@dhcp-40-8.bne.redhat.com> <1403645545.3075.3.camel@akoneru.redhat.com> <20140625064907.GC3122@dhcp-40-8.bne.redhat.com> <1403714870.6457.35.camel@localhost.localdomain> <1403888893.6624.1.camel@akoneru.redhat.com> <20140630032959.GL2417@dhcp-40-8.bne.redhat.com> <1404219166.5622.5.camel@localhost.localdomain> Message-ID: <1404226426.23998.1.camel@akoneru.redhat.com> Thanks Fraser and Ade. Pushed the patches 97 and 98-3 to master. Closed the ProfileClient ticket. #1022 Opened the following ticket on Ade's suggestion - https://fedorahosted.org/pki/ticket/1060 --Abhishek On Tue, 2014-07-01 at 20:52 +0800, Ade Lee wrote: > On Mon, 2014-06-30 at 13:29 +1000, Fraser Tweedale wrote: > > On Fri, Jun 27, 2014 at 01:08:13PM -0400, Abhishek Koneru wrote: > > > Please review the patch with fixes for Ade's comments on patch 98-2. > > > Addressed all the comments except 8 and 10. which will be done > > > separately. > > > > > > > Patch does seem to address those issues and __main__ still works > > nicely (including removing the added profile at end now - thanks). > > > > Switching to serialising the profile as JSON now means that the > > upload-xml-profile codepath is no longer exercised by __main__. Not > > a blocker I think, but if it's easy to write out a profile as XML so > > that codepath is exercised, that would be a good idea. > > ACK. Please open a ticket to add the missing XML tests/functions. We > can triage accordingly. I would suggest a milestone of 10.3. > > Ade > > > > > ACK modulo preceding comment. > > > > > > > Please apply patch 97 before applying 98-3 > > > --Abhishek > > > > > > On Thu, 2014-06-26 at 00:47 +0800, Ade Lee wrote: > > > > In profile.py: > > > > > > > > 1. Usually, we set @author and so on after the copyright notice. > > > > > > > > 2. In PolicyDefault, you have methods to add/remove/get attributes and > > > > parameters, as in: > > > > def add_attribute(self, profile_attribute): > > > > self.policy_attributes.append(profile_attribute) > > > > > > > > Change profile_attribute to policy_attribute etc. You should indicate in > > > > a docstring that the expected type for the policy_attribute etc. is a > > > > ProfileAttribute. > > > > > > > > 3. I'm not a big fan of your _send_profile_request() and > > > > _send_request_in_file() methods in that they mix create and update > > > > operations. That might make sense if the operations had a lot of common > > > > code and were doing the same HTTP operation, but otherwise is just > > > > confusing. > > > > > > > > A better way to split this up might be to define: > > > > > > > > def _send_profile_create(self, profile_data) > > > > > > > > def _send_profile_modify(self, profile_id, profile_data) > > > > > > > > @staticmethod > > > > def _get_profile_data_from_file(path_to_file, file_format) > > > > > > > > and then call these as needed. > > > > > > > > So for example, create_profile_using_file_input(self, path_to_file, > > > > file_format) -> which I would rename to create_profile_from_file(..) -> > > > > would call: > > > > profile_data = _get_profile_data_from_file(path_to_file, file_format) > > > > _send_profile_create(profile_data) > > > > > > > > 4. In the test where you add a duplicate profile, you should check for > > > > the specific exception thrown -- is there a specific exception? > > > > > > > > 5. Change the profileID in the test to MySampleProfile instead of > > > > MySampleCert. > > > > > > > > 6. In the test, we are testing file input using original.xml and > > > > modified.xml. Where do these files come from? It would be better if > > > > these files were generated as part of the previous tests - which means > > > > you need a function to save files to xml/json. You should then test xml > > > > and json. > > > > > > > > 7. The policy stuff gets confusing fast, so the example really helps. > > > > We need more function docstrings though that specify when inputs/outputs > > > > the functions have - and in particular the class of each input. > > > > > > > > 8. We also need more api documentation on the wiki as we have done in > > > > the past. This is not part of this patch of course. > > > > > > > > 9. There are Pycharm errors about unused variables. Most of these will > > > > be addressed by the refactoring I mention in point 3 above. > > > > > > > > 10. profile.py - which is all new code generates probably close to 200 > > > > new messages in pylint. We really should fix these. To keep things > > > > clear, you should do this in a separate patch. > > > > > > > > > > > > > > > > On Wed, 2014-06-25 at 16:49 +1000, Fraser Tweedale wrote: > > > > > On Tue, Jun 24, 2014 at 05:32:25PM -0400, Abhishek Koneru wrote: > > > > > > Please review the patch with fixes for comments on patch 98. > > > > > > Issues addressed: > > > > > > 1. Added a try catch block for issue 1(as noted below) caught by > > > > > > ftweedal(nice catch!). > > > > > > 2. Made the data_format mandatory for file_input methods. > > > > > > 3. Added code to create a profile with PolicyOutput and PolicySets. > > > > > > > > > > > > -- Abhishek > > > > > > Please apply patch 97 before applying this patch. > > > > > > > > > > > > > > > > ACK > > > > > > > > > > > On Fri, 2014-06-20 at 12:02 +1000, Fraser Tweedale wrote: > > > > > > > On Thu, Jun 19, 2014 at 09:58:43AM -0400, Abhishek Koneru wrote: > > > > > > > > Please review the attached patch which adds methods that allow users to > > > > > > > > pass the profile data in a file. > > > > > > > > > > > > > > > > Also attached two sample xml files, original.xml and modified.xml. > > > > > > > > Place them in /tmp before running the profile.py module. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Abhishek > > > > > > > > > > > > > > > > > > > > > > Hi Abhishek, > > > > > > > > > > > > > > Patch applied and __main__ example run. Seems to do what it says on > > > > > > > the tin; the usual "haven't used it in anger" caveat ^_^. Review > > > > > > > comments inline below. > > > > > > > > > > > > > > Fraser > > > > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > > > def modify_profile(self, profile_data): > > > > > > > > """ > > > > > > > > Modify an existing profile. > > > > > > > > """ > > > > > > > > - if profile_data is None: > > > > > > > > - raise ValueError("No ProfileData specified") > > > > > > > > + return self._send_profile_request(profile_data, 'modify') > > > > > > > > + > > > > > > > > + def _send_request_in_file(self, path_to_file, data_format, operation): > > > > > > > > + > > > > > > > > + if path_to_file is None: > > > > > > > > + raise ValueError("File path must be specified.") > > > > > > > > + > > > > > > > > + if data_format not in ['xml', 'json']: > > > > > > > > + raise ValueError("Unsupported data type: " + str(data_format)) > > > > > > > > + > > > > > > > > + if operation not in ['create', 'modify']: > > > > > > > > + raise ValueError("Invalid operation specified: " + str(operation)) > > > > > > > > + > > > > > > > > + data = None > > > > > > > > + with open(path_to_file) as input_file: > > > > > > > > + data = input_file.read() > > > > > > > > + > > > > > > > > + if data_format == 'xml': > > > > > > > > + self.headers['Content-type'] = 'application/xml' > > > > > > > > > > > > > > 1) > > > > > > > > > > > > > > Overwriting self.headers['Content-type'] is problematic. For > > > > > > > example, if the data cannot be parsed or lacks an 'id' key, an > > > > > > > exception will be raised and the ProfileClient will be stuck with > > > > > > > the wrong Content-Type header. > > > > > > > > > > > > > > Possible solutions: > > > > > > > > > > > > > > - use try/finally or a context manager to ensure the header gets > > > > > > > reset to 'application/json' even if an exception is raised. > > > > > > > - Modify the _put and _post methods to include an optional argument > > > > > > > to override the content-type header. To avoid special-cases too > > > > > > > many things, this arg could even be a dict that can be merged with > > > > > > > the default headers, e.g.: > > > > > > > > > > > > > > def _post(self, url, payload=None, headers=None): > > > > > > > self.account_client.login() > > > > > > > headers = dict(self.headers, **(headers or {})) > > > > > > > r = self.connection.post(url, payload, headers, query_params) > > > > > > > ... > > > > > > > > > > > > > > Then callers can supply custom headers for a single request > > > > > > > without potentially affecting other requests. > > > > > > > > > > > > > > > + > > > > > > > > + r = None > > > > > > > > + # Sending the data to the server. > > > > > > > > + if operation == 'create': > > > > > > > > + r = self._post(self.profiles_url, data) > > > > > > > > + else: > > > > > > > > + profile_id = None > > > > > > > > + if data_format == 'xml': > > > > > > > > + profile_id = etree.fromstring(data).get('id') > > > > > > > > + else: > > > > > > > > + profile_id = json.loads(data)['id'] > > > > > > > > + > > > > > > > > + if profile_id is None: > > > > > > > > + raise ValueError('Profile Id is not specified.') > > > > > > > > + url = self.profiles_url + '/' + profile_id > > > > > > > > + r = self._put(url, data) > > > > > > > > + > > > > > > > > + # Reset the Content-type header to json(As used by other methods). > > > > > > > > + if data_format == 'xml': > > > > > > > > + self.headers['Content-type'] = 'application/json' > > > > > > > > > > > > > > > > - url = self.profiles_url + '/' + str(profile_data.profile_id) > > > > > > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > > > > - sort_keys=True) > > > > > > > > - r = self._put(url, profile_object) > > > > > > > > return Profile.from_json(r.json()) > > > > > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > > > + def create_profile_using_file_input(self, path_to_file, data_format="xml"): > > > > > > > > + """ > > > > > > > > + Create a new profile from a profile object stored in a file. > > > > > > > > + Acceptable data formats - json, xml. > > > > > > > > + """ > > > > > > > > + return self._send_request_in_file(path_to_file, data_format, 'create') > > > > > > > > + > > > > > > > > > > > > > > 2) > > > > > > > > > > > > > > Default ``data_format="xml"`` makes it too easy for people to misuse > > > > > > > the API, e.g. a path to a JSON file, but no ``data_format`` kwarg > > > > > > > given, resulting in default "xml" being used. I suggest either > > > > > > > making it a compulsory argument, or making its default ``None`` and, > > > > > > > if no explicit ``data_format`` arg given, using the file extension. > > > > > > > > > > > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > > > + def modify_profile_using_file_input(self, path_to_file, data_format="xml"): > > > > > > > > + """ > > > > > > > > + Modify a profile from a profile object stored in a file. > > > > > > > > + Acceptable data formats - json, xml. > > > > > > > > + """ > > > > > > > > + return self._send_request_in_file(path_to_file, data_format, 'modify') > > > > > > > > + > > > > > > > > + @pki.handle_exceptions() > > > > > > > > def delete_profile(self, profile_id): > > > > > > > > """ > > > > > > > > Delete a profile. > > > > > > > > @@ -1185,6 +1253,23 @@ def main(): > > > > > > > > # pylint: disable-msg=W0703 > > > > > > > > except Exception as e: > > > > > > > > print str(e) > > > > > > > > + print > > > > > > > > + > > > > > > > > + # Creating a profile from file > > > > > > > > + print('Creating a profile using file input.') > > > > > > > > + print('------------------------------------') > > > > > > > > + original = profile_client.create_profile_using_file_input( > > > > > > > > + '/tmp/original.xml') > > > > > > > > + print(original) > > > > > > > > + print > > > > > > > > + > > > > > > > > + # Modifying a profile from file > > > > > > > > + print('Modifying a profile using file input.') > > > > > > > > + print('------------------------------------') > > > > > > > > + modified = profile_client.modify_profile_using_file_input( > > > > > > > > + '/tmp/modified.xml') > > > > > > > > + print(modified) > > > > > > > > + print > > > > > > > > > > > > > > 3) > > > > > > > > > > > > > > Nit-pick: could you delete the created profile here so that the > > > > > > > example can be run multiple times? > > > > > > > > > > > > > > > > > From 3a031e463e6fb0b5658ad9e1bc37a018c99f32de Mon Sep 17 00:00:00 2001 > > > > > > From: Abhishek Koneru > > > > > > Date: Thu, 19 Jun 2014 00:10:13 -0400 > > > > > > Subject: [PATCH] Added methods for providing file input for profile request. > > > > > > > > > > > > Added new methods to allow user to provide file input to perform > > > > > > operations like create profile/modify profile. > > > > > > The supported file formats a re xml and json. > > > > > > --- > > > > > > base/common/python/pki/__init__.py | 4 +- > > > > > > base/common/python/pki/profile.py | 265 ++++++++++++++++++++++++++++++++++--- > > > > > > 2 files changed, 252 insertions(+), 17 deletions(-) > > > > > > > > > > > > diff --git a/base/common/python/pki/__init__.py b/base/common/python/pki/__init__.py > > > > > > index e9b726cf763785b4a700ef314ff27774b13aba40..891d6ea6364b037f132ff3754b73b372c638b0f7 100644 > > > > > > --- a/base/common/python/pki/__init__.py > > > > > > +++ b/base/common/python/pki/__init__.py > > > > > > @@ -168,7 +168,7 @@ class PKIException(Exception, ResourceMessage): > > > > > > ret = cls(json_value['Message'], json_value['Code'], > > > > > > json_value['ClassName']) > > > > > > for attr in json_value['Attributes']['Attribute']: > > > > > > - print(str(attr)) > > > > > > + print str(attr) > > > > > > ret.add_attribute(attr["name"], attr["value"]) > > > > > > return ret > > > > > > > > > > > > @@ -299,7 +299,7 @@ class PropertyFile(object): > > > > > > def show(self): > > > > > > """ Show contents of property file.""" > > > > > > for line in self.lines: > > > > > > - print(line) > > > > > > + print line > > > > > > > > > > > > def insert_line(self, index, line): > > > > > > """ Insert line in property file """ > > > > > > diff --git a/base/common/python/pki/profile.py b/base/common/python/pki/profile.py > > > > > > index 4f08ee5ba9527855f63b2785c32f9aaddc8d1289..2d4ce5ace11bc6417032fb89ecf6b18707b6fca0 100644 > > > > > > --- a/base/common/python/pki/profile.py > > > > > > +++ b/base/common/python/pki/profile.py > > > > > > @@ -1,12 +1,27 @@ > > > > > > #!/usr/bin/python > > > > > > """ > > > > > > -Created on May 13,, 2014 > > > > > > + at author: Abhishek Koneru > > > > > > > > > > > > - at author: akoneru > > > > > > + This program is free software; you can redistribute it and/or modify > > > > > > + it under the terms of the GNU General Public License as published by > > > > > > + the Free Software Foundation; version 2 of the License. > > > > > > + > > > > > > + This program is distributed in the hope that it will be useful, > > > > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > > > > > + GNU General Public License for more details. > > > > > > + > > > > > > + You should have received a copy of the GNU General Public License along > > > > > > + with this program; if not, write to the Free Software Foundation, Inc., > > > > > > + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. > > > > > > + > > > > > > + Copyright (C) 2014 Red Hat, Inc. > > > > > > + All rights reserved. > > > > > > """ > > > > > > > > > > > > import json > > > > > > import types > > > > > > +from lxml import etree > > > > > > > > > > > > import pki > > > > > > import pki.client as client > > > > > > @@ -433,6 +448,38 @@ class PolicyDefault(object): > > > > > > def policy_params(self, value): > > > > > > setattr(self, 'params', value) > > > > > > > > > > > > + def add_attribute(self, profile_attribute): > > > > > > + self.policy_attributes.append(profile_attribute) > > > > > > + > > > > > > + def remove_attribute(self, profile_attribute_name): > > > > > > + for attr in self.policy_attributes: > > > > > > + if attr.name == profile_attribute_name: > > > > > > + self.policy_attributes.remove(attr) > > > > > > + break > > > > > > + > > > > > > + def get_attribute(self, profile_attribute_name): > > > > > > + for attr in self.policy_attributes: > > > > > > + if attr.name == profile_attribute_name: > > > > > > + return attr > > > > > > + > > > > > > + return None > > > > > > + > > > > > > + def add_parameter(self, profile_parameter): > > > > > > + self.policy_params.append(profile_parameter) > > > > > > + > > > > > > + def remove_parameter(self, profile_parameter_name): > > > > > > + for param in self.policy_params: > > > > > > + if param.name == profile_parameter_name: > > > > > > + self.policy_params.remove(param) > > > > > > + break > > > > > > + > > > > > > + def get_parameter(self, profile_parameter_name): > > > > > > + for param in self.policy_params: > > > > > > + if param.name == profile_parameter_name: > > > > > > + return param > > > > > > + > > > > > > + return None > > > > > > + > > > > > > @classmethod > > > > > > def from_json(cls, json_value): > > > > > > policy_def = cls() > > > > > > @@ -531,6 +578,22 @@ class PolicyConstraint(object): > > > > > > def policy_constraint_values(self, value): > > > > > > setattr(self, 'constraint', value) > > > > > > > > > > > > + def add_constraint_value(self, policy_constraint_value): > > > > > > + self.policy_constraint_values.append(policy_constraint_value) > > > > > > + > > > > > > + def remove_attribute(self, policy_constraint_value_name): > > > > > > + for attr in self.policy_constraint_values: > > > > > > + if attr.name == policy_constraint_value_name: > > > > > > + self.policy_constraint_values.remove(attr) > > > > > > + break > > > > > > + > > > > > > + def get_attribute(self, policy_constraint_value_name): > > > > > > + for constraint in self.policy_constraint_values: > > > > > > + if constraint.name == policy_constraint_value_name: > > > > > > + return constraint > > > > > > + > > > > > > + return None > > > > > > + > > > > > > @classmethod > > > > > > def from_json(cls, json_value): > > > > > > policy_constraint = cls() > > > > > > @@ -995,31 +1058,99 @@ class ProfileClient(object): > > > > > > """ > > > > > > return self._modify_profile_state(profile_id, 'disable') > > > > > > > > > > > > + def _send_profile_request(self, profile_data, operation): > > > > > > + > > > > > > + if profile_data is None: > > > > > > + raise ValueError("No ProfileData specified") > > > > > > + > > > > > > + if operation not in ['create', 'modify']: > > > > > > + raise ValueError("Invalid operation specified: " + str(operation)) > > > > > > + > > > > > > + profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > > + sort_keys=True) > > > > > > + r = None > > > > > > + if operation == 'create': > > > > > > + r = self._post(self.profiles_url, profile_object) > > > > > > + else: > > > > > > + if profile_data.profile_id is None: > > > > > > + raise ValueError("Profile Id is not specified.") > > > > > > + url = self.profiles_url + '/' + str(profile_data.profile_id) > > > > > > + r = self._put(url, profile_object) > > > > > > + > > > > > > + return Profile.from_json(r.json()) > > > > > > + > > > > > > + @pki.handle_exceptions() > > > > > > def create_profile(self, profile_data): > > > > > > """ > > > > > > Create a new profile for the given ProfileData object. > > > > > > """ > > > > > > - if profile_data is None: > > > > > > - raise ValueError("No ProfileData specified") > > > > > > - > > > > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > > - sort_keys=True) > > > > > > - r = self._post(self.profiles_url, profile_object) > > > > > > - return Profile.from_json(r.json()) > > > > > > + return self._send_profile_request(profile_data, 'create') > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > def modify_profile(self, profile_data): > > > > > > """ > > > > > > Modify an existing profile. > > > > > > """ > > > > > > - if profile_data is None: > > > > > > - raise ValueError("No ProfileData specified") > > > > > > + return self._send_profile_request(profile_data, 'modify') > > > > > > + > > > > > > + def _send_request_in_file(self, path_to_file, data_format, operation): > > > > > > + > > > > > > + if path_to_file is None: > > > > > > + raise ValueError("File path must be specified.") > > > > > > + > > > > > > + if data_format not in ['xml', 'json']: > > > > > > + raise ValueError("Unsupported data type: " + str(data_format)) > > > > > > + > > > > > > + if operation not in ['create', 'modify']: > > > > > > + raise ValueError("Invalid operation specified: " + str(operation)) > > > > > > + > > > > > > + data = None > > > > > > + with open(path_to_file) as input_file: > > > > > > + data = input_file.read() > > > > > > + > > > > > > + if data_format == 'xml': > > > > > > + self.headers['Content-type'] = 'application/xml' > > > > > > + > > > > > > + r = None > > > > > > + try: > > > > > > + # Sending the data to the server. > > > > > > + if operation == 'create': > > > > > > + r = self._post(self.profiles_url, data) > > > > > > + else: > > > > > > + profile_id = None > > > > > > + if data_format == 'xml': > > > > > > + profile_id = etree.fromstring(data).get('id') > > > > > > + else: > > > > > > + profile_id = json.loads(data)['id'] > > > > > > + > > > > > > + if profile_id is None: > > > > > > + raise ValueError('Profile Id is not specified.') > > > > > > + url = self.profiles_url + '/' + profile_id > > > > > > + r = self._put(url, data) > > > > > > + finally: > > > > > > + # Reset the Content-type header to json(As used by other methods). > > > > > > + if data_format == 'xml': > > > > > > + self.headers['Content-type'] = 'application/json' > > > > > > > > > > > > - url = self.profiles_url + '/' + str(profile_data.profile_id) > > > > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > > > > - sort_keys=True) > > > > > > - r = self._put(url, profile_object) > > > > > > return Profile.from_json(r.json()) > > > > > > > > > > > > + @pki.handle_exceptions() > > > > > > + def create_profile_using_file_input(self, path_to_file, data_format): > > > > > > + """ > > > > > > + Create a new profile from a profile object stored in a file. > > > > > > + Acceptable data formats - json, xml. > > > > > > + """ > > > > > > + return self._send_request_in_file(path_to_file, data_format, 'create') > > > > > > + > > > > > > + @pki.handle_exceptions() > > > > > > + def modify_profile_using_file_input(self, path_to_file, data_format): > > > > > > + """ > > > > > > + Modify a profile from a profile object stored in a file. > > > > > > + Acceptable data formats - json, xml. > > > > > > + """ > > > > > > + return self._send_request_in_file(path_to_file, data_format, 'modify') > > > > > > + > > > > > > + @pki.handle_exceptions() > > > > > > def delete_profile(self, profile_id): > > > > > > """ > > > > > > Delete a profile. > > > > > > @@ -1108,6 +1239,7 @@ def main(): > > > > > > renewal=False, xml_output=False, > > > > > > authorization_acl="") > > > > > > > > > > > > + # Adding a profile input > > > > > > profile_input = ProfileInput("i1", "subjectNameInputImpl") > > > > > > profile_input.add_attribute(ProfileAttribute("sn_uid")) > > > > > > profile_input.add_attribute(ProfileAttribute("sn_e")) > > > > > > @@ -1121,6 +1253,92 @@ def main(): > > > > > > > > > > > > profile_data.add_input(profile_input) > > > > > > > > > > > > + # Adding a profile output > > > > > > + profile_output = ProfileOutput("o1", name="Certificate Output", > > > > > > + class_id="certOutputImpl") > > > > > > + profile_output.add_attribute(ProfileAttribute("pretty_cert")) > > > > > > + profile_output.add_attribute(ProfileAttribute("b64_cert")) > > > > > > + > > > > > > + profile_data.add_output(profile_output) > > > > > > + > > > > > > + # Create a Policy set with a list of profile policies > > > > > > + policy_list = [] > > > > > > + > > > > > > + # Creating profile policy > > > > > > + policy_default = PolicyDefault("Subject Name Default", > > > > > > + "userSubjectNameDefaultImpl", > > > > > > + "This default populates a User-Supplied " > > > > > > + "Certificate Subject Name to the request.") > > > > > > + > > > > > > + attr_descriptor = Descriptor(syntax="string", description="Subject Name") > > > > > > + policy_attribute = ProfileAttribute("name", descriptor=attr_descriptor) > > > > > > + policy_default.add_attribute(policy_attribute) > > > > > > + > > > > > > + policy_constraint = PolicyConstraint("Subject Name Constraint", > > > > > > + "This constraint accepts the subject " > > > > > > + "name that matches UID=.*", > > > > > > + "subjectNameConstraintImpl") > > > > > > + constraint_descriptor = Descriptor(syntax="string", > > > > > > + description="Subject Name Pattern") > > > > > > + policy_constraint_value = PolicyConstraintValue("pattern", > > > > > > + "UID=.*", > > > > > > + constraint_descriptor) > > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > > + > > > > > > + policy_list.append(ProfilePolicy("1", policy_default, policy_constraint)) > > > > > > + > > > > > > + # Creating another profile policy > > > > > > + # Defining the policy default > > > > > > + policy_default = PolicyDefault("Validity Default", "validityDefaultImpl", > > > > > > + "This default populates a Certificate " > > > > > > + "Validity to the request. The default " > > > > > > + "values are Range=180 in days") > > > > > > + attr_descriptor = Descriptor(syntax="string", description="Not Before") > > > > > > + policy_attribute = ProfileAttribute("notBefore", descriptor=attr_descriptor) > > > > > > + policy_default.add_attribute(policy_attribute) > > > > > > + > > > > > > + attr_descriptor = Descriptor(syntax="string", description="Not After") > > > > > > + policy_attribute = ProfileAttribute("notAfter", descriptor=attr_descriptor) > > > > > > + policy_default.add_attribute(policy_attribute) > > > > > > + > > > > > > + profile_param = ProfileParameter("range", 180) > > > > > > + profile_param2 = ProfileParameter("startTime", 0) > > > > > > + policy_default.add_parameter(profile_param) > > > > > > + policy_default.add_parameter(profile_param2) > > > > > > + > > > > > > + #Defining the policy constraint > > > > > > + policy_constraint = PolicyConstraint("Validity Constraint", > > > > > > + "This constraint rejects the validity " > > > > > > + "that is not between 365 days.", > > > > > > + "validityConstraintImpl") > > > > > > + constraint_descriptor = Descriptor(syntax="integer", > > > > > > + description="Validity Range (in days)", > > > > > > + default_value=365) > > > > > > + policy_constraint_value = PolicyConstraintValue("range", 365, > > > > > > + constraint_descriptor) > > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > > + > > > > > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > > > > > + description="Check Not Before against" > > > > > > + " current time") > > > > > > + policy_constraint_value = PolicyConstraintValue("notBeforeCheck", False, > > > > > > + constraint_descriptor) > > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > > + > > > > > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > > > > > + description="Check Not After against" > > > > > > + " Not Before") > > > > > > + policy_constraint_value = PolicyConstraintValue("notAfterCheck", False, > > > > > > + constraint_descriptor) > > > > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > > > > + > > > > > > + policy_list.append(ProfilePolicy("2", policy_default, policy_constraint)) > > > > > > + > > > > > > + policy_set = PolicySet("userCertSet", policy_list) > > > > > > + > > > > > > + profile_data.add_policy_set(policy_set) > > > > > > + > > > > > > + # Create a new profile > > > > > > created_profile = profile_client.create_profile(profile_data) > > > > > > print(created_profile) > > > > > > print > > > > > > @@ -1185,6 +1403,23 @@ def main(): > > > > > > # pylint: disable-msg=W0703 > > > > > > except Exception as e: > > > > > > print str(e) > > > > > > + print > > > > > > + > > > > > > + # Creating a profile from file > > > > > > + print('Creating a profile using file input.') > > > > > > + print('------------------------------------') > > > > > > + original = profile_client.create_profile_using_file_input( > > > > > > + '/tmp/original.xml', 'xml') > > > > > > + print(original) > > > > > > + print > > > > > > + > > > > > > + # Modifying a profile from file > > > > > > + print('Modifying a profile using file input.') > > > > > > + print('------------------------------------') > > > > > > + modified = profile_client.modify_profile_using_file_input( > > > > > > + '/tmp/modified.xml', 'xml') > > > > > > + print(modified) > > > > > > + print > > > > > > > > > > > > > > > > > > if __name__ == "__main__": > > > > > > -- > > > > > > 1.8.5.3 > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Pki-devel mailing list > > > > > Pki-devel at redhat.com > > > > > https://www.redhat.com/mailman/listinfo/pki-devel > > > > > > > > > > > > > > > > From 6f0acdb884fa8306be8d3478a22becb301cb6d79 Mon Sep 17 00:00:00 2001 > > > From: Abhishek Koneru > > > Date: Thu, 19 Jun 2014 00:10:13 -0400 > > > Subject: [PATCH] Added methods for providing file input for profile request. > > > > > > Added new methods to allow user to provide file input to perform > > > operations like create profile/modify profile. > > > The supported file formats a re xml and json. > > > --- > > > base/common/python/pki/__init__.py | 7 +- > > > base/common/python/pki/profile.py | 636 +++++++++++++++++++++++++++++++------ > > > 2 files changed, 541 insertions(+), 102 deletions(-) > > > > > > diff --git a/base/common/python/pki/__init__.py b/base/common/python/pki/__init__.py > > > index e9b726cf763785b4a700ef314ff27774b13aba40..b610a9f8d9dc13dcdaf1e7cc920a204d0277d3f3 100644 > > > --- a/base/common/python/pki/__init__.py > > > +++ b/base/common/python/pki/__init__.py > > > @@ -168,7 +168,7 @@ class PKIException(Exception, ResourceMessage): > > > ret = cls(json_value['Message'], json_value['Code'], > > > json_value['ClassName']) > > > for attr in json_value['Attributes']['Attribute']: > > > - print(str(attr)) > > > + print str(attr) > > > ret.add_attribute(attr["name"], attr["value"]) > > > return ret > > > > > > @@ -299,7 +299,7 @@ class PropertyFile(object): > > > def show(self): > > > """ Show contents of property file.""" > > > for line in self.lines: > > > - print(line) > > > + print line > > > > > > def insert_line(self, index, line): > > > """ Insert line in property file """ > > > @@ -403,6 +403,9 @@ class Link: > > > > > > @classmethod > > > def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > link = cls() > > > for attr in attr_list: > > > setattr(link, attr, attr_list[attr]) > > > diff --git a/base/common/python/pki/profile.py b/base/common/python/pki/profile.py > > > index 4f08ee5ba9527855f63b2785c32f9aaddc8d1289..f8caf10e0a89773f3287899c68a9ce60da5cee0e 100644 > > > --- a/base/common/python/pki/profile.py > > > +++ b/base/common/python/pki/profile.py > > > @@ -1,13 +1,29 @@ > > > #!/usr/bin/python > > > """ > > > -Created on May 13,, 2014 > > > + This program is free software; you can redistribute it and/or modify > > > + it under the terms of the GNU General Public License as published by > > > + the Free Software Foundation; version 2 of the License. > > > > > > - at author: akoneru > > > + This program is distributed in the hope that it will be useful, > > > + but WITHOUT ANY WARRANTY; without even the implied warranty of > > > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > > + GNU General Public License for more details. > > > + > > > + You should have received a copy of the GNU General Public License along > > > + with this program; if not, write to the Free Software Foundation, Inc., > > > + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. > > > + > > > + Copyright (C) 2014 Red Hat, Inc. > > > + All rights reserved. > > > + > > > + @author: Abhishek Koneru > > > """ > > > > > > import json > > > +import os > > > import types > > > > > > + > > > import pki > > > import pki.client as client > > > import pki.account as account > > > @@ -36,6 +52,9 @@ class ProfileDataInfo(object): > > > > > > @classmethod > > > def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > profile_data_info = cls() > > > profile_data_info.profile_id = attr_list['profileId'] > > > profile_data_info.profile_name = attr_list['profileName'] > > > @@ -59,9 +78,9 @@ class ProfileDataInfoCollection(object): > > > return iter(self.profile_data_list) > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > ret = cls() > > > - profile_data_infos = json_value['entries'] > > > + profile_data_infos = attr_list['entries'] > > > if not isinstance(profile_data_infos, types.ListType): > > > ret.profile_data_list.append( > > > ProfileDataInfo.from_json(profile_data_infos)) > > > @@ -70,7 +89,7 @@ class ProfileDataInfoCollection(object): > > > ret.profile_data_list.append( > > > ProfileDataInfo.from_json(profile_info)) > > > > > > - links = json_value['Link'] > > > + links = attr_list['Link'] > > > if not isinstance(links, types.ListType): > > > ret.links.append(pki.Link.from_json(links)) > > > else: > > > @@ -128,6 +147,9 @@ class Descriptor(object): > > > > > > @classmethod > > > def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > descriptor = cls() > > > for attr in attr_list: > > > setattr(descriptor, attr, attr_list[attr]) > > > @@ -163,6 +185,9 @@ class ProfileAttribute(object): > > > > > > @classmethod > > > def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > attribute = cls() > > > attribute.name = attr_list['name'] > > > if 'Value' in attr_list: > > > @@ -241,15 +266,28 @@ class ProfileInput(object): > > > setattr(self, 'ConfigAttribute', value) > > > > > > def add_attribute(self, profile_attribute): > > > + """ > > > + Add a ProfileAttribute object to the attributes list. > > > + """ > > > + if not isinstance(profile_attribute, ProfileAttribute): > > > + raise ValueError("Object passed is not a ProfileAttribute.") > > > self.attributes.append(profile_attribute) > > > > > > def remove_attribute(self, profile_attribute_name): > > > + """ > > > + Remove a ProfileAttribute object with the given name from the attributes > > > + list. > > > + """ > > > for attr in self.attributes: > > > if attr.name == profile_attribute_name: > > > self.attributes.remove(attr) > > > break > > > > > > def get_attribute(self, profile_attribute_name): > > > + """ > > > + Returns a ProfileAttribute object for the given name. > > > + None, if no match. > > > + """ > > > for attr in self.attributes: > > > if attr.name == profile_attribute_name: > > > return attr > > > @@ -257,31 +295,46 @@ class ProfileInput(object): > > > return None > > > > > > def add_config_attribute(self, profile_attribute): > > > - self.attributes.append(profile_attribute) > > > + """ > > > + Add a ProfileAttribute object to the config_attributes list. > > > + """ > > > + if not isinstance(profile_attribute, ProfileAttribute): > > > + raise ValueError("Object passed is not a ProfileAttribute.") > > > + self.config_attributes.append(profile_attribute) > > > > > > def remove_config_attribute(self, config_attribute_name): > > > + """ > > > + Remove a ProfileAttribute object with the given name from the > > > + config_attributes list. > > > + """ > > > for attr in self.config_attributes: > > > if attr.name == config_attribute_name: > > > - self.attributes.remove(attr) > > > + self.config_attributes.remove(attr) > > > break > > > > > > def get_config_attribute(self, config_attribute_name): > > > - for attr in self.attributes: > > > + """ > > > + Returns a ProfileAttribute object with the given name. > > > + None, if there is no match in the config_attributes list. > > > + """ > > > + for attr in self.config_attributes: > > > if attr.name == config_attribute_name: > > > return attr > > > > > > return None > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > profile_input = cls() > > > - profile_input.profile_input_id = json_value['id'] > > > - profile_input.class_id = json_value['ClassID'] > > > - profile_input.name = json_value['Name'] > > > - if 'Text' in json_value: > > > - profile_input.text = json_value['Text'] > > > + profile_input.profile_input_id = attr_list['id'] > > > + profile_input.class_id = attr_list['ClassID'] > > > + profile_input.name = attr_list['Name'] > > > + if 'Text' in attr_list: > > > + profile_input.text = attr_list['Text'] > > > > > > - attributes = json_value['Attribute'] > > > + attributes = attr_list['Attribute'] > > > if not isinstance(attributes, types.ListType): > > > profile_input.attributes.append( > > > ProfileAttribute.from_json(attributes)) > > > @@ -290,7 +343,7 @@ class ProfileInput(object): > > > profile_input.attributes.append( > > > ProfileAttribute.from_json(profile_info)) > > > > > > - config_attributes = json_value['ConfigAttribute'] > > > + config_attributes = attr_list['ConfigAttribute'] > > > if not isinstance(config_attributes, types.ListType): > > > profile_input.config_attributes.append( > > > ProfileAttribute.from_json(config_attributes)) > > > @@ -334,15 +387,28 @@ class ProfileOutput(object): > > > setattr(self, 'classId', value) > > > > > > def add_attribute(self, profile_attribute): > > > + """ > > > + Add a ProfileAttribute object to the attributes list. > > > + """ > > > + if not isinstance(profile_attribute, ProfileAttribute): > > > + raise ValueError("Object passed is not a ProfileAttribute.") > > > self.attributes.append(profile_attribute) > > > > > > def remove_attribute(self, profile_attribute_name): > > > + """ > > > + Remove a ProfileAttribute object with the given name from the attributes > > > + list. > > > + """ > > > for attr in self.attributes: > > > if attr.name == profile_attribute_name: > > > self.attributes.remove(attr) > > > break > > > > > > def get_attribute(self, profile_attribute_name): > > > + """ > > > + Returns a ProfileAttribute object for the given name. > > > + None, if no match. > > > + """ > > > for attr in self.attributes: > > > if attr.name == profile_attribute_name: > > > return attr > > > @@ -350,14 +416,17 @@ class ProfileOutput(object): > > > return None > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > profile_output = cls() > > > - profile_output.profile_output_id = json_value['id'] > > > - profile_output.name = json_value['name'] > > > - if 'text' in json_value: > > > - profile_output.text = json_value['text'] > > > - profile_output.class_id = json_value['classId'] > > > - attributes = json_value['attributes'] > > > + profile_output.profile_output_id = attr_list['id'] > > > + profile_output.name = attr_list['name'] > > > + if 'text' in attr_list: > > > + profile_output.text = attr_list['text'] > > > + profile_output.class_id = attr_list['classId'] > > > + attributes = attr_list['attributes'] > > > if not isinstance(attributes, types.ListType): > > > profile_output.attributes.append( > > > ProfileAttribute.from_json(attributes)) > > > @@ -375,6 +444,9 @@ class ProfileParameter(object): > > > > > > @classmethod > > > def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > param = cls() > > > for attr in attr_list: > > > setattr(param, attr, attr_list[attr]) > > > @@ -433,17 +505,76 @@ class PolicyDefault(object): > > > def policy_params(self, value): > > > setattr(self, 'params', value) > > > > > > + def add_attribute(self, policy_attribute): > > > + """ > > > + Add a policy attribute to the attribute list. > > > + @param policy_attribute - A ProfileAttribute object > > > + """ > > > + if not isinstance(policy_attribute, ProfileAttribute): > > > + raise ValueError("Object passed is not a ProfileAttribute.") > > > + self.policy_attributes.append(policy_attribute) > > > + > > > + def remove_attribute(self, policy_attribute_name): > > > + """ > > > + Remove a policy attribute with the given name from the attributes list. > > > + """ > > > + for attr in self.policy_attributes: > > > + if attr.name == policy_attribute_name: > > > + self.policy_attributes.remove(attr) > > > + break > > > + > > > + def get_attribute(self, policy_attribute_name): > > > + """ > > > + Fetch the policy attribute with the given name from the attributes list. > > > + """ > > > + for attr in self.policy_attributes: > > > + if attr.name == policy_attribute_name: > > > + return attr > > > + > > > + return None > > > + > > > + def add_parameter(self, policy_parameter): > > > + """ > > > + Add a profile parameter to the parameters list. > > > + @param policy_parameter - A ProfileParameter object. > > > + """ > > > + if not isinstance(policy_parameter, ProfileParameter): > > > + raise ValueError("Object passed is not a ProfileParameter.") > > > + self.policy_params.append(policy_parameter) > > > + > > > + def remove_parameter(self, profile_parameter_name): > > > + """ > > > + Remove a profile parameter with the given name from the parameters list. > > > + """ > > > + for param in self.policy_params: > > > + if param.name == profile_parameter_name: > > > + self.policy_params.remove(param) > > > + break > > > + > > > + def get_parameter(self, profile_parameter_name): > > > + """ > > > + Fetch a profile parameter with the given name from the parameters list. > > > + """ > > > + for param in self.policy_params: > > > + if param.name == profile_parameter_name: > > > + return param > > > + > > > + return None > > > + > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > policy_def = cls() > > > - if 'id' in json_value: > > > - policy_def.name = json_value['id'] > > > - if 'classId' in json_value: > > > - policy_def.class_id = json_value['classId'] > > > - if 'description' in json_value: > > > - policy_def.description = json_value['description'] > > > - if 'policyAttribute' in json_value: > > > - attributes = json_value['policyAttribute'] > > > + if 'id' in attr_list: > > > + policy_def.name = attr_list['id'] > > > + if 'classId' in attr_list: > > > + policy_def.class_id = attr_list['classId'] > > > + if 'description' in attr_list: > > > + policy_def.description = attr_list['description'] > > > + if 'policyAttribute' in attr_list: > > > + attributes = attr_list['policyAttribute'] > > > if not isinstance(attributes, types.ListType): > > > policy_def.policy_attributes.append( > > > ProfileAttribute.from_json(attributes)) > > > @@ -452,8 +583,8 @@ class PolicyDefault(object): > > > policy_def.policy_attributes.append( > > > ProfileAttribute.from_json(attr)) > > > > > > - if 'params' in json_value: > > > - params = json_value['params'] > > > + if 'params' in attr_list: > > > + params = attr_list['params'] > > > if not isinstance(params, types.ListType): > > > policy_def.policy_params.append( > > > ProfileParameter.from_json(params)) > > > @@ -480,13 +611,16 @@ class PolicyConstraintValue(object): > > > setattr(self, 'id', value) > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > ret = cls() > > > > > > - ret.name = json_value['id'] > > > - ret.value = json_value['value'] > > > - if 'descriptor' in json_value: > > > - ret.descriptor = Descriptor.from_json(json_value['descriptor']) > > > + ret.name = attr_list['id'] > > > + ret.value = attr_list['value'] > > > + if 'descriptor' in attr_list: > > > + ret.descriptor = Descriptor.from_json(attr_list['descriptor']) > > > > > > return ret > > > > > > @@ -531,17 +665,49 @@ class PolicyConstraint(object): > > > def policy_constraint_values(self, value): > > > setattr(self, 'constraint', value) > > > > > > + def add_constraint_value(self, policy_constraint_value): > > > + """ > > > + Add a ProfileConstraintValue to the policy_constraint_values list. > > > + """ > > > + if not isinstance(policy_constraint_value, PolicyConstraintValue): > > > + raise ValueError("Object passed not of type PolicyConstraintValue") > > > + self.policy_constraint_values.append(policy_constraint_value) > > > + > > > + def remove_constraint_value(self, policy_constraint_value_name): > > > + """ > > > + Removes a PolicyConstraintValue with the given name form the > > > + policy_constraint_values list. > > > + """ > > > + for attr in self.policy_constraint_values: > > > + if attr.name == policy_constraint_value_name: > > > + self.policy_constraint_values.remove(attr) > > > + break > > > + > > > + def get_constraint_value(self, policy_constraint_value_name): > > > + """ > > > + Returns a PolicyConstraintValue object with the given name. > > > + None, if there is no match. > > > + """ > > > + for constraint in self.policy_constraint_values: > > > + if constraint.name == policy_constraint_value_name: > > > + return constraint > > > + > > > + return None > > > + > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > policy_constraint = cls() > > > - if 'id' in json_value: > > > - policy_constraint.name = json_value['id'] > > > - if 'description' in json_value: > > > - policy_constraint.description = json_value['description'] > > > - if 'classId' in json_value: > > > - policy_constraint.class_id = json_value['classId'] > > > - if 'constraint' in json_value: > > > - constraints = json_value['constraint'] > > > + if 'id' in attr_list: > > > + policy_constraint.name = attr_list['id'] > > > + if 'description' in attr_list: > > > + policy_constraint.description = attr_list['description'] > > > + if 'classId' in attr_list: > > > + policy_constraint.class_id = attr_list['classId'] > > > + if 'constraint' in attr_list: > > > + constraints = attr_list['constraint'] > > > if not isinstance(constraints, types.ListType): > > > policy_constraint.policy_constraint_values.append( > > > PolicyConstraintValue.from_json(constraints)) > > > @@ -592,9 +758,12 @@ class ProfilePolicy(object): > > > setattr(self, 'constraint', value) > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > - return cls(json_value['id'], PolicyDefault.from_json(json_value['def']), > > > - PolicyConstraint.from_json(json_value['constraint'])) > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > + return cls(attr_list['id'], PolicyDefault.from_json(attr_list['def']), > > > + PolicyConstraint.from_json(attr_list['constraint'])) > > > > > > > > > class ProfilePolicySet(object): > > > @@ -607,6 +776,9 @@ class ProfilePolicySet(object): > > > > > > @classmethod > > > def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > policy_set = cls() > > > > > > policies = attr_list['policies'] > > > @@ -649,19 +821,40 @@ class PolicySet(object): > > > setattr(self, 'value', value) > > > > > > def add_policy(self, profile_policy): > > > + """ > > > + Add a ProfilePolicy object to the policy_list > > > + """ > > > + if not isinstance(profile_policy, ProfilePolicy): > > > + raise ValueError("Object passed is not a ProfilePolicy.") > > > self.policy_list.append(profile_policy) > > > > > > def remove_policy(self, policy_id): > > > + """ > > > + Removes a ProfilePolicy with the given ID from the PolicySet. > > > + """ > > > for policy in self.policy_list: > > > if policy.policy_id == policy_id: > > > - self.policy_list.pop(policy) > > > + self.policy_list.remove(policy) > > > + break > > > + > > > + def get_policy(self, policy_id): > > > + """ > > > + Returns a ProfilePolicy object with the given profile id. > > > + """ > > > + for policy in self.policy_list: > > > + if policy.policy_id == policy_id: > > > + return policy > > > + return None > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > policy_set = cls() > > > > > > - policy_set.name = json_value['id'] > > > - policies = json_value['value'] > > > + policy_set.name = attr_list['id'] > > > + policies = attr_list['value'] > > > if not isinstance(policies, types.ListType): > > > policy_set.policy_list.append(ProfilePolicy.from_json(policies)) > > > else: > > > @@ -694,17 +887,39 @@ class PolicySetList(object): > > > setattr(self, 'PolicySet', value) > > > > > > def add_policy_set(self, policy_set): > > > + """ > > > + Add a PolicySet object to the policy_sets list. > > > + """ > > > + if not isinstance(policy_set, PolicySet): > > > + raise ValueError("Object passed is not a PolicySet.") > > > self.policy_sets.append(policy_set) > > > > > > def remove_policy_set(self, policy_set_name): > > > + """ > > > + Remove a PolicySet object with the given name from the policy_sets list. > > > + """ > > > for policy_set in self.policy_sets: > > > if policy_set.name == policy_set_name: > > > - self.policy_sets.pop(policy_set) > > > + self.policy_sets.remove(policy_set) > > > + break > > > + > > > + def get_policy_set(self, policy_set_name): > > > + """ > > > + Fetch the PolicySet object for the given name. > > > + Returns None, if not found. > > > + """ > > > + for policy_set in self.policy_sets: > > > + if policy_set.name == policy_set_name: > > > + return policy_set > > > + return None > > > > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > + if attr_list is None: > > > + return None > > > + > > > policy_set_list = cls() > > > - policy_sets = json_value['PolicySet'] > > > + policy_sets = attr_list['PolicySet'] > > > if not isinstance(policy_sets, types.ListType): > > > policy_set_list.policy_sets.append(PolicySet.from_json(policy_sets)) > > > else: > > > @@ -824,51 +1039,101 @@ class Profile(object): > > > setattr(self, 'PolicySets', value) > > > > > > def add_input(self, profile_input): > > > + """ > > > + Add a ProfileInput object to the inputs list of the Profile. > > > + """ > > > + if not isinstance(profile_input, ProfileInput): > > > + raise ValueError("Object passed is not a PolicyInput.") > > > if profile_input is None: > > > raise ValueError("No ProfileInput object provided.") > > > self.inputs.append(profile_input) > > > > > > def remove_input(self, profile_input_id): > > > + """ > > > + Remove a ProfileInput from the inputs list of the Profile. > > > + """ > > > for profile_input in self.inputs: > > > if profile_input_id == profile_input.profile_input_id: > > > - self.inputs.pop(profile_input) > > > + self.inputs.remove(profile_input) > > > + break > > > + > > > + def get_input(self, profile_input_id): > > > + """ > > > + Fetches a ProfileInput with the given ProfileInput id. > > > + Returns None, if there is no matching input. > > > + """ > > > + for profile_input in self.inputs: > > > + if profile_input_id == profile_input.profile_input_id: > > > + return profile_input > > > + return None > > > > > > def add_output(self, profile_output): > > > + """ > > > + Add a ProfileOutput object to the outputs list of the Profile. > > > + """ > > > + if not isinstance(profile_output, ProfileOutput): > > > + raise ValueError("Object passed is not a PolicyOutput.") > > > if profile_output is None: > > > raise ValueError("No ProfileOutput object provided.") > > > self.outputs.append(profile_output) > > > > > > def remove_output(self, profile_output_id): > > > + """ > > > + Remove a ProfileOutput from the outputs list of the Profile. > > > + """ > > > for profile_output in self.outputs: > > > if profile_output_id == profile_output.profile_output_id: > > > - self.inputs.pop(profile_output) > > > + self.inputs.remove(profile_output) > > > + > > > + def get_output(self, profile_output_id): > > > + """ > > > + Fetches a ProfileOutput with the given ProfileOutput id. > > > + Returns None, if there is no matching output. > > > + """ > > > + for profile_input in self.inputs: > > > + if profile_output_id == profile_input.profile_input_id: > > > + return profile_input > > > + return None > > > > > > def add_policy_set(self, policy_set): > > > + """ > > > + Add a PolicySet object to the policy_sets list of the Profile. > > > + """ > > > if policy_set is None: > > > raise ValueError("No PolicySet object provided.") > > > self.policy_set_list.add_policy_set(policy_set) > > > > > > def remove_policy_set(self, policy_set_name): > > > + """ > > > + Remove a PolicySet from the policy_sets list of the Profile. > > > + """ > > > self.policy_set_list.remove_policy_set(policy_set_name) > > > > > > + def get_policy_set(self, policy_set_name): > > > + """ > > > + Fetches a ProfileInput with the given ProfileInput id. > > > + Returns None, if there is no matching input. > > > + """ > > > + return self.policy_set_list.get_policy_set(policy_set_name) > > > + > > > @classmethod > > > - def from_json(cls, json_value): > > > + def from_json(cls, attr_list): > > > profile_data = cls() > > > - profile_data.profile_id = json_value['id'] > > > - profile_data.class_id = json_value['classId'] > > > - profile_data.name = json_value['name'] > > > - profile_data.description = json_value['description'] > > > - profile_data.enabled = json_value['enabled'] > > > - profile_data.visible = json_value['visible'] > > > - if 'enabledBy' in json_value: > > > - profile_data.enabled_by = json_value['enabledBy'] > > > - if 'authenticatorId' in json_value: > > > - profile_data.authenticator_id = json_value['authenticatorId'] > > > - profile_data.authorization_acl = json_value['authzAcl'] > > > - profile_data.renewal = json_value['renewal'] > > > - profile_data.xml_output = json_value['xmlOutput'] > > > + profile_data.profile_id = attr_list['id'] > > > + profile_data.class_id = attr_list['classId'] > > > + profile_data.name = attr_list['name'] > > > + profile_data.description = attr_list['description'] > > > + profile_data.enabled = attr_list['enabled'] > > > + profile_data.visible = attr_list['visible'] > > > + if 'enabledBy' in attr_list: > > > + profile_data.enabled_by = attr_list['enabledBy'] > > > + if 'authenticatorId' in attr_list: > > > + profile_data.authenticator_id = attr_list['authenticatorId'] > > > + profile_data.authorization_acl = attr_list['authzAcl'] > > > + profile_data.renewal = attr_list['renewal'] > > > + profile_data.xml_output = attr_list['xmlOutput'] > > > > > > - profile_inputs = json_value['Input'] > > > + profile_inputs = attr_list['Input'] > > > if not isinstance(profile_inputs, types.ListType): > > > profile_data.inputs.append(ProfileInput.from_json(profile_inputs)) > > > else: > > > @@ -876,7 +1141,7 @@ class Profile(object): > > > profile_data.inputs.append( > > > ProfileInput.from_json(profile_input)) > > > > > > - profile_outputs = json_value['Output'] > > > + profile_outputs = attr_list['Output'] > > > if not isinstance(profile_outputs, types.ListType): > > > profile_data.outputs.append( > > > ProfileOutput.from_json(profile_outputs)) > > > @@ -886,9 +1151,9 @@ class Profile(object): > > > ProfileOutput.from_json(profile_output)) > > > > > > profile_data.policy_set_list = \ > > > - PolicySetList.from_json(json_value['PolicySets']) > > > + PolicySetList.from_json(attr_list['PolicySets']) > > > > > > - profile_data.link = pki.Link.from_json(json_value['link']) > > > + profile_data.link = pki.Link.from_json(attr_list['link']) > > > > > > return profile_data > > > > > > @@ -904,6 +1169,20 @@ class Profile(object): > > > } > > > return str(attributes) > > > > > > + @staticmethod > > > + def get_profile_data_from_file(path_to_file): > > > + """ > > > + Reads the file for the serialized Profile object. > > > + Currently supports only data format in json. > > > + """ > > > + if path_to_file is None: > > > + raise ValueError("File path must be specified.") > > > + with open(path_to_file) as input_file: > > > + data = input_file.read() > > > + if data is not None: > > > + return Profile.from_json(json.loads(data)) > > > + return None > > > + > > > > > > class ProfileClient(object): > > > """ > > > @@ -995,34 +1274,67 @@ class ProfileClient(object): > > > """ > > > return self._modify_profile_state(profile_id, 'disable') > > > > > > - def create_profile(self, profile_data): > > > - """ > > > - Create a new profile for the given ProfileData object. > > > - """ > > > + def _send_profile_create(self, profile_data): > > > + > > > if profile_data is None: > > > raise ValueError("No ProfileData specified") > > > > > > profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > sort_keys=True) > > > + > > > r = self._post(self.profiles_url, profile_object) > > > + > > > return Profile.from_json(r.json()) > > > > > > - def modify_profile(self, profile_data): > > > - """ > > > - Modify an existing profile. > > > - """ > > > + def _send_profile_modify(self, profile_data): > > > if profile_data is None: > > > raise ValueError("No ProfileData specified") > > > > > > + profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > + sort_keys=True) > > > + if profile_data.profile_id is None: > > > + raise ValueError("Profile Id is not specified.") > > > url = self.profiles_url + '/' + str(profile_data.profile_id) > > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > > - sort_keys=True) > > > r = self._put(url, profile_object) > > > + > > > return Profile.from_json(r.json()) > > > > > > + @pki.handle_exceptions() > > > + def create_profile(self, profile_data): > > > + """ > > > + Create a new profile for the given Profile object. > > > + """ > > > + return self._send_profile_create(profile_data) > > > + > > > + @pki.handle_exceptions() > > > + def modify_profile(self, profile_data): > > > + """ > > > + Modify an existing profile with the given Profile object. > > > + """ > > > + return self._send_profile_modify(profile_data) > > > + > > > + def create_profile_from_file(self, path_to_file): > > > + """ > > > + Reads the file for the serialized Profile object. > > > + Performs the profile create operation. > > > + Currently supports only data format in json. > > > + """ > > > + profile_data = Profile.get_profile_data_from_file(path_to_file) > > > + return self._send_profile_create(profile_data) > > > + > > > + def modify_profile_from_file(self, path_to_file): > > > + """ > > > + Reads the file for the serialized Profile object. > > > + Performs the profile modify operation. > > > + Currently supports only data format in json. > > > + """ > > > + profile_data = Profile.get_profile_data_from_file(path_to_file) > > > + return self._send_profile_modify(profile_data) > > > + > > > + @pki.handle_exceptions() > > > def delete_profile(self, profile_id): > > > """ > > > - Delete a profile. > > > + Delete a profile with the given Profile Id. > > > """ > > > if profile_id is None: > > > raise ValueError("Profile Id must be specified.") > > > @@ -1058,6 +1370,11 @@ def main(): > > > #Initialize the ProfileClient class > > > profile_client = ProfileClient(connection) > > > > > > + # Folder to store the files generated during test > > > + file_path = '/tmp/profile_client_test/' > > > + if not os.path.exists(file_path): > > > + os.makedirs(file_path) > > > + > > > #Fetching a list of profiles > > > profile_data_infos = profile_client.list_profiles() > > > print('List of profiles:') > > > @@ -1102,12 +1419,14 @@ def main(): > > > print('-----------------------') > > > > > > profile_data = Profile(name="My Sample User Cert Enrollment", > > > - profile_id="MySampleCert", class_id="caEnrollImpl", > > > + profile_id="MySampleProfile", > > > + class_id="caEnrollImpl", > > > description="Example User Cert Enroll Impl", > > > enabled_by='admin', enabled=False, visible=False, > > > renewal=False, xml_output=False, > > > authorization_acl="") > > > > > > + # Adding a profile input > > > profile_input = ProfileInput("i1", "subjectNameInputImpl") > > > profile_input.add_attribute(ProfileAttribute("sn_uid")) > > > profile_input.add_attribute(ProfileAttribute("sn_e")) > > > @@ -1121,6 +1440,97 @@ def main(): > > > > > > profile_data.add_input(profile_input) > > > > > > + # Adding a profile output > > > + profile_output = ProfileOutput("o1", name="Certificate Output", > > > + class_id="certOutputImpl") > > > + profile_output.add_attribute(ProfileAttribute("pretty_cert")) > > > + profile_output.add_attribute(ProfileAttribute("b64_cert")) > > > + > > > + profile_data.add_output(profile_output) > > > + > > > + # Create a Policy set with a list of profile policies > > > + policy_list = [] > > > + > > > + # Creating profile policy > > > + policy_default = PolicyDefault("Subject Name Default", > > > + "userSubjectNameDefaultImpl", > > > + "This default populates a User-Supplied " > > > + "Certificate Subject Name to the request.") > > > + > > > + attr_descriptor = Descriptor(syntax="string", description="Subject Name") > > > + policy_attribute = ProfileAttribute("name", descriptor=attr_descriptor) > > > + policy_default.add_attribute(policy_attribute) > > > + > > > + policy_constraint = PolicyConstraint("Subject Name Constraint", > > > + "This constraint accepts the subject " > > > + "name that matches UID=.*", > > > + "subjectNameConstraintImpl") > > > + constraint_descriptor = Descriptor(syntax="string", > > > + description="Subject Name Pattern") > > > + policy_constraint_value = PolicyConstraintValue("pattern", > > > + "UID=.*", > > > + constraint_descriptor) > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > + > > > + policy_list.append(ProfilePolicy("1", policy_default, policy_constraint)) > > > + > > > + # Creating another profile policy > > > + # Defining the policy default > > > + policy_default = PolicyDefault("Validity Default", "validityDefaultImpl", > > > + "This default populates a Certificate " > > > + "Validity to the request. The default " > > > + "values are Range=180 in days") > > > + attr_descriptor = Descriptor(syntax="string", description="Not Before") > > > + policy_attribute = ProfileAttribute("notBefore", descriptor=attr_descriptor) > > > + policy_default.add_attribute(policy_attribute) > > > + > > > + attr_descriptor = Descriptor(syntax="string", description="Not After") > > > + policy_attribute = ProfileAttribute("notAfter", descriptor=attr_descriptor) > > > + policy_default.add_attribute(policy_attribute) > > > + > > > + profile_param = ProfileParameter("range", 180) > > > + profile_param2 = ProfileParameter("startTime", 0) > > > + policy_default.add_parameter(profile_param) > > > + policy_default.add_parameter(profile_param2) > > > + > > > + #Defining the policy constraint > > > + policy_constraint = PolicyConstraint("Validity Constraint", > > > + "This constraint rejects the validity " > > > + "that is not between 365 days.", > > > + "validityConstraintImpl") > > > + constraint_descriptor = Descriptor(syntax="integer", > > > + description="Validity Range (in days)", > > > + default_value=365) > > > + policy_constraint_value = PolicyConstraintValue("range", 365, > > > + constraint_descriptor) > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > + > > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > > + description="Check Not Before against" > > > + " current time") > > > + policy_constraint_value = PolicyConstraintValue("notBeforeCheck", False, > > > + constraint_descriptor) > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > + > > > + constraint_descriptor = Descriptor(syntax="boolean", default_value=False, > > > + description="Check Not After against" > > > + " Not Before") > > > + policy_constraint_value = PolicyConstraintValue("notAfterCheck", False, > > > + constraint_descriptor) > > > + policy_constraint.add_constraint_value(policy_constraint_value) > > > + > > > + policy_list.append(ProfilePolicy("2", policy_default, policy_constraint)) > > > + > > > + policy_set = PolicySet("userCertSet", policy_list) > > > + > > > + profile_data.add_policy_set(policy_set) > > > + > > > + # Write the profile data object to a file for testing a file input > > > + with open(file_path+'/original.json', 'w') as output_file: > > > + output_file.write(json.dumps(profile_data, > > > + cls=encoder.CustomTypeEncoder, > > > + sort_keys=True, indent=4)) > > > + # Create a new profile > > > created_profile = profile_client.create_profile(profile_data) > > > print(created_profile) > > > print > > > @@ -1131,7 +1541,7 @@ def main(): > > > > > > try: > > > profile_data = Profile(name="My Sample User Cert Enrollment", > > > - profile_id="MySampleCert", > > > + profile_id="MySampleProfile", > > > class_id="caEnrollImpl", > > > description="Example User Cert Enroll Impl", > > > enabled_by='admin', enabled=False, visible=False, > > > @@ -1151,40 +1561,66 @@ def main(): > > > profile_data.add_input(profile_input) > > > profile_client.create_profile(profile_data) > > > # pylint: disable-msg=W0703 > > > - except Exception as e: > > > - print str(e) > > > + except pki.BadRequestException as e: > > > + print 'MySampleProfile ' + str(e) > > > print > > > > > > # Modify the above created profile > > > - print('Modifying the profile MySampleCert.') > > > + print('Modifying the profile MySampleProfile.') > > > print('-----------------------------------') > > > > > > - fetch = profile_client.get_profile('MySampleCert') > > > + fetch = profile_client.get_profile('MySampleProfile') > > > profile_input2 = ProfileInput("i2", "keyGenInputImpl") > > > profile_input2.add_attribute(ProfileAttribute("cert_request_type")) > > > profile_input2.add_attribute(ProfileAttribute("cert_request")) > > > fetch.add_input(profile_input2) > > > > > > fetch.name += " (Modified)" > > > + with open(file_path+'modified.json', 'w') as output_file: > > > + output_file.write(json.dumps(fetch, cls=encoder.CustomTypeEncoder, > > > + sort_keys=True, indent=4)) > > > modified_profile = profile_client.modify_profile(fetch) > > > print(modified_profile) > > > print > > > > > > # Delete a profile > > > - print ("Deleting the profile MySampleCert.") > > > + print ("Deleting the profile MySampleProfile.") > > > print ("----------------------------------") > > > - profile_client.delete_profile('MySampleCert') > > > - print ("Deleted profile MySampleCert.") > > > + profile_client.delete_profile('MySampleProfile') > > > + print ("Deleted profile MySampleProfile.") > > > print > > > > > > # Testing deletion of a profile > > > print('Test profile deletion.') > > > print('----------------------') > > > try: > > > - profile_client.get_profile('MySampleCert') > > > + profile_client.get_profile('MySampleProfile') > > > # pylint: disable-msg=W0703 > > > - except Exception as e: > > > + except pki.ProfileNotFoundException as e: > > > print str(e) > > > + print > > > + > > > + # Creating a profile from file > > > + print('Creating a profile using file input.') > > > + print('------------------------------------') > > > + original = profile_client.create_profile_from_file( > > > + file_path + 'original.json') > > > + print(original) > > > + print > > > + > > > + # Modifying a profile from file > > > + print('Modifying a profile using file input.') > > > + print('------------------------------------') > > > + modified = profile_client.modify_profile_from_file( > > > + file_path + 'modified.json') > > > + print(modified) > > > + print > > > + > > > + # Test clean up > > > + profile_client.delete_profile('MySampleProfile') > > > + os.remove(file_path+'original.json') > > > + os.remove(file_path+'modified.json') > > > + os.removedirs(file_path) > > > > > > > > > if __name__ == "__main__": > > > -- > > > 1.8.5.3 > > > > > > > From edewata at redhat.com Wed Jul 2 00:52:31 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Tue, 01 Jul 2014 19:52:31 -0500 Subject: [Pki-devel] [PATCH] 513 Renamed CryptoUtil to CryptoProvider. Message-ID: <53B357CF.3020500@redhat.com> The CryptoUtil classes in the Python client library has been renamed to CryptoProvider for consistency with the Java client library. The cryptoutil.py module has been renamed to crypto.py. Ticket #1042 The drmtest.py works fine with these changes. -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0513-Renamed-CryptoUtil-to-CryptoProvider.patch Type: text/x-patch Size: 5586 bytes Desc: not available URL: From mharmsen at redhat.com Wed Jul 2 02:11:31 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Tue, 01 Jul 2014 19:11:31 -0700 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working Message-ID: <53B36A53.9060904@redhat.com> Please review the attached patch (based upon what was provided by Marko Karg in Bugzilla Bug #1081916 - freeipa does not install on arm architecture ) for the following TRAC ticket: * PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working CAVEAT: To test this, all I did was add the code to the template and build on my 'x86_64' Fedora 20 machine. I extracted 'AtoB' and 'BtoA' and confirmed that they now include the 'armv7l' section, and although I have attempted to request an 'arm' machine to test this out, I have not yet been able to acquire a test machine, so I do not know if the 'arch' command actually returns 'armv7l'. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 20140701-Add-arm-architecture-to-tool-scripts.patch Type: text/x-patch Size: 1192 bytes Desc: not available URL: From ftweedal at redhat.com Wed Jul 2 05:57:32 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 2 Jul 2014 15:57:32 +1000 Subject: [Pki-devel] [PATCH] 513 Renamed CryptoUtil to CryptoProvider. In-Reply-To: <53B357CF.3020500@redhat.com> References: <53B357CF.3020500@redhat.com> Message-ID: <20140702055732.GS2417@dhcp-40-8.bne.redhat.com> On Tue, Jul 01, 2014 at 07:52:31PM -0500, Endi Sukma Dewata wrote: > The CryptoUtil classes in the Python client library has been renamed > to CryptoProvider for consistency with the Java client library. The > cryptoutil.py module has been renamed to crypto.py. > > Ticket #1042 > > The drmtest.py works fine with these changes. > > -- > Endi S. Dewata Haven't run drmtest myself but it doesn't look like you've missed anything and changes apply and build fine. ACK > From 96a662d1884f3898bc5d119166854b9e4e12997b Mon Sep 17 00:00:00 2001 > From: "Endi S. Dewata" > Date: Tue, 1 Jul 2014 15:36:03 -0400 > Subject: [PATCH] Renamed CryptoUtil to CryptoProvider. > > The CryptoUtil classes in the Python client library has been renamed > to CryptoProvider for consistency with the Java client library. The > cryptoutil.py module has been renamed to crypto.py. > > Ticket #1042 > --- > base/common/python/pki/{cryptoutil.py => crypto.py} | 8 ++++---- > base/common/python/pki/key.py | 4 ++-- > base/common/python/pki/kra.py | 6 +++--- > base/kra/functional/drmtest.py | 6 +++--- > 4 files changed, 12 insertions(+), 12 deletions(-) > rename base/common/python/pki/{cryptoutil.py => crypto.py} (98%) > > diff --git a/base/common/python/pki/cryptoutil.py b/base/common/python/pki/crypto.py > similarity index 98% > rename from base/common/python/pki/cryptoutil.py > rename to base/common/python/pki/crypto.py > index 44adb3fb6f377053e85b26be1f3a9652ba1c8ef3..174e681b8fedf37973ec4d15cf0aca1e12a774ec 100644 > --- a/base/common/python/pki/cryptoutil.py > +++ b/base/common/python/pki/crypto.py > @@ -30,7 +30,7 @@ import subprocess > import tempfile > > > -class CryptoUtil(object): > +class CryptoProvider(object): > """ > Abstract class containing methods to do cryptographic operations. > """ > @@ -93,9 +93,9 @@ class CryptoUtil(object): > pass > > > -class NSSCryptoUtil(CryptoUtil): > +class NSSCryptoProvider(CryptoProvider): > """ > - Class that defines NSS implementation of CryptoUtil. > + Class that defines NSS implementation of CryptoProvider. > Requires an NSS database to have been set up and initialized. > > Note that all inputs and outputs are unencoded. > @@ -126,7 +126,7 @@ class NSSCryptoUtil(CryptoUtil): > This method expects a NSS database to have already been created at > certdb_dir with password certdb_password. > """ > - CryptoUtil.__init__(self) > + CryptoProvider.__init__(self) > self.certdb_dir = certdb_dir > self.certdb_password = certdb_password > self.nonce_iv = "e4:bb:3b:d3:c3:71:2e:58" > diff --git a/base/common/python/pki/key.py b/base/common/python/pki/key.py > index 048cc0d41a500513cf4c0f7860dcb8756fa6d714..af34a7ff42ffe288f9b1f2767989bfc2b383343d 100644 > --- a/base/common/python/pki/key.py > +++ b/base/common/python/pki/key.py > @@ -770,7 +770,7 @@ class KeyClient(object): > > 1) trans_wrapped_session_key is not provided by caller. > > - In this case, the function will call CryptoUtil methods to generate and > + In this case, the function will call CryptoProvider methods to generate and > wrap the session key. The function will return the KeyData object with > a private_data attribute which stores the unwrapped key information. > > @@ -833,7 +833,7 @@ class KeyClient(object): > > 1) A passphrase is provided by the caller. > > - In this case, CryptoUtil methods will be called to create the data to > + In this case, CryptoProvider methods will be called to create the data to > securely send the passphrase to the DRM. Basically, three pieces of > data will be sent: > > diff --git a/base/common/python/pki/kra.py b/base/common/python/pki/kra.py > index e3daabdd350383e6bd178779f8c0ea71aa0fe682..6559b986b05bc3e9049cdf7d95a8bef19a118cd8 100644 > --- a/base/common/python/pki/kra.py > +++ b/base/common/python/pki/kra.py > @@ -40,13 +40,13 @@ class KRAClient(object): > """ Constructor > > :param connection - PKIConnection object with DRM connection info. > - :param crypto - CryptoUtil object. NSSCryptoUtil is provided by > + :param crypto - CryptoProvider object. NSSCryptoProvider is provided by > default. If a different crypto implementation is > - desired, a different subclass of CryptoUtil must be > + desired, a different subclass of CryptoProvider must be > provided. > :param transport_cert_nick - identifier for the DRM transport > certificate. This will be passed to the > - CryptoUtil.get_cert() command to get a representation > + CryptoProvider.get_cert() command to get a representation > of the transport certificate usable for crypto ops. > Note that for NSS databases, the database must have been > initialized beforehand. > diff --git a/base/kra/functional/drmtest.py b/base/kra/functional/drmtest.py > index 9ef096194793a37f7ed3290d43e58a366522f718..abade351112c73e9c075631440dcae6797567971 100644 > --- a/base/kra/functional/drmtest.py > +++ b/base/kra/functional/drmtest.py > @@ -33,7 +33,7 @@ See drmtest.readme.txt. > > import base64 > import pki > -import pki.cryptoutil as cryptoutil > +import pki.crypto > import pki.key as key > import time > > @@ -80,11 +80,11 @@ def main(): > # create an NSS DB for crypto operations > certdb_dir = "/tmp/drmtest-certdb" > certdb_password = "redhat123" > - cryptoutil.NSSCryptoUtil.setup_database(certdb_dir, certdb_password, > + pki.crypto.NSSCryptoProvider.setup_database(certdb_dir, certdb_password, > over_write=True) > > #create kraclient > - crypto = cryptoutil.NSSCryptoUtil(certdb_dir, certdb_password) > + crypto = pki.crypto.NSSCryptoProvider(certdb_dir, certdb_password) > kraclient = KRAClient(connection, crypto) > keyclient = kraclient.keys > > -- > 1.8.4.2 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From ftweedal at redhat.com Wed Jul 2 06:47:49 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 2 Jul 2014 16:47:49 +1000 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working In-Reply-To: <53B36A53.9060904@redhat.com> References: <53B36A53.9060904@redhat.com> Message-ID: <20140702064749.GT2417@dhcp-40-8.bne.redhat.com> On Tue, Jul 01, 2014 at 07:11:31PM -0700, Matthew Harmsen wrote: > Please review the attached patch (based upon what was provided by Marko Karg > in Bugzilla Bug #1081916 - freeipa does not install on arm architecture > ) for the following > TRAC ticket: > > * PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working > > > CAVEAT: To test this, all I did was add the code to the template and build > on my 'x86_64' Fedora 20 machine. > I extracted 'AtoB' and 'BtoA' and confirmed that they now > include the 'armv7l' section, and although > I have attempted to request an 'arm' machine to test this > out, I have not yet been able to acquire a > test machine, so I do not know if the 'arch' command actually > returns 'armv7l'. > The person who filed this bug also provided the patch and confirmed that it worked for them. It would be nice to confirm it on ARM hardware but if we can't get some soon, it seems like it should be fine to merge. Fraser > From cc3baa89bf36250bfafbecb5736cbe4c66da9331 Mon Sep 17 00:00:00 2001 > From: Matthew Harmsen > Date: Tue, 1 Jul 2014 18:53:00 -0700 > Subject: [PATCH] Add 'arm' architecture to tool scripts. > > - Bugzilla Bug #1081916 - freeipa does not install on arm architecture > - PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working > --- > base/java-tools/templates/pki_java_command_wrapper.in | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/base/java-tools/templates/pki_java_command_wrapper.in b/base/java-tools/templates/pki_java_command_wrapper.in > index 5d28d0e..e42d191 100644 > --- a/base/java-tools/templates/pki_java_command_wrapper.in > +++ b/base/java-tools/templates/pki_java_command_wrapper.in > @@ -81,6 +81,10 @@ if [ "${OS}" = "Linux" ] ; then > LD_LIBRARY_PATH=/usr/lib64:/lib64:${LD_LIBRARY_PATH} > LD_LIBRARY_PATH=/usr/lib64/jss:${LD_LIBRARY_PATH} > export LD_LIBRARY_PATH > + elif [ "${ARCHITECTURE}" = "armv7l" ] ; then > + LD_LIBRARY_PATH=/usr/lib:/lib > + LD_LIBRARY_PATH=/usr/lib/jss:${LD_LIBRARY_PATH} > + export LD_LIBRARY_PATH > else > invalid_architecture "${ARCHITECTURE}" > exit 255 > -- > 1.9.3 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Wed Jul 2 12:39:11 2014 From: alee at redhat.com (Ade Lee) Date: Wed, 02 Jul 2014 20:39:11 +0800 Subject: [Pki-devel] [PATCH] 513 Renamed CryptoUtil to CryptoProvider. In-Reply-To: <53B357CF.3020500@redhat.com> References: <53B357CF.3020500@redhat.com> Message-ID: <1404304751.4916.0.camel@localhost.localdomain> ACK On Tue, 2014-07-01 at 19:52 -0500, Endi Sukma Dewata wrote: > The CryptoUtil classes in the Python client library has been renamed > to CryptoProvider for consistency with the Java client library. The > cryptoutil.py module has been renamed to crypto.py. > > Ticket #1042 > > The drmtest.py works fine with these changes. > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Wed Jul 2 12:41:30 2014 From: alee at redhat.com (Ade Lee) Date: Wed, 02 Jul 2014 20:41:30 +0800 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working In-Reply-To: <20140702064749.GT2417@dhcp-40-8.bne.redhat.com> References: <53B36A53.9060904@redhat.com> <20140702064749.GT2417@dhcp-40-8.bne.redhat.com> Message-ID: <1404304890.4916.1.camel@localhost.localdomain> Ditto - ACK On Wed, 2014-07-02 at 16:47 +1000, Fraser Tweedale wrote: > On Tue, Jul 01, 2014 at 07:11:31PM -0700, Matthew Harmsen wrote: > > Please review the attached patch (based upon what was provided by Marko Karg > > in Bugzilla Bug #1081916 - freeipa does not install on arm architecture > > ) for the following > > TRAC ticket: > > > > * PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working > > > > > > CAVEAT: To test this, all I did was add the code to the template and build > > on my 'x86_64' Fedora 20 machine. > > I extracted 'AtoB' and 'BtoA' and confirmed that they now > > include the 'armv7l' section, and although > > I have attempted to request an 'arm' machine to test this > > out, I have not yet been able to acquire a > > test machine, so I do not know if the 'arch' command actually > > returns 'armv7l'. > > > > The person who filed this bug also provided the patch and confirmed > that it worked for them. It would be nice to confirm it on ARM > hardware but if we can't get some soon, it seems like it should be > fine to merge. > > Fraser > > > From cc3baa89bf36250bfafbecb5736cbe4c66da9331 Mon Sep 17 00:00:00 2001 > > From: Matthew Harmsen > > Date: Tue, 1 Jul 2014 18:53:00 -0700 > > Subject: [PATCH] Add 'arm' architecture to tool scripts. > > > > - Bugzilla Bug #1081916 - freeipa does not install on arm architecture > > - PKI TRAC Ticket #935 - patch to BtoA and AtoB to get ARM working > > --- > > base/java-tools/templates/pki_java_command_wrapper.in | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/base/java-tools/templates/pki_java_command_wrapper.in b/base/java-tools/templates/pki_java_command_wrapper.in > > index 5d28d0e..e42d191 100644 > > --- a/base/java-tools/templates/pki_java_command_wrapper.in > > +++ b/base/java-tools/templates/pki_java_command_wrapper.in > > @@ -81,6 +81,10 @@ if [ "${OS}" = "Linux" ] ; then > > LD_LIBRARY_PATH=/usr/lib64:/lib64:${LD_LIBRARY_PATH} > > LD_LIBRARY_PATH=/usr/lib64/jss:${LD_LIBRARY_PATH} > > export LD_LIBRARY_PATH > > + elif [ "${ARCHITECTURE}" = "armv7l" ] ; then > > + LD_LIBRARY_PATH=/usr/lib:/lib > > + LD_LIBRARY_PATH=/usr/lib/jss:${LD_LIBRARY_PATH} > > + export LD_LIBRARY_PATH > > else > > invalid_architecture "${ARCHITECTURE}" > > exit 255 > > -- > > 1.9.3 > > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Wed Jul 2 14:42:29 2014 From: alee at redhat.com (Ade Lee) Date: Wed, 02 Jul 2014 22:42:29 +0800 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg In-Reply-To: <53AE3D7F.5000006@redhat.com> References: <53AE3D7F.5000006@redhat.com> Message-ID: <1404312149.4916.6.camel@localhost.localdomain> ACK. Looks good. Two small nits only - just fix and check in. 1. In operations, no need to further indent the comments in backup_instance_configuration_files(). 2. At the end of start_instance() , you do: return $? Why not just move that into the if .. then .. fi clause above? ie. replace : rv=$? with return $? Ade On Fri, 2014-06-27 at 20:58 -0700, Matthew Harmsen wrote: > Please review the attached patch for: > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of > CS.cfg > > This patch is based upon a previously reviewed patch for the Dogtag 9 > architecture utilized by the IPA_v2_RHEL_6_ERRATA_BRANCH, but was > modified and tested to work with the Dogtag 10.2 architecture. > > > CAVEAT 1: > > > Although this patch contains changes to multiple PKI > subsystem's 'CS.cfg' configuration files, an upgrade script > should not be specifically required for legacy instances since > the parameter that is added, > 'archive.configuration_file=true', is presumed even if the > parameter is missing (as it would be on any legacy instance). > In this case, it would only be necessary to add this parameter > to a legacy instance's CS.cfg, and set the value to 'false' in > order to turn off 'CS.cfg' configuration file archival > (explicit instructions detailing this are found in the > 'operations' script). However, if this is desired for > completeness, I don't mind adding it. > CAVEAT 2: > I had originally made the effort to attempt to have specific > crucial WARNING messages echoed to the display as well as to > the journal. I believe that this would be beneficial, as, for > example, it would immediately notify an admin that since an > error had occurred, 'CS.cfg' backups would be discontinued > until the error was corrected. My idea was to echo these > WARNING messages explicitly to stderr via redirecting them > (>&2), and adding the parameter 'StandardError=journal > +console' under the [Service] section of the > 'pki-tomcatd at pki-tomcat.service' file. Unfortunately, I was > never able to make this work - both stdout and stderr messages > were stored in the journal, but were never displayed to the > screen when typing 'systemctl restart > pki-tomcatd at pki-tomcat.service' (even after a 'systemctl > daemon-reload' had been performed). > -- Matt > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Wed Jul 2 17:17:23 2014 From: alee at redhat.com (Ade Lee) Date: Thu, 03 Jul 2014 01:17:23 +0800 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> Message-ID: <1404321443.4916.30.camel@localhost.localdomain> On Tue, 2014-07-01 at 12:42 +1000, Fraser Tweedale wrote: > On Mon, Jun 30, 2014 at 02:28:09PM -0500, Endi Sukma Dewata wrote: > > Hi, I have some comments about the design: > > http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage > > > > I think all system/default profiles should remain file-based and all custom > > profiles should be LDAP-based. It will make a clean separation: system > > profiles are owned by us (Dogtag developers), custom profiles are owned by > > the admin. > > > > The system profiles will be read-only. This way we will be able to update > > the system profiles without writing any upgrade scripts because the files > > will be updated automatically by RPM. Just one requirement, all server > > instances must be upgraded to the same version. > > > > If the admin wants to change a system profile, they can clone it into a > > custom profile and make the changes there. The custom profiles cannot have > > the same names as the system profiles, so there's won't be any > > conflict/confusion, and no need to support a "restore" command. In general > > we won't need to write upgrade scripts for custom profiles except if we > > change the LDAP schema. > > > > This was my original proposal, with the exception of making > system/default profiles read-only and cloning them to make > modifications. I think this is a good modification to that > proposal. > > Ade was in favour of moving to LDAP only, but perhaps this variation > makes using a mix of LDAP and file-based profiles more palatable? > It would be good to hear from more people on this part of the design > - to see if there's a clear winner. > The problem with the above proposal is one of migration. Customers are used to both creating new profiles and modifying existing profiles to suit their needs. If we go along with this mechanism, we would have to tell these customers that they would need to rename their existing customized profiles (and update all the relevant clients etc.) There is another problem, and that is that it is not clear that we want updates to the default profiles to be propagated to existing instances. I have looked at the profiles and there have been only a handful of changes over the last 7 years. Those changes include things like updating the default signing algorithms or the default validity. More likely than not, admins would prefer that we not change the behavior of profiles in existing instances underneath them. The changes that I have found are all behavioral - and therefore things that admin can opt out of -- or would prefer to do on their own schedule. There have been no structural changes. If there are structural changes, then we need to (and can) provide an upgrade script which would run with the automatic upgrade. An example of this would be a schema upgrade as we sort out how to represent profiles in LDAP. So, I think we should do the following: 1. Continue to provide the system profiles in files. These files will be parsed and stored in LDAP when an instance is created. 2. All profiles for an instance should live in LDAP. This makes it simple - no need to check to see if a profile is in LDAP or files or both, and which has priority etc. Tools will be provided to manage/ create/delete profiles etc. 3. Updates to system profile files will not affect the existing LDAP profiles. We can provide update scripts or manual instructions for admins to run when they opt to do so. This will be for behavioral changes. 4. Structural changes will be done using upgrade scripts using the database upgrade mechanism. This framework is something that we had planned to do already in 10.3. We already have a model on how to do this in our current upgrade framework. > > About the schema itself, I suppose we want to have something that resembles > > the actual Profile data structure (see ProfileData Java class). There should > > be an LDAP attribute for each single-valued Java attribute (e.g. name, > > description, enabled, visible). This way the profile is more manageable and > > can be queried based on these attributes. For collection attributes (e.g. > > inputs, outputs, policySets) we can use child LDAP entries to represent > > them. > > > > About the names, right now we use "profile" to refer to both cert profile in > > CA and token profile in TPS. We probably should use separate terms to > > distinguish them (e.g. CertProfile and TokenProfile). This applies to > > Java/Python class names, REST interface, CLI, LDAP schema, etc. > > > Good point. I've s/profile/certProfile/ in the schema. > > > So the LDAP entries for a profile may look like this: > > > > dn: cn=MyUserCert,ou=Profiles,ou=CA,{suffix} > > objectClass: certProfile > > cn: MyUserCert > > displayName: Manual User Dual-Use Certificate Enrollment > > description: This certificate profile is for enrolling user certificates. > > visible: true > > enabled: true > > enabledBy: admin > > > > dn: ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,{suffix} > > objectClass: organizationalUnit > > ou: Inputs > > > > dn: cn=i1,ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,o=pki-tomcat > > objectClass: pkiCertProfileInput > > cn: i1 > > classId: keyGenInputImpl > > > > Introducing this schema (and the more fine-grained functionality it > would support) could happen later. I see the benefits but I think > it's better to just get the profiles in LDAP as soon as possible, > then look at how we can break them down to enable richer features. > I would prefer to see if we could design something more fine grained at first, rather than assuming an upgrade. If nothing else, you will need to separate out whether the profile is visible and enabled and by whom. > Also, if we retain file-based profiles as you suggest, whatever > queries LDAP makes easier, we would still need to be able to do > those same queries across the file-based profiles. So while > file-based profiles remain, a finer-grained LDAP profile schema > doesn't buy us anything in terms of reducing complexity or the > amount of code to write. > > > About the REST interface & CLI, since this will be the primary way to edit > > profiles, we might want to have more granular commands to modify parts of > > the profile. Right now with ca-profile-mod command you need to send the > > entire profile in a file. It would be nice to be able to specify some > > parameters to change certain attributes only, or use separate commands to > > manage the inputs/outputs. > > > > We'll also need an interface to find existing cert records that use a > > certain profile and bulk modify them to use a different profile. This will > > be useful when you create a clone to change the system profile. > > > > Again, I agree with the benefits here, but feel that this may be out > of scope for the initial changeover to LDAP profiles, given how much > we need that for FreeIPA. > Agreed. I think this can be a later development. > > I'm not sure if we should support 10.2 -> 10.3 cloning. When we release 10.3 > > the 10.2 will still be fairly new so it might be reasonable to require all > > clones to be upgraded. It will reduce the amount of testing requirement too. > > > I'll incorporate this comment (and all your others) in the design > proposal; you should see the updated document soon. > > Cheers, > > Fraser > > > -- > > Endi S. Dewata > > > > On 6/18/2014 2:44 AM, Fraser Tweedale wrote: > > >Hi all, > > > > > >A requirement from the FreeIPA side is the ability to add and > > >customise CA profiles. Dogtag's current profile creation behaviour > > >writes the new profile to the filesystem beside the standard > > >profiles (as well as making the appropriate update to the registry, > > >etc.) > > > > > >There does not seem to be a mechanism to distribute new/modified > > >profiles to replicas - though perhaps I have missed something. > > > > > >Because this behaviour is required, unless I have overlooked > > >something or there is a better way (in which case please shout out), > > >I think it makes sense to begin a design proposal for an LDAP-based > > >profile store. > > > > > >Finally, a brief mention of some tickets related to profile storage > > >that could be good to tackle simultaneously should the proposed > > >change go ahead: > > > > > >- https://fedorahosted.org/pki/ticket/778 > > >- https://fedorahosted.org/freeipa/ticket/4002 > > > > > >_______________________________________________ > > >Pki-devel mailing list > > >Pki-devel at redhat.com > > >https://www.redhat.com/mailman/listinfo/pki-devel > > > > > From edewata at redhat.com Wed Jul 2 17:59:35 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 02 Jul 2014 12:59:35 -0500 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg In-Reply-To: <53AE3D7F.5000006@redhat.com> References: <53AE3D7F.5000006@redhat.com> Message-ID: <53B44887.7020205@redhat.com> On 6/27/2014 10:58 PM, Matthew Harmsen wrote: > CAVEAT 1: > > Although this patch contains changes to multiple PKI subsystem's > 'CS.cfg' configuration files, an upgrade script should not be > specifically required for legacy instances since the parameter that > is added, 'archive.configuration_file=true', is presumed even if the > parameter is missing (as it would be on any legacy instance). In > this case, it would only be necessary to add this parameter to a > legacy instance's CS.cfg, and set the value to 'false' in order to > turn off 'CS.cfg' configuration file archival (explicit instructions > detailing this are found in the 'operations' script). However, if > this is desired for completeness, I don't mind adding it. Since by default archive.configuration_file=true, it's not necessary to add the parameter in the CS.cfg.in for new instances. If we remove the parameter from this patch, it is not necessary to write an upgrade script since old and new CS.cfg will be identical (in terms of this parameter). We just need to document the parameter in the man page in case the admin wants to change the behavior. I think in general we shouldn't put parameters in CS.cfg that have the same value as the default value. That will defeat the purpose of having a default value in the first place, will add more things to maintain, and also will be difficult to change later because we won't be able to tell whether the value was intentionally set by the admin. This is more of a long term goal. The CS.cfg itself should not be a place to document or define default values because of the reasons above. The CS.cfg should only be used to customize Dogtag behavior if it's different from the standard behavior. If someone uses Dogtag mostly with the default settings, the CS.cfg should be very small. -- Endi S. Dewata From cfu at redhat.com Wed Jul 2 18:09:16 2014 From: cfu at redhat.com (Christina Fu) Date: Wed, 02 Jul 2014 11:09:16 -0700 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <1404321443.4916.30.camel@localhost.localdomain> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> Message-ID: <53B44ACC.4090200@redhat.com> In general, I think we all agree that putting profiles on ldap is a good idea. However, in practice, I think we want to allow admins to make changes to local files as they see fit. Please see my in-line response below. On 07/02/2014 10:17 AM, Ade Lee wrote: > On Tue, 2014-07-01 at 12:42 +1000, Fraser Tweedale wrote: >> On Mon, Jun 30, 2014 at 02:28:09PM -0500, Endi Sukma Dewata wrote: >>> Hi, I have some comments about the design: >>> http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage >>> >>> I think all system/default profiles should remain file-based and all custom >>> profiles should be LDAP-based. It will make a clean separation: system >>> profiles are owned by us (Dogtag developers), custom profiles are owned by >>> the admin. >>> >>> The system profiles will be read-only. This way we will be able to update >>> the system profiles without writing any upgrade scripts because the files >>> will be updated automatically by RPM. Just one requirement, all server >>> instances must be upgraded to the same version. >>> >>> If the admin wants to change a system profile, they can clone it into a >>> custom profile and make the changes there. The custom profiles cannot have >>> the same names as the system profiles, so there's won't be any >>> conflict/confusion, and no need to support a "restore" command. In general >>> we won't need to write upgrade scripts for custom profiles except if we >>> change the LDAP schema. >>> >> This was my original proposal, with the exception of making >> system/default profiles read-only and cloning them to make >> modifications. I think this is a good modification to that >> proposal. >> >> Ade was in favour of moving to LDAP only, but perhaps this variation >> makes using a mix of LDAP and file-based profiles more palatable? >> It would be good to hear from more people on this part of the design >> - to see if there's a clear winner. >> Perhaps we can view files v.s. ldap as "localized" v.s. "centralized" instead of "system" v.s. "customized". Again, the benefit of putting profiles in ldap is so that it can be configured once and utilized by many. My comments below are based on such view. > The problem with the above proposal is one of migration. Customers are > used to both creating new profiles and modifying existing profiles to > suit their needs. If we go along with this mechanism, we would have to > tell these customers that they would need to rename their existing > customized profiles (and update all the relevant clients etc.) > > There is another problem, and that is that it is not clear that we want > updates to the default profiles to be propagated to existing instances. > I have looked at the profiles and there have been only a handful of > changes over the last 7 years. Those changes include things like > updating the default signing algorithms or the default validity. More > likely than not, admins would prefer that we not change the behavior of > profiles in existing instances underneath them. > > The changes that I have found are all behavioral - and therefore things > that admin can opt out of -- or would prefer to do on their own > schedule. There have been no structural changes. > > If there are structural changes, then we need to (and can) provide an > upgrade script which would run with the automatic upgrade. An example > of this would be a schema upgrade as we sort out how to represent > profiles in LDAP. > > So, I think we should do the following: > 1. Continue to provide the system profiles in files. These files will > be parsed and stored in LDAP when an instance is created. if the ldap copies are the "centralized" ones, then we don't want to override them at each installation of an instance. We could give admins an option to override what's stored in ldap during configuration. When a profile with the same id exists in both ldap and local system, then by default the one on ldap takes precedence. Although we could add "per profile" config param for each instance to change the view (remember the unix ns/yp config where you can set the order of the source? so if "file" is 1st in the order, then the local ones have priority, etc.) With this, a site that has carefully crafted their profiles and stored in ldap to share among all ca instances don't have to bother changing anything for each ca instance created (because they are default to the ldap). Upgrade will also not affect the ldap copies inadvertently. > > 2. All profiles for an instance should live in LDAP. This makes it > simple - no need to check to see if a profile is in LDAP or files or > both, and which has priority etc. Tools will be provided to manage/ > create/delete profiles etc. It might be simple, but we need to think about what's the value of them existing in ldap and not being shared by others. > > 3. Updates to system profile files will not affect the existing LDAP > profiles. We can provide update scripts or manual instructions for > admins to run when they opt to do so. This will be for behavioral > changes. so, in the view I specified above, update to the "local" profiles will only affect the instance where they exist, however, we can allow option for admins to "publish" to ldap. > > 4. Structural changes will be done using upgrade scripts using the > database upgrade mechanism. This framework is something that we had > planned to do already in 10.3. We already have a model on how to do > this in our current upgrade framework. > >>> About the schema itself, I suppose we want to have something that resembles >>> the actual Profile data structure (see ProfileData Java class). There should >>> be an LDAP attribute for each single-valued Java attribute (e.g. name, >>> description, enabled, visible). This way the profile is more manageable and >>> can be queried based on these attributes. For collection attributes (e.g. >>> inputs, outputs, policySets) we can use child LDAP entries to represent >>> them. >>> >>> About the names, right now we use "profile" to refer to both cert profile in >>> CA and token profile in TPS. We probably should use separate terms to >>> distinguish them (e.g. CertProfile and TokenProfile). This applies to >>> Java/Python class names, REST interface, CLI, LDAP schema, etc. >>> >> Good point. I've s/profile/certProfile/ in the schema. >> >>> So the LDAP entries for a profile may look like this: >>> >>> dn: cn=MyUserCert,ou=Profiles,ou=CA,{suffix} >>> objectClass: certProfile >>> cn: MyUserCert >>> displayName: Manual User Dual-Use Certificate Enrollment >>> description: This certificate profile is for enrolling user certificates. >>> visible: true >>> enabled: true >>> enabledBy: admin >>> >>> dn: ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,{suffix} >>> objectClass: organizationalUnit >>> ou: Inputs >>> >>> dn: cn=i1,ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,o=pki-tomcat >>> objectClass: pkiCertProfileInput >>> cn: i1 >>> classId: keyGenInputImpl >>> >> Introducing this schema (and the more fine-grained functionality it >> would support) could happen later. I see the benefits but I think >> it's better to just get the profiles in LDAP as soon as possible, >> then look at how we can break them down to enable richer features. >> > I would prefer to see if we could design something more fine grained at > first, rather than assuming an upgrade. If nothing else, you will need > to separate out whether the profile is visible and enabled and by whom. > >> Also, if we retain file-based profiles as you suggest, whatever >> queries LDAP makes easier, we would still need to be able to do >> those same queries across the file-based profiles. So while >> file-based profiles remain, a finer-grained LDAP profile schema >> doesn't buy us anything in terms of reducing complexity or the >> amount of code to write. >> >>> About the REST interface & CLI, since this will be the primary way to edit >>> profiles, we might want to have more granular commands to modify parts of >>> the profile. Right now with ca-profile-mod command you need to send the >>> entire profile in a file. It would be nice to be able to specify some >>> parameters to change certain attributes only, or use separate commands to >>> manage the inputs/outputs. >>> >>> We'll also need an interface to find existing cert records that use a >>> certain profile and bulk modify them to use a different profile. This will >>> be useful when you create a clone to change the system profile. >>> >> Again, I agree with the benefits here, but feel that this may be out >> of scope for the initial changeover to LDAP profiles, given how much >> we need that for FreeIPA. >> > Agreed. I think this can be a later development. > >>> I'm not sure if we should support 10.2 -> 10.3 cloning. When we release 10.3 >>> the 10.2 will still be fairly new so it might be reasonable to require all >>> clones to be upgraded. It will reduce the amount of testing requirement too. >>> >> I'll incorporate this comment (and all your others) in the design >> proposal; you should see the updated document soon. >> >> Cheers, >> >> Fraser >> >>> -- >>> Endi S. Dewata >>> >>> On 6/18/2014 2:44 AM, Fraser Tweedale wrote: >>>> Hi all, >>>> >>>> A requirement from the FreeIPA side is the ability to add and >>>> customise CA profiles. Dogtag's current profile creation behaviour >>>> writes the new profile to the filesystem beside the standard >>>> profiles (as well as making the appropriate update to the registry, >>>> etc.) >>>> >>>> There does not seem to be a mechanism to distribute new/modified >>>> profiles to replicas - though perhaps I have missed something. >>>> >>>> Because this behaviour is required, unless I have overlooked >>>> something or there is a better way (in which case please shout out), >>>> I think it makes sense to begin a design proposal for an LDAP-based >>>> profile store. >>>> >>>> Finally, a brief mention of some tickets related to profile storage >>>> that could be good to tackle simultaneously should the proposed >>>> change go ahead: >>>> >>>> - https://fedorahosted.org/pki/ticket/778 >>>> - https://fedorahosted.org/freeipa/ticket/4002 >>>> >>>> _______________________________________________ >>>> Pki-devel mailing list >>>> Pki-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/pki-devel >>>> > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From edewata at redhat.com Wed Jul 2 21:02:26 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 02 Jul 2014 16:02:26 -0500 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <1404321443.4916.30.camel@localhost.localdomain> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> Message-ID: <53B47362.80609@redhat.com> On 7/2/2014 12:17 PM, Ade Lee wrote: > The problem with the above proposal is one of migration. Customers are > used to both creating new profiles and modifying existing profiles to > suit their needs. If we go along with this mechanism, we would have to > tell these customers that they would need to rename their existing > customized profiles (and update all the relevant clients etc.) Just to clarify, so there are 3 proposals: 1. Stacked profiles (Fraser's proposal) 2. Pure LDAP profiles (Ade's proposal) 3. Separate file/LDAP profiles (my proposal) As discussed over IRC, proposal #3 will be supplemented with a mechanism to allow creating aliases for backward compatibility. I'm trying to generalize the mechanism to become "profile inheritance". Basically each LDAP profile will have an optional parent. The parent can be the file-based system/default profile, or another LDAP profile. A sub-profile will inherit all attributes, except when it's explicitly declared in the sub-profile. This mechanism allows us to create just a proxy/alias, a full clone, or anything in between. For example, a proxy profile might only have a few attributes: dn: cn=caAdminCert,ou=Profiles,ou=CA,{suffix} objectClass: certProfile cn: caAdminCert parent: defaultAdminCert visible: true Let's say in the old system we have an unchanged caAdminCert and a customized caUserCert. With proposal #1 we'll have: 1. caAdminCert (file) 2. caUserCert (LDAP) 3. caUserCert (file, inaccessible via REST due to the LDAP entry) With proposal #2 we'll have: 1. caAdminCert (LDAP) 2. caUserCert (LDAP) 3. caAdminCert (file, for initial installation only) 4. caUserCert (file, for initial installation only) With proposal #3 we'll have: 1. caAdminCert -> defaultAdminCert (proxy) 2. caUserCert (LDAP) 3. defaultAdminCert (file) 4. defaultUserCert (file) The benefits of proposal #3 are: 1. The system/default profiles will be stored in /usr/share/pki. People will be naturally discouraged to change them since it will be overwritten during RPM upgrade. On the other hand, an LDAP profile, although we can flag it as system profile or make it read-only, is more tempting to change. 2. The original/unchanged system profiles will be accessible via REST as well so people can use it directly and rely on auto update. 3. Backward compatibility can be maintained with proxy profiles. These profiles will be updated automatically during RPM upgrade since it's pointing to the file-based system profiles. No need to write upgrade scripts. 4. The proxy profiles can be used to hide certain system profiles without creating a full clone. With stacking, to hide caAdminCert we probably would have to create a full clone then change the visibility, and lose the auto update ability. 5. The mechanism can be used to extend/change the behavior of the default profiles without having to create the full custom profile. For example we want to create IPA-specific caUserCert with different inputs. In this case we'll be able to just create a sub-profile with the new inputs. No need to clone the entire profile. > There is another problem, and that is that it is not clear that we want > updates to the default profiles to be propagated to existing instances. > I have looked at the profiles and there have been only a handful of > changes over the last 7 years. Those changes include things like > updating the default signing algorithms or the default validity. More > likely than not, admins would prefer that we not change the behavior of > profiles in existing instances underneath them. > > The changes that I have found are all behavioral - and therefore things > that admin can opt out of -- or would prefer to do on their own > schedule. There have been no structural changes. > > If there are structural changes, then we need to (and can) provide an > upgrade script which would run with the automatic upgrade. An example > of this would be a schema upgrade as we sort out how to represent > profiles in LDAP. As discussed, we're separating database changes into: 1. structural changes (e.g. schema change, adding/removing attributes) 2. behavioral changes (e.g. changing SHA1 to SHA512, default validity) The structural changes will be semi-automatic. The admin decides when to run them, but it's not necessary to review them one-by-one. The behavioral changes should be reviewed by a security expert to make sure that it won't affect their particular system negatively. With proposal #3 the admin can decide whether to (a) trust us (developers) to make the behavioral changes automatically by using the default profiles directly or using proxy profiles, or (b) maintain a full control of all behavioral changes using custom profiles. If all profiles are in LDAP they might be limited to option (b) because the upgrade script won't be able to tell whether the current value is a default value that we can change, or it was intentionally set by the admin. -- Endi S. Dewata From cfu at redhat.com Wed Jul 2 21:42:16 2014 From: cfu at redhat.com (Christina Fu) Date: Wed, 02 Jul 2014 14:42:16 -0700 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B47362.80609@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B47362.80609@redhat.com> Message-ID: <53B47CB8.5070802@redhat.com> IMHO, I think #3 is way too complicated. Complication invites issues and confuse people. Could we step back and try something simpler? When you copy the content of one profile and modify it to create a new one, then it's a new profile standing on its own. Why the parent-child relationship and all? Seems like an administrator's nightmare. Maybe I missed out on the irc discussion, but could you please give us a summary of the benefit and how the benefit weights against development time and administration maintenance, and support effort in the future on our end? Anyway, I hope you will consider what I said in my earlier response. I thought our goal was to provide a "centralized collection of profiles" to ease administration effort. I hope we achieve simplicity rather than create complication. It's just my personal preference. thanks, Christina On 07/02/2014 02:02 PM, Endi Sukma Dewata wrote: > On 7/2/2014 12:17 PM, Ade Lee wrote: >> The problem with the above proposal is one of migration. Customers are >> used to both creating new profiles and modifying existing profiles to >> suit their needs. If we go along with this mechanism, we would have to >> tell these customers that they would need to rename their existing >> customized profiles (and update all the relevant clients etc.) > > Just to clarify, so there are 3 proposals: > 1. Stacked profiles (Fraser's proposal) > 2. Pure LDAP profiles (Ade's proposal) > 3. Separate file/LDAP profiles (my proposal) > > As discussed over IRC, proposal #3 will be supplemented with a > mechanism to allow creating aliases for backward compatibility. I'm > trying to generalize the mechanism to become "profile inheritance". > > Basically each LDAP profile will have an optional parent. The parent > can be the file-based system/default profile, or another LDAP profile. > A sub-profile will inherit all attributes, except when it's explicitly > declared in the sub-profile. This mechanism allows us to create just a > proxy/alias, a full clone, or anything in between. For example, a > proxy profile might only have a few attributes: > > dn: cn=caAdminCert,ou=Profiles,ou=CA,{suffix} > objectClass: certProfile > cn: caAdminCert > parent: defaultAdminCert > visible: true > > Let's say in the old system we have an unchanged caAdminCert and a > customized caUserCert. With proposal #1 we'll have: > 1. caAdminCert (file) > 2. caUserCert (LDAP) > 3. caUserCert (file, inaccessible via REST due to the LDAP entry) > > With proposal #2 we'll have: > 1. caAdminCert (LDAP) > 2. caUserCert (LDAP) > 3. caAdminCert (file, for initial installation only) > 4. caUserCert (file, for initial installation only) > > With proposal #3 we'll have: > 1. caAdminCert -> defaultAdminCert (proxy) > 2. caUserCert (LDAP) > 3. defaultAdminCert (file) > 4. defaultUserCert (file) > > The benefits of proposal #3 are: > > 1. The system/default profiles will be stored in /usr/share/pki. > People will be naturally discouraged to change them since it will be > overwritten during RPM upgrade. On the other hand, an LDAP profile, > although we can flag it as system profile or make it read-only, is > more tempting to change. > > 2. The original/unchanged system profiles will be accessible via REST > as well so people can use it directly and rely on auto update. > > 3. Backward compatibility can be maintained with proxy profiles. These > profiles will be updated automatically during RPM upgrade since it's > pointing to the file-based system profiles. No need to write upgrade > scripts. > > 4. The proxy profiles can be used to hide certain system profiles > without creating a full clone. With stacking, to hide caAdminCert we > probably would have to create a full clone then change the visibility, > and lose the auto update ability. > > 5. The mechanism can be used to extend/change the behavior of the > default profiles without having to create the full custom profile. For > example we want to create IPA-specific caUserCert with different > inputs. In this case we'll be able to just create a sub-profile with > the new inputs. No need to clone the entire profile. > >> There is another problem, and that is that it is not clear that we want >> updates to the default profiles to be propagated to existing instances. >> I have looked at the profiles and there have been only a handful of >> changes over the last 7 years. Those changes include things like >> updating the default signing algorithms or the default validity. More >> likely than not, admins would prefer that we not change the behavior of >> profiles in existing instances underneath them. >> >> The changes that I have found are all behavioral - and therefore things >> that admin can opt out of -- or would prefer to do on their own >> schedule. There have been no structural changes. >> >> If there are structural changes, then we need to (and can) provide an >> upgrade script which would run with the automatic upgrade. An example >> of this would be a schema upgrade as we sort out how to represent >> profiles in LDAP. > > As discussed, we're separating database changes into: > 1. structural changes (e.g. schema change, adding/removing attributes) > 2. behavioral changes (e.g. changing SHA1 to SHA512, default validity) > > The structural changes will be semi-automatic. The admin decides when > to run them, but it's not necessary to review them one-by-one. The > behavioral changes should be reviewed by a security expert to make > sure that it won't affect their particular system negatively. > > With proposal #3 the admin can decide whether to (a) trust us > (developers) to make the behavioral changes automatically by using the > default profiles directly or using proxy profiles, or (b) maintain a > full control of all behavioral changes using custom profiles. If all > profiles are in LDAP they might be limited to option (b) because the > upgrade script won't be able to tell whether the current value is a > default value that we can change, or it was intentionally set by the > admin. > From edewata at redhat.com Thu Jul 3 02:51:12 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 02 Jul 2014 21:51:12 -0500 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B44ACC.4090200@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> Message-ID: <53B4C520.3060502@redhat.com> On 7/2/2014 1:09 PM, Christina Fu wrote: > In general, I think we all agree that putting profiles on ldap is a > good idea. However, in practice, I think we want to allow admins to > make changes to local files as they see fit. > Please see my in-line response below. This is probably something that the admins will have to adapt eventually. I think in a replicated environment we shouldn't treat each server as unique individuals with "local" settings in files (including profiles). From user's perspective all replicas should be (mostly) identical. An admin, agent, or end entity should be able to go to any replica and get the same services, see the same data, etc. For this to work consistently, we want to minimize possible misconfiguration, so the number of local files should be minimized. The configuration (including profiles) should gradually be moved to LDAP, and the CLI/UI will be the primary way to manage the configuration. To help the transition, CLI like proposed by Fraser could provide the experience of editing a local file but it's actually stored on LDAP. Also in the future the UI could provide an intuitive interface to manage the profiles, so the admins will no longer need to memorize the syntax of profile configuration file. > Perhaps we can view files v.s. ldap as "localized" v.s. "centralized" > instead of "system" v.s. "customized". > Again, the benefit of putting profiles in ldap is so that it can be > configured once and utilized by many. > My comments below are based on such view. Let's call this proposal #4: Localized & centralized profiles. If I understand this correctly, as an example server1 may have local profile1 and profile2, server2 may have local profile1 and profile3, then we have centralized LDAP profile1 and profile4. The profile1 in server1, server2, and LDAP can be completely different although they share the same name. >> 1. Continue to provide the system profiles in files. These files will >> be parsed and stored in LDAP when an instance is created. > if the ldap copies are the "centralized" ones, then we don't want to > override them at each installation of an instance. We could give admins > an option to override what's stored in ldap during configuration. I think the process to import the system profiles into LDAP should be done only once regardless of the number of replicas. Otherwise, suppose we removed one of the centralized profile (e.g. profile1), it might get added again when we install another replica. > When a profile with the same id exists in both ldap and local system, > then by default the one on ldap takes precedence. Although we could add > "per profile" config param for each instance to change the view > (remember the unix ns/yp config where you can set the order of the > source? so if "file" is 1st in the order, then the local ones have > priority, etc.) I don't have a lot of experience as system admin, but I think managing individual machines is difficult so NIS (or LDAP, NFS, DNS, etc.) is used to transition to a centrally managed system. The order of sources is set up such that eventually everything can be moved into a single source. For example, once we have NIS, we add new users in NIS instead of locally on each machine (unless absolutely necessary) even if a user might use a specific machine only. > With this, a site that has carefully crafted their profiles and stored > in ldap to share among all ca instances don't have to bother changing > anything for each ca instance created (because they are default to the > ldap). > Upgrade will also not affect the ldap copies inadvertently. >> 2. All profiles for an instance should live in LDAP. This makes it >> simple - no need to check to see if a profile is in LDAP or files or >> both, and which has priority etc. Tools will be provided to manage/ >> create/delete profiles etc. > It might be simple, but we need to think about what's the value of them > existing in ldap and not being shared by others. What is the use case for having local profiles that will not or cannot be shared with other replicas? If sharing the profile doesn't pose any problem we might as well store "local" profiles in LDAP so they can be handled more consistently. >> 3. Updates to system profile files will not affect the existing LDAP >> profiles. We can provide update scripts or manual instructions for >> admins to run when they opt to do so. This will be for behavioral >> changes. > so, in the view I specified above, update to the "local" profiles will > only affect the instance where they exist, however, we can allow option > for admins to "publish" to ldap. The problem with local profiles in files is that they are not replicated, so in case the machine is hosed, all local settings is gone, including the profiles. Unless the admin created a backup for each machine, it will be difficult to restore the machine quickly. The replicated LDAP profiles will take care of this problem automatically. In other proposals the system profiles in files are read-only and not customizable, so there's no local profiles that need to be backed up. -- Endi S. Dewata From edewata at redhat.com Thu Jul 3 02:51:46 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 02 Jul 2014 21:51:46 -0500 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B47CB8.5070802@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B47362.80609@redhat.com> <53B47CB8.5070802@redhat.com> Message-ID: <53B4C542.4080101@redhat.com> On 7/2/2014 4:42 PM, Christina Fu wrote: > IMHO, I think #3 is way too complicated. Complication invites issues > and confuse people. > > Could we step back and try something simpler? When you copy the content > of one profile and modify it to create a new one, then it's a new > profile standing on its own. Why the parent-child relationship and > all? Seems like an administrator's nightmare. Maybe I missed out on the > irc discussion, but could you please give us a summary of the benefit > and how the benefit weights against development time and administration > maintenance, and support effort in the future on our end? Proposal #3 can be simplified into #3(a) without profile inheritance. Here we'll support just the file-based system profile, proxy LDAP profile, and custom LDAP profile. For immediate purposes this should be sufficient. Profile inheritance is an idea that just came up after the IRC discussion. It would take a whole separate design page to see how it works, the potential benefits, and the impact on us. We don't have to implement this now, but later if we determine that it is useful, we can implement #3(b) with profile inheritance without changing the schema or the existing data. > Anyway, I hope you will consider what I said in my earlier response. I > thought our goal was to provide a "centralized collection of profiles" > to ease administration effort. I hope we achieve simplicity rather than > create complication. It's just my personal preference. Not disagreeing with that. Simplicity is always a goal, but sometimes the proper solution cannot be the simplest one. -- Endi S. Dewata From mharmsen at redhat.com Thu Jul 3 05:12:14 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Wed, 02 Jul 2014 22:12:14 -0700 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #832 - -Remove legacy 'systemctl' files . . . Message-ID: <53B4E62E.7090003@redhat.com> Please review the attached patch for: * PKI TRAC Ticket #832 - Remove legacy 'systemctl' files . . . This patch has been tested with both GUI and non-GUI configurations. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 20140702-Remove-legacy-systemctl-files.patch Type: text/x-patch Size: 112572 bytes Desc: not available URL: From cfu at redhat.com Thu Jul 3 16:18:59 2014 From: cfu at redhat.com (Christina Fu) Date: Thu, 03 Jul 2014 09:18:59 -0700 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B4C520.3060502@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> <53B4C520.3060502@redhat.com> Message-ID: <53B58273.60005@redhat.com> Actually, I did not know that this discussion was restricted to a replicated ("clone") environment. My view was more for a general organizational environment where there are multiple sub-ca's for different departments (which are not necessarily clones, but some could be) and there are two types of profiles: 1. centralized profiles (shared by all) 2. local profiles (customized by each department) I think this view (let's adopt your term and call it proposal #4) is more flexible in serving both non-clones and clones and yet retains the simplicity. You are right, Endi, about how "local profiles" don't necessarily have to be on the disk. It's just a personal preference that I feel most comfortable editing files directly than using ui's. I have never used the java console to edit profiles. I don't know if there are others who feel the same way. Maybe a market research on administrators is needed here. a few more comments in-line below... On 07/02/2014 07:51 PM, Endi Sukma Dewata wrote: > On 7/2/2014 1:09 PM, Christina Fu wrote: >> In general, I think we all agree that putting profiles on ldap is a >> good idea. However, in practice, I think we want to allow admins to >> make changes to local files as they see fit. >> Please see my in-line response below. > > This is probably something that the admins will have to adapt > eventually. I think in a replicated environment we shouldn't treat > each server as unique individuals with "local" settings in files > (including profiles). From user's perspective all replicas should be > (mostly) identical. An admin, agent, or end entity should be able to > go to any replica and get the same services, see the same data, etc. > For this to work consistently, we want to minimize possible > misconfiguration, so the number of local files should be minimized. > The configuration (including profiles) should gradually be moved to > LDAP, and the CLI/UI will be the primary way to manage the configuration. again, what I had in mind was a bunch of departments under the same organization where they might want to share some but customize some. > > To help the transition, CLI like proposed by Fraser could provide the > experience of editing a local file but it's actually stored on LDAP. > Also in the future the UI could provide an intuitive interface to > manage the profiles, so the admins will no longer need to memorize the > syntax of profile configuration file. Yes to the CLI's and UI's. > >> Perhaps we can view files v.s. ldap as "localized" v.s. "centralized" >> instead of "system" v.s. "customized". >> Again, the benefit of putting profiles in ldap is so that it can be >> configured once and utilized by many. >> My comments below are based on such view. > > Let's call this proposal #4: Localized & centralized profiles. If I > understand this correctly, as an example server1 may have local > profile1 and profile2, server2 may have local profile1 and profile3, > then we have centralized LDAP profile1 and profile4. The profile1 in > server1, server2, and LDAP can be completely different although they > share the same name. Yes. > >>> 1. Continue to provide the system profiles in files. These files will >>> be parsed and stored in LDAP when an instance is created. > >> if the ldap copies are the "centralized" ones, then we don't want to >> override them at each installation of an instance. We could give admins >> an option to override what's stored in ldap during configuration. > > I think the process to import the system profiles into LDAP should be > done only once regardless of the number of replicas. Otherwise, > suppose we removed one of the centralized profile (e.g. profile1), it > might get added again when we install another replica. With proper access control in place, we should allow admins to do what they want to do in a reasonably convenient way. > >> When a profile with the same id exists in both ldap and local system, >> then by default the one on ldap takes precedence. Although we could add >> "per profile" config param for each instance to change the view >> (remember the unix ns/yp config where you can set the order of the >> source? so if "file" is 1st in the order, then the local ones have >> priority, etc.) > > I don't have a lot of experience as system admin, but I think managing > individual machines is difficult so NIS (or LDAP, NFS, DNS, etc.) is > used to transition to a centrally managed system. The order of sources > is set up such that eventually everything can be moved into a single > source. For example, once we have NIS, we add new users in NIS instead > of locally on each machine (unless absolutely necessary) even if a > user might use a specific machine only. again, I wasn't thinking of a strictly cloned environment. > >> With this, a site that has carefully crafted their profiles and stored >> in ldap to share among all ca instances don't have to bother changing >> anything for each ca instance created (because they are default to the >> ldap). >> Upgrade will also not affect the ldap copies inadvertently. > >>> 2. All profiles for an instance should live in LDAP. This makes it >>> simple - no need to check to see if a profile is in LDAP or files or >>> both, and which has priority etc. Tools will be provided to manage/ >>> create/delete profiles etc. > >> It might be simple, but we need to think about what's the value of them >> existing in ldap and not being shared by others. > > What is the use case for having local profiles that will not or cannot > be shared with other replicas? If sharing the profile doesn't pose any > problem we might as well store "local" profiles in LDAP so they can be > handled more consistently. again, I wasn't thinking of a strictly cloned environment. > >>> 3. Updates to system profile files will not affect the existing LDAP >>> profiles. We can provide update scripts or manual instructions for >>> admins to run when they opt to do so. This will be for behavioral >>> changes. > >> so, in the view I specified above, update to the "local" profiles will >> only affect the instance where they exist, however, we can allow option >> for admins to "publish" to ldap. > > The problem with local profiles in files is that they are not > replicated, so in case the machine is hosed, all local settings is > gone, including the profiles. Unless the admin created a backup for > each machine, it will be difficult to restore the machine quickly. The > replicated LDAP profiles will take care of this problem automatically. > In other proposals the system profiles in files are read-only and not > customizable, so there's no local profiles that need to be backed up. > If they don't do backups, losing profiles would not be their only issue there. But anyway, like I said above, I think putting "local" profiles in the lda is fine. It's just the user experience of administration that has to change. I have no opinion on this other than stating my personal preference of editing files directly ;-) From edewata at redhat.com Thu Jul 3 16:44:15 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Jul 2014 11:44:15 -0500 Subject: [Pki-devel] [PATCH] 513 Renamed CryptoUtil to CryptoProvider. In-Reply-To: <1404304751.4916.0.camel@localhost.localdomain> References: <53B357CF.3020500@redhat.com> <1404304751.4916.0.camel@localhost.localdomain> Message-ID: <53B5885F.7010303@redhat.com> On 7/2/2014 12:57 AM, Fraser Tweedale wrote: > Haven't run drmtest myself but it doesn't look like you've missed > anything and changes apply and build fine. > > ACK On 7/2/2014 7:39 AM, Ade Lee wrote: > ACK Thanks. Pushed to master. -- Endi S. Dewata From edewata at redhat.com Thu Jul 3 16:49:16 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Jul 2014 11:49:16 -0500 Subject: [Pki-devel] [PATCH] 514 Refactored SystemCertClient.get_transport_cert(). Message-ID: <53B5898C.90106@redhat.com> To simplify the usage, the SystemCertClient.get_transport_cert() has been modified to parse and decode the PEM certificate in CertData object, store the DER certificate back into the object, and return the CertData object to the client. This way the client will have access to the certificate attributes and both PEM and DER certificates. The PKIService.sendConditionalGetResponse() has been fixed to use the requested format. This is needed to display the transport certificate properly in the browser. Ticket #1062 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0514-Refactored-SystemCertClient.get_transport_cert.patch Type: text/x-patch Size: 4915 bytes Desc: not available URL: From edewata at redhat.com Thu Jul 3 17:59:38 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Thu, 03 Jul 2014 12:59:38 -0500 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B58273.60005@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> <53B4C520.3060502@redhat.com> <53B58273.60005@redhat.com> Message-ID: <53B59A0A.3080908@redhat.com> On 7/3/2014 11:18 AM, Christina Fu wrote: > Actually, I did not know that this discussion was restricted to a > replicated ("clone") environment. My view was more for a general > organizational environment where there are multiple sub-ca's for > different departments (which are not necessarily clones, but some could > be) and there are two types of profiles: > 1. centralized profiles (shared by all) > 2. local profiles (customized by each department) > > I think this view (let's adopt your term and call it proposal #4) is > more flexible in serving both non-clones and clones and yet retains the > simplicity. Thanks for clarifying. Yes, the main purpose of this enhancement is to simplify replicating the profiles within a cluster (e.g. IPA). Proposal #4 is probably meant to be an inter-cluster profile management. Within a cluster itself, the replicas should be indistinguishable. For example: * dept1 (cluster1: server1, server2): profile1, profile2 * dept2 (cluster2: server3, server4): profile1, profile3 * shared: profile1, profile4 I think proposal #4 can be done as a separate enhancement after we address the intra-cluster management (proposal #1-3). > You are right, Endi, about how "local profiles" don't necessarily have > to be on the disk. It's just a personal preference that I feel most > comfortable editing files directly than using ui's. I have never used > the java console to edit profiles. I don't know if there are others who > feel the same way. Maybe a market research on administrators is needed > here. Here's the CLI that Fraser proposed: http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Edit_profile So instead of executing the following commands on each replica (and risking inconsistent changes): $ vi /var/lib/pki/pki-tomcat/ca/profiles/ca/caUserCert.cfg $ systemctl restart pki-tomcatd at pki-tomcat.service you can call this just once (it will use the same vi editor) from any machine: $ pki profile-edit caUserCert and it should automatically propagate the changes to all replica. Same thing with the UI, you'll only need to do the changes once. >> The problem with local profiles in files is that they are not >> replicated, so in case the machine is hosed, all local settings is >> gone, including the profiles. Unless the admin created a backup for >> each machine, it will be difficult to restore the machine quickly. The >> replicated LDAP profiles will take care of this problem automatically. >> In other proposals the system profiles in files are read-only and not >> customizable, so there's no local profiles that need to be backed up. >> > If they don't do backups, losing profiles would not be their only issue > there. I think, ideally, if we lose a replica, other replicas should be able to replace it immediately. We still need a backup for the shared data/configuration, but we shouldn't need to backup individual replica. A replica should be something that you can add/remove relatively quickly. So, as a long term goal we should gradually migrate local configuration files into LDAP, starting with the profiles. > But anyway, like I said above, I think putting "local" profiles > in the lda is fine. It's just the user experience of administration > that has to change. I have no opinion on this other than stating my > personal preference of editing files directly ;-) Is the above CLI a good substitute? -- Endi S. Dewata From cfu at redhat.com Sat Jul 5 18:27:16 2014 From: cfu at redhat.com (Christina Fu) Date: Sat, 05 Jul 2014 11:27:16 -0700 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B4C542.4080101@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B47362.80609@redhat.com> <53B47CB8.5070802@redhat.com> <53B4C542.4080101@redhat.com> Message-ID: <53B84384.4070302@redhat.com> Yes, deferring profile inheritance for later discussion would be the right approach at this time. thanks, Christina On 07/02/2014 07:51 PM, Endi Sukma Dewata wrote: > On 7/2/2014 4:42 PM, Christina Fu wrote: >> IMHO, I think #3 is way too complicated. Complication invites issues >> and confuse people. >> >> Could we step back and try something simpler? When you copy the content >> of one profile and modify it to create a new one, then it's a new >> profile standing on its own. Why the parent-child relationship and >> all? Seems like an administrator's nightmare. Maybe I missed out on the >> irc discussion, but could you please give us a summary of the benefit >> and how the benefit weights against development time and administration >> maintenance, and support effort in the future on our end? > > Proposal #3 can be simplified into #3(a) without profile inheritance. > Here we'll support just the file-based system profile, proxy LDAP > profile, and custom LDAP profile. For immediate purposes this should > be sufficient. > > Profile inheritance is an idea that just came up after the IRC > discussion. It would take a whole separate design page to see how it > works, the potential benefits, and the impact on us. We don't have to > implement this now, but later if we determine that it is useful, we > can implement #3(b) with profile inheritance without changing the > schema or the existing data. > >> Anyway, I hope you will consider what I said in my earlier response. I >> thought our goal was to provide a "centralized collection of profiles" >> to ease administration effort. I hope we achieve simplicity rather than >> create complication. It's just my personal preference. > > Not disagreeing with that. Simplicity is always a goal, but sometimes > the proper solution cannot be the simplest one. > From ftweedal at redhat.com Mon Jul 7 07:08:05 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Mon, 7 Jul 2014 17:08:05 +1000 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> Message-ID: <20140707070805.GZ2417@dhcp-40-8.bne.redhat.com> Thank you for all of the feedback on the LDAP profiles design. There were a lot of interesting questions/comments/suggestions from the Dogtag, FreeIPA and DS teams. Most of this feedback has been incorporated into the wiki. Alternative suggestions have been moved to the History section in favour of the most strongly favoured solution: LDAP-only profiles, and no automatic upgrades of default profiles. http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage The other important change to the document is more information about how refreshing the profiles will be done, when modifications are replicated from other clones. Please review that section (and the LDAP schema, as some new schema was added). Finally, I have added my planned implementation steps to the Implementation section, and without further ado, I am starting. Of course, I welcome ongoing discussion of the design; it can be tweaked as necessary. Fraser On Wed, Jun 18, 2014 at 05:44:19PM +1000, Fraser Tweedale wrote: > Hi all, > > A requirement from the FreeIPA side is the ability to add and > customise CA profiles. Dogtag's current profile creation behaviour > writes the new profile to the filesystem beside the standard > profiles (as well as making the appropriate update to the registry, > etc.) > > There does not seem to be a mechanism to distribute new/modified > profiles to replicas - though perhaps I have missed something. > > Because this behaviour is required, unless I have overlooked > something or there is a better way (in which case please shout out), > I think it makes sense to begin a design proposal for an LDAP-based > profile store. > > Finally, a brief mention of some tickets related to profile storage > that could be good to tackle simultaneously should the proposed > change go ahead: > > - https://fedorahosted.org/pki/ticket/778 > - https://fedorahosted.org/freeipa/ticket/4002 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Mon Jul 7 11:44:58 2014 From: alee at redhat.com (Ade Lee) Date: Mon, 07 Jul 2014 19:44:58 +0800 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53B59A0A.3080908@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> <53B4C520.3060502@redhat.com> <53B58273.60005@redhat.com> <53B59A0A.3080908@redhat.com> Message-ID: <1404733498.21539.13.camel@localhost.localdomain> On Thu, 2014-07-03 at 12:59 -0500, Endi Sukma Dewata wrote: > On 7/3/2014 11:18 AM, Christina Fu wrote: > > Actually, I did not know that this discussion was restricted to a > > replicated ("clone") environment. My view was more for a general > > organizational environment where there are multiple sub-ca's for > > different departments (which are not necessarily clones, but some could > > be) and there are two types of profiles: > > 1. centralized profiles (shared by all) > > 2. local profiles (customized by each department) > > > > I think this view (let's adopt your term and call it proposal #4) is > > more flexible in serving both non-clones and clones and yet retains the > > simplicity. > > Thanks for clarifying. Yes, the main purpose of this enhancement is to > simplify replicating the profiles within a cluster (e.g. IPA). Proposal > #4 is probably meant to be an inter-cluster profile management. Within a > cluster itself, the replicas should be indistinguishable. > > For example: > * dept1 (cluster1: server1, server2): profile1, profile2 > * dept2 (cluster2: server3, server4): profile1, profile3 > * shared: profile1, profile4 > > I think proposal #4 can be done as a separate enhancement after we > address the intra-cluster management (proposal #1-3). > Agreed. One of the things we would have to address, for instance, would be where the shared profiles would reside. Certainly it has to be a location that is available to all subCA's. Several possibilities come to mind: 1) under a baseDN 2) user-defined 3) in the security domain - particularly useful perhaps for storing customized system profiles to used during installation. Christina, can you open a ticket for this feature? > > You are right, Endi, about how "local profiles" don't necessarily have > > to be on the disk. It's just a personal preference that I feel most > > comfortable editing files directly than using ui's. I have never used > > the java console to edit profiles. I don't know if there are others who > > feel the same way. Maybe a market research on administrators is needed > > here. > > Here's the CLI that Fraser proposed: > http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Edit_profile > > So instead of executing the following commands on each replica (and > risking inconsistent changes): > > $ vi /var/lib/pki/pki-tomcat/ca/profiles/ca/caUserCert.cfg > $ systemctl restart pki-tomcatd at pki-tomcat.service > > you can call this just once (it will use the same vi editor) from any > machine: > > $ pki profile-edit caUserCert > > and it should automatically propagate the changes to all replica. Same > thing with the UI, you'll only need to do the changes once. > Right, the idea is for the pki utility to spawn an editor -- we would likely default to "vi", but I suppose we could allow the user to specify others too. Ideally, what you would see is exactly the same as if you were editing a file today. The console would also continue to work, because it interacts with the ProfileSubsystem - which will know how to talk to LDAP. > >> The problem with local profiles in files is that they are not > >> replicated, so in case the machine is hosed, all local settings is > >> gone, including the profiles. Unless the admin created a backup for > >> each machine, it will be difficult to restore the machine quickly. The > >> replicated LDAP profiles will take care of this problem automatically. > >> In other proposals the system profiles in files are read-only and not > >> customizable, so there's no local profiles that need to be backed up. > >> > > If they don't do backups, losing profiles would not be their only issue > > there. > > I think, ideally, if we lose a replica, other replicas should be able to > replace it immediately. We still need a backup for the shared > data/configuration, but we shouldn't need to backup individual replica. > A replica should be something that you can add/remove relatively > quickly. So, as a long term goal we should gradually migrate local > configuration files into LDAP, starting with the profiles. > > > But anyway, like I said above, I think putting "local" profiles > > in the lda is fine. It's just the user experience of administration > > that has to change. I have no opinion on this other than stating my > > personal preference of editing files directly ;-) > > Is the above CLI a good substitute? > From cfu at redhat.com Mon Jul 7 16:21:56 2014 From: cfu at redhat.com (Christina Fu) Date: Mon, 07 Jul 2014 09:21:56 -0700 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <1404733498.21539.13.camel@localhost.localdomain> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> <53B4C520.3060502@redhat.com> <53B58273.60005@redhat.com> <53B59A0A.3080908@redhat.com> <1404733498.21539.13.camel@localhost.localdomain> Message-ID: <53BAC924.9020808@redhat.com> Done. https://fedorahosted.org/pki/ticket/1067 thanks, Christina On 07/07/2014 04:44 AM, Ade Lee wrote: > On Thu, 2014-07-03 at 12:59 -0500, Endi Sukma Dewata wrote: >> On 7/3/2014 11:18 AM, Christina Fu wrote: >>> Actually, I did not know that this discussion was restricted to a >>> replicated ("clone") environment. My view was more for a general >>> organizational environment where there are multiple sub-ca's for >>> different departments (which are not necessarily clones, but some could >>> be) and there are two types of profiles: >>> 1. centralized profiles (shared by all) >>> 2. local profiles (customized by each department) >>> >>> I think this view (let's adopt your term and call it proposal #4) is >>> more flexible in serving both non-clones and clones and yet retains the >>> simplicity. >> Thanks for clarifying. Yes, the main purpose of this enhancement is to >> simplify replicating the profiles within a cluster (e.g. IPA). Proposal >> #4 is probably meant to be an inter-cluster profile management. Within a >> cluster itself, the replicas should be indistinguishable. >> >> For example: >> * dept1 (cluster1: server1, server2): profile1, profile2 >> * dept2 (cluster2: server3, server4): profile1, profile3 >> * shared: profile1, profile4 >> >> I think proposal #4 can be done as a separate enhancement after we >> address the intra-cluster management (proposal #1-3). >> > Agreed. One of the things we would have to address, for instance, would > be where the shared profiles would reside. Certainly it has to be a > location that is available to all subCA's. Several possibilities come > to mind: > 1) under a baseDN > 2) user-defined > 3) in the security domain - particularly useful perhaps for storing > customized system profiles to used during installation. > > Christina, can you open a ticket for this feature? > >>> You are right, Endi, about how "local profiles" don't necessarily have >>> to be on the disk. It's just a personal preference that I feel most >>> comfortable editing files directly than using ui's. I have never used >>> the java console to edit profiles. I don't know if there are others who >>> feel the same way. Maybe a market research on administrators is needed >>> here. >> Here's the CLI that Fraser proposed: >> http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Edit_profile >> >> So instead of executing the following commands on each replica (and >> risking inconsistent changes): >> >> $ vi /var/lib/pki/pki-tomcat/ca/profiles/ca/caUserCert.cfg >> $ systemctl restart pki-tomcatd at pki-tomcat.service >> >> you can call this just once (it will use the same vi editor) from any >> machine: >> >> $ pki profile-edit caUserCert >> >> and it should automatically propagate the changes to all replica. Same >> thing with the UI, you'll only need to do the changes once. >> > Right, the idea is for the pki utility to spawn an editor -- we would > likely default to "vi", but I suppose we could allow the user to specify > others too. Ideally, what you would see is exactly the same as if you > were editing a file today. > > The console would also continue to work, because it interacts with the > ProfileSubsystem - which will know how to talk to LDAP. > >>>> The problem with local profiles in files is that they are not >>>> replicated, so in case the machine is hosed, all local settings is >>>> gone, including the profiles. Unless the admin created a backup for >>>> each machine, it will be difficult to restore the machine quickly. The >>>> replicated LDAP profiles will take care of this problem automatically. >>>> In other proposals the system profiles in files are read-only and not >>>> customizable, so there's no local profiles that need to be backed up. >>>> >>> If they don't do backups, losing profiles would not be their only issue >>> there. >> I think, ideally, if we lose a replica, other replicas should be able to >> replace it immediately. We still need a backup for the shared >> data/configuration, but we shouldn't need to backup individual replica. >> A replica should be something that you can add/remove relatively >> quickly. So, as a long term goal we should gradually migrate local >> configuration files into LDAP, starting with the profiles. >> >>> But anyway, like I said above, I think putting "local" profiles >>> in the lda is fine. It's just the user experience of administration >>> that has to change. I have no opinion on this other than stating my >>> personal preference of editing files directly ;-) >> Is the above CLI a good substitute? >> > From cfu at redhat.com Mon Jul 7 16:31:27 2014 From: cfu at redhat.com (Christina Fu) Date: Mon, 07 Jul 2014 09:31:27 -0700 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53BAC924.9020808@redhat.com> References: <20140618074419.GM11349@dhcp-40-8.bne.redhat.com> <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> <53B4C520.3060502@redhat.com> <53B58273.60005@redhat.com> <53B59A0A.3080908@redhat.com> <1404733498.21539.13.camel@localhost.localdomain> <53BAC924.9020808@redhat.com> Message-ID: <53BACB5F.7060603@redhat.com> Oh, and Fraser, Ade, and Endi, please keep https://fedorahosted.org/pki/ticket/1067 in mind when you implement the "replication of new/modified profile" so that they can seamlessly co-exist in the future. thanks, Christina On 07/07/2014 09:21 AM, Christina Fu wrote: > Done. > https://fedorahosted.org/pki/ticket/1067 > > thanks, > Christina > > On 07/07/2014 04:44 AM, Ade Lee wrote: >> On Thu, 2014-07-03 at 12:59 -0500, Endi Sukma Dewata wrote: >>> On 7/3/2014 11:18 AM, Christina Fu wrote: >>>> Actually, I did not know that this discussion was restricted to a >>>> replicated ("clone") environment. My view was more for a general >>>> organizational environment where there are multiple sub-ca's for >>>> different departments (which are not necessarily clones, but some >>>> could >>>> be) and there are two types of profiles: >>>> 1. centralized profiles (shared by all) >>>> 2. local profiles (customized by each department) >>>> >>>> I think this view (let's adopt your term and call it proposal #4) is >>>> more flexible in serving both non-clones and clones and yet retains >>>> the >>>> simplicity. >>> Thanks for clarifying. Yes, the main purpose of this enhancement is to >>> simplify replicating the profiles within a cluster (e.g. IPA). Proposal >>> #4 is probably meant to be an inter-cluster profile management. >>> Within a >>> cluster itself, the replicas should be indistinguishable. >>> >>> For example: >>> * dept1 (cluster1: server1, server2): profile1, profile2 >>> * dept2 (cluster2: server3, server4): profile1, profile3 >>> * shared: profile1, profile4 >>> >>> I think proposal #4 can be done as a separate enhancement after we >>> address the intra-cluster management (proposal #1-3). >>> >> Agreed. One of the things we would have to address, for instance, would >> be where the shared profiles would reside. Certainly it has to be a >> location that is available to all subCA's. Several possibilities come >> to mind: >> 1) under a baseDN >> 2) user-defined >> 3) in the security domain - particularly useful perhaps for storing >> customized system profiles to used during installation. >> >> Christina, can you open a ticket for this feature? >> >>>> You are right, Endi, about how "local profiles" don't necessarily have >>>> to be on the disk. It's just a personal preference that I feel most >>>> comfortable editing files directly than using ui's. I have never used >>>> the java console to edit profiles. I don't know if there are >>>> others who >>>> feel the same way. Maybe a market research on administrators is >>>> needed >>>> here. >>> Here's the CLI that Fraser proposed: >>> http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Edit_profile >>> >>> So instead of executing the following commands on each replica (and >>> risking inconsistent changes): >>> >>> $ vi /var/lib/pki/pki-tomcat/ca/profiles/ca/caUserCert.cfg >>> $ systemctl restart pki-tomcatd at pki-tomcat.service >>> >>> you can call this just once (it will use the same vi editor) from any >>> machine: >>> >>> $ pki profile-edit caUserCert >>> >>> and it should automatically propagate the changes to all replica. Same >>> thing with the UI, you'll only need to do the changes once. >>> >> Right, the idea is for the pki utility to spawn an editor -- we would >> likely default to "vi", but I suppose we could allow the user to specify >> others too. Ideally, what you would see is exactly the same as if you >> were editing a file today. >> >> The console would also continue to work, because it interacts with the >> ProfileSubsystem - which will know how to talk to LDAP. >> >>>>> The problem with local profiles in files is that they are not >>>>> replicated, so in case the machine is hosed, all local settings is >>>>> gone, including the profiles. Unless the admin created a backup for >>>>> each machine, it will be difficult to restore the machine quickly. >>>>> The >>>>> replicated LDAP profiles will take care of this problem >>>>> automatically. >>>>> In other proposals the system profiles in files are read-only and not >>>>> customizable, so there's no local profiles that need to be backed up. >>>>> >>>> If they don't do backups, losing profiles would not be their only >>>> issue >>>> there. >>> I think, ideally, if we lose a replica, other replicas should be >>> able to >>> replace it immediately. We still need a backup for the shared >>> data/configuration, but we shouldn't need to backup individual replica. >>> A replica should be something that you can add/remove relatively >>> quickly. So, as a long term goal we should gradually migrate local >>> configuration files into LDAP, starting with the profiles. >>> >>>> But anyway, like I said above, I think putting "local" profiles >>>> in the lda is fine. It's just the user experience of administration >>>> that has to change. I have no opinion on this other than stating my >>>> personal preference of editing files directly ;-) >>> Is the above CLI a good substitute? >>> >> > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From edewata at redhat.com Mon Jul 7 18:48:41 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 07 Jul 2014 13:48:41 -0500 Subject: [Pki-devel] [PATCH] 514 Refactored SystemCertClient.get_transport_cert(). In-Reply-To: <53B5898C.90106@redhat.com> References: <53B5898C.90106@redhat.com> Message-ID: <53BAEB89.6080203@redhat.com> On 7/3/2014 11:49 AM, Endi Sukma Dewata wrote: > To simplify the usage, the SystemCertClient.get_transport_cert() > has been modified to parse and decode the PEM certificate in > CertData object, store the DER certificate back into the object, > and return the CertData object to the client. This way the client > will have access to the certificate attributes and both PEM and > DER certificates. > > The PKIService.sendConditionalGetResponse() has been fixed to use > the requested format. This is needed to display the transport > certificate properly in the browser. > > Ticket #1062 ACKed by Ade. Pushed to master. -- Endi S. Dewata From edewata at redhat.com Tue Jul 8 00:21:13 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 07 Jul 2014 19:21:13 -0500 Subject: [Pki-devel] [PATCH] 515 Fixed transport certificate delimiters. Message-ID: <53BB3979.20009@redhat.com> The REST service and client library have been fixed to use the correct delimiters for transport certificate. The REST service was also modified to insert a new line between the header and the certificate data. Ticket #1063 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0515-Fixed-transport-certificate-delimiters.patch Type: text/x-patch Size: 3006 bytes Desc: not available URL: From edewata at redhat.com Tue Jul 8 00:21:16 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 07 Jul 2014 19:21:16 -0500 Subject: [Pki-devel] [PATCH] 516 Added transport cert attributes. Message-ID: <53BB397C.1050806@redhat.com> The REST service has been modified to return additional attributes for transport certificate including serial number, issuer DN, subject DN, and resource link. Ticket #1065 -- Endi S. Dewata -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-edewata-0516-Added-transport-cert-attributes.patch Type: text/x-patch Size: 5687 bytes Desc: not available URL: From ftweedal at redhat.com Tue Jul 8 01:40:54 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 8 Jul 2014 11:40:54 +1000 Subject: [Pki-devel] replication of new/modified profiles In-Reply-To: <53BACB5F.7060603@redhat.com> References: <53B1BA49.1090809@redhat.com> <20140701024259.GO2417@dhcp-40-8.bne.redhat.com> <1404321443.4916.30.camel@localhost.localdomain> <53B44ACC.4090200@redhat.com> <53B4C520.3060502@redhat.com> <53B58273.60005@redhat.com> <53B59A0A.3080908@redhat.com> <1404733498.21539.13.camel@localhost.localdomain> <53BAC924.9020808@redhat.com> <53BACB5F.7060603@redhat.com> Message-ID: <20140708014054.GC2417@dhcp-40-8.bne.redhat.com> On Mon, Jul 07, 2014 at 09:31:27AM -0700, Christina Fu wrote: > Oh, and Fraser, Ade, and Endi, please keep > https://fedorahosted.org/pki/ticket/1067 in mind when you implement the > "replication of new/modified profile" so that they can seamlessly co-exist > in the future. > Sure thing, I will add a reference to the LDAP Profiles design document. There will be some similar questions as were raised for the hybrid LDAP/files model, i.e. questions of precedence, or whether name collisions are even allowed. I don't see any fundamental issues, though. Cheers, Fraser > thanks, > Christina > > On 07/07/2014 09:21 AM, Christina Fu wrote: > >Done. > >https://fedorahosted.org/pki/ticket/1067 > > > >thanks, > >Christina > > > >On 07/07/2014 04:44 AM, Ade Lee wrote: > >>On Thu, 2014-07-03 at 12:59 -0500, Endi Sukma Dewata wrote: > >>>On 7/3/2014 11:18 AM, Christina Fu wrote: > >>>>Actually, I did not know that this discussion was restricted to a > >>>>replicated ("clone") environment. My view was more for a general > >>>>organizational environment where there are multiple sub-ca's for > >>>>different departments (which are not necessarily clones, but some > >>>>could > >>>>be) and there are two types of profiles: > >>>>1. centralized profiles (shared by all) > >>>>2. local profiles (customized by each department) > >>>> > >>>>I think this view (let's adopt your term and call it proposal #4) is > >>>>more flexible in serving both non-clones and clones and yet retains > >>>>the > >>>>simplicity. > >>>Thanks for clarifying. Yes, the main purpose of this enhancement is to > >>>simplify replicating the profiles within a cluster (e.g. IPA). Proposal > >>>#4 is probably meant to be an inter-cluster profile management. Within > >>>a > >>>cluster itself, the replicas should be indistinguishable. > >>> > >>>For example: > >>>* dept1 (cluster1: server1, server2): profile1, profile2 > >>>* dept2 (cluster2: server3, server4): profile1, profile3 > >>>* shared: profile1, profile4 > >>> > >>>I think proposal #4 can be done as a separate enhancement after we > >>>address the intra-cluster management (proposal #1-3). > >>> > >>Agreed. One of the things we would have to address, for instance, would > >>be where the shared profiles would reside. Certainly it has to be a > >>location that is available to all subCA's. Several possibilities come > >>to mind: > >>1) under a baseDN > >>2) user-defined > >>3) in the security domain - particularly useful perhaps for storing > >>customized system profiles to used during installation. > >> > >>Christina, can you open a ticket for this feature? > >> > >>>>You are right, Endi, about how "local profiles" don't necessarily have > >>>>to be on the disk. It's just a personal preference that I feel most > >>>>comfortable editing files directly than using ui's. I have never used > >>>>the java console to edit profiles. I don't know if there are others > >>>>who > >>>>feel the same way. Maybe a market research on administrators is > >>>>needed > >>>>here. > >>>Here's the CLI that Fraser proposed: > >>>http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Edit_profile > >>> > >>>So instead of executing the following commands on each replica (and > >>>risking inconsistent changes): > >>> > >>> $ vi /var/lib/pki/pki-tomcat/ca/profiles/ca/caUserCert.cfg > >>> $ systemctl restart pki-tomcatd at pki-tomcat.service > >>> > >>>you can call this just once (it will use the same vi editor) from any > >>>machine: > >>> > >>> $ pki profile-edit caUserCert > >>> > >>>and it should automatically propagate the changes to all replica. Same > >>>thing with the UI, you'll only need to do the changes once. > >>> > >>Right, the idea is for the pki utility to spawn an editor -- we would > >>likely default to "vi", but I suppose we could allow the user to specify > >>others too. Ideally, what you would see is exactly the same as if you > >>were editing a file today. > >> > >>The console would also continue to work, because it interacts with the > >>ProfileSubsystem - which will know how to talk to LDAP. > >> > >>>>>The problem with local profiles in files is that they are not > >>>>>replicated, so in case the machine is hosed, all local settings is > >>>>>gone, including the profiles. Unless the admin created a backup for > >>>>>each machine, it will be difficult to restore the machine quickly. > >>>>>The > >>>>>replicated LDAP profiles will take care of this problem > >>>>>automatically. > >>>>>In other proposals the system profiles in files are read-only and not > >>>>>customizable, so there's no local profiles that need to be backed up. > >>>>> > >>>>If they don't do backups, losing profiles would not be their only > >>>>issue > >>>>there. > >>>I think, ideally, if we lose a replica, other replicas should be able > >>>to > >>>replace it immediately. We still need a backup for the shared > >>>data/configuration, but we shouldn't need to backup individual replica. > >>>A replica should be something that you can add/remove relatively > >>>quickly. So, as a long term goal we should gradually migrate local > >>>configuration files into LDAP, starting with the profiles. > >>> > >>>>But anyway, like I said above, I think putting "local" profiles > >>>>in the lda is fine. It's just the user experience of administration > >>>>that has to change. I have no opinion on this other than stating my > >>>>personal preference of editing files directly ;-) > >>>Is the above CLI a good substitute? > >>> > >> > > > >_______________________________________________ > >Pki-devel mailing list > >Pki-devel at redhat.com > >https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From ftweedal at redhat.com Tue Jul 8 07:55:37 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 8 Jul 2014 17:55:37 +1000 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg In-Reply-To: <53AE3D7F.5000006@redhat.com> References: <53AE3D7F.5000006@redhat.com> Message-ID: <20140708075537.GD2417@dhcp-40-8.bne.redhat.com> There seems to be an selinux issue with this change. When I spawned a new instance, it was not premitted to create the CS.cfg.bak symlink on startup (and startup failed as a result). It's the end of the day and I didn't get to the bottom of it (I have little prior experience with selinux) but it seems specifically related to symlinks - when I changed the `ln -s' to a `cp' in scripts/operations:1569 everything works OK. So I'll leave it that for today; if anyone has any pointers (or patches) that would be great, otherwise I'll press on tomorrow morning. Cheers, Fraser On Fri, Jun 27, 2014 at 08:58:55PM -0700, Matthew Harmsen wrote: > Please review the attached patch for: > > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg > > > This patch is based upon a previously reviewed patch for the Dogtag 9 > architecture utilized by the IPA_v2_RHEL_6_ERRATA_BRANCH, but was modified > and tested to work with the Dogtag 10.2 architecture. > > CAVEAT 1: > > Although this patch contains changes to multiple PKI subsystem's > 'CS.cfg' configuration files, an upgrade script should not be > specifically required for legacy instances since the parameter that > is added, 'archive.configuration_file=true', is presumed even if the > parameter is missing (as it would be on any legacy instance). In > this case, it would only be necessary to add this parameter to a > legacy instance's CS.cfg, and set the value to 'false' in order to > turn off 'CS.cfg' configuration file archival (explicit instructions > detailing this are found in the 'operations' script). However, if > this is desired for completeness, I don't mind adding it. > > CAVEAT 2: > > I had originally made the effort to attempt to have specific crucial > WARNING messages echoed to the display as well as to the journal. I > believe that this would be beneficial, as, for example, it would > immediately notify an admin that since an error had occurred, > 'CS.cfg' backups would be discontinued until the error was > corrected. My idea was to echo these WARNING messages explicitly to > stderr via redirecting them (>&2), and adding the parameter > 'StandardError=journal+console' under the [Service] section of the > 'pki-tomcatd at pki-tomcat.service' file. Unfortunately, I was never > able to make this work - both stdout and stderr messages were stored > in the journal, but were never displayed to the screen when typing > 'systemctl restart pki-tomcatd at pki-tomcat.service' (even after a > 'systemctl daemon-reload' had been performed). > > -- Matt > From 22242207fd6403dd65f777691ae1bfd0a2aed678 Mon Sep 17 00:00:00 2001 > From: Matthew Harmsen > Date: Fri, 27 Jun 2014 20:35:04 -0700 > Subject: [PATCH] Backup and Archive CS.cfg > > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg > --- > base/ca/shared/conf/CS.cfg.in | 1 + > base/kra/shared/conf/CS.cfg.in | 1 + > base/ocsp/shared/conf/CS.cfg.in | 1 + > base/server/scripts/operations | 211 +++++++++++++++++++++++++++++++++- > base/tks/shared/conf/CS.cfg.in | 1 + > base/tps-tomcat/shared/conf/CS.cfg.in | 1 + > 6 files changed, 215 insertions(+), 1 deletion(-) > > diff --git a/base/ca/shared/conf/CS.cfg.in b/base/ca/shared/conf/CS.cfg.in > index 90fb2d2..4ab8974 100644 > --- a/base/ca/shared/conf/CS.cfg.in > +++ b/base/ca/shared/conf/CS.cfg.in > @@ -159,6 +159,7 @@ accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluato > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > accessEvaluator.impl.user_origreq.class=com.netscape.cms.evaluators.UserOrigReqAccessEvaluator > +archive.configuration_file=true > auths._000=## > auths._001=## new authentication > auths._002=## > diff --git a/base/kra/shared/conf/CS.cfg.in b/base/kra/shared/conf/CS.cfg.in > index d8b5951..5febae8 100644 > --- a/base/kra/shared/conf/CS.cfg.in > +++ b/base/kra/shared/conf/CS.cfg.in > @@ -135,6 +135,7 @@ CrossCertPair.ldap=internaldb > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > +archive.configuration_file=true > auths._000=## > auths._001=## new authentication > auths._002=## > diff --git a/base/ocsp/shared/conf/CS.cfg.in b/base/ocsp/shared/conf/CS.cfg.in > index ace7f54..9f92ebf 100644 > --- a/base/ocsp/shared/conf/CS.cfg.in > +++ b/base/ocsp/shared/conf/CS.cfg.in > @@ -121,6 +121,7 @@ CrossCertPair.ldap=internaldb > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > +archive.configuration_file=true > auths._000=## > auths._001=## new authentication > auths._002=## > diff --git a/base/server/scripts/operations b/base/server/scripts/operations > index bfd2de8..bff3573 100644 > --- a/base/server/scripts/operations > +++ b/base/server/scripts/operations > @@ -1413,6 +1413,189 @@ verify_symlinks() > return 0 > } > > +backup_instance_configuration_files() > +{ > + declare -a pki_subsystems=('ca' > + 'kra' > + 'ocsp' > + 'tks' > + 'tps') > + > + # Utilize an identical timestamp on archives for each PKI subsystem > + # residing within the same instance to mark a common archival time > + timestamp=`date +%Y%m%d%H%M%S` > + > + # Automatically enable timestamped archives > + # > + # NOTE: To disable this feature for a particular PKI subsystem > + # within an instance, edit that PKI subsystem's 'CS.cfg' file > + # within the instance: > + # > + # If the 'archive.configuration_file' parameter exists, > + # change it to 'archive.configuration_file=false'. > + # > + # However, if the 'archive.configuration_file' parameter does > + # not exist, simply add 'archive.configuration_file=false' > + # to the 'CS.cfg'. > + # > + # In either case, it is unnecessary to restart the instance, > + # as each instance's 'CS.cfg' file is always processed every > + # time an instance is restarted. > + # > + backup_errors=0 > + for pki in "${pki_subsystems[@]}" > + do > + config_dir=${PKI_INSTANCE_PATH}/conf/${pki} > + > + # Check to see if this PKI subsystem exists within this instance > + if [ ! -d ${config_dir} ] ; then > + continue > + fi > + > + # Compute uppercase representation of this PKI subsystem > + PKI=${pki^^} > + > + # Backup parameters > + pki_instance_configuration_file=${config_dir}/CS.cfg > + backup_file=${config_dir}/CS.cfg.bak > + saved_backup_file=${config_dir}/CS.cfg.bak.saved > + > + # Check for an empty 'CS.cfg' > + # > + # NOTE: 'CS.cfg' is always a regular file > + # > + if [ ! -s ${pki_instance_configuration_file} ] ; then > + # Issue a warning that the 'CS.cfg' is empty > + echo "WARNING: The '${pki_instance_configuration_file}' is empty!" > + echo " ${PKI} backups will be discontinued until this" > + echo " issue has been resolved!" > + $((backup_errors++)) > + continue > + fi > + > + # Make certain that a previous attempt to backup 'CS.cfg' has not failed > + # (i. e. - 'CS.cfg.bak.saved' exists) > + # > + # NOTE: 'CS.cfg.bak.saved' is always a regular file > + # > + if [ -f ${saved_backup_file} ] ; then > + # 'CS.cfg.bak.saved' is a regular file or a symlink > + echo "WARNING: Since the file '${saved_backup_file}' exists, a" > + echo " previous backup attempt has failed! ${PKI} backups" > + echo " will be discontinued until this issue has been resolved!" > + $((backup_errors++)) > + continue > + fi > + > + # If present, compare 'CS.cfg' to 'CS.cfg.bak' to see if it is necessary > + # to backup 'CS.cfg'. 'CS.cfg.bak' may be a regular file, a > + # symlink, or a dangling symlink > + # > + # NOTE: 'CS.cfg.bak' may be a regular file, a symlink, or a > + # dangling symlink > + # > + if [ -f ${backup_file} ] ; then > + # 'CS.cfg.bak' is a regular file or a symlink > + cmp --silent ${pki_instance_configuration_file} ${backup_file} > + rv=$? > + if [ $rv -eq 0 ] ; then > + # 'CS.cfg' is identical to 'CS.cfg.bak'; > + # no need to archive or backup 'CS.cfg' > + continue > + fi > + > + # Since it is known that the previous 'CS.cfg.bak' file exists, and > + # and it is either a symlink or a regular file, save the previous > + # 'CS.cfg.bak' to 'CS.cfg.bak.saved' > + # > + # NOTE: If switching between simply creating backups to generating > + # timestamped archives, the previous 'CS.cfg.bak' that > + # existed as a regular file will NOT be archived! > + # > + if [ -h ${backup_file} ] ; then > + # 'CS.cfg.bak' is a symlink > + # (i. e. - copy the timestamped archive to a regular file) > + cp ${backup_file} ${saved_backup_file} > + > + # remove the 'CS.cfg.bak' symlink > + rm ${backup_file} > + else > + # 'CS.cfg.bak' is a regular file > + # (i. e. - simply rename the regular file) > + mv ${backup_file} ${saved_backup_file} > + fi > + elif [ -h ${backup_file} ] ; then > + # 'CS.cfg.bak' is a dangling symlink > + echo "WARNING: The file '${backup_file}' is a dangling symlink" > + echo " which suggests that the previous backup file has" > + echo " been removed! ${PKI} backups will be discontinued" > + echo " until this issue has been resolved!" > + $((backup_errors++)) > + continue > + fi > + > + # Check 'CS.cfg' for 'archive.configuration_file' parameter > + # to see if timestamped archives should be disabled > + archive_configuration_file="true" > + line=`grep -e '^[ \t]*archive.configuration_file[ \t]*=' ${pki_instance_configuration_file}` > + if [ "${line}" != "" ] ; then > + archive_configuration_file=`echo "${line}" | sed -e 's/^[^=]*[ \t]*=[ \t]*\(.*\)/\1/' -e 's/[ \t]*$//'` > + fi > + > + # Backup 'CS.cfg' > + if [ "${archive_configuration_file}" != "true" ] ; then > + # Always backup 'CS.cfg' to 'CS.cfg.bak' > + cp -b ${pki_instance_configuration_file} ${backup_file} > + else > + # Archive parameters > + archive_dir=${config_dir}/archives > + archived_file=${archive_dir}/CS.cfg.bak.${timestamp} > + > + # If not present, create an archives directory for this 'CS.cfg' > + if [ ! -d ${archive_dir} ] ; then > + mkdir -p ${archive_dir} > + fi > + > + # Archive 'CS.cfg' to 'CS.cfg.bak.${timestamp}' > + cp -a ${pki_instance_configuration_file} ${archived_file} > + if [ ! -s ${archived_file} ] ; then > + # Issue a warning that the archived backup failed > + echo "WARNING: Failed to archive '${pki_instance_configuration_file}' to '${archived_file}'!" > + $((backup_errors++)) > + continue > + fi > + > + # Always create 'CS.cfg.bak' by linking to this archived file > + ln -s ${archived_file} ${backup_file} > + > + # Report that 'CS.cfg' has been successfully archived > + echo "SUCCESS: Successfully archived '${archived_file}'" > + fi > + > + # Check that a non-empty 'CS.cfg.bak' symlink or regular file exists > + if [ ! -s ${backup_file} ] ; then > + # Issue a warning that the backup failed > + echo "WARNING: Failed to backup '${pki_instance_configuration_file}' to '${backup_file}'!" > + $((backup_errors++)) > + continue > + else > + # Report that 'CS.cfg' has been successfully backed up > + echo "SUCCESS: Successfully backed up '${backup_file}'" > + fi > + > + # Since 'CS.cfg' was backed up successfully, remove 'CS.cfg.bak.saved' > + if [ -f ${saved_backup_file} ] ; then > + rm ${saved_backup_file} > + fi > + done > + > + if [ ${backup_errors} -ne 0 ]; then > + return 1 > + fi > + > + return 0 > +} > + > start_instance() > { > rv=0 > @@ -1453,8 +1636,34 @@ start_instance() > return 6 > else > # 0 success > - return 0 > + > + # Always create a backup of each PKI subsystem's 'CS.cfg' file > + # within an instance. > + # > + # For every backup failure detected within a PKI subsystem within > + # an instance, a warning message will be issued, and an error code > + # of 1 will be returned. > + # > + # Note that until they have been resolved, every previous backup > + # failures of any PKI subsystem within an instance will also issue > + # a warning message and return an error code of 1. Backups of that > + # particular instance's PKI subsystem will be suspended until this > + # error has been addressed. > + # > + # By default, unless they have been explicitly disabled, > + # a timestamped archive of each PKI subsystem's 'CS.cfg' file > + # within an instance will also be created. Note that a single > + # timestamp will be utlized across each PKI subsystem within > + # an instance for each invocation of this function. > + # > + # When enabled, any timestamped archive failures also issue a > + # warning message and return an error code of 1. > + # > + backup_instance_configuration_files > + rv=$? > fi > + > + return $? > } > > # function used in debian to find the correct jdk > diff --git a/base/tks/shared/conf/CS.cfg.in b/base/tks/shared/conf/CS.cfg.in > index 4d32f6e..bd2858d 100644 > --- a/base/tks/shared/conf/CS.cfg.in > +++ b/base/tks/shared/conf/CS.cfg.in > @@ -112,6 +112,7 @@ CrossCertPair.ldap=internaldb > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > +archive.configuration_file=true > auths._000=## > auths._001=## new authentication > auths._002=## > diff --git a/base/tps-tomcat/shared/conf/CS.cfg.in b/base/tps-tomcat/shared/conf/CS.cfg.in > index b4b1941..57a7866 100644 > --- a/base/tps-tomcat/shared/conf/CS.cfg.in > +++ b/base/tps-tomcat/shared/conf/CS.cfg.in > @@ -4,6 +4,7 @@ _002=## > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > +archive.configuration_file=true > applet._000=######################################### > applet._001=# applet information > applet._002=# SAF Key: > -- > 1.9.3 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Tue Jul 8 11:49:29 2014 From: alee at redhat.com (Ade Lee) Date: Tue, 08 Jul 2014 19:49:29 +0800 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg In-Reply-To: <20140708075537.GD2417@dhcp-40-8.bne.redhat.com> References: <53AE3D7F.5000006@redhat.com> <20140708075537.GD2417@dhcp-40-8.bne.redhat.com> Message-ID: <1404820169.3945.7.camel@localhost.localdomain> Fraser, What is likely needed is a rule permitting the pki_tomcat_t type to create links in the config directory. To get the exact rule needed, please do the following: 1. set selinux to permissive mode (setenforce 0) 2. clear the audit log - cat /dev/null > /var/log/audit/audit.log 3. start the server (with the original script). Make sure to remove the copy you have placed there. 4. The instance should start. 5. Check to see what rule is needed: audit2allow -R -i /var/log/audit/audit.log audit2allow -R /var/log/audit/audit.log 6. File a BZ against selinux-policy in Fedora 20/rawhide, providing the above output. In 10.x, our selinux policy is managed by the system selinux policy. Ade On Tue, 2014-07-08 at 17:55 +1000, Fraser Tweedale wrote: > There seems to be an selinux issue with this change. When I spawned > a new instance, it was not premitted to create the CS.cfg.bak > symlink on startup (and startup failed as a result). > > It's the end of the day and I didn't get to the bottom of it (I have > little prior experience with selinux) but it seems specifically > related to symlinks - when I changed the `ln -s' to a `cp' in > scripts/operations:1569 everything works OK. > > So I'll leave it that for today; if anyone has any pointers (or > patches) that would be great, otherwise I'll press on tomorrow > morning. > > Cheers, > > Fraser > > On Fri, Jun 27, 2014 at 08:58:55PM -0700, Matthew Harmsen wrote: > > Please review the attached patch for: > > > > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg > > > > > > This patch is based upon a previously reviewed patch for the Dogtag 9 > > architecture utilized by the IPA_v2_RHEL_6_ERRATA_BRANCH, but was modified > > and tested to work with the Dogtag 10.2 architecture. > > > > CAVEAT 1: > > > > Although this patch contains changes to multiple PKI subsystem's > > 'CS.cfg' configuration files, an upgrade script should not be > > specifically required for legacy instances since the parameter that > > is added, 'archive.configuration_file=true', is presumed even if the > > parameter is missing (as it would be on any legacy instance). In > > this case, it would only be necessary to add this parameter to a > > legacy instance's CS.cfg, and set the value to 'false' in order to > > turn off 'CS.cfg' configuration file archival (explicit instructions > > detailing this are found in the 'operations' script). However, if > > this is desired for completeness, I don't mind adding it. > > > > CAVEAT 2: > > > > I had originally made the effort to attempt to have specific crucial > > WARNING messages echoed to the display as well as to the journal. I > > believe that this would be beneficial, as, for example, it would > > immediately notify an admin that since an error had occurred, > > 'CS.cfg' backups would be discontinued until the error was > > corrected. My idea was to echo these WARNING messages explicitly to > > stderr via redirecting them (>&2), and adding the parameter > > 'StandardError=journal+console' under the [Service] section of the > > 'pki-tomcatd at pki-tomcat.service' file. Unfortunately, I was never > > able to make this work - both stdout and stderr messages were stored > > in the journal, but were never displayed to the screen when typing > > 'systemctl restart pki-tomcatd at pki-tomcat.service' (even after a > > 'systemctl daemon-reload' had been performed). > > > > -- Matt > > > From 22242207fd6403dd65f777691ae1bfd0a2aed678 Mon Sep 17 00:00:00 2001 > > From: Matthew Harmsen > > Date: Fri, 27 Jun 2014 20:35:04 -0700 > > Subject: [PATCH] Backup and Archive CS.cfg > > > > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg > > --- > > base/ca/shared/conf/CS.cfg.in | 1 + > > base/kra/shared/conf/CS.cfg.in | 1 + > > base/ocsp/shared/conf/CS.cfg.in | 1 + > > base/server/scripts/operations | 211 +++++++++++++++++++++++++++++++++- > > base/tks/shared/conf/CS.cfg.in | 1 + > > base/tps-tomcat/shared/conf/CS.cfg.in | 1 + > > 6 files changed, 215 insertions(+), 1 deletion(-) > > > > diff --git a/base/ca/shared/conf/CS.cfg.in b/base/ca/shared/conf/CS.cfg.in > > index 90fb2d2..4ab8974 100644 > > --- a/base/ca/shared/conf/CS.cfg.in > > +++ b/base/ca/shared/conf/CS.cfg.in > > @@ -159,6 +159,7 @@ accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluato > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > accessEvaluator.impl.user_origreq.class=com.netscape.cms.evaluators.UserOrigReqAccessEvaluator > > +archive.configuration_file=true > > auths._000=## > > auths._001=## new authentication > > auths._002=## > > diff --git a/base/kra/shared/conf/CS.cfg.in b/base/kra/shared/conf/CS.cfg.in > > index d8b5951..5febae8 100644 > > --- a/base/kra/shared/conf/CS.cfg.in > > +++ b/base/kra/shared/conf/CS.cfg.in > > @@ -135,6 +135,7 @@ CrossCertPair.ldap=internaldb > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > +archive.configuration_file=true > > auths._000=## > > auths._001=## new authentication > > auths._002=## > > diff --git a/base/ocsp/shared/conf/CS.cfg.in b/base/ocsp/shared/conf/CS.cfg.in > > index ace7f54..9f92ebf 100644 > > --- a/base/ocsp/shared/conf/CS.cfg.in > > +++ b/base/ocsp/shared/conf/CS.cfg.in > > @@ -121,6 +121,7 @@ CrossCertPair.ldap=internaldb > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > +archive.configuration_file=true > > auths._000=## > > auths._001=## new authentication > > auths._002=## > > diff --git a/base/server/scripts/operations b/base/server/scripts/operations > > index bfd2de8..bff3573 100644 > > --- a/base/server/scripts/operations > > +++ b/base/server/scripts/operations > > @@ -1413,6 +1413,189 @@ verify_symlinks() > > return 0 > > } > > > > +backup_instance_configuration_files() > > +{ > > + declare -a pki_subsystems=('ca' > > + 'kra' > > + 'ocsp' > > + 'tks' > > + 'tps') > > + > > + # Utilize an identical timestamp on archives for each PKI subsystem > > + # residing within the same instance to mark a common archival time > > + timestamp=`date +%Y%m%d%H%M%S` > > + > > + # Automatically enable timestamped archives > > + # > > + # NOTE: To disable this feature for a particular PKI subsystem > > + # within an instance, edit that PKI subsystem's 'CS.cfg' file > > + # within the instance: > > + # > > + # If the 'archive.configuration_file' parameter exists, > > + # change it to 'archive.configuration_file=false'. > > + # > > + # However, if the 'archive.configuration_file' parameter does > > + # not exist, simply add 'archive.configuration_file=false' > > + # to the 'CS.cfg'. > > + # > > + # In either case, it is unnecessary to restart the instance, > > + # as each instance's 'CS.cfg' file is always processed every > > + # time an instance is restarted. > > + # > > + backup_errors=0 > > + for pki in "${pki_subsystems[@]}" > > + do > > + config_dir=${PKI_INSTANCE_PATH}/conf/${pki} > > + > > + # Check to see if this PKI subsystem exists within this instance > > + if [ ! -d ${config_dir} ] ; then > > + continue > > + fi > > + > > + # Compute uppercase representation of this PKI subsystem > > + PKI=${pki^^} > > + > > + # Backup parameters > > + pki_instance_configuration_file=${config_dir}/CS.cfg > > + backup_file=${config_dir}/CS.cfg.bak > > + saved_backup_file=${config_dir}/CS.cfg.bak.saved > > + > > + # Check for an empty 'CS.cfg' > > + # > > + # NOTE: 'CS.cfg' is always a regular file > > + # > > + if [ ! -s ${pki_instance_configuration_file} ] ; then > > + # Issue a warning that the 'CS.cfg' is empty > > + echo "WARNING: The '${pki_instance_configuration_file}' is empty!" > > + echo " ${PKI} backups will be discontinued until this" > > + echo " issue has been resolved!" > > + $((backup_errors++)) > > + continue > > + fi > > + > > + # Make certain that a previous attempt to backup 'CS.cfg' has not failed > > + # (i. e. - 'CS.cfg.bak.saved' exists) > > + # > > + # NOTE: 'CS.cfg.bak.saved' is always a regular file > > + # > > + if [ -f ${saved_backup_file} ] ; then > > + # 'CS.cfg.bak.saved' is a regular file or a symlink > > + echo "WARNING: Since the file '${saved_backup_file}' exists, a" > > + echo " previous backup attempt has failed! ${PKI} backups" > > + echo " will be discontinued until this issue has been resolved!" > > + $((backup_errors++)) > > + continue > > + fi > > + > > + # If present, compare 'CS.cfg' to 'CS.cfg.bak' to see if it is necessary > > + # to backup 'CS.cfg'. 'CS.cfg.bak' may be a regular file, a > > + # symlink, or a dangling symlink > > + # > > + # NOTE: 'CS.cfg.bak' may be a regular file, a symlink, or a > > + # dangling symlink > > + # > > + if [ -f ${backup_file} ] ; then > > + # 'CS.cfg.bak' is a regular file or a symlink > > + cmp --silent ${pki_instance_configuration_file} ${backup_file} > > + rv=$? > > + if [ $rv -eq 0 ] ; then > > + # 'CS.cfg' is identical to 'CS.cfg.bak'; > > + # no need to archive or backup 'CS.cfg' > > + continue > > + fi > > + > > + # Since it is known that the previous 'CS.cfg.bak' file exists, and > > + # and it is either a symlink or a regular file, save the previous > > + # 'CS.cfg.bak' to 'CS.cfg.bak.saved' > > + # > > + # NOTE: If switching between simply creating backups to generating > > + # timestamped archives, the previous 'CS.cfg.bak' that > > + # existed as a regular file will NOT be archived! > > + # > > + if [ -h ${backup_file} ] ; then > > + # 'CS.cfg.bak' is a symlink > > + # (i. e. - copy the timestamped archive to a regular file) > > + cp ${backup_file} ${saved_backup_file} > > + > > + # remove the 'CS.cfg.bak' symlink > > + rm ${backup_file} > > + else > > + # 'CS.cfg.bak' is a regular file > > + # (i. e. - simply rename the regular file) > > + mv ${backup_file} ${saved_backup_file} > > + fi > > + elif [ -h ${backup_file} ] ; then > > + # 'CS.cfg.bak' is a dangling symlink > > + echo "WARNING: The file '${backup_file}' is a dangling symlink" > > + echo " which suggests that the previous backup file has" > > + echo " been removed! ${PKI} backups will be discontinued" > > + echo " until this issue has been resolved!" > > + $((backup_errors++)) > > + continue > > + fi > > + > > + # Check 'CS.cfg' for 'archive.configuration_file' parameter > > + # to see if timestamped archives should be disabled > > + archive_configuration_file="true" > > + line=`grep -e '^[ \t]*archive.configuration_file[ \t]*=' ${pki_instance_configuration_file}` > > + if [ "${line}" != "" ] ; then > > + archive_configuration_file=`echo "${line}" | sed -e 's/^[^=]*[ \t]*=[ \t]*\(.*\)/\1/' -e 's/[ \t]*$//'` > > + fi > > + > > + # Backup 'CS.cfg' > > + if [ "${archive_configuration_file}" != "true" ] ; then > > + # Always backup 'CS.cfg' to 'CS.cfg.bak' > > + cp -b ${pki_instance_configuration_file} ${backup_file} > > + else > > + # Archive parameters > > + archive_dir=${config_dir}/archives > > + archived_file=${archive_dir}/CS.cfg.bak.${timestamp} > > + > > + # If not present, create an archives directory for this 'CS.cfg' > > + if [ ! -d ${archive_dir} ] ; then > > + mkdir -p ${archive_dir} > > + fi > > + > > + # Archive 'CS.cfg' to 'CS.cfg.bak.${timestamp}' > > + cp -a ${pki_instance_configuration_file} ${archived_file} > > + if [ ! -s ${archived_file} ] ; then > > + # Issue a warning that the archived backup failed > > + echo "WARNING: Failed to archive '${pki_instance_configuration_file}' to '${archived_file}'!" > > + $((backup_errors++)) > > + continue > > + fi > > + > > + # Always create 'CS.cfg.bak' by linking to this archived file > > + ln -s ${archived_file} ${backup_file} > > + > > + # Report that 'CS.cfg' has been successfully archived > > + echo "SUCCESS: Successfully archived '${archived_file}'" > > + fi > > + > > + # Check that a non-empty 'CS.cfg.bak' symlink or regular file exists > > + if [ ! -s ${backup_file} ] ; then > > + # Issue a warning that the backup failed > > + echo "WARNING: Failed to backup '${pki_instance_configuration_file}' to '${backup_file}'!" > > + $((backup_errors++)) > > + continue > > + else > > + # Report that 'CS.cfg' has been successfully backed up > > + echo "SUCCESS: Successfully backed up '${backup_file}'" > > + fi > > + > > + # Since 'CS.cfg' was backed up successfully, remove 'CS.cfg.bak.saved' > > + if [ -f ${saved_backup_file} ] ; then > > + rm ${saved_backup_file} > > + fi > > + done > > + > > + if [ ${backup_errors} -ne 0 ]; then > > + return 1 > > + fi > > + > > + return 0 > > +} > > + > > start_instance() > > { > > rv=0 > > @@ -1453,8 +1636,34 @@ start_instance() > > return 6 > > else > > # 0 success > > - return 0 > > + > > + # Always create a backup of each PKI subsystem's 'CS.cfg' file > > + # within an instance. > > + # > > + # For every backup failure detected within a PKI subsystem within > > + # an instance, a warning message will be issued, and an error code > > + # of 1 will be returned. > > + # > > + # Note that until they have been resolved, every previous backup > > + # failures of any PKI subsystem within an instance will also issue > > + # a warning message and return an error code of 1. Backups of that > > + # particular instance's PKI subsystem will be suspended until this > > + # error has been addressed. > > + # > > + # By default, unless they have been explicitly disabled, > > + # a timestamped archive of each PKI subsystem's 'CS.cfg' file > > + # within an instance will also be created. Note that a single > > + # timestamp will be utlized across each PKI subsystem within > > + # an instance for each invocation of this function. > > + # > > + # When enabled, any timestamped archive failures also issue a > > + # warning message and return an error code of 1. > > + # > > + backup_instance_configuration_files > > + rv=$? > > fi > > + > > + return $? > > } > > > > # function used in debian to find the correct jdk > > diff --git a/base/tks/shared/conf/CS.cfg.in b/base/tks/shared/conf/CS.cfg.in > > index 4d32f6e..bd2858d 100644 > > --- a/base/tks/shared/conf/CS.cfg.in > > +++ b/base/tks/shared/conf/CS.cfg.in > > @@ -112,6 +112,7 @@ CrossCertPair.ldap=internaldb > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > +archive.configuration_file=true > > auths._000=## > > auths._001=## new authentication > > auths._002=## > > diff --git a/base/tps-tomcat/shared/conf/CS.cfg.in b/base/tps-tomcat/shared/conf/CS.cfg.in > > index b4b1941..57a7866 100644 > > --- a/base/tps-tomcat/shared/conf/CS.cfg.in > > +++ b/base/tps-tomcat/shared/conf/CS.cfg.in > > @@ -4,6 +4,7 @@ _002=## > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > +archive.configuration_file=true > > applet._000=######################################### > > applet._001=# applet information > > applet._002=# SAF Key: > > -- > > 1.9.3 > > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From alee at redhat.com Tue Jul 8 11:56:52 2014 From: alee at redhat.com (Ade Lee) Date: Tue, 08 Jul 2014 19:56:52 +0800 Subject: [Pki-devel] [PATCH] 516 Added transport cert attributes. In-Reply-To: <53BB397C.1050806@redhat.com> References: <53BB397C.1050806@redhat.com> Message-ID: <1404820612.3945.8.camel@localhost.localdomain> ACK On Mon, 2014-07-07 at 19:21 -0500, Endi Sukma Dewata wrote: > The REST service has been modified to return additional attributes > for transport certificate including serial number, issuer DN, > subject DN, and resource link. > > Ticket #1065 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From akoneru at redhat.com Tue Jul 8 13:29:05 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Tue, 08 Jul 2014 09:29:05 -0400 Subject: [Pki-devel] [PATCH] 515 Fixed transport certificate delimiters. In-Reply-To: <53BB3979.20009@redhat.com> References: <53BB3979.20009@redhat.com> Message-ID: <1404826145.25632.0.camel@akoneru.redhat.com> ACK. On Mon, 2014-07-07 at 19:21 -0500, Endi Sukma Dewata wrote: > The REST service and client library have been fixed to use the correct > delimiters for transport certificate. > > The REST service was also modified to insert a new line between the > header and the certificate data. > > Ticket #1063 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From ftweedal at redhat.com Wed Jul 9 09:15:43 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 9 Jul 2014 19:15:43 +1000 Subject: [Pki-devel] [PATCH] PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg In-Reply-To: <1404820169.3945.7.camel@localhost.localdomain> References: <53AE3D7F.5000006@redhat.com> <20140708075537.GD2417@dhcp-40-8.bne.redhat.com> <1404820169.3945.7.camel@localhost.localdomain> Message-ID: <20140709091543.GE2417@dhcp-40-8.bne.redhat.com> On Tue, Jul 08, 2014 at 07:49:29PM +0800, Ade Lee wrote: > Fraser, > > What is likely needed is a rule permitting the pki_tomcat_t type to > create links in the config directory. > > To get the exact rule needed, please do the following: > > 1. set selinux to permissive mode (setenforce 0) > 2. clear the audit log - cat /dev/null > /var/log/audit/audit.log > 3. start the server (with the original script). Make sure to remove the > copy you have placed there. > 4. The instance should start. > 5. Check to see what rule is needed: > > audit2allow -R -i /var/log/audit/audit.log > audit2allow -R /var/log/audit/audit.log > > 6. File a BZ against selinux-policy in Fedora 20/rawhide, providing the > above output. In 10.x, our selinux policy is managed by the system > selinux policy. > > Ade Thanks for the tips Ade. I have filed a bug[1]. I also blogged about this experience[2]. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1117673 [2] http://blog-ftweedal.rhcloud.com/2014/07/diagnosing-a-dogtag-selinux-issue/ Cheers, Fraser > > On Tue, 2014-07-08 at 17:55 +1000, Fraser Tweedale wrote: > > There seems to be an selinux issue with this change. When I spawned > > a new instance, it was not premitted to create the CS.cfg.bak > > symlink on startup (and startup failed as a result). > > > > It's the end of the day and I didn't get to the bottom of it (I have > > little prior experience with selinux) but it seems specifically > > related to symlinks - when I changed the `ln -s' to a `cp' in > > scripts/operations:1569 everything works OK. > > > > So I'll leave it that for today; if anyone has any pointers (or > > patches) that would be great, otherwise I'll press on tomorrow > > morning. > > > > Cheers, > > > > Fraser > > > > On Fri, Jun 27, 2014 at 08:58:55PM -0700, Matthew Harmsen wrote: > > > Please review the attached patch for: > > > > > > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg > > > > > > > > > This patch is based upon a previously reviewed patch for the Dogtag 9 > > > architecture utilized by the IPA_v2_RHEL_6_ERRATA_BRANCH, but was modified > > > and tested to work with the Dogtag 10.2 architecture. > > > > > > CAVEAT 1: > > > > > > Although this patch contains changes to multiple PKI subsystem's > > > 'CS.cfg' configuration files, an upgrade script should not be > > > specifically required for legacy instances since the parameter that > > > is added, 'archive.configuration_file=true', is presumed even if the > > > parameter is missing (as it would be on any legacy instance). In > > > this case, it would only be necessary to add this parameter to a > > > legacy instance's CS.cfg, and set the value to 'false' in order to > > > turn off 'CS.cfg' configuration file archival (explicit instructions > > > detailing this are found in the 'operations' script). However, if > > > this is desired for completeness, I don't mind adding it. > > > > > > CAVEAT 2: > > > > > > I had originally made the effort to attempt to have specific crucial > > > WARNING messages echoed to the display as well as to the journal. I > > > believe that this would be beneficial, as, for example, it would > > > immediately notify an admin that since an error had occurred, > > > 'CS.cfg' backups would be discontinued until the error was > > > corrected. My idea was to echo these WARNING messages explicitly to > > > stderr via redirecting them (>&2), and adding the parameter > > > 'StandardError=journal+console' under the [Service] section of the > > > 'pki-tomcatd at pki-tomcat.service' file. Unfortunately, I was never > > > able to make this work - both stdout and stderr messages were stored > > > in the journal, but were never displayed to the screen when typing > > > 'systemctl restart pki-tomcatd at pki-tomcat.service' (even after a > > > 'systemctl daemon-reload' had been performed). > > > > > > -- Matt > > > > > From 22242207fd6403dd65f777691ae1bfd0a2aed678 Mon Sep 17 00:00:00 2001 > > > From: Matthew Harmsen > > > Date: Fri, 27 Jun 2014 20:35:04 -0700 > > > Subject: [PATCH] Backup and Archive CS.cfg > > > > > > * PKI TRAC Ticket #899 - RFE - ipa-server should keep backup of CS.cfg > > > --- > > > base/ca/shared/conf/CS.cfg.in | 1 + > > > base/kra/shared/conf/CS.cfg.in | 1 + > > > base/ocsp/shared/conf/CS.cfg.in | 1 + > > > base/server/scripts/operations | 211 +++++++++++++++++++++++++++++++++- > > > base/tks/shared/conf/CS.cfg.in | 1 + > > > base/tps-tomcat/shared/conf/CS.cfg.in | 1 + > > > 6 files changed, 215 insertions(+), 1 deletion(-) > > > > > > diff --git a/base/ca/shared/conf/CS.cfg.in b/base/ca/shared/conf/CS.cfg.in > > > index 90fb2d2..4ab8974 100644 > > > --- a/base/ca/shared/conf/CS.cfg.in > > > +++ b/base/ca/shared/conf/CS.cfg.in > > > @@ -159,6 +159,7 @@ accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluato > > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > > accessEvaluator.impl.user_origreq.class=com.netscape.cms.evaluators.UserOrigReqAccessEvaluator > > > +archive.configuration_file=true > > > auths._000=## > > > auths._001=## new authentication > > > auths._002=## > > > diff --git a/base/kra/shared/conf/CS.cfg.in b/base/kra/shared/conf/CS.cfg.in > > > index d8b5951..5febae8 100644 > > > --- a/base/kra/shared/conf/CS.cfg.in > > > +++ b/base/kra/shared/conf/CS.cfg.in > > > @@ -135,6 +135,7 @@ CrossCertPair.ldap=internaldb > > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > > +archive.configuration_file=true > > > auths._000=## > > > auths._001=## new authentication > > > auths._002=## > > > diff --git a/base/ocsp/shared/conf/CS.cfg.in b/base/ocsp/shared/conf/CS.cfg.in > > > index ace7f54..9f92ebf 100644 > > > --- a/base/ocsp/shared/conf/CS.cfg.in > > > +++ b/base/ocsp/shared/conf/CS.cfg.in > > > @@ -121,6 +121,7 @@ CrossCertPair.ldap=internaldb > > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > > +archive.configuration_file=true > > > auths._000=## > > > auths._001=## new authentication > > > auths._002=## > > > diff --git a/base/server/scripts/operations b/base/server/scripts/operations > > > index bfd2de8..bff3573 100644 > > > --- a/base/server/scripts/operations > > > +++ b/base/server/scripts/operations > > > @@ -1413,6 +1413,189 @@ verify_symlinks() > > > return 0 > > > } > > > > > > +backup_instance_configuration_files() > > > +{ > > > + declare -a pki_subsystems=('ca' > > > + 'kra' > > > + 'ocsp' > > > + 'tks' > > > + 'tps') > > > + > > > + # Utilize an identical timestamp on archives for each PKI subsystem > > > + # residing within the same instance to mark a common archival time > > > + timestamp=`date +%Y%m%d%H%M%S` > > > + > > > + # Automatically enable timestamped archives > > > + # > > > + # NOTE: To disable this feature for a particular PKI subsystem > > > + # within an instance, edit that PKI subsystem's 'CS.cfg' file > > > + # within the instance: > > > + # > > > + # If the 'archive.configuration_file' parameter exists, > > > + # change it to 'archive.configuration_file=false'. > > > + # > > > + # However, if the 'archive.configuration_file' parameter does > > > + # not exist, simply add 'archive.configuration_file=false' > > > + # to the 'CS.cfg'. > > > + # > > > + # In either case, it is unnecessary to restart the instance, > > > + # as each instance's 'CS.cfg' file is always processed every > > > + # time an instance is restarted. > > > + # > > > + backup_errors=0 > > > + for pki in "${pki_subsystems[@]}" > > > + do > > > + config_dir=${PKI_INSTANCE_PATH}/conf/${pki} > > > + > > > + # Check to see if this PKI subsystem exists within this instance > > > + if [ ! -d ${config_dir} ] ; then > > > + continue > > > + fi > > > + > > > + # Compute uppercase representation of this PKI subsystem > > > + PKI=${pki^^} > > > + > > > + # Backup parameters > > > + pki_instance_configuration_file=${config_dir}/CS.cfg > > > + backup_file=${config_dir}/CS.cfg.bak > > > + saved_backup_file=${config_dir}/CS.cfg.bak.saved > > > + > > > + # Check for an empty 'CS.cfg' > > > + # > > > + # NOTE: 'CS.cfg' is always a regular file > > > + # > > > + if [ ! -s ${pki_instance_configuration_file} ] ; then > > > + # Issue a warning that the 'CS.cfg' is empty > > > + echo "WARNING: The '${pki_instance_configuration_file}' is empty!" > > > + echo " ${PKI} backups will be discontinued until this" > > > + echo " issue has been resolved!" > > > + $((backup_errors++)) > > > + continue > > > + fi > > > + > > > + # Make certain that a previous attempt to backup 'CS.cfg' has not failed > > > + # (i. e. - 'CS.cfg.bak.saved' exists) > > > + # > > > + # NOTE: 'CS.cfg.bak.saved' is always a regular file > > > + # > > > + if [ -f ${saved_backup_file} ] ; then > > > + # 'CS.cfg.bak.saved' is a regular file or a symlink > > > + echo "WARNING: Since the file '${saved_backup_file}' exists, a" > > > + echo " previous backup attempt has failed! ${PKI} backups" > > > + echo " will be discontinued until this issue has been resolved!" > > > + $((backup_errors++)) > > > + continue > > > + fi > > > + > > > + # If present, compare 'CS.cfg' to 'CS.cfg.bak' to see if it is necessary > > > + # to backup 'CS.cfg'. 'CS.cfg.bak' may be a regular file, a > > > + # symlink, or a dangling symlink > > > + # > > > + # NOTE: 'CS.cfg.bak' may be a regular file, a symlink, or a > > > + # dangling symlink > > > + # > > > + if [ -f ${backup_file} ] ; then > > > + # 'CS.cfg.bak' is a regular file or a symlink > > > + cmp --silent ${pki_instance_configuration_file} ${backup_file} > > > + rv=$? > > > + if [ $rv -eq 0 ] ; then > > > + # 'CS.cfg' is identical to 'CS.cfg.bak'; > > > + # no need to archive or backup 'CS.cfg' > > > + continue > > > + fi > > > + > > > + # Since it is known that the previous 'CS.cfg.bak' file exists, and > > > + # and it is either a symlink or a regular file, save the previous > > > + # 'CS.cfg.bak' to 'CS.cfg.bak.saved' > > > + # > > > + # NOTE: If switching between simply creating backups to generating > > > + # timestamped archives, the previous 'CS.cfg.bak' that > > > + # existed as a regular file will NOT be archived! > > > + # > > > + if [ -h ${backup_file} ] ; then > > > + # 'CS.cfg.bak' is a symlink > > > + # (i. e. - copy the timestamped archive to a regular file) > > > + cp ${backup_file} ${saved_backup_file} > > > + > > > + # remove the 'CS.cfg.bak' symlink > > > + rm ${backup_file} > > > + else > > > + # 'CS.cfg.bak' is a regular file > > > + # (i. e. - simply rename the regular file) > > > + mv ${backup_file} ${saved_backup_file} > > > + fi > > > + elif [ -h ${backup_file} ] ; then > > > + # 'CS.cfg.bak' is a dangling symlink > > > + echo "WARNING: The file '${backup_file}' is a dangling symlink" > > > + echo " which suggests that the previous backup file has" > > > + echo " been removed! ${PKI} backups will be discontinued" > > > + echo " until this issue has been resolved!" > > > + $((backup_errors++)) > > > + continue > > > + fi > > > + > > > + # Check 'CS.cfg' for 'archive.configuration_file' parameter > > > + # to see if timestamped archives should be disabled > > > + archive_configuration_file="true" > > > + line=`grep -e '^[ \t]*archive.configuration_file[ \t]*=' ${pki_instance_configuration_file}` > > > + if [ "${line}" != "" ] ; then > > > + archive_configuration_file=`echo "${line}" | sed -e 's/^[^=]*[ \t]*=[ \t]*\(.*\)/\1/' -e 's/[ \t]*$//'` > > > + fi > > > + > > > + # Backup 'CS.cfg' > > > + if [ "${archive_configuration_file}" != "true" ] ; then > > > + # Always backup 'CS.cfg' to 'CS.cfg.bak' > > > + cp -b ${pki_instance_configuration_file} ${backup_file} > > > + else > > > + # Archive parameters > > > + archive_dir=${config_dir}/archives > > > + archived_file=${archive_dir}/CS.cfg.bak.${timestamp} > > > + > > > + # If not present, create an archives directory for this 'CS.cfg' > > > + if [ ! -d ${archive_dir} ] ; then > > > + mkdir -p ${archive_dir} > > > + fi > > > + > > > + # Archive 'CS.cfg' to 'CS.cfg.bak.${timestamp}' > > > + cp -a ${pki_instance_configuration_file} ${archived_file} > > > + if [ ! -s ${archived_file} ] ; then > > > + # Issue a warning that the archived backup failed > > > + echo "WARNING: Failed to archive '${pki_instance_configuration_file}' to '${archived_file}'!" > > > + $((backup_errors++)) > > > + continue > > > + fi > > > + > > > + # Always create 'CS.cfg.bak' by linking to this archived file > > > + ln -s ${archived_file} ${backup_file} > > > + > > > + # Report that 'CS.cfg' has been successfully archived > > > + echo "SUCCESS: Successfully archived '${archived_file}'" > > > + fi > > > + > > > + # Check that a non-empty 'CS.cfg.bak' symlink or regular file exists > > > + if [ ! -s ${backup_file} ] ; then > > > + # Issue a warning that the backup failed > > > + echo "WARNING: Failed to backup '${pki_instance_configuration_file}' to '${backup_file}'!" > > > + $((backup_errors++)) > > > + continue > > > + else > > > + # Report that 'CS.cfg' has been successfully backed up > > > + echo "SUCCESS: Successfully backed up '${backup_file}'" > > > + fi > > > + > > > + # Since 'CS.cfg' was backed up successfully, remove 'CS.cfg.bak.saved' > > > + if [ -f ${saved_backup_file} ] ; then > > > + rm ${saved_backup_file} > > > + fi > > > + done > > > + > > > + if [ ${backup_errors} -ne 0 ]; then > > > + return 1 > > > + fi > > > + > > > + return 0 > > > +} > > > + > > > start_instance() > > > { > > > rv=0 > > > @@ -1453,8 +1636,34 @@ start_instance() > > > return 6 > > > else > > > # 0 success > > > - return 0 > > > + > > > + # Always create a backup of each PKI subsystem's 'CS.cfg' file > > > + # within an instance. > > > + # > > > + # For every backup failure detected within a PKI subsystem within > > > + # an instance, a warning message will be issued, and an error code > > > + # of 1 will be returned. > > > + # > > > + # Note that until they have been resolved, every previous backup > > > + # failures of any PKI subsystem within an instance will also issue > > > + # a warning message and return an error code of 1. Backups of that > > > + # particular instance's PKI subsystem will be suspended until this > > > + # error has been addressed. > > > + # > > > + # By default, unless they have been explicitly disabled, > > > + # a timestamped archive of each PKI subsystem's 'CS.cfg' file > > > + # within an instance will also be created. Note that a single > > > + # timestamp will be utlized across each PKI subsystem within > > > + # an instance for each invocation of this function. > > > + # > > > + # When enabled, any timestamped archive failures also issue a > > > + # warning message and return an error code of 1. > > > + # > > > + backup_instance_configuration_files > > > + rv=$? > > > fi > > > + > > > + return $? > > > } > > > > > > # function used in debian to find the correct jdk > > > diff --git a/base/tks/shared/conf/CS.cfg.in b/base/tks/shared/conf/CS.cfg.in > > > index 4d32f6e..bd2858d 100644 > > > --- a/base/tks/shared/conf/CS.cfg.in > > > +++ b/base/tks/shared/conf/CS.cfg.in > > > @@ -112,6 +112,7 @@ CrossCertPair.ldap=internaldb > > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > > +archive.configuration_file=true > > > auths._000=## > > > auths._001=## new authentication > > > auths._002=## > > > diff --git a/base/tps-tomcat/shared/conf/CS.cfg.in b/base/tps-tomcat/shared/conf/CS.cfg.in > > > index b4b1941..57a7866 100644 > > > --- a/base/tps-tomcat/shared/conf/CS.cfg.in > > > +++ b/base/tps-tomcat/shared/conf/CS.cfg.in > > > @@ -4,6 +4,7 @@ _002=## > > > accessEvaluator.impl.group.class=com.netscape.cms.evaluators.GroupAccessEvaluator > > > accessEvaluator.impl.ipaddress.class=com.netscape.cms.evaluators.IPAddressAccessEvaluator > > > accessEvaluator.impl.user.class=com.netscape.cms.evaluators.UserAccessEvaluator > > > +archive.configuration_file=true > > > applet._000=######################################### > > > applet._001=# applet information > > > applet._002=# SAF Key: > > > -- > > > 1.9.3 > > > > > > > > _______________________________________________ > > > Pki-devel mailing list > > > Pki-devel at redhat.com > > > https://www.redhat.com/mailman/listinfo/pki-devel > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > > From edewata at redhat.com Wed Jul 9 22:51:08 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 09 Jul 2014 17:51:08 -0500 Subject: [Pki-devel] [PATCH] 515 Fixed transport certificate delimiters. In-Reply-To: <1404826145.25632.0.camel@akoneru.redhat.com> References: <53BB3979.20009@redhat.com> <1404826145.25632.0.camel@akoneru.redhat.com> Message-ID: <53BDC75C.2090100@redhat.com> On 7/8/2014 8:29 AM, Abhishek Koneru wrote: > ACK. Thanks. Pushed to master. -- Endi S. Dewata From edewata at redhat.com Wed Jul 9 22:51:13 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 09 Jul 2014 17:51:13 -0500 Subject: [Pki-devel] [PATCH] 516 Added transport cert attributes. In-Reply-To: <1404820612.3945.8.camel@localhost.localdomain> References: <53BB397C.1050806@redhat.com> <1404820612.3945.8.camel@localhost.localdomain> Message-ID: <53BDC761.3040809@redhat.com> On 7/8/2014 6:56 AM, Ade Lee wrote: > ACK Thanks. Pushed to master. -- Endi S. Dewata From akoneru at redhat.com Thu Jul 10 16:59:18 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Thu, 10 Jul 2014 12:59:18 -0400 Subject: [Pki-devel] [PATCH] 100 Refactoring profile.py and fixes a few issues Message-ID: <1405011558.3955.13.camel@akoneru.redhat.com> The attached patch contains the following changes. -- Refactoring profile.py and replacing the usage of property with a dictionary for attribute name conversions(when sending the objects back to server). -- Replace the logic of traversing the dict of NOTYPES with a search in NOTYPES.itervalues for the instance of an object. The traversal method causes an issue in-case of inherited classes as isinstance(cert_review_response, obj)) is true for obj= CertEnrolmentRequest/CertReviewResponse. Since a dict is not an ordered data structure, we cannot traverse it correctly. This creates problems during attr name conversion. This can still be modified by using a Set for NOTYPES and TYPES and adding the class objects to the sets and performing a lookup. Since this requires changes across the python code, i think we should take this up in a separate patch. -- Rewrote the attr_name_conversion method in encoder.py to return a new dict with the changes attribute names rather than make the changes to the object's __dict__. This would allow re-usage of the object. Otherwise an AttributeError will be raised when accessing an attribute. --Abhishek -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-akoneru-0100-Refactoring-ProfileClient-to-remove-the-property-fie.patch Type: text/x-patch Size: 28908 bytes Desc: not available URL: From ftweedal at redhat.com Fri Jul 11 04:57:28 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Fri, 11 Jul 2014 14:57:28 +1000 Subject: [Pki-devel] [PATCH] 100 Refactoring profile.py and fixes a few issues In-Reply-To: <1405011558.3955.13.camel@akoneru.redhat.com> References: <1405011558.3955.13.camel@akoneru.redhat.com> Message-ID: <20140711045728.GJ2417@dhcp-40-8.bne.redhat.com> A few nits. Comments inline. On Thu, Jul 10, 2014 at 12:59:18PM -0400, Abhishek Koneru wrote: > The attached patch contains the following changes. > > -- Refactoring profile.py and replacing the usage of property with a > dictionary for attribute name conversions(when sending the objects back > to server). > > -- Replace the logic of traversing the dict of NOTYPES with a search in > NOTYPES.itervalues for the instance of an object. The traversal method > causes an issue in-case of inherited classes as > isinstance(cert_review_response, obj)) is true for obj= > CertEnrolmentRequest/CertReviewResponse. Since a dict is not an ordered > data structure, we cannot traverse it correctly. This creates problems > during attr name conversion. > > This can still be modified by using a Set for NOTYPES and TYPES and > adding the class objects to the sets and performing a lookup. > Since this requires changes across the python code, i think we should > take this up in a separate patch. > > -- Rewrote the attr_name_conversion method in encoder.py to return a new > dict with the changes attribute names rather than make the changes to > the object's __dict__. This would allow re-usage of the object. > Otherwise an AttributeError will be raised when accessing an attribute. > > --Abhishek > >From b58f2dae6bda1c5b1734b3a9ba646884578e36a9 Mon Sep 17 00:00:00 2001 > From: Abhishek Koneru > Date: Mon, 30 Jun 2014 16:04:34 -0400 > Subject: [PATCH] Refactoring ProfileClient to remove the property fields. > > Replaced the usage of python property feature with a dict > for attribute name conversion. > > Fixed an issue caused to traversing the NOTYPES dict in > encoder.py to find the instance of an object. The traversal > causes an issue in the presence of subclassing. > > Modified method attr_name_conversion to return a new dictionary with > modified attribute names rather than making changes to the object's > __dict__. > --- > base/common/python/pki/__init__.py | 10 +- > base/common/python/pki/cert.py | 5 +- > base/common/python/pki/encoder.py | 20 +- > base/common/python/pki/profile.py | 462 +++++++++++-------------------------- > 4 files changed, 150 insertions(+), 347 deletions(-) > > diff --git a/base/common/python/pki/__init__.py b/base/common/python/pki/__init__.py > index 6f3aa62537d31df974bb732ad7b792bdf550ade7..4ca1e92e0ccc372d7845363a8a5614bb1b9623cf 100644 > --- a/base/common/python/pki/__init__.py > +++ b/base/common/python/pki/__init__.py > @@ -224,18 +224,18 @@ class UserNotFoundException(ResourceNotFoundException): > EXCEPTION_MAPPINGS = { > "com.netscape.certsrv.base.BadRequestException": BadRequestException, > "com.netscape.certsrv.base.ConflictingOperationException": > - ConflictingOperationException, > + ConflictingOperationException, > "com.netscape.certsrv.base.ForbiddenException": ForbiddenException, > "com.netscape.certsrv.base.HTTPGoneException": HTTPGoneException, > "com.netscape.certsrv.base.ResourceNotFoundException": > - ResourceNotFoundException, > + ResourceNotFoundException, > "com.netscape.certsrv.cert.CertNotFoundException": CertNotFoundException, > "com.netscape.certsrv.group.GroupNotFoundException": GroupNotFoundException, > "com.netscape.certsrv.key.KeyNotFoundException": KeyNotFoundException, > "com.netscape.certsrv.profile.ProfileNotFoundException": > - ProfileNotFoundException, > + ProfileNotFoundException, > "com.netscape.certsrv.request.RequestNotFoundException": > - RequestNotFoundException, > + RequestNotFoundException, > "com.netscape.certsrv.base.UserNotFoundException": UserNotFoundException, > "com.netscape.certsrv.base.PKIException": PKIException} > > @@ -392,7 +392,7 @@ class PropertyFile(object): > return None > > > -class Link: > +class Link(object): > """ > Stores the information of the resteasy's Link object sent by the server > for a resource. > diff --git a/base/common/python/pki/cert.py b/base/common/python/pki/cert.py > index 3e4ba058b3a6bb043b25f8bece4a702330706003..1882aa5c05392b2bb8e9c97933ff35098fa7f3f5 100644 > --- a/base/common/python/pki/cert.py > +++ b/base/common/python/pki/cert.py > @@ -610,6 +610,7 @@ class CertClient(object): > > url = self.cert_url + '/' + str(cert_serial_number) > r = self.connection.get(url, self.headers) > + #print r.json() Could you remove this and the subsequent commented-out debug code? > return CertData.from_json(r.json()) > > @pki.handle_exceptions() > @@ -787,6 +788,7 @@ class CertClient(object): > review_response = json.dumps(cert_review_response, > cls=encoder.CustomTypeEncoder, > sort_keys=True) > + #print review_response > r = self.connection.post(url, review_response, headers=self.headers) > return r > > @@ -888,7 +890,7 @@ class CertClient(object): > return copy.deepcopy(self.enrollment_templates[profile_id]) > url = self.cert_requests_url + '/profiles/' + str(profile_id) > r = self.connection.get(url, self.headers) > - > + #print r.json() > #Caching the enrollment template object in-memory for future use. > enrollment_template = CertEnrollmentRequest.from_json(r.json()) > self.enrollment_templates[profile_id] = enrollment_template > @@ -925,6 +927,7 @@ class CertClient(object): > request_object = json.dumps(enrollment_request, > cls=encoder.CustomTypeEncoder, > sort_keys=True) > + #print request_object > r = self.connection.post(self.cert_requests_url, request_object, > self.headers) > return CertRequestInfoCollection.from_json(r.json()) > diff --git a/base/common/python/pki/encoder.py b/base/common/python/pki/encoder.py > index 06a23250ed6f1a2835d2ac830f6b653a6f9ec0ee..68e4c957db59d88351d4ce88529ceb6ba805c242 100644 > --- a/base/common/python/pki/encoder.py > +++ b/base/common/python/pki/encoder.py > @@ -34,21 +34,23 @@ class CustomTypeEncoder(json.JSONEncoder): > for k, v in TYPES.items(): > if isinstance(obj, v): > return {k: obj.__dict__} > - for k, v in NOTYPES.items(): > - if isinstance(obj, v): > - return self.attr_name_conversion(obj.__dict__, v) > + if type(obj) in NOTYPES.itervalues(): > + return self.attr_name_conversion(obj.__dict__, type(obj)) > return json.JSONEncoder.default(self, obj) > > @staticmethod > def attr_name_conversion(attr_dict, object_class): > if not hasattr(object_class, 'json_attribute_names'): > return attr_dict > - for k, v in object_class.json_attribute_names.items(): > - if v in attr_dict: > - value = attr_dict[v] > - del attr_dict[v] > - attr_dict[k] = value > - return attr_dict > + reverse_dict = {v: k for k,v in > + object_class.json_attribute_names.items()} items() constructs a list that then gets thrown away. iteritems() returns a `dictview' object, which is more efficient. (compatibility note: in Python 3.x there is no `iteritems' but `items' is the dictview method). > + new_dict = dict() > + for k, v in attr_dict.items(): > + if k in reverse_dict: > + new_dict[reverse_dict[k]] = v > + else: > + new_dict[k] = v > + return new_dict > > > def CustomTypeDecoder(dct): > diff --git a/base/common/python/pki/profile.py b/base/common/python/pki/profile.py > index f8caf10e0a89773f3287899c68a9ce60da5cee0e..c888d38b46bce27e2d2dbbca66fdd58e4af13ef2 100644 > --- a/base/common/python/pki/profile.py > +++ b/base/common/python/pki/profile.py > @@ -33,6 +33,11 @@ import pki.encoder as encoder > class ProfileDataInfo(object): > """Stores information about a profile""" > > + json_attribute_names = { > + 'profileId': 'profile_id', 'profileName': 'profile_name', > + 'profileDescription': 'profile_description', 'profileURL': 'profile_url' > + } > + > def __init__(self): > self.profile_id = None > self.profile_name = None > @@ -56,10 +61,12 @@ class ProfileDataInfo(object): > return None > > profile_data_info = cls() > - profile_data_info.profile_id = attr_list['profileId'] > - profile_data_info.profile_name = attr_list['profileName'] > - profile_data_info.profile_description = attr_list['profileDescription'] > - profile_data_info.profile_url = attr_list['profileURL'] > + for k, v in attr_list.items(): > + if k in ProfileDataInfo.json_attribute_names: > + setattr(profile_data_info, > + ProfileDataInfo.json_attribute_names[k], v) > + else: > + setattr(profile_data_info, k, v) > > return profile_data_info > > @@ -106,6 +113,11 @@ class Descriptor(object): > a profile attribute. > """ > > + json_attribute_names = { > + 'Syntax': 'syntax', 'Description': 'description', > + 'Constraint': 'constraint', 'DefaultValue': 'default_value' > + } > + > def __init__(self, syntax=None, constraint=None, description=None, > default_value=None): > self.syntax = syntax > @@ -113,46 +125,18 @@ class Descriptor(object): > self.description = description > self.default_value = default_value > > - @property > - def syntax(self): > - return getattr(self, 'Syntax', None) > - > - @syntax.setter > - def syntax(self, value): > - setattr(self, 'Syntax', value) > - > - @property > - def constraint(self): > - return getattr(self, 'Constraint', None) > - > - @constraint.setter > - def constraint(self, value): > - setattr(self, 'Constraint', value) > - > - @property > - def description(self): > - return getattr(self, 'Description', None) > - > - @description.setter > - def description(self, value): > - setattr(self, 'Description', value) > - > - @property > - def default_value(self): > - return getattr(self, 'DefaultValue', None) > - > - @default_value.setter > - def default_value(self, value): > - setattr(self, 'DefaultValue', value) > - > @classmethod > def from_json(cls, attr_list): > if attr_list is None: > return None > > descriptor = cls() > - for attr in attr_list: > - setattr(descriptor, attr, attr_list[attr]) > + for k, v in attr_list.items(): > + if k in Descriptor.json_attribute_names: > + setattr(descriptor, > + Descriptor.json_attribute_names[k], v) > + else: > + setattr(descriptor, k, v) > > return descriptor > > @@ -161,28 +145,15 @@ class ProfileAttribute(object): > """ > Represents a profile attribute of a ProfileInput. > """ > + json_attribute_names = { > + 'Value': 'value', 'Descriptor': 'descriptor' > + } > > def __init__(self, name=None, value=None, descriptor=None): > self.name = name > self.value = value > self.descriptor = descriptor > > - @property > - def descriptor(self): > - return getattr(self, 'Descriptor') > - > - @descriptor.setter > - def descriptor(self, value): > - setattr(self, 'Descriptor', value) > - > - @property > - def value(self): > - return getattr(self, 'Value') > - > - @value.setter > - def value(self, value): > - setattr(self, 'Value', value) > - > @classmethod > def from_json(cls, attr_list): > if attr_list is None: > @@ -205,6 +176,12 @@ class ProfileInput(object): > Ex. Subject name, Requestor Information etc. > """ > > + json_attribute_names = { > + 'id': 'profile_input_id', 'ClassID': 'class_id', 'Name': 'name', > + 'Text': 'text', 'Attribute': 'attributes', > + 'ConfigAttribute': 'config_attributes' > + } > + > def __init__(self, profile_input_id=None, class_id=None, name=None, > text=None, attributes=None, config_attributes=None): > > @@ -214,56 +191,12 @@ class ProfileInput(object): > self.text = text > if attributes is None: > self.attributes = [] > + else: > + self.attributes = attributes > if config_attributes is None: > self.config_attributes = [] > - > - @property > - def profile_input_id(self): > - return getattr(self, 'id') > - > - @profile_input_id.setter > - def profile_input_id(self, value): > - setattr(self, 'id', value) > - > - @property > - def class_id(self): > - return getattr(self, 'ClassID', None) > - > - @class_id.setter > - def class_id(self, value): > - setattr(self, 'ClassID', value) > - > - @property > - def name(self): > - return getattr(self, 'Name', None) > - > - @name.setter > - def name(self, value): > - setattr(self, 'Name', value) > - > - @property > - def text(self): > - return getattr(self, 'Text', None) > - > - @text.setter > - def text(self, value): > - setattr(self, 'Text', value) > - > - @property > - def attributes(self): > - return getattr(self, 'Attribute') > - > - @attributes.setter > - def attributes(self, value): > - setattr(self, 'Attribute', value) > - > - @property > - def config_attributes(self): > - return getattr(self, 'ConfigAttribute') > - > - @config_attributes.setter > - def config_attributes(self, value): > - setattr(self, 'ConfigAttribute', value) > + else: > + self.config_attributes = config_attributes > > def add_attribute(self, profile_attribute): > """ > @@ -328,11 +261,14 @@ class ProfileInput(object): > if attr_list is None: > return None > profile_input = cls() > - profile_input.profile_input_id = attr_list['id'] > - profile_input.class_id = attr_list['ClassID'] > - profile_input.name = attr_list['Name'] > - if 'Text' in attr_list: > - profile_input.text = attr_list['Text'] > + > + for k, v in attr_list.items(): > + if k not in ['Attribute', 'ConfigAttribute']: > + if k in ProfileInput.json_attribute_names: > + setattr(profile_input, > + ProfileInput.json_attribute_names[k], v) > + else: > + setattr(profile_input, k, v) > > attributes = attr_list['Attribute'] > if not isinstance(attributes, types.ListType): > @@ -361,6 +297,10 @@ class ProfileOutput(object): > using a profile. > """ > > + json_attribute_names = { > + 'id': 'profile_output_id', 'classId': 'class_id' > + } > + > def __init__(self, profile_output_id=None, name=None, text=None, > class_id=None, attributes=None): > self.profile_output_id = profile_output_id > @@ -369,22 +309,8 @@ class ProfileOutput(object): > self.class_id = class_id > if attributes is None: > self.attributes = [] > - > - @property > - def profile_output_id(self): > - return getattr(self, 'id') > - > - @profile_output_id.setter > - def profile_output_id(self, value): > - setattr(self, 'id', value) > - > - @property > - def class_id(self): > - return getattr(self, 'classId', None) > - > - @class_id.setter > - def class_id(self, value): > - setattr(self, 'classId', value) > + else: > + self.attributes = attributes > > def add_attribute(self, profile_attribute): > """ > @@ -421,11 +347,14 @@ class ProfileOutput(object): > return None > > profile_output = cls() > - profile_output.profile_output_id = attr_list['id'] > - profile_output.name = attr_list['name'] > - if 'text' in attr_list: > - profile_output.text = attr_list['text'] > - profile_output.class_id = attr_list['classId'] > + for k, v in attr_list.items(): > + if k not in ['attributes']: > + if k in ProfileOutput.json_attribute_names: > + setattr(profile_output, > + ProfileOutput.json_attribute_names[k], v) > + else: > + setattr(profile_output, k, v) > + > attributes = attr_list['attributes'] > if not isinstance(attributes, types.ListType): > profile_output.attributes.append( > @@ -459,6 +388,11 @@ class PolicyDefault(object): > specific ProfileInput. > """ > > + json_attribute_names = { > + 'id': 'name', 'classId': 'class_id', > + 'policyAttribute': 'policy_attributes', 'params': 'policy_params' > + } > + > def __init__(self, name=None, class_id=None, description=None, > policy_attributes=None, policy_params=None): > self.name = name > @@ -473,38 +407,6 @@ class PolicyDefault(object): > else: > self.policy_params = policy_params > > - @property > - def name(self): > - return getattr(self, 'id') > - > - @name.setter > - def name(self, value): > - setattr(self, 'id', value) > - > - @property > - def class_id(self): > - return getattr(self, 'classId') > - > - @class_id.setter > - def class_id(self, value): > - setattr(self, 'classId', value) > - > - @property > - def policy_attributes(self): > - return getattr(self, 'policyAttribute') > - > - @policy_attributes.setter > - def policy_attributes(self, value): > - setattr(self, 'policyAttribute', value) > - > - @property > - def policy_params(self): > - return getattr(self, 'params') > - > - @policy_params.setter > - def policy_params(self, value): > - setattr(self, 'params', value) > - > def add_attribute(self, policy_attribute): > """ > Add a policy attribute to the attribute list. > @@ -567,12 +469,14 @@ class PolicyDefault(object): > return None > > policy_def = cls() > - if 'id' in attr_list: > - policy_def.name = attr_list['id'] > - if 'classId' in attr_list: > - policy_def.class_id = attr_list['classId'] > - if 'description' in attr_list: > - policy_def.description = attr_list['description'] > + for k, v in attr_list.items(): > + if k not in ['policyAttribute', 'params']: > + if k in PolicyDefault.json_attribute_names: > + setattr(policy_def, > + PolicyDefault.json_attribute_names[k], v) > + else: > + setattr(policy_def, k, v) > + > if 'policyAttribute' in attr_list: > attributes = attr_list['policyAttribute'] > if not isinstance(attributes, types.ListType): > @@ -597,6 +501,9 @@ class PolicyDefault(object): > > > class PolicyConstraintValue(object): > + """ > + Represents a PolicyConstraintValue > + """ > def __init__(self, name=None, value=None, descriptor=None): > self.name = name > self.value = value > @@ -616,7 +523,6 @@ class PolicyConstraintValue(object): > return None > > ret = cls() > - > ret.name = attr_list['id'] > ret.value = attr_list['value'] > if 'descriptor' in attr_list: > @@ -631,6 +537,11 @@ class PolicyConstraint(object): > ProfileInput used by a certificate enrollment request. > """ > > + json_attribute_names = { > + 'id': 'name', 'classId': 'class_id', > + 'constraint': 'policy_constraint_values' > + } > + > def __init__(self, name=None, description=None, class_id=None, > policy_constraint_values=None): > self.name = name > @@ -641,33 +552,9 @@ class PolicyConstraint(object): > else: > self.policy_constraint_values = policy_constraint_values > > - @property > - def name(self): > - return getattr(self, 'id') > - > - @name.setter > - def name(self, value): > - setattr(self, 'id', value) > - > - @property > - def class_id(self): > - return getattr(self, 'classId') > - > - @class_id.setter > - def class_id(self, value): > - setattr(self, 'classId', value) > - > - @property > - def policy_constraint_values(self): > - return getattr(self, 'constraint') > - > - @policy_constraint_values.setter > - def policy_constraint_values(self, value): > - setattr(self, 'constraint', value) > - > def add_constraint_value(self, policy_constraint_value): > """ > - Add a ProfileConstraintValue to the policy_constraint_values list. > + Add a PolicyConstraintValue to the policy_constraint_values list. > """ > if not isinstance(policy_constraint_value, PolicyConstraintValue): > raise ValueError("Object passed not of type PolicyConstraintValue") > @@ -700,20 +587,22 @@ class PolicyConstraint(object): > return None > > policy_constraint = cls() > - if 'id' in attr_list: > - policy_constraint.name = attr_list['id'] > - if 'description' in attr_list: > - policy_constraint.description = attr_list['description'] > - if 'classId' in attr_list: > - policy_constraint.class_id = attr_list['classId'] > + for k, v in attr_list.items(): > + if k not in ['constraint']: > + if k in PolicyConstraint.json_attribute_names: > + setattr(policy_constraint, > + PolicyConstraint.json_attribute_names[k], v) > + else: > + setattr(policy_constraint, k, v) > + > if 'constraint' in attr_list: > constraints = attr_list['constraint'] > if not isinstance(constraints, types.ListType): > - policy_constraint.policy_constraint_values.append( > + policy_constraint.add_constraint_value( > PolicyConstraintValue.from_json(constraints)) > else: > for constraint in constraints: > - policy_constraint.policy_constraint_values.append( > + policy_constraint.add_constraint_value( > PolicyConstraintValue.from_json(constraint)) > > return policy_constraint > @@ -727,43 +616,31 @@ class ProfilePolicy(object): > for an enrollment request. > """ > > + json_attribute_names = { > + 'id': 'policy_id', 'def': 'policy_default', > + 'constraint': 'policy_constraint' > + } > + > def __init__(self, policy_id=None, policy_default=None, > policy_constraint=None): > self.policy_id = policy_id > self.policy_default = policy_default > self.policy_constraint = policy_constraint > > - @property > - def policy_id(self): > - return getattr(self, 'id') > - > - @policy_id.setter > - def policy_id(self, value): > - setattr(self, 'id', value) > - > - @property > - def policy_default(self): > - return getattr(self, 'def') > - > - @policy_default.setter > - def policy_default(self, value): > - setattr(self, 'def', value) > - > - @property > - def policy_constraint(self): > - return getattr(self, 'constraint') > - > - @policy_constraint.setter > - def policy_constraint(self, value): > - setattr(self, 'constraint', value) > - > @classmethod > def from_json(cls, attr_list): > if attr_list is None: > return None > + policy = cls() > > - return cls(attr_list['id'], PolicyDefault.from_json(attr_list['def']), > - PolicyConstraint.from_json(attr_list['constraint'])) > + policy.policy_id = attr_list['id'] > + if 'def' in attr_list: > + policy.policy_default = PolicyDefault.from_json(attr_list['def']) > + if 'constraint' in attr_list: > + policy.policy_constraint = \ > + PolicyConstraint.from_json(attr_list['constraint']) > + > + return policy > > > class ProfilePolicySet(object): > @@ -797,6 +674,10 @@ class PolicySet(object): > policy name and the ProfilePolicy object. > """ > > + json_attribute_names = { > + 'id': 'name', 'value': 'policy_list' > + } > + > def __init__(self, name=None, policy_list=None): > self.name = name > if policy_list is None: > @@ -804,22 +685,6 @@ class PolicySet(object): > else: > self.policy_list = policy_list > > - @property > - def name(self): > - return getattr(self, 'id') > - > - @name.setter > - def name(self, value): > - setattr(self, 'id', value) > - > - @property > - def policy_list(self): > - return getattr(self, 'value') > - > - @policy_list.setter > - def policy_list(self, value): > - setattr(self, 'value', value) > - > def add_policy(self, profile_policy): > """ > Add a ProfilePolicy object to the policy_list > @@ -917,6 +782,7 @@ class PolicySetList(object): > def from_json(cls, attr_list): > if attr_list is None: > return None > + #print attr_list > > policy_set_list = cls() > policy_sets = attr_list['PolicySet'] > @@ -935,6 +801,13 @@ class Profile(object): > This class represents an enrollment profile. > """ > > + json_attribute_names = { > + 'id': 'profile_id', 'classId': 'class_id', 'enabledBy': 'enabled_by', > + 'authenticatorId': 'authenticator_id', 'authzAcl': 'authorization_acl', > + 'xmlOutput': 'xml_output', 'Input': 'inputs', 'Output': 'outputs', > + 'PolicySets': 'policy_set_list' > + } > + > def __init__(self, profile_id=None, class_id=None, name=None, > description=None, enabled=None, visible=None, enabled_by=None, > authenticator_id=None, authorization_acl=None, renewal=None, > @@ -966,78 +839,6 @@ class Profile(object): > self.policy_set_list = policy_set_list > self.link = link > > - @property > - def profile_id(self): > - return getattr(self, 'id') > - > - @profile_id.setter > - def profile_id(self, value): > - setattr(self, 'id', value) > - > - @property > - def class_id(self): > - return getattr(self, 'classId') > - > - @class_id.setter > - def class_id(self, value): > - setattr(self, 'classId', value) > - > - @property > - def enabled_by(self): > - return getattr(self, 'enabledBy') > - > - @enabled_by.setter > - def enabled_by(self, value): > - setattr(self, 'enabledBy', value) > - > - @property > - def authenticator_id(self): > - return getattr(self, 'authenticatorId') > - > - @authenticator_id.setter > - def authenticator_id(self, value): > - setattr(self, 'authenticatorId', value) > - > - @property > - def authorization_acl(self): > - return getattr(self, 'authzAcl') > - > - @authorization_acl.setter > - def authorization_acl(self, value): > - setattr(self, 'authzAcl', value) > - > - @property > - def xml_output(self): > - return getattr(self, 'xmlOutput') > - > - @xml_output.setter > - def xml_output(self, value): > - setattr(self, 'xmlOutput', value) > - > - @property > - def inputs(self): > - return getattr(self, 'Input') > - > - @inputs.setter > - def inputs(self, value): > - setattr(self, 'Input', value) > - > - @property > - def outputs(self): > - return getattr(self, 'Output') > - > - @outputs.setter > - def outputs(self, value): > - setattr(self, 'Output', value) > - > - @property > - def policy_set_list(self): > - return getattr(self, 'PolicySets') > - > - @policy_set_list.setter > - def policy_set_list(self, value): > - setattr(self, 'PolicySets', value) > - > def add_input(self, profile_input): > """ > Add a ProfileInput object to the inputs list of the Profile. > @@ -1119,19 +920,13 @@ class Profile(object): > @classmethod > def from_json(cls, attr_list): > profile_data = cls() > - profile_data.profile_id = attr_list['id'] > - profile_data.class_id = attr_list['classId'] > - profile_data.name = attr_list['name'] > - profile_data.description = attr_list['description'] > - profile_data.enabled = attr_list['enabled'] > - profile_data.visible = attr_list['visible'] > - if 'enabledBy' in attr_list: > - profile_data.enabled_by = attr_list['enabledBy'] > - if 'authenticatorId' in attr_list: > - profile_data.authenticator_id = attr_list['authenticatorId'] > - profile_data.authorization_acl = attr_list['authzAcl'] > - profile_data.renewal = attr_list['renewal'] > - profile_data.xml_output = attr_list['xmlOutput'] > + for k, v in attr_list.items(): > + if k not in ['Input', 'Output', 'PolicySets']: > + if k in Profile.json_attribute_names: > + setattr(profile_data, > + Profile.json_attribute_names[k], v) > + else: > + setattr(profile_data, k, v) > > profile_inputs = attr_list['Input'] > if not isinstance(profile_inputs, types.ListType): > @@ -1244,6 +1039,7 @@ class ProfileClient(object): > raise ValueError("Profile ID must be specified.") > url = self.profiles_url + '/' + str(profile_id) > r = self._get(url) > + #print r.json() > return Profile.from_json(r.json()) > > def _modify_profile_state(self, profile_id, action): > @@ -1289,11 +1085,11 @@ class ProfileClient(object): > def _send_profile_modify(self, profile_data): > if profile_data is None: > raise ValueError("No ProfileData specified") > - > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > - sort_keys=True) > if profile_data.profile_id is None: > raise ValueError("Profile Id is not specified.") > + profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > + sort_keys=True) > + #print profile_object > url = self.profiles_url + '/' + str(profile_data.profile_id) > r = self._put(url, profile_object) > > @@ -1413,7 +1209,7 @@ def main(): > print(' Profile ID: ' + profile_data.profile_id) > print(' Is profile enabled? ' + str(profile.enabled)) > print > - > + #profile_client.delete_profile('MySampleProfile') > # Create a new sample profile > print('Creating a new profile:') > print('-----------------------') > @@ -1576,10 +1372,12 @@ def main(): > fetch.add_input(profile_input2) > > fetch.name += " (Modified)" > + modified_profile = profile_client.modify_profile(fetch) > + > with open(file_path+'modified.json', 'w') as output_file: > output_file.write(json.dumps(fetch, cls=encoder.CustomTypeEncoder, > sort_keys=True, indent=4)) > - modified_profile = profile_client.modify_profile(fetch) > + > print(modified_profile) > print > > -- > 1.8.5.3 > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From akoneru at redhat.com Fri Jul 11 19:28:30 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Fri, 11 Jul 2014 15:28:30 -0400 Subject: [Pki-devel] [PATCH] 100 Refactoring profile.py and fixes a few issues In-Reply-To: <20140711045728.GJ2417@dhcp-40-8.bne.redhat.com> References: <1405011558.3955.13.camel@akoneru.redhat.com> <20140711045728.GJ2417@dhcp-40-8.bne.redhat.com> Message-ID: <1405106910.3288.3.camel@akoneru.redhat.com> Thanks Fraser and Ade. Fixed the issues and pushed to master. -- Abhishek On Fri, 2014-07-11 at 14:57 +1000, Fraser Tweedale wrote: > A few nits. Comments inline. > > On Thu, Jul 10, 2014 at 12:59:18PM -0400, Abhishek Koneru wrote: > > The attached patch contains the following changes. > > > > -- Refactoring profile.py and replacing the usage of property with a > > dictionary for attribute name conversions(when sending the objects back > > to server). > > > > -- Replace the logic of traversing the dict of NOTYPES with a search in > > NOTYPES.itervalues for the instance of an object. The traversal method > > causes an issue in-case of inherited classes as > > isinstance(cert_review_response, obj)) is true for obj= > > CertEnrolmentRequest/CertReviewResponse. Since a dict is not an ordered > > data structure, we cannot traverse it correctly. This creates problems > > during attr name conversion. > > > > This can still be modified by using a Set for NOTYPES and TYPES and > > adding the class objects to the sets and performing a lookup. > > Since this requires changes across the python code, i think we should > > take this up in a separate patch. > > > > -- Rewrote the attr_name_conversion method in encoder.py to return a new > > dict with the changes attribute names rather than make the changes to > > the object's __dict__. This would allow re-usage of the object. > > Otherwise an AttributeError will be raised when accessing an attribute. > > > > --Abhishek > > > >From b58f2dae6bda1c5b1734b3a9ba646884578e36a9 Mon Sep 17 00:00:00 2001 > > From: Abhishek Koneru > > Date: Mon, 30 Jun 2014 16:04:34 -0400 > > Subject: [PATCH] Refactoring ProfileClient to remove the property fields. > > > > Replaced the usage of python property feature with a dict > > for attribute name conversion. > > > > Fixed an issue caused to traversing the NOTYPES dict in > > encoder.py to find the instance of an object. The traversal > > causes an issue in the presence of subclassing. > > > > Modified method attr_name_conversion to return a new dictionary with > > modified attribute names rather than making changes to the object's > > __dict__. > > --- > > base/common/python/pki/__init__.py | 10 +- > > base/common/python/pki/cert.py | 5 +- > > base/common/python/pki/encoder.py | 20 +- > > base/common/python/pki/profile.py | 462 +++++++++++-------------------------- > > 4 files changed, 150 insertions(+), 347 deletions(-) > > > > diff --git a/base/common/python/pki/__init__.py b/base/common/python/pki/__init__.py > > index 6f3aa62537d31df974bb732ad7b792bdf550ade7..4ca1e92e0ccc372d7845363a8a5614bb1b9623cf 100644 > > --- a/base/common/python/pki/__init__.py > > +++ b/base/common/python/pki/__init__.py > > @@ -224,18 +224,18 @@ class UserNotFoundException(ResourceNotFoundException): > > EXCEPTION_MAPPINGS = { > > "com.netscape.certsrv.base.BadRequestException": BadRequestException, > > "com.netscape.certsrv.base.ConflictingOperationException": > > - ConflictingOperationException, > > + ConflictingOperationException, > > "com.netscape.certsrv.base.ForbiddenException": ForbiddenException, > > "com.netscape.certsrv.base.HTTPGoneException": HTTPGoneException, > > "com.netscape.certsrv.base.ResourceNotFoundException": > > - ResourceNotFoundException, > > + ResourceNotFoundException, > > "com.netscape.certsrv.cert.CertNotFoundException": CertNotFoundException, > > "com.netscape.certsrv.group.GroupNotFoundException": GroupNotFoundException, > > "com.netscape.certsrv.key.KeyNotFoundException": KeyNotFoundException, > > "com.netscape.certsrv.profile.ProfileNotFoundException": > > - ProfileNotFoundException, > > + ProfileNotFoundException, > > "com.netscape.certsrv.request.RequestNotFoundException": > > - RequestNotFoundException, > > + RequestNotFoundException, > > "com.netscape.certsrv.base.UserNotFoundException": UserNotFoundException, > > "com.netscape.certsrv.base.PKIException": PKIException} > > > > @@ -392,7 +392,7 @@ class PropertyFile(object): > > return None > > > > > > -class Link: > > +class Link(object): > > """ > > Stores the information of the resteasy's Link object sent by the server > > for a resource. > > diff --git a/base/common/python/pki/cert.py b/base/common/python/pki/cert.py > > index 3e4ba058b3a6bb043b25f8bece4a702330706003..1882aa5c05392b2bb8e9c97933ff35098fa7f3f5 100644 > > --- a/base/common/python/pki/cert.py > > +++ b/base/common/python/pki/cert.py > > @@ -610,6 +610,7 @@ class CertClient(object): > > > > url = self.cert_url + '/' + str(cert_serial_number) > > r = self.connection.get(url, self.headers) > > + #print r.json() > > Could you remove this and the subsequent commented-out debug code? > > > return CertData.from_json(r.json()) > > > > @pki.handle_exceptions() > > @@ -787,6 +788,7 @@ class CertClient(object): > > review_response = json.dumps(cert_review_response, > > cls=encoder.CustomTypeEncoder, > > sort_keys=True) > > + #print review_response > > r = self.connection.post(url, review_response, headers=self.headers) > > return r > > > > @@ -888,7 +890,7 @@ class CertClient(object): > > return copy.deepcopy(self.enrollment_templates[profile_id]) > > url = self.cert_requests_url + '/profiles/' + str(profile_id) > > r = self.connection.get(url, self.headers) > > - > > + #print r.json() > > #Caching the enrollment template object in-memory for future use. > > enrollment_template = CertEnrollmentRequest.from_json(r.json()) > > self.enrollment_templates[profile_id] = enrollment_template > > @@ -925,6 +927,7 @@ class CertClient(object): > > request_object = json.dumps(enrollment_request, > > cls=encoder.CustomTypeEncoder, > > sort_keys=True) > > + #print request_object > > r = self.connection.post(self.cert_requests_url, request_object, > > self.headers) > > return CertRequestInfoCollection.from_json(r.json()) > > diff --git a/base/common/python/pki/encoder.py b/base/common/python/pki/encoder.py > > index 06a23250ed6f1a2835d2ac830f6b653a6f9ec0ee..68e4c957db59d88351d4ce88529ceb6ba805c242 100644 > > --- a/base/common/python/pki/encoder.py > > +++ b/base/common/python/pki/encoder.py > > @@ -34,21 +34,23 @@ class CustomTypeEncoder(json.JSONEncoder): > > for k, v in TYPES.items(): > > if isinstance(obj, v): > > return {k: obj.__dict__} > > - for k, v in NOTYPES.items(): > > - if isinstance(obj, v): > > - return self.attr_name_conversion(obj.__dict__, v) > > + if type(obj) in NOTYPES.itervalues(): > > + return self.attr_name_conversion(obj.__dict__, type(obj)) > > return json.JSONEncoder.default(self, obj) > > > > @staticmethod > > def attr_name_conversion(attr_dict, object_class): > > if not hasattr(object_class, 'json_attribute_names'): > > return attr_dict > > - for k, v in object_class.json_attribute_names.items(): > > - if v in attr_dict: > > - value = attr_dict[v] > > - del attr_dict[v] > > - attr_dict[k] = value > > - return attr_dict > > + reverse_dict = {v: k for k,v in > > + object_class.json_attribute_names.items()} > > items() constructs a list that then gets thrown away. iteritems() > returns a `dictview' object, which is more efficient. > (compatibility note: in Python 3.x there is no `iteritems' but > `items' is the dictview method). > > > + new_dict = dict() > > + for k, v in attr_dict.items(): > > + if k in reverse_dict: > > + new_dict[reverse_dict[k]] = v > > + else: > > + new_dict[k] = v > > + return new_dict > > > > > > def CustomTypeDecoder(dct): > > diff --git a/base/common/python/pki/profile.py b/base/common/python/pki/profile.py > > index f8caf10e0a89773f3287899c68a9ce60da5cee0e..c888d38b46bce27e2d2dbbca66fdd58e4af13ef2 100644 > > --- a/base/common/python/pki/profile.py > > +++ b/base/common/python/pki/profile.py > > @@ -33,6 +33,11 @@ import pki.encoder as encoder > > class ProfileDataInfo(object): > > """Stores information about a profile""" > > > > + json_attribute_names = { > > + 'profileId': 'profile_id', 'profileName': 'profile_name', > > + 'profileDescription': 'profile_description', 'profileURL': 'profile_url' > > + } > > + > > def __init__(self): > > self.profile_id = None > > self.profile_name = None > > @@ -56,10 +61,12 @@ class ProfileDataInfo(object): > > return None > > > > profile_data_info = cls() > > - profile_data_info.profile_id = attr_list['profileId'] > > - profile_data_info.profile_name = attr_list['profileName'] > > - profile_data_info.profile_description = attr_list['profileDescription'] > > - profile_data_info.profile_url = attr_list['profileURL'] > > + for k, v in attr_list.items(): > > + if k in ProfileDataInfo.json_attribute_names: > > + setattr(profile_data_info, > > + ProfileDataInfo.json_attribute_names[k], v) > > + else: > > + setattr(profile_data_info, k, v) > > > > return profile_data_info > > > > @@ -106,6 +113,11 @@ class Descriptor(object): > > a profile attribute. > > """ > > > > + json_attribute_names = { > > + 'Syntax': 'syntax', 'Description': 'description', > > + 'Constraint': 'constraint', 'DefaultValue': 'default_value' > > + } > > + > > def __init__(self, syntax=None, constraint=None, description=None, > > default_value=None): > > self.syntax = syntax > > @@ -113,46 +125,18 @@ class Descriptor(object): > > self.description = description > > self.default_value = default_value > > > > - @property > > - def syntax(self): > > - return getattr(self, 'Syntax', None) > > - > > - @syntax.setter > > - def syntax(self, value): > > - setattr(self, 'Syntax', value) > > - > > - @property > > - def constraint(self): > > - return getattr(self, 'Constraint', None) > > - > > - @constraint.setter > > - def constraint(self, value): > > - setattr(self, 'Constraint', value) > > - > > - @property > > - def description(self): > > - return getattr(self, 'Description', None) > > - > > - @description.setter > > - def description(self, value): > > - setattr(self, 'Description', value) > > - > > - @property > > - def default_value(self): > > - return getattr(self, 'DefaultValue', None) > > - > > - @default_value.setter > > - def default_value(self, value): > > - setattr(self, 'DefaultValue', value) > > - > > @classmethod > > def from_json(cls, attr_list): > > if attr_list is None: > > return None > > > > descriptor = cls() > > - for attr in attr_list: > > - setattr(descriptor, attr, attr_list[attr]) > > + for k, v in attr_list.items(): > > + if k in Descriptor.json_attribute_names: > > + setattr(descriptor, > > + Descriptor.json_attribute_names[k], v) > > + else: > > + setattr(descriptor, k, v) > > > > return descriptor > > > > @@ -161,28 +145,15 @@ class ProfileAttribute(object): > > """ > > Represents a profile attribute of a ProfileInput. > > """ > > + json_attribute_names = { > > + 'Value': 'value', 'Descriptor': 'descriptor' > > + } > > > > def __init__(self, name=None, value=None, descriptor=None): > > self.name = name > > self.value = value > > self.descriptor = descriptor > > > > - @property > > - def descriptor(self): > > - return getattr(self, 'Descriptor') > > - > > - @descriptor.setter > > - def descriptor(self, value): > > - setattr(self, 'Descriptor', value) > > - > > - @property > > - def value(self): > > - return getattr(self, 'Value') > > - > > - @value.setter > > - def value(self, value): > > - setattr(self, 'Value', value) > > - > > @classmethod > > def from_json(cls, attr_list): > > if attr_list is None: > > @@ -205,6 +176,12 @@ class ProfileInput(object): > > Ex. Subject name, Requestor Information etc. > > """ > > > > + json_attribute_names = { > > + 'id': 'profile_input_id', 'ClassID': 'class_id', 'Name': 'name', > > + 'Text': 'text', 'Attribute': 'attributes', > > + 'ConfigAttribute': 'config_attributes' > > + } > > + > > def __init__(self, profile_input_id=None, class_id=None, name=None, > > text=None, attributes=None, config_attributes=None): > > > > @@ -214,56 +191,12 @@ class ProfileInput(object): > > self.text = text > > if attributes is None: > > self.attributes = [] > > + else: > > + self.attributes = attributes > > if config_attributes is None: > > self.config_attributes = [] > > - > > - @property > > - def profile_input_id(self): > > - return getattr(self, 'id') > > - > > - @profile_input_id.setter > > - def profile_input_id(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def class_id(self): > > - return getattr(self, 'ClassID', None) > > - > > - @class_id.setter > > - def class_id(self, value): > > - setattr(self, 'ClassID', value) > > - > > - @property > > - def name(self): > > - return getattr(self, 'Name', None) > > - > > - @name.setter > > - def name(self, value): > > - setattr(self, 'Name', value) > > - > > - @property > > - def text(self): > > - return getattr(self, 'Text', None) > > - > > - @text.setter > > - def text(self, value): > > - setattr(self, 'Text', value) > > - > > - @property > > - def attributes(self): > > - return getattr(self, 'Attribute') > > - > > - @attributes.setter > > - def attributes(self, value): > > - setattr(self, 'Attribute', value) > > - > > - @property > > - def config_attributes(self): > > - return getattr(self, 'ConfigAttribute') > > - > > - @config_attributes.setter > > - def config_attributes(self, value): > > - setattr(self, 'ConfigAttribute', value) > > + else: > > + self.config_attributes = config_attributes > > > > def add_attribute(self, profile_attribute): > > """ > > @@ -328,11 +261,14 @@ class ProfileInput(object): > > if attr_list is None: > > return None > > profile_input = cls() > > - profile_input.profile_input_id = attr_list['id'] > > - profile_input.class_id = attr_list['ClassID'] > > - profile_input.name = attr_list['Name'] > > - if 'Text' in attr_list: > > - profile_input.text = attr_list['Text'] > > + > > + for k, v in attr_list.items(): > > + if k not in ['Attribute', 'ConfigAttribute']: > > + if k in ProfileInput.json_attribute_names: > > + setattr(profile_input, > > + ProfileInput.json_attribute_names[k], v) > > + else: > > + setattr(profile_input, k, v) > > > > attributes = attr_list['Attribute'] > > if not isinstance(attributes, types.ListType): > > @@ -361,6 +297,10 @@ class ProfileOutput(object): > > using a profile. > > """ > > > > + json_attribute_names = { > > + 'id': 'profile_output_id', 'classId': 'class_id' > > + } > > + > > def __init__(self, profile_output_id=None, name=None, text=None, > > class_id=None, attributes=None): > > self.profile_output_id = profile_output_id > > @@ -369,22 +309,8 @@ class ProfileOutput(object): > > self.class_id = class_id > > if attributes is None: > > self.attributes = [] > > - > > - @property > > - def profile_output_id(self): > > - return getattr(self, 'id') > > - > > - @profile_output_id.setter > > - def profile_output_id(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def class_id(self): > > - return getattr(self, 'classId', None) > > - > > - @class_id.setter > > - def class_id(self, value): > > - setattr(self, 'classId', value) > > + else: > > + self.attributes = attributes > > > > def add_attribute(self, profile_attribute): > > """ > > @@ -421,11 +347,14 @@ class ProfileOutput(object): > > return None > > > > profile_output = cls() > > - profile_output.profile_output_id = attr_list['id'] > > - profile_output.name = attr_list['name'] > > - if 'text' in attr_list: > > - profile_output.text = attr_list['text'] > > - profile_output.class_id = attr_list['classId'] > > + for k, v in attr_list.items(): > > + if k not in ['attributes']: > > + if k in ProfileOutput.json_attribute_names: > > + setattr(profile_output, > > + ProfileOutput.json_attribute_names[k], v) > > + else: > > + setattr(profile_output, k, v) > > + > > attributes = attr_list['attributes'] > > if not isinstance(attributes, types.ListType): > > profile_output.attributes.append( > > @@ -459,6 +388,11 @@ class PolicyDefault(object): > > specific ProfileInput. > > """ > > > > + json_attribute_names = { > > + 'id': 'name', 'classId': 'class_id', > > + 'policyAttribute': 'policy_attributes', 'params': 'policy_params' > > + } > > + > > def __init__(self, name=None, class_id=None, description=None, > > policy_attributes=None, policy_params=None): > > self.name = name > > @@ -473,38 +407,6 @@ class PolicyDefault(object): > > else: > > self.policy_params = policy_params > > > > - @property > > - def name(self): > > - return getattr(self, 'id') > > - > > - @name.setter > > - def name(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def class_id(self): > > - return getattr(self, 'classId') > > - > > - @class_id.setter > > - def class_id(self, value): > > - setattr(self, 'classId', value) > > - > > - @property > > - def policy_attributes(self): > > - return getattr(self, 'policyAttribute') > > - > > - @policy_attributes.setter > > - def policy_attributes(self, value): > > - setattr(self, 'policyAttribute', value) > > - > > - @property > > - def policy_params(self): > > - return getattr(self, 'params') > > - > > - @policy_params.setter > > - def policy_params(self, value): > > - setattr(self, 'params', value) > > - > > def add_attribute(self, policy_attribute): > > """ > > Add a policy attribute to the attribute list. > > @@ -567,12 +469,14 @@ class PolicyDefault(object): > > return None > > > > policy_def = cls() > > - if 'id' in attr_list: > > - policy_def.name = attr_list['id'] > > - if 'classId' in attr_list: > > - policy_def.class_id = attr_list['classId'] > > - if 'description' in attr_list: > > - policy_def.description = attr_list['description'] > > + for k, v in attr_list.items(): > > + if k not in ['policyAttribute', 'params']: > > + if k in PolicyDefault.json_attribute_names: > > + setattr(policy_def, > > + PolicyDefault.json_attribute_names[k], v) > > + else: > > + setattr(policy_def, k, v) > > + > > if 'policyAttribute' in attr_list: > > attributes = attr_list['policyAttribute'] > > if not isinstance(attributes, types.ListType): > > @@ -597,6 +501,9 @@ class PolicyDefault(object): > > > > > > class PolicyConstraintValue(object): > > + """ > > + Represents a PolicyConstraintValue > > + """ > > def __init__(self, name=None, value=None, descriptor=None): > > self.name = name > > self.value = value > > @@ -616,7 +523,6 @@ class PolicyConstraintValue(object): > > return None > > > > ret = cls() > > - > > ret.name = attr_list['id'] > > ret.value = attr_list['value'] > > if 'descriptor' in attr_list: > > @@ -631,6 +537,11 @@ class PolicyConstraint(object): > > ProfileInput used by a certificate enrollment request. > > """ > > > > + json_attribute_names = { > > + 'id': 'name', 'classId': 'class_id', > > + 'constraint': 'policy_constraint_values' > > + } > > + > > def __init__(self, name=None, description=None, class_id=None, > > policy_constraint_values=None): > > self.name = name > > @@ -641,33 +552,9 @@ class PolicyConstraint(object): > > else: > > self.policy_constraint_values = policy_constraint_values > > > > - @property > > - def name(self): > > - return getattr(self, 'id') > > - > > - @name.setter > > - def name(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def class_id(self): > > - return getattr(self, 'classId') > > - > > - @class_id.setter > > - def class_id(self, value): > > - setattr(self, 'classId', value) > > - > > - @property > > - def policy_constraint_values(self): > > - return getattr(self, 'constraint') > > - > > - @policy_constraint_values.setter > > - def policy_constraint_values(self, value): > > - setattr(self, 'constraint', value) > > - > > def add_constraint_value(self, policy_constraint_value): > > """ > > - Add a ProfileConstraintValue to the policy_constraint_values list. > > + Add a PolicyConstraintValue to the policy_constraint_values list. > > """ > > if not isinstance(policy_constraint_value, PolicyConstraintValue): > > raise ValueError("Object passed not of type PolicyConstraintValue") > > @@ -700,20 +587,22 @@ class PolicyConstraint(object): > > return None > > > > policy_constraint = cls() > > - if 'id' in attr_list: > > - policy_constraint.name = attr_list['id'] > > - if 'description' in attr_list: > > - policy_constraint.description = attr_list['description'] > > - if 'classId' in attr_list: > > - policy_constraint.class_id = attr_list['classId'] > > + for k, v in attr_list.items(): > > + if k not in ['constraint']: > > + if k in PolicyConstraint.json_attribute_names: > > + setattr(policy_constraint, > > + PolicyConstraint.json_attribute_names[k], v) > > + else: > > + setattr(policy_constraint, k, v) > > + > > if 'constraint' in attr_list: > > constraints = attr_list['constraint'] > > if not isinstance(constraints, types.ListType): > > - policy_constraint.policy_constraint_values.append( > > + policy_constraint.add_constraint_value( > > PolicyConstraintValue.from_json(constraints)) > > else: > > for constraint in constraints: > > - policy_constraint.policy_constraint_values.append( > > + policy_constraint.add_constraint_value( > > PolicyConstraintValue.from_json(constraint)) > > > > return policy_constraint > > @@ -727,43 +616,31 @@ class ProfilePolicy(object): > > for an enrollment request. > > """ > > > > + json_attribute_names = { > > + 'id': 'policy_id', 'def': 'policy_default', > > + 'constraint': 'policy_constraint' > > + } > > + > > def __init__(self, policy_id=None, policy_default=None, > > policy_constraint=None): > > self.policy_id = policy_id > > self.policy_default = policy_default > > self.policy_constraint = policy_constraint > > > > - @property > > - def policy_id(self): > > - return getattr(self, 'id') > > - > > - @policy_id.setter > > - def policy_id(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def policy_default(self): > > - return getattr(self, 'def') > > - > > - @policy_default.setter > > - def policy_default(self, value): > > - setattr(self, 'def', value) > > - > > - @property > > - def policy_constraint(self): > > - return getattr(self, 'constraint') > > - > > - @policy_constraint.setter > > - def policy_constraint(self, value): > > - setattr(self, 'constraint', value) > > - > > @classmethod > > def from_json(cls, attr_list): > > if attr_list is None: > > return None > > + policy = cls() > > > > - return cls(attr_list['id'], PolicyDefault.from_json(attr_list['def']), > > - PolicyConstraint.from_json(attr_list['constraint'])) > > + policy.policy_id = attr_list['id'] > > + if 'def' in attr_list: > > + policy.policy_default = PolicyDefault.from_json(attr_list['def']) > > + if 'constraint' in attr_list: > > + policy.policy_constraint = \ > > + PolicyConstraint.from_json(attr_list['constraint']) > > + > > + return policy > > > > > > class ProfilePolicySet(object): > > @@ -797,6 +674,10 @@ class PolicySet(object): > > policy name and the ProfilePolicy object. > > """ > > > > + json_attribute_names = { > > + 'id': 'name', 'value': 'policy_list' > > + } > > + > > def __init__(self, name=None, policy_list=None): > > self.name = name > > if policy_list is None: > > @@ -804,22 +685,6 @@ class PolicySet(object): > > else: > > self.policy_list = policy_list > > > > - @property > > - def name(self): > > - return getattr(self, 'id') > > - > > - @name.setter > > - def name(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def policy_list(self): > > - return getattr(self, 'value') > > - > > - @policy_list.setter > > - def policy_list(self, value): > > - setattr(self, 'value', value) > > - > > def add_policy(self, profile_policy): > > """ > > Add a ProfilePolicy object to the policy_list > > @@ -917,6 +782,7 @@ class PolicySetList(object): > > def from_json(cls, attr_list): > > if attr_list is None: > > return None > > + #print attr_list > > > > policy_set_list = cls() > > policy_sets = attr_list['PolicySet'] > > @@ -935,6 +801,13 @@ class Profile(object): > > This class represents an enrollment profile. > > """ > > > > + json_attribute_names = { > > + 'id': 'profile_id', 'classId': 'class_id', 'enabledBy': 'enabled_by', > > + 'authenticatorId': 'authenticator_id', 'authzAcl': 'authorization_acl', > > + 'xmlOutput': 'xml_output', 'Input': 'inputs', 'Output': 'outputs', > > + 'PolicySets': 'policy_set_list' > > + } > > + > > def __init__(self, profile_id=None, class_id=None, name=None, > > description=None, enabled=None, visible=None, enabled_by=None, > > authenticator_id=None, authorization_acl=None, renewal=None, > > @@ -966,78 +839,6 @@ class Profile(object): > > self.policy_set_list = policy_set_list > > self.link = link > > > > - @property > > - def profile_id(self): > > - return getattr(self, 'id') > > - > > - @profile_id.setter > > - def profile_id(self, value): > > - setattr(self, 'id', value) > > - > > - @property > > - def class_id(self): > > - return getattr(self, 'classId') > > - > > - @class_id.setter > > - def class_id(self, value): > > - setattr(self, 'classId', value) > > - > > - @property > > - def enabled_by(self): > > - return getattr(self, 'enabledBy') > > - > > - @enabled_by.setter > > - def enabled_by(self, value): > > - setattr(self, 'enabledBy', value) > > - > > - @property > > - def authenticator_id(self): > > - return getattr(self, 'authenticatorId') > > - > > - @authenticator_id.setter > > - def authenticator_id(self, value): > > - setattr(self, 'authenticatorId', value) > > - > > - @property > > - def authorization_acl(self): > > - return getattr(self, 'authzAcl') > > - > > - @authorization_acl.setter > > - def authorization_acl(self, value): > > - setattr(self, 'authzAcl', value) > > - > > - @property > > - def xml_output(self): > > - return getattr(self, 'xmlOutput') > > - > > - @xml_output.setter > > - def xml_output(self, value): > > - setattr(self, 'xmlOutput', value) > > - > > - @property > > - def inputs(self): > > - return getattr(self, 'Input') > > - > > - @inputs.setter > > - def inputs(self, value): > > - setattr(self, 'Input', value) > > - > > - @property > > - def outputs(self): > > - return getattr(self, 'Output') > > - > > - @outputs.setter > > - def outputs(self, value): > > - setattr(self, 'Output', value) > > - > > - @property > > - def policy_set_list(self): > > - return getattr(self, 'PolicySets') > > - > > - @policy_set_list.setter > > - def policy_set_list(self, value): > > - setattr(self, 'PolicySets', value) > > - > > def add_input(self, profile_input): > > """ > > Add a ProfileInput object to the inputs list of the Profile. > > @@ -1119,19 +920,13 @@ class Profile(object): > > @classmethod > > def from_json(cls, attr_list): > > profile_data = cls() > > - profile_data.profile_id = attr_list['id'] > > - profile_data.class_id = attr_list['classId'] > > - profile_data.name = attr_list['name'] > > - profile_data.description = attr_list['description'] > > - profile_data.enabled = attr_list['enabled'] > > - profile_data.visible = attr_list['visible'] > > - if 'enabledBy' in attr_list: > > - profile_data.enabled_by = attr_list['enabledBy'] > > - if 'authenticatorId' in attr_list: > > - profile_data.authenticator_id = attr_list['authenticatorId'] > > - profile_data.authorization_acl = attr_list['authzAcl'] > > - profile_data.renewal = attr_list['renewal'] > > - profile_data.xml_output = attr_list['xmlOutput'] > > + for k, v in attr_list.items(): > > + if k not in ['Input', 'Output', 'PolicySets']: > > + if k in Profile.json_attribute_names: > > + setattr(profile_data, > > + Profile.json_attribute_names[k], v) > > + else: > > + setattr(profile_data, k, v) > > > > profile_inputs = attr_list['Input'] > > if not isinstance(profile_inputs, types.ListType): > > @@ -1244,6 +1039,7 @@ class ProfileClient(object): > > raise ValueError("Profile ID must be specified.") > > url = self.profiles_url + '/' + str(profile_id) > > r = self._get(url) > > + #print r.json() > > return Profile.from_json(r.json()) > > > > def _modify_profile_state(self, profile_id, action): > > @@ -1289,11 +1085,11 @@ class ProfileClient(object): > > def _send_profile_modify(self, profile_data): > > if profile_data is None: > > raise ValueError("No ProfileData specified") > > - > > - profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > - sort_keys=True) > > if profile_data.profile_id is None: > > raise ValueError("Profile Id is not specified.") > > + profile_object = json.dumps(profile_data, cls=encoder.CustomTypeEncoder, > > + sort_keys=True) > > + #print profile_object > > url = self.profiles_url + '/' + str(profile_data.profile_id) > > r = self._put(url, profile_object) > > > > @@ -1413,7 +1209,7 @@ def main(): > > print(' Profile ID: ' + profile_data.profile_id) > > print(' Is profile enabled? ' + str(profile.enabled)) > > print > > - > > + #profile_client.delete_profile('MySampleProfile') > > # Create a new sample profile > > print('Creating a new profile:') > > print('-----------------------') > > @@ -1576,10 +1372,12 @@ def main(): > > fetch.add_input(profile_input2) > > > > fetch.name += " (Modified)" > > + modified_profile = profile_client.modify_profile(fetch) > > + > > with open(file_path+'modified.json', 'w') as output_file: > > output_file.write(json.dumps(fetch, cls=encoder.CustomTypeEncoder, > > sort_keys=True, indent=4)) > > - modified_profile = profile_client.modify_profile(fetch) > > + > > print(modified_profile) > > print > > > > -- > > 1.8.5.3 > > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > From akoneru at redhat.com Mon Jul 14 02:15:19 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Sun, 13 Jul 2014 22:15:19 -0400 Subject: [Pki-devel] PTO Monday 07/14/2014 Message-ID: <1405304119.3100.6.camel@akoneru.redhat.com> Will be on PTO for 1 day as i will be out of town. Thanks, Abhishek From ftweedal at redhat.com Mon Jul 14 05:25:10 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Mon, 14 Jul 2014 15:25:10 +1000 Subject: [Pki-devel] importing profiles into database Message-ID: <20140714052510.GM2417@dhcp-40-8.bne.redhat.com> Hi all, As part of http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage I will be adding a script to import the file-based CA profiles into the database. Are the upgrade scriptlets the appropriate way to do this, or should I take a different approach? Regards, Fraser From alee at redhat.com Mon Jul 14 07:48:21 2014 From: alee at redhat.com (Ade Lee) Date: Mon, 14 Jul 2014 03:48:21 -0400 (EDT) Subject: [Pki-devel] [PATCH] 226 - allow databse to be installed as subtree of existing ldap tree In-Reply-To: <1568139936.7660540.1405323989632.JavaMail.zimbra@redhat.com> Message-ID: <1281079198.7660642.1405324101924.JavaMail.zimbra@redhat.com> Add ability to create database as subtree of existing tree This patch adds the ability to create a subsystem that uses an existing subtree to create the internal basedn. This is useful for instance, for IPA which will use the original o=ipaca as the top level DN for a KRA, which will be situated at o=ipadrm, o=ipaca. The patch also allows such a system to be cloned, but not to setup the replication agreements, on the assumption that the data is already being replicated at the top-level DN or some higher level. The patch also contains some minor cleanups - removing unused imports and removal of an invalid reference in the python code. Ticket 1051 Note: Changes to the man pages will be submitted in a separate follow-on patch. Please review, Ade -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-vakwetu-0226-Add-ability-to-create-database-as-subtree-of-existin.patch Type: text/x-patch Size: 41515 bytes Desc: not available URL: From alee at redhat.com Mon Jul 14 09:45:52 2014 From: alee at redhat.com (Ade Lee) Date: Mon, 14 Jul 2014 05:45:52 -0400 (EDT) Subject: [Pki-devel] [PATCH] - Add DRM to IPA In-Reply-To: <1100861497.7686981.1405330421267.JavaMail.zimbra@redhat.com> Message-ID: <1922084174.7689340.1405331152213.JavaMail.zimbra@redhat.com> Hi all, I have rebased all the previous patches against master, and have squashed them all into a single patch. Its a large patch, but as many folks have already reviewed the constituent precursor patches, most if it should be familiar and easier to review. The main difference with what was specified before is that the DRM database is installed as a subtree to o=ipaca. This means that no new replication agreements will be needed to replicate DRM data. Replication agreements set up for the Dogtag CA will automatically replicate DRM data. In order for this patch to work, a new 10.2 build of Dogtag 10.2 is needed - with specific changes to allow the ability to install a database as a subtree of an existing tree. At this time, these changes have not yet been checked into the dogtag source. You can obtain such a build from: http://copr.fedoraproject.org/coprs/vakwetu/dogtag/build/21936/ Please review, Thanks, Ade -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-a-DRM-to-IPA.patch Type: text/x-patch Size: 143139 bytes Desc: not available URL: From ftweedal at redhat.com Tue Jul 15 07:03:47 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 15 Jul 2014 17:03:47 +1000 Subject: [Pki-devel] LDAP profiles progress update (and patch preview) Message-ID: <20140715070347.GO2417@dhcp-40-8.bne.redhat.com> Hi all, Drafts of my first two patches in for LDAP profiles are attached. There are several more patches yet to come, including switching the ProfileSubsystem from files to LDAP, which is almost complete. I am not as far along in the implementation as I hoped for this week, but not too far behind. My main stumbling blocks last week were typos and other invalid things in my schema definition, and the Dogtag SELinux doing that rounds that culminated in a Bugzilla ticket for selinux-policy-targeted[1] and a blog post[2] about it. Patch 0004 add the LDAP schema for profile profiles, and patch 0005 add an LDAPConfigStore class that profiles will use to load and save their configuration to the database. Feedback is most welcome. Cheers, Fraser [1] https://bugzilla.redhat.com/show_bug.cgi?id=1117673 [2] http://blog-ftweedal.rhcloud.com/2014/07/diagnosing-a-dogtag-selinux-issue/ -------------- next part -------------- >From 693bbdc19a1ee398dc2f8572fb64ab15cba89d92 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Mon, 7 Jul 2014 23:35:35 -0400 Subject: [PATCH] add schema for LDAP-based profiles --- base/ca/shared/conf/db.ldif | 9 +++++++++ base/ca/shared/conf/schema.ldif | 30 ++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/base/ca/shared/conf/db.ldif b/base/ca/shared/conf/db.ldif index 00fa919b7df38ed97f0bc21b5616a9998845c7d4..b0f36b6d85f70900d0b2c5e0ee92d9fb8c1cb3e8 100644 --- a/base/ca/shared/conf/db.ldif +++ b/base/ca/shared/conf/db.ldif @@ -160,4 +160,13 @@ objectClass: top objectClass: organizationalUnit ou: certificateRepository +dn: ou=certProfiles,{rootSuffix} +objectClass: top +objectClass: organizationalUnit +ou: certProfiles +dn: cn=certProfilesInfo,{rootSuffix} +objectClass: top +objectClass: certProfilesInfo +cn: certProfilesInfo +certProfilesLastModified: 197001010000Z diff --git a/base/ca/shared/conf/schema.ldif b/base/ca/shared/conf/schema.ldif index 70578e21ce4e102909a1b7b45fa84c184a997bdf..4f74869da1e46b39469dd17ddb2517e111300b43 100644 --- a/base/ca/shared/conf/schema.ldif +++ b/base/ca/shared/conf/schema.ldif @@ -487,3 +487,33 @@ dn: cn=schema changetype: modify add: objectClasses objectClasses: ( securityDomainSessionEntry-oid NAME 'securityDomainSessionEntry' DESC 'CMS defined class' SUP top STRUCTURAL MUST ( cn $ host $ uid $ cmsUserGroup $ dateOfCreate ) X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( classId-oid NAME 'classId' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( certProfileIsDefault-oid NAME 'certProfileIsDefault' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( certProfileConfig-oid NAME 'certProfileConfig' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: objectClasses +objectClasses: ( certProfile-oid NAME 'certProfile' DESC 'CMS defined class' SUP top STRUCTURAL MUST cn MAY ( classId $ certProfileIsDefault $ certProfileConfig ) X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( certProfilesLastModified-oid NAME 'certProfilesLastModified' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: objectClasses +objectClasses: ( certProfilesInfo-oid NAME 'certProfilesInfo' DESC 'CMS defined class' SUP top STRUCTURAL MUST cn MAY certProfilesLastModified X-ORIGIN 'user defined' ) -- 1.9.3 -------------- next part -------------- >From fc39af6f9d884ec3c1609c5ab249397e8b68a78a Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Tue, 15 Jul 2014 02:48:35 -0400 Subject: [PATCH] add LDAPConfigStore class The LDAPConfigStore class is an IConfigStore that reads and writes its configuration to a given attribute and DN in an LDAP database. --- .../com/netscape/cmscore/base/LDAPConfigStore.java | 114 +++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java diff --git a/base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java b/base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java new file mode 100644 index 0000000000000000000000000000000000000000..7d08fe9d3b75ceb954f3bd515eec5076d0af2db9 --- /dev/null +++ b/base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java @@ -0,0 +1,114 @@ +// --- BEGIN COPYRIGHT BLOCK --- +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; version 2 of the License. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// +// (C) 2007, 2014 Red Hat, Inc. +// All rights reserved. +// --- END COPYRIGHT BLOCK --- + +package com.netscape.cmscore.base; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.InputStream; +import java.io.IOException; +import java.util.Map; + +import netscape.ldap.LDAPAttribute; +import netscape.ldap.LDAPConnection; +import netscape.ldap.LDAPEntry; +import netscape.ldap.LDAPModification; + +import com.netscape.certsrv.apps.CMS; +import com.netscape.certsrv.base.EBaseException; +import com.netscape.certsrv.base.IConfigStore; +import com.netscape.certsrv.ldap.ILdapConnFactory; +import com.netscape.cmsutil.util.Utils; + +/** + * LDAPConfigStore: + * Extends PropConfigStore with methods to load/save from/to file for + * persistent storage. This is a configuration store agent who + * reads data from an LDAP entry. + *

+ * + * @version $Revision$, $Date$ + * @see PropConfigStore + */ +public class LDAPConfigStore extends PropConfigStore implements IConfigStore { + + private ILdapConnFactory mDbFactory; + private String mDn; + private String mAttr; + + /** + * + */ + private static final long serialVersionUID = 3642124526598175633L; + + /** + * Constructs a file configuration store. + *

+ * + * @param dbFactory Database connection factory + * @param dn Distinguished name of record containing config store + * @param attr Name of attribute containing config store + * + * @exception EBaseException failed to create file configuration + */ + public LDAPConfigStore(ILdapConnFactory dbFactory, String dn, String attr) + throws EBaseException { + super(null); // top-level store without a name + + mDbFactory = dbFactory; + mDn = dn; + mAttr = attr; + + LDAPConnection conn = mDbFactory.getConn(); + + String[] attrs = {mAttr}; + LDAPEntry ldapEntry = conn.read(mDn, attrs); + InputStream data = new ByteArrayInputStream( (byte[]) + ldapEntry.getAttribute(mAttr).getByteValues().nextElement()); + load(data); + + conn.disconnect(); + } + + /** + * The original config file is copied to + * .. + * Commits the current properties to the configuration file. + *

+ * + * @param backup + */ + public void commit(boolean createBackup) throws EBaseException { + if (createBackup) { + // TODO back it up, perhaps to a user-specified attribute, + // or `mAttr + "_backup"`. + } + + LDAPConnection conn = mDbFactory.getConn(); + + ByteArrayOutputStream data = new ByteArrayOutputStream(); + save(data, null); + + LDAPAttribute ldapAttr = new LDAPAttribute(mAttr, data.toByteArray()); + LDAPModification ldapMod = + new LDAPModification(LDAPModification.REPLACE, ldapAttr); + conn.modify(mDn, ldapMod); + + conn.disconnect(); + } +} -- 1.9.3 From mharmsen at redhat.com Wed Jul 16 02:37:22 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Tue, 15 Jul 2014 19:37:22 -0700 Subject: [Pki-devel] [PATCH] Removed 'java-atk-wrapper' dependency from 'pki-server' Message-ID: <53C5E562.2020302@redhat.com> After conversations with other members of the Dogtag team, it was determined that 'java-atk-wrapper' may have once been needed, but was probably no longer required. Per their suggestion, I removed the dependency, and successfully built the following scratch builds on Koji: Fedora 20: *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147156 *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147161 (x86_64) Fedora 21: *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147431 *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147435 (x86_64) Additionally, as this was a runtime requirement, after removing the 'java-atk-wrapper' package, I successfully downloaded, installed, configured, and tested the Fedora 20 RPMS. Please review the attached patch. -- Matt -------------- next part -------------- A non-text attachment was scrubbed... Name: 20140715-Removed-java-atk-wrapper-dependency.patch Type: text/x-patch Size: 1689 bytes Desc: not available URL: From edewata at redhat.com Wed Jul 16 15:06:57 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 16 Jul 2014 10:06:57 -0500 Subject: [Pki-devel] [PATCH] 226 - allow databse to be installed as subtree of existing ldap tree In-Reply-To: <1281079198.7660642.1405324101924.JavaMail.zimbra@redhat.com> References: <1281079198.7660642.1405324101924.JavaMail.zimbra@redhat.com> Message-ID: <53C69511.5030808@redhat.com> On 7/14/2014 2:48 AM, Ade Lee wrote: > Add ability to create database as subtree of existing tree > > This patch adds the ability to create a subsystem that uses > an existing subtree to create the internal basedn. This is useful > for instance, for IPA which will use the original o=ipaca as the > top level DN for a KRA, which will be situated at o=ipadrm, o=ipaca. > > The patch also allows such a system to be cloned, but not to setup the > replication agreements, on the assumption that the data is already being > replicated at the top-level DN or some higher level. > > The patch also contains some minor cleanups - removing unused imports and > removal of an invalid reference in the python code. > > Ticket 1051 > > Note: Changes to the man pages will be submitted in a separate follow-on patch. > > Please review, > Ade Some minor stuff: 1. The extra parentheses in the following line are redundant: return (isClone!= null) && (isClone.equalsIgnoreCase("true")); 2. The "DN" in the following method names can be removed because it's redundant/confusing. The methods return LDAPEntry, not a DN. * getMappingDNEntry() -> getMappingEntry() * getBaseDNEntry() -> getBaseEntry() 3. Similarly, the following methods also create LDAP entries so they can also be renamed for clarity. * createBaseDN() -> createBaseEntry() * createDatabase() -> createDatabaseEntry() * createDatabaseMapping() -> createDatabaseMappingEntry() 4. The method getIsClone()/setIsClone() can be renamed to isClone()/setClone() to be more consistent with Java convention. It should not conflict with the attribute isClone. Regardless, it works when tested with IPA with the DRM patches. ACK. -- Endi S. Dewata From edewata at redhat.com Wed Jul 16 15:12:01 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 16 Jul 2014 10:12:01 -0500 Subject: [Pki-devel] [Freeipa-devel] [PATCH] - Add DRM to IPA In-Reply-To: <1922084174.7689340.1405331152213.JavaMail.zimbra@redhat.com> References: <1922084174.7689340.1405331152213.JavaMail.zimbra@redhat.com> Message-ID: <53C69641.2010004@redhat.com> On 7/14/2014 4:45 AM, Ade Lee wrote: > Hi all, > > I have rebased all the previous patches against master, and have squashed them all into a single patch. > Its a large patch, but as many folks have already reviewed the constituent precursor patches, most if it > should be familiar and easier to review. > > The main difference with what was specified before is that the DRM database is installed as a subtree > to o=ipaca. This means that no new replication agreements will be needed to replicate DRM data. > Replication agreements set up for the Dogtag CA will automatically replicate DRM data. > > In order for this patch to work, a new 10.2 build of Dogtag 10.2 is needed - with specific changes to > allow the ability to install a database as a subtree of an existing tree. At this time, these > changes have not yet been checked into the dogtag source. You can obtain such a build from: > > http://copr.fedoraproject.org/coprs/vakwetu/dogtag/build/21936/ > > Please review, > > Thanks, > Ade Some comments/questions: 1. The suffix for the DRM is o=ipadrm,o=ipaca. It's probably better to change it to ou=drm,o=ipaca since another "ipa" under o=ipaca would be redundant. In the future we might want to migrate the current CA entries into ou=ca,o=ipaca subtree so that ou=ca and ou=drm will be at the same level, and keep o=ipaca as the parent tree for Dogtag subsystems. Alternatively, we probably could merge o=ipaca and o=ipadrm since the structure of each tree seems to have been designed to share the user and groups, but still maintain separate structure for CA/KRA-specific storage. The current Dogtag probably doesn't support this, but it's a possibility with additional works. 2. If a clone doesn't have DRM installed but it's getting replicated DRM data, is there any concern? 3. The Dogtag dependency should be updated to 10.2. Also the dogtag_version and DOGTAG_VERSION variables are probably not granular enough to detect the minor version. This message should be updated too: Dogtag must be version 10.1 or above to install DRM 4. It's probably unnecessary to override the following methods in CAInstance since they only call the base methods. * enable() * start_instance() * stop_instance() * restart_instance() * http_proxy() 5. The following code in ipaserver/plugins/dogtag.py will no longer work due to a recent change in Dogtag: transport_cert = kraclient.system_certs.get_transport_cert() tcert = transport_cert[ len(pki.CERT_HEADER): len(transport_cert) - len(pki.CERT_FOOTER)] crypto.import_cert( self.transport_nick, base64.decodestring(tcert), "u,u,u") This is how it's used now in drmtest.py: transport_cert = kraclient.system_certs.get_transport_cert() print "Subject DN: " + transport_cert.subject_dn print transport_cert.encoded crypto.import_cert(transport_nick, transport_cert, "u,u,u") 6. The code in ipaserver/install/drminstance.py creates a file /tmp/drm.p12. How long will this file stay in the /tmp folder? Should it be moved into a more permanent location? If it's a temporary file, can we use the python tempfile module? -- Endi S. Dewata From edewata at redhat.com Wed Jul 16 19:11:41 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 16 Jul 2014 14:11:41 -0500 Subject: [Pki-devel] [PATCH] Removed 'java-atk-wrapper' dependency from 'pki-server' In-Reply-To: <53C5E562.2020302@redhat.com> References: <53C5E562.2020302@redhat.com> Message-ID: <53C6CE6D.8040602@redhat.com> On 7/15/2014 9:37 PM, Matthew Harmsen wrote: > After conversations with other members of the Dogtag team, it was > determined that 'java-atk-wrapper' may have once been needed, > but was probably no longer required. > > Per their suggestion, I removed the dependency, and successfully built > the following scratch builds on Koji: > > Fedora 20: > *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147156 > *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147161 > (x86_64) > Fedora 21: > *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147431 > *http://koji.fedoraproject.org/koji/taskinfo?taskID=7147435 > (x86_64) > > Additionally, as this was a runtime requirement, after removing the > 'java-atk-wrapper' package, I successfully downloaded, installed, > configured, and tested the Fedora 20 RPMS. > > Please review the attached patch. > > -- Matt ACK. The changes look fine and java-headless is actually available on F20. As you mentioned, there's a separate bug for the java-atk-wrapper removal: https://bugzilla.redhat.com/show_bug.cgi?id=1120045 -- Endi S. Dewata From alee at redhat.com Thu Jul 17 13:10:06 2014 From: alee at redhat.com (Ade Lee) Date: Thu, 17 Jul 2014 21:10:06 +0800 Subject: [Pki-devel] [PATCH] 226 - allow databse to be installed as subtree of existing ldap tree In-Reply-To: <53C69511.5030808@redhat.com> References: <1281079198.7660642.1405324101924.JavaMail.zimbra@redhat.com> <53C69511.5030808@redhat.com> Message-ID: <1405602606.1885.0.camel@localhost.localdomain> On Wed, 2014-07-16 at 10:06 -0500, Endi Sukma Dewata wrote: > On 7/14/2014 2:48 AM, Ade Lee wrote: > > Add ability to create database as subtree of existing tree > > > > This patch adds the ability to create a subsystem that uses > > an existing subtree to create the internal basedn. This is useful > > for instance, for IPA which will use the original o=ipaca as the > > top level DN for a KRA, which will be situated at o=ipadrm, o=ipaca. > > > > The patch also allows such a system to be cloned, but not to setup the > > replication agreements, on the assumption that the data is already being > > replicated at the top-level DN or some higher level. > > > > The patch also contains some minor cleanups - removing unused imports and > > removal of an invalid reference in the python code. > > > > Ticket 1051 > > > > Note: Changes to the man pages will be submitted in a separate follow-on patch. > > > > Please review, > > Ade > > Some minor stuff: > > 1. The extra parentheses in the following line are redundant: > > return (isClone!= null) && (isClone.equalsIgnoreCase("true")); > > 2. The "DN" in the following method names can be removed because it's > redundant/confusing. The methods return LDAPEntry, not a DN. > * getMappingDNEntry() -> getMappingEntry() > * getBaseDNEntry() -> getBaseEntry() > > 3. Similarly, the following methods also create LDAP entries so they can > also be renamed for clarity. > * createBaseDN() -> createBaseEntry() > * createDatabase() -> createDatabaseEntry() > * createDatabaseMapping() -> createDatabaseMappingEntry() > > 4. The method getIsClone()/setIsClone() can be renamed to > isClone()/setClone() to be more consistent with Java convention. It > should not conflict with the attribute isClone. > > Regardless, it works when tested with IPA with the DRM patches. ACK. > Thanks. Made suggested changes above. Pushed to master. Ade From mharmsen at redhat.com Thu Jul 17 23:46:33 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Thu, 17 Jul 2014 16:46:33 -0700 Subject: [Pki-devel] [PATCH] Display list of profiles by default Message-ID: <53C86059.9080405@redhat.com> Please review the following patch which addresses: * PKI TRAC Ticket #992 - pki cert-request-profile-find doesn't display list of profiles by default -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 20140717-Remove-profile-ID-argument.patch Type: text/x-patch Size: 1518 bytes Desc: not available URL: From akoneru at redhat.com Fri Jul 18 15:28:46 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Fri, 18 Jul 2014 11:28:46 -0400 Subject: [Pki-devel] [PATCH] Display list of profiles by default In-Reply-To: <53C86059.9080405@redhat.com> References: <53C86059.9080405@redhat.com> Message-ID: <1405697326.25231.0.camel@akoneru.redhat.com> ACK. On Thu, 2014-07-17 at 16:46 -0700, Matthew Harmsen wrote: > Please review the following patch which addresses: > * PKI TRAC Ticket #992 - pki cert-request-profile-find doesn't > display list of profiles by default > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From cfu at redhat.com Fri Jul 18 16:18:01 2014 From: cfu at redhat.com (Christina Fu) Date: Fri, 18 Jul 2014 09:18:01 -0700 Subject: [Pki-devel] [PATCH] pki-cfu-0020-TPSRewrite-credential-match-fix.patch Message-ID: <53C948B9.2090300@redhat.com> This patch is to fix an issue that Jack reported some time ago regarding authentication in Java TPS. It was speculated that it had to do with ESC and tpsclient not expecting same credential names, etc. (note: I understand that Jack worked out something that was to get him through what he was working on, but it was not meant as a permanent solution, hence my investigation and separate patch) I spent some time investigating to find out the root cause and a final solution. Here is what I discovered: In the old TPS, in case of "ExtendedLoginRequest", the uid/pwd are called "UID" and "PASSWORD"; where as in case of "LoginRequest", the uid/pwd are called "screen_name" and "password" In tpsclient, BOTH "ExtendedLoginResponse" and "LoginResponse" are coded to put "screen_name". My suspicion is with the old TPS the "ExtendedLoginRequest" never worked. So, in summary, it's the difference between "extended" and "not extended" rather than the clients. The clients just happened to specify extended/not extended by default so it appears that way. Note: In this patch, I added changes to tpsclient in an attempt to make extendedLogin work but I learned later from jack that tpsclient never does that, so please ignore that part of the patch. I made some attempt to restore that part of tpsclient with a new patch but couldn't get it. I will sort it out with our git experts later. This works with tpsclient, but I am still waiting for jack to test it out with ESC. thanks, Christina From cfu at redhat.com Fri Jul 18 16:22:55 2014 From: cfu at redhat.com (Christina Fu) Date: Fri, 18 Jul 2014 09:22:55 -0700 Subject: [Pki-devel] [PATCH] pki-cfu-0020-TPSRewrite-credential-match-fix.patch In-Reply-To: <53C948B9.2090300@redhat.com> References: <53C948B9.2090300@redhat.com> Message-ID: <53C949DF.3040409@redhat.com> and here is the patch. On 07/18/2014 09:18 AM, Christina Fu wrote: > This patch is to fix an issue that Jack reported some time ago > regarding authentication in Java TPS. > It was speculated that it had to do with ESC and tpsclient not > expecting same credential names, etc. > (note: I understand that Jack worked out something that was to get him > through what he was working on, but it was not meant as a permanent > solution, hence my investigation and separate patch) > > I spent some time investigating to find out the root cause and a final > solution. > > Here is what I discovered: > In the old TPS, in case of "ExtendedLoginRequest", the uid/pwd are > called "UID" and "PASSWORD"; > where as in case of "LoginRequest", the uid/pwd are called > "screen_name" and "password" > > In tpsclient, BOTH "ExtendedLoginResponse" and "LoginResponse" are > coded to put "screen_name". My suspicion is with the old TPS the > "ExtendedLoginRequest" never worked. > > So, in summary, it's the difference between "extended" and "not > extended" rather than the clients. The clients just happened to > specify extended/not extended by default so it appears that way. > > Note: In this patch, I added changes to tpsclient in an attempt to > make extendedLogin work but I learned later from jack that tpsclient > never does that, so please ignore that part of the patch. I made some > attempt to restore that part of tpsclient with a new patch but > couldn't get it. I will sort it out with our git experts later. > > This works with tpsclient, but I am still waiting for jack to test it > out with ESC. > > thanks, > Christina > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-cfu-0020-TPSRewrite-credential-match-fix.patch Type: text/x-patch Size: 6081 bytes Desc: not available URL: From jmagne at redhat.com Fri Jul 18 18:26:59 2014 From: jmagne at redhat.com (John Magne) Date: Fri, 18 Jul 2014 14:26:59 -0400 (EDT) Subject: [Pki-devel] [pki-devel][PATCH] 0015-First-cut-of-enrollment-feature.patch In-Reply-To: <759961650.14172280.1405707869965.JavaMail.zimbra@redhat.com> Message-ID: <203323816.14176175.1405708019746.JavaMail.zimbra@redhat.com> First cut of enrollment feature. The following features implemented for enrollment. 1. Standard enrollment of a list of RSA certificates. 2. Certificates are only done with token side keygen. 3. Minimual enrollment based pin reset functionality implemented to create a pin for the enrolled token. 4. Much work done to the PKCS11 object code, which allows us to write the compressed object blob to the token, allowing coolkey to access it and use the certs and keys on the token. 5. Tested with Bob Relyea's "smartcard" utility to prove that signing and encryption operations worked as expected. 6. Some work done to get authentication working with esc. 7. Created of stub of standalone Pin Reset Processor. Now it returns an error from esc but the pin reset command is accepted. To Do. 1. We need to support server side keygen. 2. Symmetric Key Changeover in another ticket. 3. Finish up the stand alone Pin Reset Processor in another ticket. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0015-First-cut-of-enrollment-feature.patch Type: text/x-patch Size: 78125 bytes Desc: not available URL: From rcritten at redhat.com Fri Jul 18 20:32:05 2014 From: rcritten at redhat.com (Rob Crittenden) Date: Fri, 18 Jul 2014 16:32:05 -0400 Subject: [Pki-devel] [Freeipa-devel] [PATCH] - Add DRM to IPA In-Reply-To: <1922084174.7689340.1405331152213.JavaMail.zimbra@redhat.com> References: <1922084174.7689340.1405331152213.JavaMail.zimbra@redhat.com> Message-ID: <53C98445.9000900@redhat.com> Ade Lee wrote: > Hi all, > > I have rebased all the previous patches against master, and have squashed them all into a single patch. > Its a large patch, but as many folks have already reviewed the constituent precursor patches, most if it > should be familiar and easier to review. > > The main difference with what was specified before is that the DRM database is installed as a subtree > to o=ipaca. This means that no new replication agreements will be needed to replicate DRM data. > Replication agreements set up for the Dogtag CA will automatically replicate DRM data. > > In order for this patch to work, a new 10.2 build of Dogtag 10.2 is needed - with specific changes to > allow the ability to install a database as a subtree of an existing tree. At this time, these > changes have not yet been checked into the dogtag source. You can obtain such a build from: > > http://copr.fedoraproject.org/coprs/vakwetu/dogtag/build/21936/ > > Please review, The BuildRequires needs to be updated to avoid a bunch of lint errors: ./make-lint ************* Module ipaserver.plugins.dogtag ipaserver/plugins/dogtag.py:249: [E0611(no-name-in-module), ] No name 'crypto' in module 'pki') ipaserver/plugins/dogtag.py:250: [E0611(no-name-in-module), ] No name 'key' in module 'pki') ipaserver/plugins/dogtag.py:251: [E0611(no-name-in-module), ] No name 'kra' in module 'pki') ipaserver/plugins/dogtag.py:1952: [E1101(no-member), drm._setup] Instance of 'PKIConnection' has no 'set_authentication_cert' member) ipaserver/plugins/dogtag.py:1963: [E1101(no-member), drm._setup] Module 'pki' has no 'CERT_HEADER' member) ipaserver/plugins/dogtag.py:1964: [E1101(no-member), drm._setup] Module 'pki' has no 'CERT_FOOTER' member) ************* Module ipaserver.install.dogtaginstance ipaserver/install/dogtaginstance.py:71: [E1101(no-member), get_security_domain] Instance of 'SecurityDomainClient' has no 'get_security_domain_info' member) I forget what back and forth we had on DRM vs no DRM by default, but is it right to have no option at all to add one at install time, ala DNS? My first install failed: Jul 18 15:38:36 ipa.example.com pkidaemon[16516]: ln: failed to create symbolic link ?/var/lib/pki/pki-tomcat/conf/ca/CS.cfg.bak?: Permission denied Jul 18 15:38:36 ipa.example.com pkidaemon[16516]: SUCCESS: Successfully archived '/var/lib/pki/pki-tomcat/conf/ca/archives/CS.cfg.bak.20140718153836' Jul 18 15:38:36 ipa.example.com pkidaemon[16516]: WARNING: Failed to backup '/var/lib/pki/pki-tomcat/conf/ca/CS.cfg' to '/var/lib/pki/pki-tomcat/conf/ca/CS.cfg.bak'! Jul 18 15:38:36 ipa.example.com pkidaemon[16516]: /usr/share/pki/scripts/operations: line 1579: 0: command not found Jul 18 15:38:36 ipa.example.com systemd[1]: pki-tomcatd at pki-tomcat.service: control process exited, code=exited status=1 Jul 18 15:38:36 ipa.example.com systemd[1]: Failed to start PKI Tomcat Server pki-tomcat. This is due to SELinux issues: type=AVC msg=audit(1405712316.049:1656): avc: denied { setfscreate } for pid=16702 comm="cp" scontext=system_u:system_r:pki_tomcat_t:s0 tcontext=system_u:system_r:pki_tomcat_t:s0 tclass=process type=AVC msg=audit(1405712316.049:1657): avc: denied { relabelfrom } for pid=16702 comm="cp" name="CS.cfg.bak.20140718153836" dev="dm-0" ino=431097 scontext=system_u:system_r:pki_tomcat_t:s0 tcontext=system_u:object_r:pki_tomcat_etc_rw_t:s0 tclass=file type=AVC msg=audit(1405712316.050:1658): avc: denied { create } for pid=16703 comm="ln" name="CS.cfg.bak" scontext=system_u:system_r:pki_tomcat_t:s0 tcontext=system_u:object_r:pki_tomcat_etc_rw_t:s0 tclass=lnk_file I put it into permissive and continued. The installer still references backing up /root/drmcert.p12 but it isn't created by default. The estimate for configuring the DRM seems off. On my VM it took 126 seconds, not 210. Mileage may vary but since my box was the source for all the other timings :-) On the plus side the DRM seems to work. I used the ca-agent cert and was able to follow the steps at https://access.redhat.com/documentation/en-US/Red_Hat_Certificate_System/8.1/html/Admin_Guide/Testing_the_Key_Archival_and_Recovery_Setup.html to issue and recover a key. rob From cfu at redhat.com Sat Jul 19 01:16:20 2014 From: cfu at redhat.com (Christina Fu) Date: Fri, 18 Jul 2014 18:16:20 -0700 Subject: [Pki-devel] [PATCH] pki-cfu-0020-TPSRewrite-credential-match-fix.patch In-Reply-To: <53C949DF.3040409@redhat.com> References: <53C948B9.2090300@redhat.com> <53C949DF.3040409@redhat.com> Message-ID: <53C9C6E4.8050102@redhat.com> Thanks to jack for helping me test. After much consideration, I think I will add my changes on top of his last patch instead to make things speedier on both sides. So, please don't review this patch... thanks, Christina On 07/18/2014 09:22 AM, Christina Fu wrote: > and here is the patch. > > On 07/18/2014 09:18 AM, Christina Fu wrote: >> This patch is to fix an issue that Jack reported some time ago >> regarding authentication in Java TPS. >> It was speculated that it had to do with ESC and tpsclient not >> expecting same credential names, etc. >> (note: I understand that Jack worked out something that was to get >> him through what he was working on, but it was not meant as a >> permanent solution, hence my investigation and separate patch) >> >> I spent some time investigating to find out the root cause and a >> final solution. >> >> Here is what I discovered: >> In the old TPS, in case of "ExtendedLoginRequest", the uid/pwd are >> called "UID" and "PASSWORD"; >> where as in case of "LoginRequest", the uid/pwd are called >> "screen_name" and "password" >> >> In tpsclient, BOTH "ExtendedLoginResponse" and "LoginResponse" are >> coded to put "screen_name". My suspicion is with the old TPS the >> "ExtendedLoginRequest" never worked. >> >> So, in summary, it's the difference between "extended" and "not >> extended" rather than the clients. The clients just happened to >> specify extended/not extended by default so it appears that way. >> >> Note: In this patch, I added changes to tpsclient in an attempt to >> make extendedLogin work but I learned later from jack that tpsclient >> never does that, so please ignore that part of the patch. I made >> some attempt to restore that part of tpsclient with a new patch but >> couldn't get it. I will sort it out with our git experts later. >> >> This works with tpsclient, but I am still waiting for jack to test it >> out with ESC. >> >> thanks, >> Christina >> >> _______________________________________________ >> Pki-devel mailing list >> Pki-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/pki-devel > > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ftweedal at redhat.com Mon Jul 21 02:13:47 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Mon, 21 Jul 2014 12:13:47 +1000 Subject: [Pki-devel] ProfileSubsystem configuration with LDAPConnection Message-ID: <20140721021347.GT2417@dhcp-40-8.bne.redhat.com> With the introduction of LDAP-based profiles, the ProfileSubsystem needs access to the profile configuration. When spawning a new instance, the CMS tries to start its subsystems, but the ProfileSubsystem cannot start because it requires the LDAP connection details, which are not yet configured (this action is performed by SystemConfigService.configureDatabase). OK, so what to do? I see two options: 1) Remove the profile subsystem from the initial CS.cfg, so that it doesn't start up. Add it back into CS.cfg as a configuration step; on the next startup, it will run and be happy, because the database configuration is there. 2) Handle the absense of database configuration in ProfileSubsystem itself. That is, keep track of whether initialisation has been successfully performed, and try again "just-in-time" when it is needed. This probably violates semantics of the IProfileSubsystem API. Feedback or other ideas are appreciated. I'm going to push ahead with option (1). Both options feel like hacks but (2) seems like a worse hack ^_^ Cheers, Fraser From edewata at redhat.com Mon Jul 21 14:12:17 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Mon, 21 Jul 2014 09:12:17 -0500 Subject: [Pki-devel] ProfileSubsystem configuration with LDAPConnection In-Reply-To: <20140721021347.GT2417@dhcp-40-8.bne.redhat.com> References: <20140721021347.GT2417@dhcp-40-8.bne.redhat.com> Message-ID: <53CD1FC1.8040208@redhat.com> On 7/20/2014 9:13 PM, Fraser Tweedale wrote: > With the introduction of LDAP-based profiles, the ProfileSubsystem > needs access to the profile configuration. When spawning a new > instance, the CMS tries to start its subsystems, but the > ProfileSubsystem cannot start because it requires the LDAP > connection details, which are not yet configured (this action is > performed by SystemConfigService.configureDatabase). > > OK, so what to do? I see two options: > > 1) Remove the profile subsystem from the initial CS.cfg, so that it > doesn't start up. Add it back into CS.cfg as a configuration step; > on the next startup, it will run and be happy, because the database > configuration is there. > > 2) Handle the absense of database configuration in ProfileSubsystem > itself. That is, keep track of whether initialisation has been > successfully performed, and try again "just-in-time" when it is > needed. This probably violates semantics of the IProfileSubsystem > API. > > Feedback or other ideas are appreciated. I'm going to push ahead > with option (1). Both options feel like hacks but (2) seems like a > worse hack ^_^ Agree with option 1, but instead of removing the profile subsystem from the list we probably can also add something like an 'enabled' parameter. The profile subsystem should be disabled by default, so it will be skipped during initialization. After the configuration the profile can be enabled and supplied with the proper database configuration. -- Endi S. Dewata From ftweedal at redhat.com Tue Jul 22 04:52:55 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Tue, 22 Jul 2014 14:52:55 +1000 Subject: [Pki-devel] ProfileSubsystem configuration with LDAPConnection In-Reply-To: <53CD1FC1.8040208@redhat.com> References: <20140721021347.GT2417@dhcp-40-8.bne.redhat.com> <53CD1FC1.8040208@redhat.com> Message-ID: <20140722045255.GU2417@dhcp-40-8.bne.redhat.com> On Mon, Jul 21, 2014 at 09:12:17AM -0500, Endi Sukma Dewata wrote: > On 7/20/2014 9:13 PM, Fraser Tweedale wrote: > >With the introduction of LDAP-based profiles, the ProfileSubsystem > >needs access to the profile configuration. When spawning a new > >instance, the CMS tries to start its subsystems, but the > >ProfileSubsystem cannot start because it requires the LDAP > >connection details, which are not yet configured (this action is > >performed by SystemConfigService.configureDatabase). > > > >OK, so what to do? I see two options: > > > >1) Remove the profile subsystem from the initial CS.cfg, so that it > >doesn't start up. Add it back into CS.cfg as a configuration step; > >on the next startup, it will run and be happy, because the database > >configuration is there. > > > >2) Handle the absense of database configuration in ProfileSubsystem > >itself. That is, keep track of whether initialisation has been > >successfully performed, and try again "just-in-time" when it is > >needed. This probably violates semantics of the IProfileSubsystem > >API. > > > >Feedback or other ideas are appreciated. I'm going to push ahead > >with option (1). Both options feel like hacks but (2) seems like a > >worse hack ^_^ > > Agree with option 1, but instead of removing the profile subsystem from the > list we probably can also add something like an 'enabled' parameter. > > The profile subsystem should be disabled by default, so it will be skipped > during initialization. After the configuration the profile can be enabled > and supplied with the proper database configuration. Thanks Endi. Having the 'enabled' parameter was a very good idea - it turned out to be much easier to implement than removing the parameter (which broke the dynamic subsystem loading code). Cheers, Fraser > -- > Endi S. Dewata From akoneru at redhat.com Tue Jul 22 16:31:40 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Tue, 22 Jul 2014 12:31:40 -0400 Subject: [Pki-devel] [PATCH] 96-2 Updated man page for pki key CLI commands (Ticket 945) In-Reply-To: <53895084.9040003@redhat.com> References: <1401480833.5379.1.camel@akoneru.redhat.com> <53895084.9040003@redhat.com> Message-ID: <1406046700.26559.3.camel@akoneru.redhat.com> Please review the patch with changes suggested by Matt. Added the examples section to explain the usage of agent authentication and a section to explain the details of the parameters used in the templates(asked b mrniranjan) on IRC. -- Abhishek On Fri, 2014-05-30 at 20:46 -0700, Matthew Harmsen wrote: > On 05/30/14 13:13, Abhishek Koneru wrote: > > > Please review the patch which updates the man pages for the pki key CLI > > commands. > > > > --Abhishek > > > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > Abhishek, > > What is there, is fine. However, the man page as is, is not very > useful since it presumes a great deal of knowledge! > > I would strongly urge you to provide an EXAMPLES section utilizing > sample agent authentication. > > For example, at the very least, please provide the most basic scenario > of showing exactly what one would specify in a default installation of > a CA and KRA to simply perform a "key-find" and a "key-show" using > client certification. > > I would also suggest that you add your name to the list of Authors of > this man page. > > -- Matt -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-akoneru-96-2-Updated-man-page-for-pki-key-commands.patch Type: text/x-patch Size: 16524 bytes Desc: not available URL: From akoneru at redhat.com Tue Jul 22 16:25:24 2014 From: akoneru at redhat.com (Abhishek Koneru) Date: Tue, 22 Jul 2014 12:25:24 -0400 Subject: [Pki-devel] [PATCH] 96-2 Updated man page for pki key CLI commands (Ticket 945) In-Reply-To: <53895084.9040003@redhat.com> References: <1401480833.5379.1.camel@akoneru.redhat.com> <53895084.9040003@redhat.com> Message-ID: <1406046324.26559.2.camel@akoneru.redhat.com> Please review the patch with changes suggested by Matt. Added the examples section to explain the usage of agent authentication and a section to explain the details of the parameters used in the templates(asked b mrniranjan) on IRC. -- Abhishek On Fri, 2014-05-30 at 20:46 -0700, Matthew Harmsen wrote: > On 05/30/14 13:13, Abhishek Koneru wrote: > > > Please review the patch which updates the man pages for the pki key CLI > > commands. > > > > --Abhishek > > > > > > _______________________________________________ > > Pki-devel mailing list > > Pki-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/pki-devel > Abhishek, > > What is there, is fine. However, the man page as is, is not very > useful since it presumes a great deal of knowledge! > > I would strongly urge you to provide an EXAMPLES section utilizing > sample agent authentication. > > For example, at the very least, please provide the most basic scenario > of showing exactly what one would specify in a default installation of > a CA and KRA to simply perform a "key-find" and a "key-show" using > client certification. > > I would also suggest that you add your name to the list of Authors of > this man page. > > -- Matt -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-akoneru-96-2-Updated-man-page-for-pki-key-commands.patch Type: text/x-patch Size: 16524 bytes Desc: not available URL: From cfu at redhat.com Wed Jul 23 00:28:54 2014 From: cfu at redhat.com (Christina Fu) Date: Tue, 22 Jul 2014 17:28:54 -0700 Subject: [Pki-devel] [pki-devel][PATCH] 0015-First-cut-of-enrollment-feature.patch In-Reply-To: <203323816.14176175.1405708019746.JavaMail.zimbra@redhat.com> References: <203323816.14176175.1405708019746.JavaMail.zimbra@redhat.com> Message-ID: <53CF01C6.5040905@redhat.com> This is the bulk of the work that writes objects to the token with a lot of goodies. The following are my my review comments for 0015-First-cut-of-end-to-end-enrollment-feature.patch * CreatePinAPDU.java - It appears that the old TPS has SetP1(p1). Is there a reason why you removed that here? * SecureChannel.java -appendKeyCapabilities: the getting keyCapabilities part from the config is very repetitive. You might want to write a convenience routine to iterate through all capabilities. - for most perms, you did: + TPSBuffer perms = new TPSBuffer(); + + perms.add((byte) 0xff); + perms.add((byte) 0xff); + perms.add((byte) 0x40); + perms.add((byte) 0x00); + perms.add((byte) 0x40); + perms.add((byte) 0x00); but for "createCertificate" you did: + byte[] perms = { (byte) 0xff, (byte) 0xff, 0x40, 0x00, 0x40, 0x00 }; + + TPSBuffer permissions = new TPSBuffer(perms); both achieve the same thing, but you probably want to be consistent. The 2nd way seems cleaner - writeObject() throws TPSStatus.STATUS_ERROR_MAC_ENROLL_PDU in case of response checkResult failure. Is it always the same error at each call? How about createObject()? * PKCS11Obj - addObjectSpec() - I think it'd be more useful if you put objectId in the the debug message when one duplicated object is removed. - getRawHeaderData() - typo "returing" *TPSProcessor - mapPattern() - some debug messages call themselves buildTokenLabel instead... - mapPattern() - I am not sure it is what you wanted. You had + if ((nextPos - firstPos) <= 1) { + return "token"; + } that will literally return "token" if the pattern turns out to be a string of non-tokens (not pattern). What you want to do is to return the exact same pattern if it contains no '$'. * makeKeyIDFromPublicKeyInfo() - Most our existing code call MessageDigest like this: MessageDigest md = MessageDigest.getInstance("SHA1"); I'm curious if it makes any difference if you specify the provider or not? I'm also not so sure "mozilla" is the appropriate name for the digesgt (I know we call our JSS provider "Mozilla-JSS", but a message digest is just a message digest) * about authentication: per our discussion, with what we have now, the CS.cfg configuration needs to be manually changed and server restarted between swapping the clients between ESC and tpsclient. I will just add my changes after your patch is checked in. Christina On 07/18/2014 11:26 AM, John Magne wrote: > First cut of enrollment feature. > > The following features implemented for enrollment. > > 1. Standard enrollment of a list of RSA certificates. > 2. Certificates are only done with token side keygen. > 3. Minimual enrollment based pin reset functionality implemented to create > a pin for the enrolled token. > 4. Much work done to the PKCS11 object code, which allows us to write the > compressed object blob to the token, allowing coolkey to access it and use > the certs and keys on the token. > 5. Tested with Bob Relyea's "smartcard" utility to prove that signing and encryption > operations worked as expected. > 6. Some work done to get authentication working with esc. > 7. Created of stub of standalone Pin Reset Processor. Now it returns an error from > esc but the pin reset command is accepted. > > To Do. > > 1. We need to support server side keygen. > 2. Symmetric Key Changeover in another ticket. > 3. Finish up the stand alone Pin Reset Processor in another ticket. > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ftweedal at redhat.com Wed Jul 23 01:27:11 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 23 Jul 2014 11:27:11 +1000 Subject: [Pki-devel] CLI for editing profiles Message-ID: <20140723012711.GW2417@dhcp-40-8.bne.redhat.com> Along with LDAP profiles, we will be adding modules to the CLI for adding and editing profiles in the ConfigStore format that was used for file-based profiles. For more info, see: http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Command-line_utilities There is an existing CLI for adding and modifying profiles, in the XML format, e.g. ``pki ca profile add caCustomProfile.xml``. The XML format carries information including the profile ID and class_id, but these data must be supplied out-of-band when dealing with the ConfigStore format. Because of this, I intend to: - add new commands to the existing profile CLI for working with the "raw" (i.e., ConfigStore) format, e.g. "edit-raw", "add-raw". Where necessary, these commands will take compulsory ``--profile-id`` and/or ``--class-id`` arguments, to account for the absense of such information in the profile ConfigStore format; and - transport this information in the XML format - not in the "raw" format - so that it will be unnecessary to make changes to ProfileClient or the ProfileService API. As usual, I welcome feedback - especially if you feel I am going the wrong way ^_^ From ftweedal at redhat.com Wed Jul 23 07:09:24 2014 From: ftweedal at redhat.com (Fraser Tweedale) Date: Wed, 23 Jul 2014 17:09:24 +1000 Subject: [Pki-devel] CLI for editing profiles In-Reply-To: <20140723012711.GW2417@dhcp-40-8.bne.redhat.com> References: <20140723012711.GW2417@dhcp-40-8.bne.redhat.com> Message-ID: <20140723070924.GX2417@dhcp-40-8.bne.redhat.com> On Wed, Jul 23, 2014 at 11:27:11AM +1000, Fraser Tweedale wrote: > Along with LDAP profiles, we will be adding modules to the CLI for > adding and editing profiles in the ConfigStore format that was used > for file-based profiles. For more info, see: > > http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Command-line_utilities > > There is an existing CLI for adding and modifying profiles, in the > XML format, e.g. ``pki ca profile add caCustomProfile.xml``. The > XML format carries information including the profile ID and > class_id, but these data must be supplied out-of-band when dealing > with the ConfigStore format. > > Because of this, I intend to: > > - add new commands to the existing profile CLI for working with the > "raw" (i.e., ConfigStore) format, e.g. "edit-raw", "add-raw". > Where necessary, these commands will take compulsory > ``--profile-id`` and/or ``--class-id`` arguments, to account for > the absense of such information in the profile ConfigStore format; > > and > > - transport this information in the XML format - not in the "raw" > format - so that it will be unnecessary to make changes to > ProfileClient or the ProfileService API. > > As usual, I welcome feedback - especially if you feel I am going the > wrong way ^_^ > An update: due to the various Profile and ProfileInput/Output/Policy classes not being distributed in the pki-tools package (or dependencies thereof), I /did/ end up adding methods to the REST API. The `pki ca profile show-raw ` command is implemented in the attached patch 0009. I expected I will complete the edit-raw and add-raw commands tomorrow, and after that will move on to testing replication and polling/monitoring for changes to profiles. Cheers, Fraser > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- >From 019e82e4987659f1f1e066dff943355a254f2179 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Mon, 7 Jul 2014 23:35:35 -0400 Subject: [PATCH] add schema for LDAP-based profiles --- base/ca/shared/conf/db.ldif | 9 +++++++++ base/ca/shared/conf/schema.ldif | 30 ++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/base/ca/shared/conf/db.ldif b/base/ca/shared/conf/db.ldif index 00fa919b7df38ed97f0bc21b5616a9998845c7d4..b0f36b6d85f70900d0b2c5e0ee92d9fb8c1cb3e8 100644 --- a/base/ca/shared/conf/db.ldif +++ b/base/ca/shared/conf/db.ldif @@ -160,4 +160,13 @@ objectClass: top objectClass: organizationalUnit ou: certificateRepository +dn: ou=certProfiles,{rootSuffix} +objectClass: top +objectClass: organizationalUnit +ou: certProfiles +dn: cn=certProfilesInfo,{rootSuffix} +objectClass: top +objectClass: certProfilesInfo +cn: certProfilesInfo +certProfilesLastModified: 197001010000Z diff --git a/base/ca/shared/conf/schema.ldif b/base/ca/shared/conf/schema.ldif index 70578e21ce4e102909a1b7b45fa84c184a997bdf..4f74869da1e46b39469dd17ddb2517e111300b43 100644 --- a/base/ca/shared/conf/schema.ldif +++ b/base/ca/shared/conf/schema.ldif @@ -487,3 +487,33 @@ dn: cn=schema changetype: modify add: objectClasses objectClasses: ( securityDomainSessionEntry-oid NAME 'securityDomainSessionEntry' DESC 'CMS defined class' SUP top STRUCTURAL MUST ( cn $ host $ uid $ cmsUserGroup $ dateOfCreate ) X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( classId-oid NAME 'classId' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( certProfileIsDefault-oid NAME 'certProfileIsDefault' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( certProfileConfig-oid NAME 'certProfileConfig' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: objectClasses +objectClasses: ( certProfile-oid NAME 'certProfile' DESC 'CMS defined class' SUP top STRUCTURAL MUST cn MAY ( classId $ certProfileIsDefault $ certProfileConfig ) X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: attributeTypes +attributeTypes: ( certProfilesLastModified-oid NAME 'certProfilesLastModified' DESC 'CMS defined attribute' SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 X-ORIGIN 'user defined' ) + +dn: cn=schema +changetype: modify +add: objectClasses +objectClasses: ( certProfilesInfo-oid NAME 'certProfilesInfo' DESC 'CMS defined class' SUP top STRUCTURAL MUST cn MAY certProfilesLastModified X-ORIGIN 'user defined' ) -- 1.9.3 -------------- next part -------------- >From 732286e87aadc1bca90232b4af8c29106cbb27eb Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Tue, 15 Jul 2014 02:48:35 -0400 Subject: [PATCH] add LDAPConfigStore class The LDAPConfigStore class is an IConfigStore that reads and writes its configuration to a given attribute and DN in an LDAP database. --- .../com/netscape/cmscore/base/LDAPConfigStore.java | 190 +++++++++++++++++++++ 1 file changed, 190 insertions(+) create mode 100644 base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java diff --git a/base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java b/base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java new file mode 100644 index 0000000000000000000000000000000000000000..8ff477bb3cdad02092c56834e043bcd6e0108625 --- /dev/null +++ b/base/server/cmscore/src/com/netscape/cmscore/base/LDAPConfigStore.java @@ -0,0 +1,190 @@ +// --- BEGIN COPYRIGHT BLOCK --- +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; version 2 of the License. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// +// (C) 2007, 2014 Red Hat, Inc. +// All rights reserved. +// --- END COPYRIGHT BLOCK --- + +package com.netscape.cmscore.base; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.InputStream; +import java.io.IOException; + +import netscape.ldap.LDAPAttribute; +import netscape.ldap.LDAPAttributeSet; +import netscape.ldap.LDAPConnection; +import netscape.ldap.LDAPEntry; +import netscape.ldap.LDAPException; +import netscape.ldap.LDAPModification; + +import com.netscape.certsrv.base.EBaseException; +import com.netscape.certsrv.base.IConfigStore; +import com.netscape.certsrv.ldap.ILdapConnFactory; + +/** + * LDAPConfigStore: + * Extends PropConfigStore with methods to load/save from/to file for + * persistent storage. This is a configuration store agent who + * reads data from an LDAP entry. + *

+ * + * @version $Revision$, $Date$ + * @see PropConfigStore + */ +public class LDAPConfigStore extends PropConfigStore implements IConfigStore { + + private ILdapConnFactory mDbFactory; + private String mDn; + private String mAttr; + private LDAPAttribute[] mAttrs; + private boolean mInDatabase; + + /** + * + */ + private static final long serialVersionUID = 3642124526598175633L; + + /** + * Constructs a file configuration store. + *

+ * + * @param dbFactory Database connection factory + * @param cn Common name of record containing config store + * @param dn Distinguished name of record containing config store + * @param attr Name of attribute containing config store + * @param attrs Set of initial attributes if creating the entry. Should + * contain cn, objectclass and possibly other attributes. + * + * @exception EBaseException failed to create file configuration + */ + public LDAPConfigStore( + ILdapConnFactory dbFactory, + String dn, LDAPAttribute[] attrs, String attr + ) throws EBaseException { + super(null); // top-level store without a name + + mDbFactory = dbFactory; + mDn = dn; + mAttrs = attrs; + mAttr = attr; + + LDAPConnection conn = mDbFactory.getConn(); + + String[] readAttrs = {mAttr}; + try { + LDAPEntry ldapEntry = conn.read(mDn, readAttrs); + + InputStream data = new ByteArrayInputStream( (byte[]) + ldapEntry.getAttribute(mAttr).getByteValues().nextElement()); + load(data); + } + catch (LDAPException e) { + // if there is no such object, we will create it on commit() + if (e.getLDAPResultCode() != LDAPException.NO_SUCH_OBJECT) { + throw new EBaseException( + "Error reading LDAPConfigStore '" + + mDn + "': " + e.toString() + ); + } + } + catch (IOException e) { + throw new EBaseException( + "Error reading LDAPConfigStore '" + + mDn + "': " + e.toString() + ); + } + finally { + mDbFactory.returnConn(conn); + } + } + + /** + * Commit the configuration to the database. + * + * All uses of LDAPProfileStore at time of writing call with + * backup=false, so the argument is ignored. + * + * If backup becomes necessary, the constructor should be + * modified to take a String backupAttr, and the existing + * content be copied to that attribute. + * + * @param backup Ignored. + */ + public void commit(boolean createBackup) throws EBaseException { + ByteArrayOutputStream data = new ByteArrayOutputStream(); + save(data, null); + + LDAPAttribute configAttr = new LDAPAttribute(mAttr, data.toByteArray()); + + LDAPConnection conn = mDbFactory.getConn(); + + // first attempt to modify; if modification fails (due + // to no such object), try and add the entry instead. + try { + try { + commitModify(conn, configAttr); + } + catch (LDAPException e) { + if (e.getLDAPResultCode() == LDAPException.NO_SUCH_OBJECT) { + commitAdd(conn, configAttr); + } + else { + throw e; + } + } + } + catch (LDAPException e) { + throw new EBaseException( + "Error writing LDAPConfigStore '" + + mDn + "': " + e.toString() + ); + } + finally { + mDbFactory.returnConn(conn); + } + } + + /** + * Update the record via an LDAPModification. + * + * @param conn LDAP connection. + * @param configAttr Config store attribute. + * @return true on success, false if the entry does not exist. + */ + private void commitModify(LDAPConnection conn, LDAPAttribute configAttr) + throws LDAPException + { + LDAPModification ldapMod = + new LDAPModification(LDAPModification.REPLACE, configAttr); + conn.modify(mDn, ldapMod); + } + + /** + * Add the LDAPEntry via LDAPConnection.add. + * + * @param conn LDAP connection. + * @param configAttr Config store attribute. + * @return true on success, false if the entry already exists. + */ + private void commitAdd(LDAPConnection conn, LDAPAttribute configAttr) + throws LDAPException + { + LDAPAttributeSet attrSet = new LDAPAttributeSet(mAttrs); + attrSet.add(configAttr); + LDAPEntry ldapEntry = new LDAPEntry(mDn, attrSet); + conn.add(ldapEntry); + } +} -- 1.9.3 -------------- next part -------------- >From 392f22cd77497434e6db9af73399b4593124fdb1 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Thu, 17 Jul 2014 00:24:06 -0400 Subject: [PATCH] change ProfileSubsystem to use LDAP database --- .../dogtagpki/server/ca/rest/ProfileService.java | 14 +- base/server/cmsbundle/src/UserMessages.properties | 2 + .../com/netscape/cmscore/base/FileConfigStore.java | 4 +- .../netscape/cmscore/profile/ProfileSubsystem.java | 175 ++++++++++++--------- 4 files changed, 108 insertions(+), 87 deletions(-) diff --git a/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java b/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java index 3b2f8a50ebcd18fe0098b2e92e0300645b904fa3..cf0d4db7f8bed44baf6d2abbf70a61406ddb4b7c 100644 --- a/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java +++ b/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java @@ -18,7 +18,6 @@ package org.dogtagpki.server.ca.rest; -import java.io.File; import java.io.IOException; import java.net.URI; import java.security.Principal; @@ -244,7 +243,7 @@ public class ProfileService extends PKIService implements ProfileResource { data.setAuthenticatorId(profile.getAuthenticatorId()); data.setAuthzAcl(profile.getAuthzAcl()); - data.setClassId(cs.getString(profileId + ".class_id")); + data.setClassId(ps.getProfileClassId(profileId)); data.setDescription(profile.getDescription(getLocale(headers))); data.setEnabled(ps.isProfileEnable(profileId)); data.setEnabledBy(ps.getProfileEnableBy(profileId)); @@ -472,18 +471,15 @@ public class ProfileService extends PKIService implements ProfileResource { auditParams.put("description", data.getDescription()); auditParams.put("visible", Boolean.toString(data.isVisible())); - String config = CMS.getConfigStore().getString("instanceRoot") + "/ca/profiles/ca/" + - profileId + ".cfg"; - File configFile = new File(config); - configFile.createNewFile(); IPluginInfo info = registry.getPluginInfo("profile", data.getClassId()); - profile = ps.createProfile(profileId, data.getClassId(), info.getClassName(), config); + String dn = "cn=" + profileId + ",ou=certProfiles," + + CMS.getConfigStore().getString("internaldb.basedn"); + profile = ps.createProfile(profileId, data.getClassId(), info.getClassName(), dn); profile.setName(getLocale(headers), data.getName()); profile.setDescription(getLocale(headers), data.getDescription()); profile.setVisible(data.isVisible()); profile.getConfigStore().commit(false); - ps.createProfileConfig(profileId, data.getClassId(), config); if (profile instanceof IProfileEx) { // populates profile specific plugins such as @@ -504,7 +500,7 @@ public class ProfileService extends PKIService implements ProfileResource { return createCreatedResponse(profileData, profileData.getLink().getHref()); - } catch (EBaseException | IOException e) { + } catch (EBaseException e) { CMS.debug("createProfile: error in creating profile: " + e); e.printStackTrace(); diff --git a/base/server/cmsbundle/src/UserMessages.properties b/base/server/cmsbundle/src/UserMessages.properties index fe43094e6b2a0531502570bc626da557fc9061ae..cd7fa18bfb1e17bc4ab4aa9e0dac06f815861291 100644 --- a/base/server/cmsbundle/src/UserMessages.properties +++ b/base/server/cmsbundle/src/UserMessages.properties @@ -754,6 +754,8 @@ CMS_PROFILE_CONFIG_KEY_USAGE_EXTENSION_CHECKING=Allow duplicate subject names wi CMS_PROFILE_INTERNAL_ERROR=Profile internal error: {0} CMS_PROFILE_DENY_OPERATION=Not authorized to do this operation. CMS_PROFILE_DELETE_ENABLEPROFILE=Cannot delete enabled profile: {0} +CMS_PROFILE_DELETE_UNKNOWNPROFILE=Cannot delete unknown profile: {0} +CMS_PROFILE_DELETE_DATABASEERROR=Failed to delete profile: {0} CMS_PROFILE_INVALID_REQUEST=Invalid Request CMS_PROFILE_EMPTY_REQUEST_TYPE=Request type is not specified. Check your profile input. CMS_PROFILE_CREATE_POLICY_FAILED=Failed to create profile policy: {0} diff --git a/base/server/cmscore/src/com/netscape/cmscore/base/FileConfigStore.java b/base/server/cmscore/src/com/netscape/cmscore/base/FileConfigStore.java index b77f86d781995e27bb0fe16135fc45a7d6fc4da3..4f8cb2743fdecc354338042a5219a9aaf6e27880 100644 --- a/base/server/cmscore/src/com/netscape/cmscore/base/FileConfigStore.java +++ b/base/server/cmscore/src/com/netscape/cmscore/base/FileConfigStore.java @@ -33,12 +33,10 @@ import com.netscape.cmsutil.util.Utils; /** * FileConfigStore: - * Extends HashConfigStore with methods to load/save from/to file for + * Extends PropConfigStore with methods to load/save from/to file for * persistent storage. This is a configuration store agent who * reads data from a file. *

- * Note that a LdapConfigStore can be implemented so that it reads the configuration stores from the Ldap directory. - *

* * @version $Revision$, $Date$ * @see PropConfigStore diff --git a/base/server/cmscore/src/com/netscape/cmscore/profile/ProfileSubsystem.java b/base/server/cmscore/src/com/netscape/cmscore/profile/ProfileSubsystem.java index 27e72352ef22c742b5ea09a180d440d58452dd49..aaa3b29b46ae4579a59cee9f5f7bab750a40a057 100644 --- a/base/server/cmscore/src/com/netscape/cmscore/profile/ProfileSubsystem.java +++ b/base/server/cmscore/src/com/netscape/cmscore/profile/ProfileSubsystem.java @@ -17,26 +17,30 @@ // --- END COPYRIGHT BLOCK --- package com.netscape.cmscore.profile; -import java.io.File; import java.util.Enumeration; import java.util.Hashtable; -import java.util.StringTokenizer; import java.util.Vector; +import netscape.ldap.LDAPAttribute; +import netscape.ldap.LDAPConnection; +import netscape.ldap.LDAPEntry; +import netscape.ldap.LDAPException; +import netscape.ldap.LDAPSearchResults; + import com.netscape.certsrv.apps.CMS; import com.netscape.certsrv.base.EBaseException; import com.netscape.certsrv.base.IConfigStore; import com.netscape.certsrv.base.ISubsystem; +import com.netscape.certsrv.ldap.ELdapException; +import com.netscape.certsrv.ldap.ILdapConnFactory; import com.netscape.certsrv.profile.EProfileException; import com.netscape.certsrv.profile.IProfile; import com.netscape.certsrv.profile.IProfileSubsystem; import com.netscape.certsrv.registry.IPluginInfo; import com.netscape.certsrv.registry.IPluginRegistry; +import com.netscape.cmscore.base.LDAPConfigStore; public class ProfileSubsystem implements IProfileSubsystem { - private static final String PROP_LIST = "list"; - private static final String PROP_CLASS_ID = "class_id"; - private static final String PROP_CONFIG = "config"; private static final String PROP_CHECK_OWNER = "checkOwner"; private static final String PROP_ENABLE = "enable"; @@ -45,9 +49,12 @@ public class ProfileSubsystem implements IProfileSubsystem { private IConfigStore mConfig = null; @SuppressWarnings("unused") private ISubsystem mOwner; - private Vector mProfileIds = new Vector(); - private Hashtable mProfiles = new Hashtable(); - private Hashtable mProfileClassIds = new Hashtable(); + private Vector mProfileIds; + private Hashtable mProfiles; + private Hashtable mProfileClassIds; + private Hashtable mProfileDNs; + + private ILdapConnFactory dbFactory; /** * Retrieves the name of this subsystem. @@ -74,9 +81,21 @@ public class ProfileSubsystem implements IProfileSubsystem { public void init(ISubsystem owner, IConfigStore config) throws EBaseException { CMS.debug("ProfileSubsystem: start init"); + + // (re)init member collections + mProfileIds = new Vector(); + mProfiles = new Hashtable(); + mProfileClassIds = new Hashtable(); + mProfileDNs = new Hashtable(); + IPluginRegistry registry = (IPluginRegistry) CMS.getSubsystem(CMS.SUBSYSTEM_REGISTRY); + IConfigStore cs = CMS.getConfigStore(); + IConfigStore dbCfg = cs.getSubStore("internaldb"); + dbFactory = CMS.getLdapBoundConnFactory(); + dbFactory.init(dbCfg); + mConfig = config; mOwner = owner; @@ -88,24 +107,45 @@ public class ProfileSubsystem implements IProfileSubsystem { // *.profile2.config=config/profiles/profile2.cfg // read profile id, implementation, and its configuration files - String ids = config.getString(PROP_LIST, ""); - StringTokenizer st = new StringTokenizer(ids, ","); - - while (st.hasMoreTokens()) { - String id = st.nextToken(); - IConfigStore subStore = config.getSubStore(id); - String classid = subStore.getString(PROP_CLASS_ID); - IPluginInfo info = registry.getPluginInfo("profile", classid); - if (info == null) { - throw new EBaseException("No plugins for type : profile, with id " + classid); - } - String configPath = subStore.getString(PROP_CONFIG); + String basedn = cs.getString("internaldb.basedn"); + String dn = "ou=certProfiles," + basedn; + LDAPConnection conn = dbFactory.getConn(); + + String[] attrs = {"cn", "classId"}; + try { + LDAPSearchResults ldapProfiles = conn.search( + dn, LDAPConnection.SCOPE_ONE, "(objectclass=*)", attrs, false); + + while (ldapProfiles.hasMoreElements()) { + LDAPEntry ldapProfile = ldapProfiles.next(); + + String id = (String) + ldapProfile.getAttribute("cn").getStringValues().nextElement(); + + String classid = (String) + ldapProfile.getAttribute("classId").getStringValues().nextElement(); - CMS.debug("Start Profile Creation - " + id + " " + classid + " " + info.getClassName()); - createProfile(id, classid, info.getClassName(), - configPath); + IPluginInfo info = registry.getPluginInfo("profile", classid); + if (info == null) { + throw new EBaseException("No plugins for type : profile, with id " + classid); + } - CMS.debug("Done Profile Creation - " + id); + CMS.debug("Start Profile Creation - " + id + " " + classid + " " + info.getClassName()); + + createProfile(id, classid, info.getClassName(), ldapProfile.getDN()); + + CMS.debug("Done Profile Creation - " + id); + } + } + catch (LDAPException e) { + throw new EBaseException("Error reading profiles: " + e.toString()); + } + finally { + try { + dbFactory.returnConn(conn); + } catch (Exception e) { + throw new EProfileException("Error releasing the ldap connection" + e.toString()); + } } Enumeration ee = getProfileIds(); @@ -121,20 +161,27 @@ public class ProfileSubsystem implements IProfileSubsystem { * Creates a profile instance. */ public IProfile createProfile(String id, String classid, String className, - String configPath) + String dn) throws EProfileException { - IProfile profile = null; - try { - profile = (IProfile) Class.forName(className).newInstance(); - IConfigStore subStoreConfig = CMS.createFileConfigStore(configPath); + String[] objectClasses = {"top", "certProfile"}; + LDAPAttribute[] createAttrs = { + new LDAPAttribute("objectclass", objectClasses), + new LDAPAttribute("cn", id), + new LDAPAttribute("classId", classid) + }; + + IConfigStore subStoreConfig = new LDAPConfigStore( + dbFactory, dn, createAttrs, "certProfileConfig"); CMS.debug("ProfileSubsystem: initing " + className); + IProfile profile = (IProfile) Class.forName(className).newInstance(); profile.setId(id); profile.init(this, subStoreConfig); mProfileIds.addElement(id); mProfiles.put(id, profile); mProfileClassIds.put(id, classid); + mProfileDNs.put(id, dn); return profile; } catch (Exception e) { // throw exceptions @@ -145,62 +192,46 @@ public class ProfileSubsystem implements IProfileSubsystem { } public void deleteProfile(String id, String configPath) throws EProfileException { - if (isProfileEnable(id)) { throw new EProfileException("CMS_PROFILE_DELETE_ENABLEPROFILE"); } - String ids = ""; - try { - ids = mConfig.getString(PROP_LIST, ""); - } catch (Exception e) { + String dn = mProfileDNs.get(id); + if (dn == null) { + throw new EProfileException("CMS_PROFILE_DELETE_UNKNOWNPROFILE"); } - StringTokenizer tokenizer = new StringTokenizer(ids, ","); - StringBuffer list = new StringBuffer(); - - while (tokenizer.hasMoreTokens()) { - String element = tokenizer.nextToken(); - - if (!element.equals(id)) { - list.append(element + ","); + LDAPConnection conn; + try { + conn = dbFactory.getConn(); + } + catch (ELdapException e) { + throw new EProfileException("Error acquiring the ldap connection" + e.toString()); + } + try { + conn.delete(dn); + } + catch (LDAPException e) { + throw new EProfileException("CMS_PROFILE_DELETE_DATABASEERROR"); + } + finally { + try { + dbFactory.returnConn(conn); + } catch (Exception e) { + throw new EProfileException("Error releasing the ldap connection" + e.toString()); } } - if (list.length() != 0) - list.deleteCharAt(list.length() - 1); - mConfig.putString(PROP_LIST, list.toString()); - mConfig.removeSubStore(id); - File file1 = new File(configPath); - - if (!file1.delete()) { - CMS.debug("ProfileSubsystem: deleteProfile: Cannot delete the configuration file : " + configPath); - } mProfileIds.removeElement(id); mProfiles.remove(id); mProfileClassIds.remove(id); - try { - CMS.getConfigStore().commit(false); - } catch (Exception e) { - } + mProfileDNs.remove(id); } public void createProfileConfig(String id, String classId, String configPath) throws EProfileException { - try { - if (mProfiles.size() > 0) { - mConfig.putString(PROP_LIST, - mConfig.getString(PROP_LIST) + "," + id); - } else { - mConfig.putString(PROP_LIST, id); - } - mConfig.putString(id + "." + PROP_CLASS_ID, classId); - mConfig.putString(id + "." + PROP_CONFIG, configPath); - CMS.getConfigStore().commit(true); - } catch (EBaseException e) { - CMS.debug(e.toString()); - } + // nothing to do } /** @@ -219,6 +250,7 @@ public class ProfileSubsystem implements IProfileSubsystem { mProfileIds.clear(); mProfiles.clear(); mProfileClassIds.clear(); + mProfileDNs.clear(); } /** @@ -231,13 +263,6 @@ public class ProfileSubsystem implements IProfileSubsystem { return mConfig; } - /** - * Adds a profile. - */ - public void addProfile(String id, IProfile profile) - throws EProfileException { - } - public boolean isProfileEnable(String id) { IProfile profile = mProfiles.get(id); String enable = null; -- 1.9.3 -------------- next part -------------- >From 4813bf7d9052bbf6d1d68771f05329891c1acbc4 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Tue, 22 Jul 2014 00:03:47 -0400 Subject: [PATCH] add configuration to disable a dynamic subsystem Add the ability to disable a dynamic subsystem by configuring it with `enabled=false' in CS.cfg. Subsystems are enabled by default. This will be used during `pkispawn' to disable the ProfileSubsystem until the database configuration is established. --- .../src/com/netscape/cmscore/apps/CMSEngine.java | 36 +++++++++++++--------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/base/server/cmscore/src/com/netscape/cmscore/apps/CMSEngine.java b/base/server/cmscore/src/com/netscape/cmscore/apps/CMSEngine.java index 68c64824e37bcad282a5bbeabf6b943fabf39481..b6e92bfbdd372d3980c584ba8c9571a4ec52ec8d 100644 --- a/base/server/cmscore/src/com/netscape/cmscore/apps/CMSEngine.java +++ b/base/server/cmscore/src/com/netscape/cmscore/apps/CMSEngine.java @@ -208,25 +208,25 @@ public class CMSEngine implements ICMSEngine { // static subsystems - must be singletons private static SubsystemInfo[] mStaticSubsystems = { new SubsystemInfo( - Debug.ID, Debug.getInstance()), + Debug.ID, Debug.getInstance(), true), new SubsystemInfo(LogSubsystem.ID, - LogSubsystem.getInstance()), + LogSubsystem.getInstance(), true), new SubsystemInfo( - JssSubsystem.ID, JssSubsystem.getInstance()), + JssSubsystem.ID, JssSubsystem.getInstance(), true), new SubsystemInfo( - DBSubsystem.ID, DBSubsystem.getInstance()), + DBSubsystem.ID, DBSubsystem.getInstance(), true), new SubsystemInfo( - UGSubsystem.ID, UGSubsystem.getInstance()), + UGSubsystem.ID, UGSubsystem.getInstance(), true), new SubsystemInfo( - PluginRegistry.ID, new PluginRegistry()), + PluginRegistry.ID, new PluginRegistry(), true), new SubsystemInfo( - OidLoaderSubsystem.ID, OidLoaderSubsystem.getInstance()), + OidLoaderSubsystem.ID, OidLoaderSubsystem.getInstance(), true), new SubsystemInfo( - X500NameSubsystem.ID, X500NameSubsystem.getInstance()), + X500NameSubsystem.ID, X500NameSubsystem.getInstance(), true), // skip TP subsystem; // problem in needing dbsubsystem in constructor. and it's not used. new SubsystemInfo( - RequestSubsystem.ID, RequestSubsystem.getInstance()), + RequestSubsystem.ID, RequestSubsystem.getInstance(), true), }; // dynamic subsystems are loaded at init time, not neccessarily singletons. @@ -235,11 +235,11 @@ public class CMSEngine implements ICMSEngine { // final static subsystems - must be singletons. private static SubsystemInfo[] mFinalSubsystems = { new SubsystemInfo( - AuthSubsystem.ID, AuthSubsystem.getInstance()), + AuthSubsystem.ID, AuthSubsystem.getInstance(), true), new SubsystemInfo( - AuthzSubsystem.ID, AuthzSubsystem.getInstance()), + AuthzSubsystem.ID, AuthzSubsystem.getInstance(), true), new SubsystemInfo( - JobsScheduler.ID, JobsScheduler.getInstance()), + JobsScheduler.ID, JobsScheduler.getInstance(), true), }; private static final int IP = 0; @@ -892,6 +892,7 @@ public class CMSEngine implements ICMSEngine { ssconfig.getSubStore(String.valueOf(i)); String id = config.getString(PROP_ID); String classname = config.getString(PROP_CLASS); + boolean enabled = config.getBoolean("enabled", true); ISubsystem ss = null; try { @@ -906,7 +907,7 @@ public class CMSEngine implements ICMSEngine { throw new EBaseException( CMS.getUserMessage("CMS_BASE_LOAD_FAILED_1", id, e.toString())); } - mDynSubsystems[i] = new SubsystemInfo(id, ss); + mDynSubsystems[i] = new SubsystemInfo(id, ss, enabled); Debug.trace("loaded dyn subsystem " + id); } } @@ -928,6 +929,10 @@ public class CMSEngine implements ICMSEngine { IConfigStore ssConfig = mConfig.getSubStore(id); CMS.debug("CMSEngine: initSubsystem id=" + id); + if (!ssinfo.mEnabled) { + CMS.debug("CMSEngine: subsystem disabled id=" + id); + return; + } if (doSetId) ss.setId(id); CMS.debug("CMSEngine: ready to init id=" + id); @@ -2000,10 +2005,11 @@ class WarningListener implements ILogEventListener { class SubsystemInfo { public final String mId; public final ISubsystem mInstance; + public final boolean mEnabled; - public SubsystemInfo(String id, ISubsystem ssInstance) { + public SubsystemInfo(String id, ISubsystem ssInstance, boolean enabled) { mId = id; mInstance = ssInstance; + mEnabled = enabled; } - } -- 1.9.3 -------------- next part -------------- >From fe1c42feb64fdc21b185e2aa62c58c2908b5fd2d Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Fri, 18 Jul 2014 02:01:58 -0400 Subject: [PATCH] import profiles when spawning CA instance --- base/ca/shared/conf/CS.cfg.in | 1 + .../server/ca/rest/CAInstallerService.java | 118 +++++++++++++++++++++ 2 files changed, 119 insertions(+) diff --git a/base/ca/shared/conf/CS.cfg.in b/base/ca/shared/conf/CS.cfg.in index 4ab8974e6340d81d23bb7f5ea05a07b0936b6463..13ed9c4b9f43f8e76c6f5faffbc78c4a61ccf718 100644 --- a/base/ca/shared/conf/CS.cfg.in +++ b/base/ca/shared/conf/CS.cfg.in @@ -1139,6 +1139,7 @@ subsystem.0.class=com.netscape.ca.CertificateAuthority subsystem.0.id=ca subsystem.1.class=com.netscape.cmscore.profile.ProfileSubsystem subsystem.1.id=profile +subsystem.1.enabled=false subsystem.2.class=com.netscape.cmscore.selftests.SelfTestSubsystem subsystem.2.id=selftests subsystem.3.class=com.netscape.cmscore.cert.CrossCertPairSubsystem diff --git a/base/ca/src/org/dogtagpki/server/ca/rest/CAInstallerService.java b/base/ca/src/org/dogtagpki/server/ca/rest/CAInstallerService.java index bb823eece4729599b6badd9ca0e24ef560b9f279..431a0c548818c1810051d47a6484520dccd8e142 100644 --- a/base/ca/src/org/dogtagpki/server/ca/rest/CAInstallerService.java +++ b/base/ca/src/org/dogtagpki/server/ca/rest/CAInstallerService.java @@ -17,13 +17,27 @@ // --- END COPYRIGHT BLOCK --- package org.dogtagpki.server.ca.rest; +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.util.StringTokenizer; + +import netscape.ldap.LDAPAttribute; + import org.dogtagpki.server.rest.SystemConfigService; import com.netscape.certsrv.apps.CMS; import com.netscape.certsrv.base.EBaseException; +import com.netscape.certsrv.base.IConfigStore; import com.netscape.certsrv.base.PKIException; +import com.netscape.certsrv.ldap.ELdapException; +import com.netscape.certsrv.ldap.ILdapConnFactory; +import com.netscape.certsrv.registry.IPluginInfo; +import com.netscape.certsrv.registry.IPluginRegistry; import com.netscape.certsrv.system.ConfigurationRequest; import com.netscape.cms.servlet.csadmin.ConfigurationUtils; +import com.netscape.cmscore.base.LDAPConfigStore; + /** * @author alee @@ -64,5 +78,109 @@ public class CAInstallerService extends SystemConfigService { CMS.debug(e); throw new PKIException("Errors in determining if security domain host is a master CA"); } + + try { + cs.remove("subsystem.1.enabled"); + } catch (Exception e) { + CMS.debug(e); + throw new PKIException("Error enabling ProfileSubsystem"); + } + } + + @Override + public void initializeDatabase(ConfigurationRequest data) { + super.initializeDatabase(data); + + if (!data.isClone()) { + try { + importProfiles("/usr/share/pki"); + } catch (Exception e) { + throw new PKIException("Error importing profiles."); + } + } + } + + /** + * Import profiles from the filesystem into the database. + * + * @param configRoot Where to look for the profile files. For a + * fresh installation this should be + * "/usr/share/pki". For existing installations it + * should be CMS.getConfigStore().getString("instanceRoot"). + * + */ + public static void importProfiles(String configRoot) + throws EBaseException, ELdapException { + IPluginRegistry registry = (IPluginRegistry) + CMS.getSubsystem(CMS.SUBSYSTEM_REGISTRY); + IConfigStore cfg = CMS.getConfigStore(); + IConfigStore profileCfg = cfg.getSubStore("profile"); + String profileIds = profileCfg.getString("list", ""); + StringTokenizer st = new StringTokenizer(profileIds, ","); + + IConfigStore dbCfg = cfg.getSubStore("internaldb"); + ILdapConnFactory dbFactory = CMS.getLdapBoundConnFactory(); + dbFactory.init(dbCfg); + + while (st.hasMoreTokens()) { + String profileId = st.nextToken(); + IConfigStore profileSubCfg = profileCfg.getSubStore(profileId); + String classId = profileSubCfg.getString("class_id", ""); + try { + IPluginInfo info = registry.getPluginInfo("profile", classId); + if (info == null) { + throw new EBaseException("No plugins for type : profile, with id " + classId); + } + String className = info.getClassName(); + + String profilePath = configRoot + "/ca/profiles/ca/" + profileId + ".cfg"; + CMS.debug("Importing profile '" + profileId + "' from " + profilePath); + importProfile(dbFactory, classId, profileId, profilePath); + } + catch (EBaseException e) { + CMS.debug("Error importing profile '" + profileId + "': " + e.toString()); + throw e; + } + } + } + + /** + * Import one profile from the filesystem into the database. + * + * @param dbFactory LDAP connection factory. + * @param profileFile The profile to import. + */ + public static void importProfile( + ILdapConnFactory dbFactory, String classId, + String profileId, String profilePath) + throws EBaseException { + + IConfigStore cfg = CMS.getConfigStore(); + String basedn = cfg.getString("internaldb.basedn", ""); + + String dn = "cn=" + profileId + ",ou=certProfiles," + basedn; + + String[] objectClasses = {"top", "certProfile"}; + LDAPAttribute[] createAttrs = { + new LDAPAttribute("objectclass", objectClasses), + new LDAPAttribute("cn", profileId), + new LDAPAttribute("classId", classId) + }; + + IConfigStore configStore = new LDAPConfigStore( + dbFactory, dn, createAttrs, "certProfileConfig"); + + try { + FileInputStream input = new FileInputStream(profilePath); + configStore.load(input); + } + catch (FileNotFoundException e) { + throw new EBaseException("Could not find file for profile: " + profileId); + } + catch (IOException e) { + throw new EBaseException("Error loading data for profile: " + profileId); + } + + configStore.commit(false /* no backup */); } } -- 1.9.3 -------------- next part -------------- >From 7eca0dd3f819588d53e71dea84ce23a21186c586 Mon Sep 17 00:00:00 2001 From: Fraser Tweedale Date: Wed, 23 Jul 2014 02:40:07 -0400 Subject: [PATCH] add "pki ca profile show-raw" CLI command --- .../dogtagpki/server/ca/rest/ProfileService.java | 47 +++++----- .../netscape/certsrv/profile/ProfileClient.java | 5 + .../netscape/certsrv/profile/ProfileResource.java | 8 +- .../com/netscape/cmstools/profile/ProfileCLI.java | 1 + .../cmstools/profile/ProfileShowRawCLI.java | 102 +++++++++++++++++++++ 5 files changed, 141 insertions(+), 22 deletions(-) create mode 100644 base/java-tools/src/com/netscape/cmstools/profile/ProfileShowRawCLI.java diff --git a/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java b/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java index cf0d4db7f8bed44baf6d2abbf70a61406ddb4b7c..9d851145f5e681f1044f81654130a2d0114ce36d 100644 --- a/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java +++ b/base/ca/src/org/dogtagpki/server/ca/rest/ProfileService.java @@ -18,6 +18,7 @@ package org.dogtagpki.server.ca.rest; +import java.io.ByteArrayOutputStream; import java.io.IOException; import java.net.URI; import java.security.Principal; @@ -163,9 +164,7 @@ public class ProfileService extends PKIService implements ProfileResource { return createOKResponse(infos); } - @Override - public Response retrieveProfile(String profileId) throws ProfileNotFoundException { - ProfileData data = null; + private IProfile getProfile(String profileId) throws ProfileNotFoundException { boolean visibleOnly = true; if (profileId == null) { @@ -185,24 +184,12 @@ public class ProfileService extends PKIService implements ProfileResource { visibleOnly = false; } - Enumeration profileIds = ps.getProfileIds(); - - IProfile profile = null; - if (profileIds != null) { - while (profileIds.hasMoreElements()) { - String id = profileIds.nextElement(); - - if (id.equals(profileId)) { - - try { - profile = ps.getProfile(profileId); - } catch (EProfileException e) { - e.printStackTrace(); - throw new ProfileNotFoundException(profileId); - } - break; - } - } + IProfile profile; + try { + profile = ps.getProfile(profileId); + } catch (EProfileException e) { + e.printStackTrace(); + throw new ProfileNotFoundException(profileId); } if (profile == null) { @@ -213,6 +200,14 @@ public class ProfileService extends PKIService implements ProfileResource { throw new ProfileNotFoundException(profileId); } + return profile; + } + + @Override + public Response retrieveProfile(String profileId) throws ProfileNotFoundException { + IProfile profile = getProfile(profileId); + + ProfileData data = null; try { data = createProfileData(profileId); } catch (EBaseException e) { @@ -228,6 +223,16 @@ public class ProfileService extends PKIService implements ProfileResource { return createOKResponse(data); } + @Override + public Response retrieveProfileRaw(String profileId) + throws ProfileNotFoundException { + IProfile profile = getProfile(profileId); + ByteArrayOutputStream data = new ByteArrayOutputStream(); + profile.getConfigStore().save(data, null); + return createOKResponse(data.toByteArray()); + } + + public ProfileData createProfileData(String profileId) throws EBaseException { IProfile profile; diff --git a/base/common/src/com/netscape/certsrv/profile/ProfileClient.java b/base/common/src/com/netscape/certsrv/profile/ProfileClient.java index 51d159aca687719a2dace939da5b09c4809872ec..94b087230d46db342dff365da3c57734d98f02dc 100644 --- a/base/common/src/com/netscape/certsrv/profile/ProfileClient.java +++ b/base/common/src/com/netscape/certsrv/profile/ProfileClient.java @@ -45,6 +45,11 @@ public class ProfileClient extends Client { return client.getEntity(response, ProfileData.class); } + public byte[] retrieveProfileRaw(String id) { + Response response = profileClient.retrieveProfileRaw(id); + return client.getEntity(response, byte[].class); + } + public ProfileDataInfos listProfiles(Integer start, Integer size) { Response response = profileClient.listProfiles(start, size); return client.getEntity(response, ProfileDataInfos.class); diff --git a/base/common/src/com/netscape/certsrv/profile/ProfileResource.java b/base/common/src/com/netscape/certsrv/profile/ProfileResource.java index 87449b27e749c6088f65b53192eb5ac101263f1e..d0b84eeb61688fe4bbc107c2bdd368f848e568cd 100644 --- a/base/common/src/com/netscape/certsrv/profile/ProfileResource.java +++ b/base/common/src/com/netscape/certsrv/profile/ProfileResource.java @@ -31,6 +31,12 @@ public interface ProfileResource { @ACLMapping("profiles.read") public Response retrieveProfile(@PathParam("id") String id); + @GET + @Path("{id}/raw") + @ClientResponseType(entityType=byte[].class) + @ACLMapping("profiles.read") + public Response retrieveProfileRaw(@PathParam("id") String id); + @POST @ClientResponseType(entityType=ProfileData.class) @ACLMapping("profiles.create") @@ -53,4 +59,4 @@ public interface ProfileResource { @ClientResponseType(entityType=Void.class) @ACLMapping("profiles.delete") public Response deleteProfile(@PathParam("id") String id); -} \ No newline at end of file +} diff --git a/base/java-tools/src/com/netscape/cmstools/profile/ProfileCLI.java b/base/java-tools/src/com/netscape/cmstools/profile/ProfileCLI.java index 732b597afd1dbb5b9440d451f34b2f39e20fb904..a57c741946b40f0cc02a10acaf6d895081b151d9 100644 --- a/base/java-tools/src/com/netscape/cmstools/profile/ProfileCLI.java +++ b/base/java-tools/src/com/netscape/cmstools/profile/ProfileCLI.java @@ -30,6 +30,7 @@ public class ProfileCLI extends CLI { addModule(new ProfileFindCLI(this)); addModule(new ProfileShowCLI(this)); + addModule(new ProfileShowRawCLI(this)); addModule(new ProfileAddCLI(this)); addModule(new ProfileModifyCLI(this)); addModule(new ProfileRemoveCLI(this)); diff --git a/base/java-tools/src/com/netscape/cmstools/profile/ProfileShowRawCLI.java b/base/java-tools/src/com/netscape/cmstools/profile/ProfileShowRawCLI.java new file mode 100644 index 0000000000000000000000000000000000000000..a15c4e08629a759a2cd66ecfa79578fd9eab6e2e --- /dev/null +++ b/base/java-tools/src/com/netscape/cmstools/profile/ProfileShowRawCLI.java @@ -0,0 +1,102 @@ +//--- BEGIN COPYRIGHT BLOCK --- +//This program is free software; you can redistribute it and/or modify +//it under the terms of the GNU General Public License as published by +//the Free Software Foundation; version 2 of the License. +// +//This program is distributed in the hope that it will be useful, +//but WITHOUT ANY WARRANTY; without even the implied warranty of +//MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +//GNU General Public License for more details. +// +//You should have received a copy of the GNU General Public License along +//with this program; if not, write to the Free Software Foundation, Inc., +//51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +// +//(C) 2014 Red Hat, Inc. +//All rights reserved. +//--- END COPYRIGHT BLOCK --- + +package com.netscape.cmstools.profile; + +import java.io.FileOutputStream; +import java.util.Arrays; + +import org.apache.commons.cli.CommandLine; +import org.apache.commons.cli.Option; +import org.apache.commons.cli.ParseException; + +import com.netscape.certsrv.profile.ProfileData; +import com.netscape.cmstools.cli.CLI; +import com.netscape.cmstools.cli.MainCLI; + +public class ProfileShowRawCLI extends CLI { + + public ProfileCLI profileCLI; + + public ProfileShowRawCLI(ProfileCLI profileCLI) { + super("show-raw", "Show profiles (config-store format)", profileCLI); + this.profileCLI = profileCLI; + + createOptions(); + } + + public void printHelp() { + formatter.printHelp(getFullName() + " [OPTIONS...]", options); + } + + public void createOptions() { + Option option = new Option(null, "output", true, "Output filename"); + option.setArgName("filename"); + options.addOption(option); + } + + public void execute(String[] args) throws Exception { + // Always check for "--help" prior to parsing + if (Arrays.asList(args).contains("--help")) { + // Display usage + printHelp(); + System.exit(0); + } + + CommandLine cmd = null; + + try { + cmd = parser.parse(options, args); + } catch (ParseException e) { + System.err.println("Error: " + e.getMessage()); + printHelp(); + System.exit(-1); + } + + String[] cmdArgs = cmd.getArgs(); + + if (cmdArgs.length < 1) { + System.err.println("Error: No Profile ID specified."); + printHelp(); + System.exit(-1); + } + + String profileId = cmdArgs[0]; + + String filename = null; + if (cmd.hasOption("output")) { + filename = cmd.getOptionValue("output"); + + if (filename == null || filename.trim().length() == 0) { + System.err.println("Error: Missing output file name."); + printHelp(); + System.exit(-1); + } + } + + byte[] profileConfig = profileCLI.profileClient.retrieveProfileRaw(profileId); + + MainCLI.printMessage("Profile \"" + profileId + "\""); + + if (filename != null) { + (new FileOutputStream(filename)).write(profileConfig); + } else { + System.out.println(new String(profileConfig)); + } + } +} -- 1.9.3 From jmagne at redhat.com Wed Jul 23 22:12:47 2014 From: jmagne at redhat.com (John Magne) Date: Wed, 23 Jul 2014 18:12:47 -0400 (EDT) Subject: [Pki-devel] [pki-devel][PATCH] 0015-First-cut-of-enrollment-feature.patch In-Reply-To: <53CF01C6.5040905@redhat.com> References: <203323816.14176175.1405708019746.JavaMail.zimbra@redhat.com> <53CF01C6.5040905@redhat.com> Message-ID: <528459832.19914528.1406153567074.JavaMail.zimbra@redhat.com> Based on CFU's conditional verbal ACK, addressed the issues and pushed to master. Ticket # 885. ----- Original Message ----- > From: "Christina Fu" > To: pki-devel at redhat.com > Sent: Tuesday, July 22, 2014 5:28:54 PM > Subject: Re: [Pki-devel] [pki-devel][PATCH] 0015-First-cut-of-enrollment-feature.patch > > This is the bulk of the work that writes objects to the token with a lot of > goodies. > > The following are my my review comments for > 0015-First-cut-of-end-to-end-enrollment-feature.patch > > * CreatePinAPDU.java - It appears that the old TPS has SetP1(p1). Is there a > reason why you removed that here? > > * SecureChannel.java > -appendKeyCapabilities: the getting keyCapabilities part from the config is > very repetitive. You might want to write a convenience routine to iterate > through all capabilities. > - for most perms, you did: > + TPSBuffer perms = new TPSBuffer(); > + > + perms.add((byte) 0xff); > + perms.add((byte) 0xff); > + perms.add((byte) 0x40); > + perms.add((byte) 0x00); > + perms.add((byte) 0x40); > + perms.add((byte) 0x00); > but for "createCertificate" you did: > + byte[] perms = { (byte) 0xff, (byte) 0xff, 0x40, 0x00, 0x40, 0x00 }; > + > + TPSBuffer permissions = new TPSBuffer(perms); > > both achieve the same thing, but you probably want to be consistent. The 2nd > way seems cleaner > > - writeObject() throws TPSStatus.STATUS_ERROR_MAC_ENROLL_PDU in case of > response checkResult failure. Is it always the same error at each call? > How about createObject()? > > * PKCS11Obj > - addObjectSpec() - I think it'd be more useful if you put objectId in the > the debug message when one duplicated object is removed. > - getRawHeaderData() - typo "returing" > > *TPSProcessor > - mapPattern() - some debug messages call themselves buildTokenLabel > instead... > - mapPattern() - I am not sure it is what you wanted. You had > + if ((nextPos - firstPos) <= 1) { > + return "token"; > + } > that will literally return "token" if the pattern turns out to be a string of > non-tokens (not pattern). > What you want to do is to return the exact same pattern if it contains no > '$'. > > * makeKeyIDFromPublicKeyInfo() - Most our existing code call MessageDigest > like this: > MessageDigest md = MessageDigest.getInstance("SHA1"); > I'm curious if it makes any difference if you specify the provider or not? > I'm also not so sure "mozilla" is the appropriate name for the digesgt (I > know we call our JSS provider "Mozilla-JSS", but a message digest is just a > message digest) > > * about authentication: per our discussion, with what we have now, the CS.cfg > configuration needs to be manually changed and server restarted between > swapping the clients between ESC and tpsclient. I will just add my changes > after your patch is checked in. > > Christina > > > On 07/18/2014 11:26 AM, John Magne wrote: > > > > First cut of enrollment feature. > > The following features implemented for enrollment. > > 1. Standard enrollment of a list of RSA certificates. > 2. Certificates are only done with token side keygen. > 3. Minimual enrollment based pin reset functionality implemented to create > a pin for the enrolled token. > 4. Much work done to the PKCS11 object code, which allows us to write the > compressed object blob to the token, allowing coolkey to access it and use > the certs and keys on the token. > 5. Tested with Bob Relyea's "smartcard" utility to prove that signing and > encryption > operations worked as expected. > 6. Some work done to get authentication working with esc. > 7. Created of stub of standalone Pin Reset Processor. Now it returns an error > from > esc but the pin reset command is accepted. > > To Do. > > 1. We need to support server side keygen. > 2. Symmetric Key Changeover in another ticket. > 3. Finish up the stand alone Pin Reset Processor in another ticket. > > > _______________________________________________ > Pki-devel mailing list Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From cfu at redhat.com Thu Jul 24 18:37:18 2014 From: cfu at redhat.com (Christina Fu) Date: Thu, 24 Jul 2014 11:37:18 -0700 Subject: [Pki-devel] [PATCH] pki-cfu-0021-authentication-fix-fixed-issue-relating-to-authentic.patch Message-ID: <53D1525E.7000307@redhat.com> authentication fix: fixed issue relating to authentication that switching between ESC and tpsclient needs to change CS.cfg param value and restart TPS. This fix makes the issue go away. The actual issue is the differences between "loginRequest" and "extendedLoginRequeest". thanks, Christina From cfu at redhat.com Thu Jul 24 18:46:38 2014 From: cfu at redhat.com (Christina Fu) Date: Thu, 24 Jul 2014 11:46:38 -0700 Subject: [Pki-devel] [PATCH] pki-cfu-0021-authentication-fix-fixed-issue-relating-to-authentic.patch In-Reply-To: <53D1525E.7000307@redhat.com> References: <53D1525E.7000307@redhat.com> Message-ID: <53D1548E.1080303@redhat.com> patch attached. On 07/24/2014 11:37 AM, Christina Fu wrote: > authentication fix: fixed issue relating to authentication that > switching between ESC and tpsclient needs to change CS.cfg param value > and restart TPS. This fix makes the issue go away. > The actual issue is the differences between "loginRequest" and > "extendedLoginRequeest". > > thanks, > Christina > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-cfu-0022-authentication-fix-fixed-issue-relating-to-authentic.patch Type: text/x-patch Size: 11027 bytes Desc: not available URL: From jmagne at redhat.com Thu Jul 24 21:54:37 2014 From: jmagne at redhat.com (John Magne) Date: Thu, 24 Jul 2014 17:54:37 -0400 (EDT) Subject: [Pki-devel] [PATCH] pki-cfu-0021-authentication-fix-fixed-issue-relating-to-authentic.patch In-Reply-To: <53D1548E.1080303@redhat.com> References: <53D1525E.7000307@redhat.com> <53D1548E.1080303@redhat.com> Message-ID: <1481077938.21866796.1406238877580.JavaMail.zimbra@redhat.com> Looks fine: Conditional ACK based on minor suggestion: Note the following: + String name; + if (extendedLogin == true) + name = auth.getCredMap_extlogin(cred); + else + name = auth.getCredMap_login(cred); Note we have two methods each for the two different kinds of credMaps for both setters and getters. My quick suggestion is to just make one getter and setter and pass in the "extendedLogin" boolean and let the internal method decide what to do. You have another method already that uses this, it may make things a bit more uniform. ----- Original Message ----- From: "Christina Fu" To: pki-devel at redhat.com Sent: Thursday, July 24, 2014 11:46:38 AM Subject: Re: [Pki-devel] [PATCH] pki-cfu-0021-authentication-fix-fixed-issue-relating-to-authentic.patch patch attached. On 07/24/2014 11:37 AM, Christina Fu wrote: > authentication fix: fixed issue relating to authentication that > switching between ESC and tpsclient needs to change CS.cfg param value > and restart TPS. This fix makes the issue go away. > The actual issue is the differences between "loginRequest" and > "extendedLoginRequeest". > > thanks, > Christina > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel _______________________________________________ Pki-devel mailing list Pki-devel at redhat.com https://www.redhat.com/mailman/listinfo/pki-devel From cfu at redhat.com Thu Jul 24 23:04:13 2014 From: cfu at redhat.com (Christina Fu) Date: Thu, 24 Jul 2014 16:04:13 -0700 Subject: [Pki-devel] [PATCH] pki-cfu-0021-authentication-fix-fixed-issue-relating-to-authentic.patch In-Reply-To: <1481077938.21866796.1406238877580.JavaMail.zimbra@redhat.com> References: <53D1525E.7000307@redhat.com> <53D1548E.1080303@redhat.com> <1481077938.21866796.1406238877580.JavaMail.zimbra@redhat.com> Message-ID: <53D190ED.8000002@redhat.com> Thanks to jack for the review and test from ESC. Addressed and pushed to master per conditional ACK: commit d088bab2e335d266c465c333dc3f7c316c30bd46 Christina On 07/24/2014 02:54 PM, John Magne wrote: > Looks fine: > > Conditional ACK based on minor suggestion: > > > Note the following: > > + String name; > + if (extendedLogin == true) > + name = auth.getCredMap_extlogin(cred); > + else > + name = auth.getCredMap_login(cred); > > Note we have two methods each for the two different kinds of credMaps > for both setters and getters. > > My quick suggestion is to just make one getter and setter and pass in > the "extendedLogin" boolean and let the internal method decide what to do. > > You have another method already that uses this, it may make things a bit more uniform. > > > > > ----- Original Message ----- > From: "Christina Fu" > To: pki-devel at redhat.com > Sent: Thursday, July 24, 2014 11:46:38 AM > Subject: Re: [Pki-devel] [PATCH] pki-cfu-0021-authentication-fix-fixed-issue-relating-to-authentic.patch > > patch attached. > > On 07/24/2014 11:37 AM, Christina Fu wrote: >> authentication fix: fixed issue relating to authentication that >> switching between ESC and tpsclient needs to change CS.cfg param value >> and restart TPS. This fix makes the issue go away. >> The actual issue is the differences between "loginRequest" and >> "extendedLoginRequeest". >> >> thanks, >> Christina >> >> _______________________________________________ >> Pki-devel mailing list >> Pki-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/pki-devel > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From mharmsen at redhat.com Fri Jul 25 01:51:40 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Thu, 24 Jul 2014 18:51:40 -0700 Subject: [Pki-devel] [PATCH] 96-2 Updated man page for pki key CLI commands (Ticket 945) In-Reply-To: <1406046324.26559.2.camel@akoneru.redhat.com> References: <1401480833.5379.1.camel@akoneru.redhat.com> <53895084.9040003@redhat.com> <1406046324.26559.2.camel@akoneru.redhat.com> Message-ID: <53D1B82C.6020102@redhat.com> On 07/22/14 09:25, Abhishek Koneru wrote: > Please review the patch with changes suggested by Matt. > Added the examples section to explain the usage of agent authentication > and a section to explain the details of the parameters used in the > templates(asked b mrniranjan) on IRC. > > -- Abhishek > On Fri, 2014-05-30 at 20:46 -0700, Matthew Harmsen wrote: >> On 05/30/14 13:13, Abhishek Koneru wrote: >> >>> Please review the patch which updates the man pages for the pki key CLI >>> commands. >>> >>> --Abhishek >>> >>> >>> _______________________________________________ >>> Pki-devel mailing list >>> Pki-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/pki-devel >> Abhishek, >> >> What is there, is fine. However, the man page as is, is not very >> useful since it presumes a great deal of knowledge! >> >> I would strongly urge you to provide an EXAMPLES section utilizing >> sample agent authentication. >> >> For example, at the very least, please provide the most basic scenario >> of showing exactly what one would specify in a default installation of >> a CA and KRA to simply perform a "key-find" and a "key-show" using >> client certification. >> >> I would also suggest that you add your name to the list of Authors of >> this man page. >> >> -- Matt Awesome examples! ACK after addressing the following typos/suggestions: * Under 'Archiving a key' you have the following sentences (I think that it will read better if you simply delete the first sentence): Currently, there are no command options to archive a symmetric key. A symmetric key can be archived using the "archiveKey" request template. To archive a secret using the request template stored in a file: *pki key-archive --input * * Re-word the following lines (modifying them as below): The following *pki* client examples show the usage of the above operations for a basic CA and KRA server installation. A basic installation of CA and KRA servers can be done by running pkispawn in interactive mode and selecting the default parameters (see the section *INTERACTIVE MODE* in pkispawn(8)) or using a configuration file with basic parameters(see the section *EXAMPLES* in pkispawn(8)). Running the following commands will set up the NSS database for use by a *pki* client, and import the agent's certificate into the database, and list information (including the nickname) of the certificate stored in the database. The third command shows the information about the imported certificate (including the nickname). * Utilize either ** or ** within all of the various commands, but not both. * Since you do not provide instructions for importing the CA certificate, you may want to inform them that they may get a WARNING that an UNTRUSTED ISSUER was encountered, and that they will be prompted to import the CA certificate: WARNING: UNTRUSTED ISSUER encountered on 'CN=server.example.com,O=example.com Security Domain' indicates a non-trusted CA cert 'CN=CA Signing Certificate,O=example.com Security Domain' Import CA certificate (Y/n)? Y CA server URI [http://server.example.com:8080/ca]: To address this issue, I would suggest adding the following text located after "For demonstration purposes..." and before "To list all the keys...": When issuing the first command, a user may be greeted with a warning message which indicates that an untrusted issuer was encountered. Simply reply 'Y' to import the CA certificate, and, presuming that the displayed CA server URI is valid, press the carriage return. * Since the installation can only be performed by a root user, this file must be copied to a location where other users can access it, with valid permissions. * (remove --clientKeyID and change "--algorithm" to "--key-algorithm"): pki -d -c -n key-generate --clientKeyID vek123456 --key-algorithm DES3 --usages encrypt,decrypt * Inthe case of the above mentioned examples, the encryption and decryption of the secrets is done internally by the Dogtag client API. * But, applications using the CLI framework to create various requests and also use local encryption, so the xml templates can be used to supply data tothe create a request. * (key archival template): pki key-template-show archiveKey --output-file -- dataType - Type of the data to be stored which can be symmetricKey/passphrase/asymmetricKey. -- pkiArchiveOptions - An object of type PKIArchiveOptions provided by the NSS/JSS library to securely transport a secret encoded in Base64 format. * (key retrieval template): pki key-template-show retrieveKey --output-file -- sessionWrappedpassphrase - Base64 encoded string of - Passphrase encrypted with a session key. ALSO: *The order inside of the downloaded template (e. g. - nonceData) differs from the description -- make the order identical.* The downloaded template contains a typo of 'recoring' which should be 'recovering'. * (symmetric key generation): pki key-template-show generateKey --output-file To create a key generation request using the template file: *NOTE: When using the "key-generate" command, it did not recognize the "--input" option, and would therefore fail to utilize the specified template. If this is a bug, please file a new PKI TRAC Ticket.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From mharmsen at redhat.com Fri Jul 25 02:10:02 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Thu, 24 Jul 2014 19:10:02 -0700 Subject: [Pki-devel] [PATCH] Fix independent pkispawn installation and configuration Message-ID: <53D1BC7A.7000704@redhat.com> Please review the following attached patch which addresses: * PKI TRAC Ticket #905 - 2 Step Configuration of CA instance using pkispawn fails This was tested by first "installing" a CA using the following command: * *pkispawn -s CA -f ca-gui-install.cfg -vvv* where 'ca-gui-install.cfg' contained the following: [DEFAULT] pki_admin_password= pki_client_pkcs12_password= pki_skip_configuration=True Then, the CA was 'configured' by running the following command: * *pkispawn -s CA -f ca-gui-config.cfg -vvv* where 'ca-gui-config.cfg' contained the following: [DEFAULT] pki_admin_password= pki_client_pkcs12_password= pki_ds_password= pki_security_domain_password= pki_client_database_purge=False pki_skip_installation=True -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 20140724-Fix-independent-pkispawn-installation-and-configurat.patch Type: text/x-patch Size: 6835 bytes Desc: not available URL: From jmagne at redhat.com Fri Jul 25 18:37:31 2014 From: jmagne at redhat.com (John Magne) Date: Fri, 25 Jul 2014 14:37:31 -0400 (EDT) Subject: [Pki-devel] [pki-devel][PATCH] 0016-Implement-Symmetric-Key-Changeover-Feature.patch In-Reply-To: <2111872220.22918567.1406313440370.JavaMail.zimbra@redhat.com> Message-ID: <743434400.22919826.1406313451291.JavaMail.zimbra@redhat.com> [PATCH] Implement Symmetric Key Changeover Feature 1. Created code to actually change the key over to a new key set. 2. Made calls to the code from the format and enrollment operations, the Pin Reset processor will do so when completed. 3. Tested with real card. Was able to change to key set #2 and then back to stock Ticket #878. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0016-Implement-Symmetric-Key-Changeover-Feature.patch Type: text/x-patch Size: 16153 bytes Desc: not available URL: From cfu at redhat.com Sat Jul 26 01:51:54 2014 From: cfu at redhat.com (Christina Fu) Date: Fri, 25 Jul 2014 18:51:54 -0700 Subject: [Pki-devel] [pki-devel][PATCH] 0016-Implement-Symmetric-Key-Changeover-Feature.patch In-Reply-To: <743434400.22919826.1406313451291.JavaMail.zimbra@redhat.com> References: <743434400.22919826.1406313451291.JavaMail.zimbra@redhat.com> Message-ID: <53D309BA.9050404@redhat.com> ACK. However, for next patch: * I think as long as they still apply, you should put the original comments (or the improvement of which) in the same place. For instance, the original putKeys() puts the sample and layout of the key data at top of the function. Your putKyes() has no comments at all. Come to think of it, I might have missed this (lack of comments) in my previous reviews. If you spot them, please put them in the future patchs. Christina On 07/25/2014 11:37 AM, John Magne wrote: > [PATCH] Implement Symmetric Key Changeover Feature > > 1. Created code to actually change the key over to a new key set. > 2. Made calls to the code from the format and enrollment operations, > the Pin Reset processor will do so when completed. > 3. Tested with real card. Was able to change to key set #2 and then back to stock > > Ticket #878. > > > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jmagne at redhat.com Sat Jul 26 01:56:58 2014 From: jmagne at redhat.com (John Magne) Date: Fri, 25 Jul 2014 21:56:58 -0400 (EDT) Subject: [Pki-devel] [pki-devel][PATCH] 0016-Implement-Symmetric-Key-Changeover-Feature.patch In-Reply-To: <53D309BA.9050404@redhat.com> References: <743434400.22919826.1406313451291.JavaMail.zimbra@redhat.com> <53D309BA.9050404@redhat.com> Message-ID: <1817711564.23150740.1406339818791.JavaMail.zimbra@redhat.com> Based on cfu's ack, pushed to master. Will deal with missing comment blocks conveniently in next submission. ----- Original Message ----- > From: "Christina Fu" > To: pki-devel at redhat.com > Sent: Friday, July 25, 2014 6:51:54 PM > Subject: Re: [Pki-devel] [pki-devel][PATCH] 0016-Implement-Symmetric-Key-Changeover-Feature.patch > > ACK. > > However, for next patch: > * I think as long as they still apply, you should put the original comments > (or the improvement of which) in the same place. > For instance, the original putKeys() puts the sample and layout of the key > data at top of the function. Your putKyes() has no comments at all. > Come to think of it, I might have missed this (lack of comments) in my > previous reviews. If you spot them, please put them in the future patchs. > > Christina > > > On 07/25/2014 11:37 AM, John Magne wrote: > > > > [PATCH] Implement Symmetric Key Changeover Feature > > 1. Created code to actually change the key over to a new key set. > 2. Made calls to the code from the format and enrollment operations, > the Pin Reset processor will do so when completed. > 3. Tested with real card. Was able to change to key set #2 and then back to > stock > > Ticket #878. > > > _______________________________________________ > Pki-devel mailing list Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel From mharmsen at redhat.com Sat Jul 26 02:02:16 2014 From: mharmsen at redhat.com (Matthew Harmsen) Date: Fri, 25 Jul 2014 19:02:16 -0700 Subject: [Pki-devel] [PATCH] Remove ACL mapping to user from error messages Message-ID: <53D30C28.6090608@redhat.com> Please review the following attached patch (using the attached test procedure) which addresses: * PKI TRAC Ticket #965 - Improve error message - remove ACL mapping to the user -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 20140725-Remove-ACL-mapping-to-user-from-error-messages.patch Type: text/x-patch Size: 6161 bytes Desc: not available URL: -------------- next part -------------- ================================================== Test procedure used to verify PKI TRAC Ticket #965 ================================================== - installed a default CA server using 'pkispawn' on a Fedora 20 machine - created a client NSS database for use by 'pki' client: # certutil -N -d - imported the CA server Admin P12 file into this client NSS database: # pk12util -i /root/.dogtag/pki-tomcat/ca_admin_cert.p12 -d - obtained the Nickname of the CA server Admin certificate # certutil -L -d (e. g. - "PKI Administrator for example.com") NOTE: May be prompted during execution of the first test command regarding a WARNING from an untrusted issuer. Merely answer 'Y' to import the CA Certificate, and press presuming that the correct URL is displayed. - execute the 'user-find' command: # pki -d . -c -n "PKI Administrator for example.com" user-find ----------------- 3 entries matched ----------------- User ID: CA-server.example.com-8443 Full name: CA-server.example.com-8443 User ID: caadmin Full name: caadmin User ID: pkidbuser Full name: pkidbuser ---------------------------- Number of entries returned 3 ---------------------------- - execute the 'user-show' command: # pki -d . -c -n "PKI Administrator for example.com" user-show caadmin -------------- User "caadmin" -------------- User ID: caadmin Full name: caadmin Email: caadmin at example.com Type: adminType State: 1 - launched 'pkiconsole' to deny authorization for these commands: # pkiconsole https://server.example.com:8443/ca User ID: caadmin Password: Select 'Configuration' tab Highlight 'Access Control List' from left-side menu Select 'Access Control List' tab Highlight 'certServer.ca.users' Resource Name Press the 'Edit' button Highlight 'allow (execute) group='Administrators' Under 'ACI entries' Press the 'Edit' button Select the 'Deny' radio button right of 'Access' press the 'OK' button on the 'ACI Editor' Press the 'OK' button on the 'Access Control Editor' NOTE: Leave 'pkiconsole' running . . . - execute the 'user-find' command again: # pki -d . -c -n "PKI Administrator for example.com" user-find ForbiddenException: Authorization failed on resource: certServer.ca.users, operation: execute - execute the 'user-show' command again: # pki -d . -c -n "PKI Administrator for example.com" user-show caadmin ForbiddenException: Authorization failed on resource: certServer.ca.users, operation: execute - Built and applied my patch and restarted the CA server: # systemctl daemon-reload # systemctl restart pki-tomcatd at pki-tomcat.service - Created a tail of the 'ca_audit' log: # tail -f /var/log/pki/pki-tomcat/ca/signedAudit/ca_audit - execute the 'user-find' command one more time: # pki -d . -c -n "PKI Administrator for example.com" user-find ForbiddenException: Authorization Error # tail -f /var/log/pki/pki-tomcat/ca/signedAudit/ca_audit 0.http-bio-8443-exec-3 - [25/Jul/2014:18:37:14 PDT] [14] [6] Authorization failed on resource: certServer.ca.users, operation: execute - execute the 'user-show' command one more time: # pki -d . -c -n "PKI Administrator for example.com" user-show caadmin ForbiddenException: Authorization Error # tail -f /var/log/pki/pki-tomcat/ca/signedAudit/ca_audit 0.http-bio-8443-exec-9 - [25/Jul/2014:18:38:48 PDT] [14] [6] Authorization failed on resource: certServer.ca.users, operation: execute - re-allow authorization using the running 'pkiconsole': Select 'Configuration' tab Highlight 'Access Control List' from left-side menu Select 'Access Control List' tab Highlight 'certServer.ca.users' Resource Name Press the 'Edit' button Highlight 'allow (execute) group='Administrators' Under 'ACI entries' Press the 'Edit' button Select the 'Allow' radio button right of 'Access' press the 'OK' button on the 'ACI Editor' Press the 'OK' button on the 'Access Control Editor' - execute the 'user-find' command one last time: # pki -d . -c -n "PKI Administrator for example.com" user-find ----------------- 3 entries matched ----------------- User ID: CA-server.example.com-8443 Full name: CA-server.example.com-8443 User ID: caadmin Full name: caadmin User ID: pkidbuser Full name: pkidbuser ---------------------------- Number of entries returned 3 ---------------------------- NOTE: Nothing appears in 'ca_audit' related to this command. - execute the 'user-show' command one more time: # pki -d . -c -n "PKI Administrator for example.com" user-show caadmin -------------- User "caadmin" -------------- User ID: caadmin Full name: caadmin Email: caadmin at example.com Type: adminType State: 1 NOTE: Nothing appears in 'ca_audit' related to this command. From cfu at redhat.com Mon Jul 28 17:13:32 2014 From: cfu at redhat.com (Christina Fu) Date: Mon, 28 Jul 2014 10:13:32 -0700 Subject: [Pki-devel] [PATCH] Remove ACL mapping to user from error messages In-Reply-To: <53D30C28.6090608@redhat.com> References: <53D30C28.6090608@redhat.com> Message-ID: <53D684BC.3030808@redhat.com> Here are my review comments per discussion: * The exception message with less detail looks fine * First thing I noticed is that the "signed audit" messages don't conform to the format. Looking closely, I see that you have picked up an outdated interface. The real signed auditor is supposed to be called by doing: IAuditor auditor = CMS.getAuditor(); The authz fail event is supposed to be LOGGING_SIGNED_AUDIT_AUTHZ_FAIL_4 and the call is done as: auditMessage = CMS.getLogMessage( LOGGING_SIGNED_AUDIT_AUTHZ_FAIL, auditSubjectID, ILogger.FAILURE, auditACLResource, auditOperation); audit(auditMessage); where audit is resolved to auditor.log(auditMessage); See AdminServlet.java for example. Anyway, all the CS servlets do auditing that way, and so the REST interface should do it the same way. So, instead of adding audit messages in the authorization modules, I suggest you 1. put the message in debug log instead 2. If it does not exist, file a ticket for REST interface to do signed auditing Christina On 07/25/2014 07:02 PM, Matthew Harmsen wrote: > Please review the following attached patch (using the attached test > procedure) which addresses: > > * PKI TRAC Ticket #965 - Improve error message - remove ACL mapping > to the user > > > > > _______________________________________________ > Pki-devel mailing list > Pki-devel at redhat.com > https://www.redhat.com/mailman/listinfo/pki-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From edewata at redhat.com Wed Jul 30 21:13:33 2014 From: edewata at redhat.com (Endi Sukma Dewata) Date: Wed, 30 Jul 2014 16:13:33 -0500 Subject: [Pki-devel] CLI for editing profiles In-Reply-To: <20140723070924.GX2417@dhcp-40-8.bne.redhat.com> References: <20140723012711.GW2417@dhcp-40-8.bne.redhat.com> <20140723070924.GX2417@dhcp-40-8.bne.redhat.com> Message-ID: <53D95FFD.4070408@redhat.com> On 7/23/2014 2:09 AM, Fraser Tweedale wrote: > On Wed, Jul 23, 2014 at 11:27:11AM +1000, Fraser Tweedale wrote: >> Along with LDAP profiles, we will be adding modules to the CLI for >> adding and editing profiles in the ConfigStore format that was used >> for file-based profiles. For more info, see: >> >> http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage#Command-line_utilities >> >> There is an existing CLI for adding and modifying profiles, in the >> XML format, e.g. ``pki ca profile add caCustomProfile.xml``. The >> XML format carries information including the profile ID and >> class_id, but these data must be supplied out-of-band when dealing >> with the ConfigStore format. >> >> Because of this, I intend to: >> >> - add new commands to the existing profile CLI for working with the >> "raw" (i.e., ConfigStore) format, e.g. "edit-raw", "add-raw". >> Where necessary, these commands will take compulsory >> ``--profile-id`` and/or ``--class-id`` arguments, to account for >> the absense of such information in the profile ConfigStore format; >> >> and >> >> - transport this information in the XML format - not in the "raw" >> format - so that it will be unnecessary to make changes to >> ProfileClient or the ProfileService API. >> >> As usual, I welcome feedback - especially if you feel I am going the >> wrong way ^_^ >> > > An update: due to the various Profile and ProfileInput/Output/Policy > classes not being distributed in the pki-tools package (or > dependencies thereof), I /did/ end up adding methods to the REST > API. > > The `pki ca profile show-raw ` command is implemented in > the attached patch 0009. I expected I will complete the edit-raw > and add-raw commands tomorrow, and after that will move on to > testing replication and polling/monitoring for changes to profiles. > > Cheers, > > Fraser Some comments/questions: 1. About the the new ou=certProfiles entry, We probably should move it into the ou=ca subtree because that subtree already contains other CA-specific LDAP entries such as ou=certificateRepository. 2. Is certProfilesLastModified really necessary? Each LDAP entry already has a modifyTimestamp attributes. To be accurate, the certProfilesLastModified has to be updated whenever the profiles are imported or updated. There seems to be no code for that yet, and there's no guarantee it won't get out of sync (e.g. server crash, direct LDAP modification). 3. If certProfilesLastModified is really necessary, instead of adding a separate cn=certProfilesInfo entry just to hold this attribute, we probably can move it into ou=certProfiles either by extending the organizationalUnit object class or adding an auxiliary object class. We probably should make it an optional attribute (i.e. no attribute means never updated). 4. I'm not sure if we really need the certProfileIsDefault attribute. There's no guarantee that the value will be accurate because someone could modify the certProfileConfig directly and forget to update certProfileIsDefault. Also, if we need to do any profile upgrade, it should be based on the actual configuration parameters in certProfileConfig, not based on the certProfileIsDefault. 5. In LDAPConfigStore let's not use the 'm' prefix for the field names. It's a legacy convention and we should not use it for newer code. 6. The mInDatabase in LDAPConfigStore doesn't seem to be used anywhere. 7. In LDAPConfigStore the certProfileConfig attribute is read as binary using getByteValues(). Is there a specific reason? Can we use getStringValues() since the content is a string (i.e. property file)? 8. The ProfileService constructs the profile DN. I think this level of details should be hidden in the ProfileSubsystem layer. The ProfileService shouldn't need to know where/how the profiles are stored. 9. The ProfileAdminServlet calls ProfileSubsystem.createProfile() but it supplies a config path instead of a DN. As in #8, I think the caller should only specify the profile ID, it's up to the ProfileSubsystem to determine the actual location (i.e. profile DN). 10. The mProfileDNs in ProfileSubsystem shouldn't be necessary because we can generate the DN from the profileId since it's a flat tree. 11. The profile subsystem index is hard-coded in the following line: cs.remove("subsystem.1.enabled"); Although this code is modifying the standard CS.cfg, it would be better to do a search the index, then make the modification with the index. int index = ... find subsystem index where id=profile ... cs.remove("subsystem." + index + ".enabled"); This will avoid problems if we ever add a default subsystem in the future. 12. The profile-show-raw is essentially the same as profile-show, except the output format is different. I'd suggest merging them into profile-show and use an option like --raw to show the raw profile. About the out-of-band class ID, I think we should just map it into a 'virtual' property in the raw profile (e.g. class_id=...). The property will only exist if the client pulls the raw profile. As already in done the patch, in the database it will be stored in classId attribute instead of certProfileConfig. In the long term all properties will be converted into separate attributes like this, and the 'raw' format should only exist as a legacy interface. 13. The ProfileResource.retrieveProfileRaw() returns a byte[]. Although it works just fine for this purpose, I think returning a Properties would make it more developer-friendly. That way the output is readily usable, and it takes care of any CR/LF issue across different platforms. Maybe call it getProfileProperties()? 14. The CAInstallerService.importProfiles() and importProfile() don't need to be static. As virtual methods it will have access to other member attributes/methods already defined in the class and its super classes (e.g. config store). A long term goal is to eliminate the use of static methods in CMS (it's not very modular). 15. Minor. I see two styles of writing the try-catch block in the patch: try { ... } catch (...) { ... } and try { ... } catch (...) { ... } I think the first one is more common. 16. Minor. Instead of adding the enabled parameter to the existing SubsystemInfo constructor (and having to modify all invocations), it's also possible to create a new constructor with the enabled parameter and call it from the old constructor with enabled set to true. -- Endi S. Dewata From cfu at redhat.com Thu Jul 31 22:14:27 2014 From: cfu at redhat.com (Christina Fu) Date: Thu, 31 Jul 2014 15:14:27 -0700 Subject: [Pki-devel] [PATCH] first cut of tokendb management, policy, and activities Message-ID: <53DABFC3.5020704@redhat.com> This patch is the first cut of tokendb management, policy, and activities. It * adds and updates token entries in the tokendb * checks policies when necessarily (note: bug filed: https://fedorahosted.org/pki/ticket/1085 * logs activities Certificate related tokendb updates will be in the next patch. There may still be a few more activities to be logged (such as pin reset, etc, once complete.). Policy checks re ticket#1085 will be provided after ticket 1085 is addressed. Please review. thanks, Christina -------------- next part -------------- A non-text attachment was scrubbed... Name: pki-cfu-0023-ticket-882-tokendb-management-policy-and-activities-.patch Type: text/x-patch Size: 62575 bytes Desc: not available URL: