[Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18

[Ade Lee]

On Wed, 2012-08-15 at 16:34 +0200, Martin Kosek wrote:
> On 08/15/2012 03:54 PM, Ade Lee wrote:
> > On Wed, 2012-08-15 at 13:24 +0200, Martin Kosek wrote:
> >> On 08/08/2012 10:05 PM, Ade Lee wrote:
> >>> Hi, 
> >>>
> >>> Dogtag 10 is being released on f18, and has a number of changes that
> >>> will affect IPA.  In particular, the following changes will affect
> >>> current IPA code. 
> >>>
> >>> * The directory layout of the dogtag instance has changed.  Instead of
> >>> using separate tomcat instances to host different subsystems, the
> >>> standard dogtag installation will allow one to install a CA. KRA, OCSP
> >>> and TKS within the same instance.  There have been corresponding changes
> >>> in the directory layout, as well as the default instance name
> >>> (pki-tomcat instead of pki-ca), and startup daemon (pki-tomcatd, instead
> >>> of pki-cad, pki-krad etc.) 
> >>>
> >>> * The default instance will use only four ports (HTTPS, HTTP, AJP and
> >>> tomcat shutdown port) rather than the 6 previously used.  The default
> >>> ports will be changed to the standard tomcat ports.  As these ports are
> >>> local to the ipa server machine, this should not cause too much
> >>> disruption. 
> >>>
> >>> * There is a new single step installer written in python.
> >>> (pkispawn/destroy) vs. pkicreate/pkisilent/pkiremove.
> >>>
> >>> * Dogtag 10 runs on tomcat7 - with a new corresponding version of
> >>> tomcatjss.
> >>>
> >>> The attached patch integrates all the above changes in IPA installation
> >>> and maintenance code.  Once the patch is applied, users will be able to:
> >>>
> >>> 1. run ipa-server-install to completion on f18 with dogtag 10.
> >>> 2. install a new replica on f18 on dogtag 10.
> >>> 3. upgrade an f17 machine with an existing IPA instance to f18/ dogtag
> >>> 10 - and have that old-style dogtag instance continue to run correctly.
> >>> This will require the installation of the latest version of tomcatjss as
> >>> well as the installation of tomcat6.  The old-style instance will
> >>> continue to use tomcat6.
> >>> 4. in addition, the new cert renewal code has been patched and should
> >>> continue to work.
> >>>
> >>> What is not yet completed / supported:
> >>>
> >>> 1. Installation with an external CA is not yet completed in the new
> >>> installer.  We plan to complete this soon.
> >>>
> >>> 2. There is some IPA upgrade code that has not yet been touched
> >>> (install/tools/ipa-upgradeconfig).
> >>>
> >>> 3. A script needs to be written to allow admins to convert their
> >>> old-style dogtag instances to new style instances, as well as code to
> >>> periodically prompt admins to do this.
> >>>
> >>> 4. Installation of old-style instances using pkicreate/pkisilent on
> >>> dogtag 10 will no longer be supported, and will be disabled soon.
> >>>
> >>> 5.  The pki-selinux policy has been updated to reflect these changes,
> >>> but is still in flux.  In fact, it is our intention to place the dogtag
> >>> selinux policy in the base selinux policy for f18.  In the meantime, it
> >>> may be necessary to run installs in permissive mode.
> >>>
> >>> The dogtag 10 code will be released shortly into f18.  Prior to that
> >>> though, we have placed the new dogtag 10 and tomcatjss code in a
> >>> developer repo that is located at 
> >>> http://nkinder.fedorapeople.org/dogtag-devel/
> >>>
> >>> Testing can be done on both f18 and f17 - although the target platform -
> >>> and the only platform for which official builds will be created is f18.
> >>>
> >>> Thanks, 
> >>> Ade
> >>>
> >>
> >> Hi Ade,
> >>
> >> Thanks for the patch, I started with review and integration tests (currently
> >> running on Fedora 17 with Nathan's repo).
> >>
> >> Installation on single master was smooth, it worked just fine, even with
> >> enforced SELinux, without any error - kudos to you and the whole dogtag team.
> >> The resulting logs and the structure of your log directory seems improved. I
> >> believe that the brand new Python installers will make it easier to debug
> >> issues with dogtag installation when they come.  When I tried our unit tests or
> >> some simple cert operation, it worked fine as well.
> >>
> >> Now the bad news, or rather few issues and suggestions I found:
> >>
> >> 1) As we already discussed on IRC, tomcat 7 was not pulled automatically on
> >> Fedora 17 when I updated pki-ca, you somewhere miss a Requires.
> >>
> > We have a dogtag patch that is currently in review that will address
> > this.  Once this is in, tomcatjss >=7.0.0 will be required for f17+,
> > rather than f18+
> > 
> >> 2) I had installed IPA with dogtag10 on master. However, CA installation on a
> >> replica (ipa-ca-install) with dogtag9 failed - is this expectable?
> >>
> > Yes.  The current IPA patch is designed to work with dogtag 10 only,
> > which will be officially available on f18+.  So if you update to dogtag
> > 10, you must have this patch and visa versa.  We probably need to add
> > the relevant requires to IPA to enforce this.
> > 
> > If you have an existing dogtag 9 instance, and you upgrade to the new
> > dogtag 10 and patched IPA, then that instance will continue to work.
> > But any new instances would be created using dogtag 10.
> > 
> >> 3) I had installed IPA with dogtag10 on master. Replica had dogtag10 as well
> >> and I got the following error:
> >>
> >> # ipa-ca-install /home/mkosek/replica-info-vm-114.idm.lab.bos.redhat.com.gpg
> >> ...
> >> Configuring certificate server: Estimated time 3 minutes 30 seconds
> >>   [1/14]: creating certificate server user
> >>   [2/14]: configuring certificate server instance
> >>
> >> Your system may be partly configured.
> >> Run /usr/sbin/ipa-server-install --uninstall to clean up.
> >>
> >> Unexpected error - see /var/log/ipareplica-ca-install.log for details:
> >> IOError: [Errno 2] No such file or directory:
> >> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12'
> >>
> >> Root cause:
> >> ...
> >>   File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line
> >> 625, in __spawn_instance
> >>     "/root/cacert.p12")
> >> ...
> >>
> > I need to look into this.  I had fixed ipa-replica-install, rather than
> > ipa-ca-install to create replicas.  I didn't know ipa-ca-install
> > existed!  Should not be too bad to fix though - most likely just need to
> > move files to the right place.
> 
> Ok, thanks! Btw. CA on replica can be either installed during
> ipa-replica-install time (when --setup-ca option is passed, you probably used
> that one) and the aforementioned ipa-ca-install run after ipa-replica-install.
> 
I will be testing this out again.  As ipa-ca-install uses the same code
as ipa-replica-install, I would have expected it to work.  Did you try
it with selinux in permissive mode?

> > 
> >> 4) What ports does replica need to be open on the master when installing a CA?
> >> Currently, ipa-replica-conncheck does a port check and checks only for 7389,
> >> i.e. a port of backend LDAP instance for CA.
> >>
> > Well, right now - the ports required are 80,443 (to get to the CA) and
> > 7389  (for the backend LDAP of the CA).  Later, when I submit the patch
> > to merge the CA and IPA databases, the ldap port required will be 389.
> 
> 
> Then it is OK, these all ports are already covered in ipa-replica-conncheck.
> 
> > 
> >>
> >> Now to the patch itself.
> >>
> >> 5) The patch needs a rebase
> >>
> > Yup - will rebase in my next submission.
> > 
> >> 6) The patch itself looks and works fine (with the issues I wrote above), but
> >> there is currently a lot of hard-coded file paths, instance names or ports. I
> >> think it would be really great to move most of this to the platform code
> >> (ipapython/platform/). I think that ideal solution would allow us to choose if
> >> we want to build FreeIPA with dogtag9 or dogtag10 environment in a build time.
> >>
> >> I am not saying that it is something you would need to do, its just a proposal
> >> also for other developers. This step would enable us to:
> >> a) Consolidate CA ports, file paths, instance names etc. to one place
> >> b) Easily build IPA to non-Fedora platforms where we still need to use dogtag9
> >>
> > 
> > This is a great idea.  The hard-coded paths etc. are all following the
> > conventions that were already present.  I think you should open a ticket
> > for this - and we should address this in a separate patch.
> 
> I will let other developers comment on this proposal, if we get some agreement,
> I will open a ticket.
> 
> I have also found 2 more issues:
> 
> 7) pki-deploy package does not require any other pki-* package, this does not
> look ok. This way I was able to have pki-ca-9.* and pki-deploy-10.* installed
> at one time. I doubt it would work that way.
> 
We have opened a dogtag ticket to address this. Some kind of dependency
will be added so that pki-deploy and pki-common-10 are co-dependencies.

> 8) Did you test upgrade from installed IPA+dogtag9 to patchedIPA+dogtag10? I
> did that on Fedora 17 and pki-ca did not start after upgrade. Attaching logs my
> VM after I tried to (re)start pki-ca.
> 
I did test this.  From the logs though, this looks very much like it is
not starting because of selinux.  Can you try to restart with selinux
permissive?  I fully expect there to be selinux issues here as some
changes are required in the base policy where the default ports are
defined.


> Martin