[Pulp-list] High Available Pulp Setup

[Arnold Bechtoldt]

Hi Josh,

Can you promote a Pulp child to be a parent and use it to sync repos
with upstream repos?

> This scenario requires no shared storage or fancy/complicated clustering.

The systems which host Pulp won't have local disks/RAIDs big enough to
store 300-500 GB data. Yes, this isn't necessarily helpful.


Arnold

-- 
Arnold Bechtoldt
IT Engineering & Operations

inovex GmbH

Zur Gießerei 16
D-76227 Karlsruhe
Tel: 07231 31 91 0
Fax: 07231 31 91 91
Mobil: 0173 3181 117
arnold.bechtoldt at inovex.de
www.inovex.de

Sitz der Gesellschaft: Pforzheim
AG Mannheim, HRB 502126
Geschäftsführer: Stephan Müller

Am 27.11.13 21:12, schrieb Baird, Josh:
> I think you are overthinking this a little bit.  Pulp has a "nodes" concept which can provide native replication of repository to one or more child/parent nodes.  Here's how we do it:
> 
> * One Pulp server in each datacenter
>   * "Parent" pulp server in primary DC
>   * "Child" pulp server in secondary DCs
> * Content is synced to the parent Pulp server from various repositories
> * Content is then automatically replicated per a replication schedule to each child node
> * Clients point to their nearest Pulp server
>   * This is done via intelligent DNS (F5 BIG-IP GTM) that hands out the IP address for the nearest Pulp server depending on the source of the DNS query.
> 
> I don't see a need to have more than one Pulp server in any given datacenter.  One server can easily handle the load for one datacenter.  If it goes down, our BIG-IP device notices the failure and starts handing out another Pulp server that is healthy in another datacenter.  Our datacenters are very well connected so bandwidth is not a concern.  This scenario requires no shared storage or fancy/complicated clustering.
> 
> Also, Pulp, as of now, will not be able to handle Debian based repositories.
> 
> Thanks,
> 
> Josh
> 
> -----Original Message-----
> From: pulp-list-bounces at redhat.com [mailto:pulp-list-bounces at redhat.com] On Behalf Of Arnold Bechtoldt
> Sent: Wednesday, November 27, 2013 3:07 PM
> To: pulp-list at redhat.com
> Subject: [Pulp-list] High Available Pulp Setup
> 
> Hey,
> 
> we want to setup Pulp with a two-side HA concept.
> 
> There will be two servers in each of two DCs. Two per DC to do a fast failover within a DC, another (identically configured) two in the second DC to be able two work when the first DC is completly down.
> 
> Repositories to be mirorred:
> 
> * RHEL server with additional repositories/channels
> * EPEL
> * Foreman (low prio)
> * Puppet Labs (yum.puppetlabs.com)
> * rpm repos of some hardware vendors
> * rpm repos of some software community projects
> * several rpm repos of own software
> 
> and the same required for Ubuntu and maybe SLES (ASAP).
> 
> 
> Geo-redundant SAN (both DCs) via NFS is available.
> 
> When I understood Pulp correctly, Pulp requires mainly httpd with mod_wsgi, mongodb and storage (/var/lib/pulp/contents) for pulp-server and any host for pulp-admin. pulp-consumer is currently not planned for use.
> 
> 
> Besides to the node feature there are no docs concerning pulp HA on the web (or PEBKAC) - I would add some as soon I am able to.
> 
> We have tested Pulp to mirror the repos mentioned above and cloned some, too.
> Some questions remained be open:
> 
> * do I need 4 x independent storage space?
> * do I have to manage 2 or 4 pulp servers with the same content/sync-tasks/clone-tasks? note: every server must be able to provide current mirrors of upstream in a short time (5-10 min) after a failover
> * is it a expected behaviour that pulp doesn't re-download missing contents to /var/lib/pulp/contents/ of a repo (intentionally removed some)?
> * is there a way to import contents of a repo (mirror) in another pulp server with the same repo settings/parameters?
> * does a mongodb replication (master->3 x slave) make sense?
> 
> notice: Pulp needs to be run on only one system at the same time.
> Active/Active over both DCs isn't a must. The release of packages of the most important mirrors to the consuming hosts will be staged.
> 
> 
> Thank you for developing Pulp and giving your ideas to this topic.
> 
> 
> Arnold
> 
> --
> Arnold Bechtoldt
> IT Engineering & Operations
> 
> inovex GmbH
> 
> Zur Gießerei 16
> D-76227 Karlsruhe
> Tel: 07231 31 91 0
> Fax: 07231 31 91 91
> Mobil: 0173 3181 117
> arnold.bechtoldt at inovex.de
> www.inovex.de
> 
> Sitz der Gesellschaft: Pforzheim
> AG Mannheim, HRB 502126
> Geschäftsführer: Stephan Müller
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 552 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20131127/1dd04cc1/attachment.sig>