[Pulp-list] Using pulp to create monthly releases

[Barnaby Court]

Hi Mike,

When you say it's a full day effort right now, what is it that is causing it to be a full day effort? Is it personal involvement or the amount of time that Pulp is spending on processing? If it is personal involvement I would highly recommend repo groups so that you can configure all the repositories you are mirroring once and then export them in one action. If it Pulp processing time it would be helpful to understand how many repositories, how many units of each type (roughly) in each repository and which parameters you have enabled on all your repositories. Some parameters greatly increase the processing time. The biggest is generating the sqlite files. If that is turned on it can greatly increase the time required to publish. Also, is your internet connected Pulp server using local disk for the /var/lib/pulp directory or is it using NFS? Thanks!

-Barnaby

----- Original Message -----
From: "Mike Griffin" <mgriffin at griftech.net>
To: "Randy Barlow" <rbarlow at redhat.com>
Cc: pulp-list at redhat.com
Sent: Friday, May 15, 2015 4:42:39 PM
Subject: Re: [Pulp-list] Using pulp to create monthly releases

What I'm struggling with is how I can export new/updated packages of a repo on my internet-connected pulp server and on the downstream server re-create the full release-repo, using the combination of existing packages on the downstream server and the updates from the upstream server. This task would be easy if I could just export the entire release-repo every month and import it downstream. It's just that due to the number of distros we support, that gets to be unwieldy very quickly. Even with just the new/updated packages every month, it's a full day effort now. 



On Fri, May 15, 2015 at 2:21 PM, Randy Barlow < rbarlow at redhat.com > wrote: 


-----BEGIN PGP SIGNED MESSAGE----- 
Hash: SHA512 

On 05/14/2015 07:21 PM, Mike Griffin wrote: 
> Maybe I can illustrate my requirements a little easier. For 
> example: During my first month of April, package A-1.0, B-1.0, and 
> C-1.0 are downloaded via the nightly sync. When I create my 
> release, all 3 packages appear in my export and I move them 
> downstream. 
> 
> The next month when I create my release-set, package A is updated 
> to version 2.0, B is static, C is removed upstream and D1.0 is 
> released. Since I have my nightly sync set to "remove-missing true" 
> and "retain-old-count 0", A1.0 and C1.0 should be removed from the 
> sync repo and therefore removed from the release-set. I want to 
> move A2.0 and D1.0 downstream, copy B1.0 that already exists 
> downstream and re-create the release directory there (A2.0, B1.0, 
> D1.0). 

Hi Mike, 

Perhaps you can achieve the above by removing all packages from the 
destination repo, and then copying all packages from the source. So 
long as no publish happens in-between, the final publish will 
atomically switch (i.e., no clients will ever see the empty repo). 
Does that make sense? 

- -- 
Randy Barlow 
-----BEGIN PGP SIGNATURE----- 
Version: GnuPG v2 

iQIcBAEBCgAGBQJVVjlBAAoJEIyFaKUJtmpicv8P/3lbQ52FTs9YXzyhb80gxCwV 
8NwqW3y0ZVKjyK4ZK1ML9J/26ig7Bz4e4rgBvjJokuzcI2AdyoPCJtuGcxucFa0l 
DZA0T5VtNlpggEET9LoVeb23C9b/mQma54NmVkuzUDBlyyqDAHL+zK11Hs+SYp3/ 
LSq+p/QT6TKYI/GqjoTVZYGfKpVvpjt4GBERP6+K4YRhKD2x3m7R0zlpQQZ2++aq 
JN/+UCGuPjEBxXUZOIsLLUUuTgswWwih1Qghs3ZCxxotoSw2IfMXmf3Zl4W+0TS8 
U46lh9CKavmPx01e7fcPmeZlObZBpSpFwqFp7PFzmOeWAwEWxTLptyCT618/RAKM 
KvXY0aYT2DWGQ9MQFiXr93MoB4+Ckc7DHjv/DW2b/9rKGp7T4HF/Y9BzEwzDE1ua 
wArN8fIRXfSJWbdbOldl97i8SSroagTrqrbnpRtxa4CBdgc7i+36Ct6RnCSq59OL 
srFTEmwap78x9N9AntjaWUhFbNlai54VU8AYbkPavVOP0QLz8Vzgsaj3SJ+2dBJ7 
xTEhQRbyoVnSu3FDNY1AkQP9Rr0ZlQSUYBgVhXWhwaZU9breltMMGfX3PkC0M8iu 
jj/K6MmKPwp2fhn7TmvWDDaBnwPJqvAqGlKhUXlHN3vGkoVAngcoPvaThz/td2Na 
MWR1e2U5y8NrwJ/ml9QA 
=yMlk 
-----END PGP SIGNATURE----- 

_______________________________________________ 
Pulp-list mailing list 
Pulp-list at redhat.com 
https://www.redhat.com/mailman/listinfo/pulp-list 



-- 
Regards, 
Mike Griffin 

Ever grateful, ever true 

_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list