From jortel at redhat.com Tue Feb 1 15:04:27 2011 From: jortel at redhat.com (Jeff Ortel) Date: Tue, 01 Feb 2011 09:04:27 -0600 Subject: [Pulp-list] gofer 0.15 Message-ID: <4D4820FB.2080008@redhat.com> Pulp has been updated to require gofer 0.15. This is due to a plugin descriptor change. Under [messaging], the 'enabled' property has been removed and 'url', 'cacert' & 'clientcert' are now supported. Also, the gofer-lib package has been renamed to python-gofer and ruby-gofer has been added. From jason.dobies at redhat.com Tue Feb 1 18:14:09 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Tue, 01 Feb 2011 13:14:09 -0500 Subject: [Pulp-list] New Pulp installation - unknown errors In-Reply-To: <4D46E904.7080009@redhat.com> References: <4D401E39.4050405@redhat.com> <260775611.288418.1296055729310.JavaMail.root@mail-2.01.com> <4D46E904.7080009@redhat.com> Message-ID: <4D484D71.2070602@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 FYI, these CentOS instructions can also be found on pulpproject.org: http://pulpproject.org/ug/UGInstallCentOS.html Notes have been added to the installation guide to indicate their need on CentOS installations as well. On 01/31/2011 11:53 AM, Jeff Ortel wrote: > Kevin and Christian, > > The FAQ [1] has been updated with instructions for building the QPID > rpms from source on (and for) CentOS. If you run into any issues or > have any questions, please let me know. > > Hope this helps. > > Regards, > > Jeff > > [1] > https://fedorahosted.org/pulp/wiki/UGFAQ#HowdoIbuildtheQPIDrpmsforCentOS5 > > > > On 01/26/2011 01:38 PM, Masopust, Christian wrote: >> >> Hello Kevin, >> >> yes, you're right, Scientific Linux (the "CERN-distro") contains them. >> I'd pull the source-rpms >> for my RHEL6 system from there and build it myself. >> >> christian >> >> -----Urspr?ngliche Nachricht----- >> Von: pulp-list-bounces at redhat.com >> [mailto:pulp-list-bounces at redhat.com] Im Auftrag von Kevin Holmes >> Gesendet: Mittwoch, 26. J?nner 2011 16:29 >> An: pulp-list at redhat.com >> Betreff: Re: [Pulp-list] New Pulp installation - unknown errors >> >> I can confirm the Mongo directory problem happens with CentOS 5.5 - >> however that was fixed quickly thanks to your FAQ :) >> >> Another problem is the lack of a suitable CentOS repository for >> qpid-cpp-server* packages. I believe the CERN distribution includes >> them, but CentOS does not. >> >> I haven't made any progress on this, looking forward to seeing what >> Jay finds. >> >> - Kevin >> >> ----- Original Message ----- >> From: "Jay Dobies" >> To: pulp-list at redhat.com >> Sent: Wednesday, January 26, 2011 8:14:33 AM >> Subject: Re: [Pulp-list] New Pulp installation - unknown errors >> > What Christian mentioned is a potential problem; some packagings of > Mongo do not correctly install. Check out this entry on the FAQ: > http://pulpproject.org/ug/UGFAQ.html#MongodwontstartwheninstalledfromEL5 > > I'm going to set up a CentOS guest this morning and see if there are any > steps specific to CentOS that need to be taken. > > > On 01/25/2011 05:56 PM, Kevin Holmes wrote: >>>> >>>> For Python: >>>> >>>> Python 2.4.3 (#1, Nov 11 2010, 13:30:19) >>>> [GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2 >>>> >>>> Operating system is CentOS: >>>> >>>> Linux blueberry 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST >>>> 2011 x86_64 x86_64 x86_64 GNU/Linux >>>> >>>> I just updated to the testing repository and ran a yum update. That >>>> was successful, the error has changed now: >>>> >>>> [root at blueberry log]# pulp-admin auth login -u admin -p admin >>>> error: operation failed:>>> 2.0//EN"> >>>> >>>> 500 Internal Server Error >>>> >>>>

Internal Server Error

>>>>

The server encountered an internal error or >>>> misconfiguration and was unable to complete >>>> your request.

>>>>

Please contact the server administrator, >>>> root at localhost and inform them of the time the error occurred, >>>> and anything you might have done that may have >>>> caused the error.

>>>>

More information about this error may be available >>>> in the server error log.

>>>>

Apache/2.2.3 (CentOS) Server at blueberry Port 443

>>>> >>>> >>>> Apache Logs: >>>> >>>> ==> ssl_access_log<== >>>> 127.0.0.1 - - [25/Jan/2011:17:31:22 -0500] "GET >>>> /pulp/api//users/admin_certificate/ HTTP/1.1" 500 609 >>>> >>>> ==> ssl_error_log<== >>>> [Tue Jan 25 17:33:22 2011] [error] [client 127.0.0.1] Premature end >>>> of script headers: webservices.wsgi >>>> >>>> ==> ssl_request_log<== >>>> [25/Jan/2011:17:31:22 -0500] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "GET >>>> /pulp/api//users/admin_certificate/ HTTP/1.1" 609 >>>> >>>> ==> access_log<== >>>> 127.0.0.1 - - [25/Jan/2011:17:34:01 -0500] "GET / HTTP/1.1" 200 1301 >>>> "-" "check_http/v1.4.15 (nagios-plugins 1.4.15)" >>>> >>>> Just using curl gives me the same outcome: >>>> >>>> [root at blueberry pulp]# curl -k -u admin:admin >>>> https://blueberry/pulp/api/repositories >>>> >>>> >>>> 500 Internal Server Error >>>> >>>>

Internal Server Error

>>>>

The server encountered an internal error or >>>> misconfiguration and was unable to complete >>>> your request.

>>>>

Please contact the server administrator, >>>> root at localhost and inform them of the time the error occurred, >>>> and anything you might have done that may have >>>> caused the error.

>>>>

More information about this error may be available >>>> in the server error log.

>>>>

Apache/2.2.3 (CentOS) Server at blueberry Port 443

>>>> >>>> >>>> Apache logs: >>>> >>>> ==> ssl_access_log<== >>>> 127.0.0.1 - - [25/Jan/2011:17:41:14 -0500] "GET >>>> /pulp/api/repositories HTTP/1.1" 500 609 >>>> >>>> ==> ssl_error_log<== >>>> [Tue Jan 25 17:43:14 2011] [error] [client 127.0.0.1] Premature end >>>> of script headers: webservices.wsgi >>>> >>>> ==> ssl_request_log<== >>>> [25/Jan/2011:17:41:14 -0500] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "GET >>>> /pulp/api/repositories HTTP/1.1" 609 >>>> >>>> >>>> So it's looking like that probably fixed the original problem. But >>>> now I may have hit some kind of configuration issue? The errors here >>>> are not very descriptive. Your help is appreciated. I've gone through >>>> the install page and followed the directions as closely as possible. >>>> >>>> - Kevin >>>> >>>> ----- Original Message ----- >>>> From: "Jay Dobies" >>>> To: pulp-list at redhat.com >>>> Sent: Tuesday, January 25, 2011 4:06:19 PM >>>> Subject: Re: [Pulp-list] New Pulp installation - unknown errors >>>> >>>> This looks like a python version issue. What version of python are you >>>> using? What operating system? >>>> >>>> If it's python 2.4, we've only just sorted these issues out in our >>>> testing build (built yesterday, as compared to the community release >>>> which does not contain the fixes). >>>> >>>> The testing repository is hosted along side the community release (be >>>> sure to append the appropriate os/arch to the following URL): >>>> http://repos.fedorapeople.org/repos/pulp/pulp/testing/ >>>> >>>> And the installation instructions: >>>> https://fedorahosted.org/pulp/wiki/UGInstallation >>>> >>>> Please let me know if you keep running into issues. >>>> >>>> >>>> >>>> On 01/25/2011 04:01 PM, Kevin Holmes wrote: >>>>> Hi everyone, >>>> >>>>> New Pulp install and I'm having a problem. >>>> >>>>> I cannot seem to issue any commands with the pulp cli tools. >>>> >>>>> For instance: >>>> >>>>> [root at blueberry pub]# pulp-admin auth login -u admin -p admin >>>>> Traceback (most recent call last): >>>>> File "/usr/bin/pulp-admin", line 147, in ? >>>>> admin.main() >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/cli/base.py", >>>>> line >>>>> 98, in main >>>>> command.main(args[1:]) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/core/base.py", >>>>> line >>>>> 118, in main >>>>> action.main(args[1:]) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/core/base.py", >>>>> line >>>>> 229, in main >>>>> self.run() >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/core/auth.py", >>>>> line >>>>> 47, in run >>>>> cert_dict = self.authconn.admin_certificate() >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/connection.py", >>>>> line 671, in admin_certificate >>>>> return self.conn.request_get(method) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/connection.py", >>>>> line 126, in request_get >>>>> return self._request("GET", method) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/connection.py", >>>>> line 114, in _request >>>>> self.validateResponse(response) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/connection.py", >>>>> line 122, in validateResponse >>>>> raise RestlibException(response.status, response.read()) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/connection.py", >>>>> line 50, in __init__ >>>>> super(RestlibException, self).__init__(code, msg) >>>>> TypeError: super() argument 1 must be type, not classobj >>>> >>>>> Corresponding Apache logs: >>>> >>>>> ==> ssl_access_log<== >>>>> 127.0.0.1 - - [25/Jan/2011:15:46:47 -0500] "GET >>>>> /pulp/api//users/admin_certificate/ HTTP/1.1" 500 609 >>>> >>>>> ==> ssl_error_log<== >>>>> [Tue Jan 25 15:48:47 2011] [error] [client 127.0.0.1] Premature end of >>>>> script headers: webservices.wsgi >>>> >>>>> ==> ssl_request_log<== >>>>> [25/Jan/2011:15:46:47 -0500] 127.0.0.1 TLSv1 DHE-RSA-AES256-SHA "GET >>>>> /pulp/api//users/admin_certificate/ HTTP/1.1" 609 >>>> >>>> >>>>> Running user update also fails, but nothing shows up in my Apache logs. >>>> >>>>> [root at blueberry pub]# pulp-admin user update >>>>> Traceback (most recent call last): >>>>> File "/usr/bin/pulp-admin", line 147, in ? >>>>> admin.main() >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/cli/base.py", >>>>> line >>>>> 98, in main >>>>> command.main(args[1:]) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/core/base.py", >>>>> line >>>>> 118, in main >>>>> action.main(args[1:]) >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/core/base.py", >>>>> line >>>>> 228, in main >>>>> self.setup_connections() >>>>> File "/usr/lib/python2.4/site-packages/pulp/client/core/user.py", >>>>> line >>>>> 37, in setup_connections >>>>> system_exit(-1, ce.message) >>>>> AttributeError: CredentialError instance has no attribute 'message' >>>> >>>>> So far this seems like a really cool project. I hope to get it working >>>>> on our systems in the future. >>>> >>>>> - Kevin >>>> >>>> >>>> >>>>> _______________________________________________ >>>>> Pulp-list mailing list >>>>> Pulp-list at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/pulp-list >>>> >>>> > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list > > >> _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list >> _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list >> _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNSE1xAAoJEOMmcTqOSQHC9CcIAIi0nFXbazDfk9llSNAXayVQ IR/F8GkP3lT/FlK8zi01EehzOqv48MmREiwblHzzSNlJtmWFBz+9mCCy20mB6l4i 4g43Ee1kcM6APYAm92+rbTwNSTNJBRVgIwjDJfzue8ngfG0+Kodrd1eyP4diFbez pkYEyn7JjrAWpyXor1dXPKfK8R3pzVZ9rM1qFsjBg6HvJu6jYuXFNsTOOl90lmg3 agKXC25KcDS9j6rIbs1BuwaVfsKatp5H2aNH//lyk7ijFAtCGpsBy7bX6iA3nqRp fxdhuWaho3Ie5+uDtGUd13xxjkX4W8v77Gk2yKA9ydbWmw3INUuIoA/YGno8gOk= =hMwW -----END PGP SIGNATURE----- From jmatthew at redhat.com Tue Feb 1 19:37:07 2011 From: jmatthew at redhat.com (John Matthews) Date: Tue, 1 Feb 2011 14:37:07 -0500 (EST) Subject: [Pulp-list] repo["packages"] change pushed and pulp-migrate --force required Message-ID: <1767232867.47576.1296589027065.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Changes have been pushed for the repository data model. "packages" is now a list and not a dictionary. The pulp-migrate script has been updated to fix existing data. If you are running pulp from git you will need to re-run pulp-migrate: sudo /usr/bin/pulp-migrate --force If you see any problems please let me know From jason.dobies at redhat.com Wed Feb 2 15:39:07 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 02 Feb 2011 10:39:07 -0500 Subject: [Pulp-list] Dates Message-ID: <4D497A9B.4090001@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Consumer history is completely broken because the CLI fails to parse the timestamp. I looked through the diff for json_utils.py and found: - - Parses the pymongo.json_util encoding of a datetime instance. The instance is - - encoded as a 64-bit unsigned integer for milliseconds since epoch, according to - - JSON strict mode. This method also assumes the strict mode packaging of a dict - - using $date as the key for this integer. + Parses the unit timestamp string - - Example: - - { "$date" : 1283505509571} - - - - @param date_string: JSON encoded datetime instance + @param date_string: unix timestamp string @type date_string: dict with the date integer stored at $date @return: python object representing the date @rtype: L{datetime.datetime} instance ''' - - return datetime.utcfromtimestamp(float(date_string["$date"]) / 1000.0) \ No newline at end of file + return datetime.utcfromtimestamp(float(date_string)) There are a few issues: - - The method was changed to take the string itself and not a dict. This is fine, but the param docs were not updated to reflect that (they still say to pass in a dict) and existing calls into this method were not changed. - - Why were the docs describing the format of the datetime instance removed in favor of a less-useful "Parses the unit timestamp string"? It's one thing for us to have poor docs, its another to actively work towards removing the few docs we already have. - - Why was the actual logic change made? Even if I fix the call into this from consumer history, it cannot parse the date. Is there a different way we need to store it in the DB? This is what I get with this change that removes the division by 1000: File "/home/jdob/code/pulp/src/pulp/client/json_utils.py", line 32, in parse_date return datetime.utcfromtimestamp(float(date_string)) ValueError: year is out of range Can we PLEASE try to be careful when changing method signatures like this? A simple grep to see if any calls were passing in a dict would have prevented the entire consumer history feature from breaking. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNSXqbAAoJEOMmcTqOSQHCuMMH/jaE+Fd5+unayP036S+gT8fq uIs7qLNPo4oHUl4rIVkIAHeBiBno1j7D7XSpAzEHhbQUAkCm3Y47WNMgq5F7C/Th RnmCv7ZCY2b2W7C10rJ+Lhtn0gaNtj7+pCBC9LF0LFMV/WhAAtS51SixJ+xRvm5B oI+WpbZa6vYhh9kl3iETfVnhGi0mQVk0iAqwo9gI1ycsQ8c9+O7UeSixPg3mb0tT Q3H53Md6KaGxT/9XC84tEwx8cdk5tOKoID+K3aMGJtNRzDbaMyz8wQXrMBhoPxLF 2658XGAr/nauBFaDF8vfJ6HyDP15TG1rEnn9SMSrlTPUf9QxZXuwD/uj9hOKlwg= =dmnr -----END PGP SIGNATURE----- From jason.dobies at redhat.com Wed Feb 2 20:07:12 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 02 Feb 2011 15:07:12 -0500 Subject: [Pulp-list] New Testing Build: 0.134 Message-ID: <4D49B970.2020209@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 = Versions = Pulp: 0.134 Grinder: 0.77 Gofer: 0.16 - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNSblwAAoJEOMmcTqOSQHCheQH/2XGNI1KEZ6fX8y+r1FGVMeO t0AgMohC3GrVkVegjFirsARAAGMAnJKYmX/0e2emWxkaSLH1CzueRNOc7JzUnUyL JS63aT16PE7R5pSNZzq8G4JZfzlfmBqi6xIKbVmavGQ6Nkkd9bMrNrzhcHSzY0xU +HsGSEOfdFe3LGlAf5q99cAwCK/iiGfwzC45asMRdM5lKB1lIzjor+Q4s0DkykPP YEfHjCQHs5aScWCx2LoQf+dWdwPRTr3cIY5jd2DFcCQicLZvonpmoD521xi90gcc 4JvVMdtUEVRVG89EUpzDFvqhumcV5knI7hKEoyfbSHMUbEbcRdnqHy+1TBSOSfM= =A7jX -----END PGP SIGNATURE----- From jason.dobies at redhat.com Wed Feb 2 20:49:46 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 02 Feb 2011 15:49:46 -0500 Subject: [Pulp-list] New Gofer Build: 0.17 Message-ID: <4D49C36A.4080701@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Had to do a second build to pick up a new version of gofer (the previous had upgrade issues due to renamed RPMs). I haven't tested this so reply back with any issues. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNScNqAAoJEOMmcTqOSQHCx5QH/0bwyH5CMozHppaT3g57RO57 ezgC/+IkAK0LBZ1WJV3vq53gV5SmNa2gvsLXA83ANaVLSC0nijAC2Fs3pbPmdCQ8 7S2nLQHoL3KNprcy/auneDKepFkvqINMeZ97lL7BDInY4lQ+5wtihxldAE/z1wdX pClK8SDyYOohOJeCYrV2g/zEvy2s58DfLe/QWRaznA/SE/dyX42zyI8NjfTtlm4z 80JZIOjJbtVZoO4MXzaP6pSm2Zn6r/4+/O5BMQU6SDTZJtblpcs0q6sET90lov2v M7/8KCklE8h3X7Ige6wjIiuFStzIGU+hwUl6f6ifmeFQ8qqsgfd6vn3zZta7MqI= =7AXg -----END PGP SIGNATURE----- From jortel at redhat.com Wed Feb 2 21:43:29 2011 From: jortel at redhat.com (Jeff Ortel) Date: Wed, 02 Feb 2011 15:43:29 -0600 Subject: [Pulp-list] Proof UG Additions Message-ID: <4D49D001.3060509@redhat.com> All, I noticed that the 'repo update' and 'repo listkeys' commands are missing from the User Guide so I added them. Please review [1] at your convenience. Thanks, Jeff [1] https://fedorahosted.org/pulp/wiki/UGRepo#crud From jason.dobies at redhat.com Thu Feb 3 15:26:13 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 03 Feb 2011 10:26:13 -0500 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D49D001.3060509@redhat.com> References: <4D49D001.3060509@redhat.com> Message-ID: <4D4AC915.4060005@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/02/2011 04:43 PM, Jeff Ortel wrote: > All, > > I noticed that the 'repo update' and 'repo listkeys' commands are > missing from the User Guide so I added them. Please review [1] at your > convenience. > > Thanks, > > Jeff > > [1] https://fedorahosted.org/pulp/wiki/UGRepo#crud Thanks for doing this. A few questions on behavior based on reading it: - - It says we can update the symlinks flag on a repo. What happens if the repo was already synchronized? Will we remove the packages from the repo and replace them with symlinks? Will future syncs on that repo result in a hybrid of full packages and symlinks? - - When changing the relative path, is that something we need to broadcast out to all bound consumers so they can update their .repo files? If it is, that's a bug; we don't have any hooks in the update API method. - - You can update a group ID with --groupid, but how do you remove it from a group? - - When specifying to remove GPG keys, the docs say to specify the GPG key file. What if they don't have the file anymore, they can't remove that GPG key? - - When changing an existing --cacert, --cert, or --key is the old one deleted from the Pulp server? Looking at the code not only do we not delete the old ones, it doesn't look like we write the new ones to disk. Can someone more familiar with that area confirm this is a bug? - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNSskVAAoJEOMmcTqOSQHC2LoH/jE61DMQMay+ppxgsd/iQe9a LfQBni8gvGnAZ/e3XPYeqpElo6SiL5n2Q9C/+hVUuVGiCf2UsByR3RYBCwaDSP0B AUijbF2EJnrjYy1JXCJBQpuusV/fg3ZhoIU5djoi1E7P/xxHOKB7OP88jN1Z7g+P 4CiicYWOe/bcood357Je5Z+W2ZcUjIqtGDIiuEbbMeI+1AYhxGYZBBYbcprl+znl sO4fA5SgAJIrlwTmbBn/f//1A4IZ5nuagoxFm/92HuNAsuQa6omEjQjeljL/jvHa cwA5TT9tqTfElrPKps7A2e+91KLxXGwu8IphwNLjVeoriLQhGoepbxAjOSm5MSI= =xNhh -----END PGP SIGNATURE----- From jortel at redhat.com Thu Feb 3 16:05:51 2011 From: jortel at redhat.com (Jeff Ortel) Date: Thu, 03 Feb 2011 10:05:51 -0600 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D4AC915.4060005@redhat.com> References: <4D49D001.3060509@redhat.com> <4D4AC915.4060005@redhat.com> Message-ID: <4D4AD25F.3090304@redhat.com> On 02/03/2011 09:26 AM, Jay Dobies wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 02/02/2011 04:43 PM, Jeff Ortel wrote: >> All, >> >> I noticed that the 'repo update' and 'repo listkeys' commands are >> missing from the User Guide so I added them. Please review [1] at your >> convenience. >> >> Thanks, >> >> Jeff >> >> [1] https://fedorahosted.org/pulp/wiki/UGRepo#crud > > Thanks for doing this. A few questions on behavior based on reading it: Thanks for the review. Between writing this up and working on https://bugzilla.redhat.com/show_bug.cgi?id=664040, I found that we permit changes to the repo that really breaks the repo in many ways. Assuming there are no valid use cases for changing the relative path after the repo has been populated, I propose following restrictions on 'repo update'. If the repo has been populated (synced): --relativepath (option rejected) --symlinks (option rejected) --feed (rejected if relative path (part) changed) The main use case for changing the feed is that it was entered incorrectly when the repo was created. User tries to sync and it barfs. Then, the user corrects the feed URL, retries the sync and all is good. A secondary use case is that the user wants to sync the repo from a different mirror. To support this, we permit the --feed so long as the relative path remains the same. > > - - It says we can update the symlinks flag on a repo. What happens if the > repo was already synchronized? Will we remove the packages from the repo > and replace them with symlinks? Will future syncs on that repo result in > a hybrid of full packages and symlinks? Probably. > > - - When changing the relative path, is that something we need to > broadcast out to all bound consumers so they can update their .repo > files? If it is, that's a bug; we don't have any hooks in the update API > method. Yup, this is busted. > > - - You can update a group ID with --groupid, but how do you remove it > from a group? > > - - When specifying to remove GPG keys, the docs say to specify the GPG > key file. What if they don't have the file anymore, they can't remove > that GPG key? I can update the doc. Basically the user uses 'repo listkeys' and specifies one of the file names listed. > > - - When changing an existing --cacert, --cert, or --key is the old one > deleted from the Pulp server? Looking at the code not only do we not > delete the old ones, it doesn't look like we write the new ones to disk. > Can someone more familiar with that area confirm this is a bug? > > > - -- > Jay Dobies > RHCE# 805008743336126 > Freenode: jdob > http://pulpproject.org > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJNSskVAAoJEOMmcTqOSQHC2LoH/jE61DMQMay+ppxgsd/iQe9a > LfQBni8gvGnAZ/e3XPYeqpElo6SiL5n2Q9C/+hVUuVGiCf2UsByR3RYBCwaDSP0B > AUijbF2EJnrjYy1JXCJBQpuusV/fg3ZhoIU5djoi1E7P/xxHOKB7OP88jN1Z7g+P > 4CiicYWOe/bcood357Je5Z+W2ZcUjIqtGDIiuEbbMeI+1AYhxGYZBBYbcprl+znl > sO4fA5SgAJIrlwTmbBn/f//1A4IZ5nuagoxFm/92HuNAsuQa6omEjQjeljL/jvHa > cwA5TT9tqTfElrPKps7A2e+91KLxXGwu8IphwNLjVeoriLQhGoepbxAjOSm5MSI= > =xNhh > -----END PGP SIGNATURE----- > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Thu Feb 3 16:38:38 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 03 Feb 2011 11:38:38 -0500 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D4AD25F.3090304@redhat.com> References: <4D49D001.3060509@redhat.com> <4D4AC915.4060005@redhat.com> <4D4AD25F.3090304@redhat.com> Message-ID: <4D4ADA0E.70905@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> Between writing this up and working on >> https://bugzilla.redhat.com/show_bug.cgi?id=664040, I found that we >> permit changes to the repo that really breaks the repo in many ways. >> Assuming there are no valid use cases for changing the relative path >> after the repo has been populated, I propose following restrictions on >> 'repo update'. > >> If the repo has been populated (synced): > >> --relativepath (option rejected) > >> --symlinks (option rejected) > >> --feed (rejected if relative path (part) changed) > >> The main use case for changing the feed is that it was entered >> incorrectly when the repo was created. User tries to sync and it >> barfs. Then, the user corrects the feed URL, retries the sync and all >> is good. I agree with the main use case and I like the idea of "you can make more changes before you do the initial sync". The feed/relativepath relationship you described may be confusing to users. We may want to make it simpler and more coarse-grained by saying they have to make sure to lock in all their values before synchronizing. If it really becomes an issue in the future we can address it. >> A secondary use case is that the user wants to sync the repo from a >> different mirror. To support this, we permit the --feed so long as the >> relative path remains the same. Ahh, ok. That's a good use case and covered by your rules. Lock in the relative path before synchronizing, but you can change the feed later. What about changing feed types? I can't think of any issues in the feed being a whole new type on an update, but figured I'd throw it out there. > - It says we can update the symlinks flag on a repo. What happens if > the > repo was already synchronized? Will we remove the packages from the repo > and replace them with symlinks? Will future syncs on that repo result in > a hybrid of full packages and symlinks? > >> Probably. Ok, that's bad, but will be covered by your rules above. > - When changing the relative path, is that something we need to > broadcast out to all bound consumers so they can update their .repo > files? If it is, that's a bug; we don't have any hooks in the update API > method. > >> Yup, this is busted. Is there a bug on it? > - You can update a group ID with --groupid, but how do you remove it > from a group? I'm going to guess this is a bug and I'll file it. > - When specifying to remove GPG keys, the docs say to specify the GPG > key file. What if they don't have the file anymore, they can't remove > that GPG key? > >> I can update the doc. Basically the user uses 'repo listkeys' and >> specifies one of the file names listed. Cool, sounds good. I just double checked and the --help is correct, so it should just be a wiki fix. > - When changing an existing --cacert, --cert, or --key is the old one > deleted from the Pulp server? Looking at the code not only do we not > delete the old ones, it doesn't look like we write the new ones to disk. > Can someone more familiar with that area confirm this is a bug? I'm going to file this as a bug too. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNStoOAAoJEOMmcTqOSQHCVzgIALOxA4MqVrPcjKWH/IqrN4uy dCQR4CdkRS3O5hJ29PUKnFZHVgkmCwMZUwmuGCzDnmDvYxMSP3q8NrCEzpLwafo+ 5jmyo9qoqkpQNQ7ZxVvEYVghilp+WEXZqteOu1Yh0lm+B4QwcXoRiImFT/vMnN+s wmKlYkTFuzEJS99iJeD6+4kw8BH2/5vEvxOigtvwsJ0/b2k/V/9MwvcT9FHGKJNc jsPNlJzeK7GmkYhht1CUvVK0isJruCYX9YtF6M8xLUrEEKav1VMFn8/9LRbCqQlk 6z0Cow0918dzNbnyFKGxgDNXFHQvWNo7uMAqkYYftL8q67kPHQ6lj9qiQYsk2KA= =aqfb -----END PGP SIGNATURE----- From jortel at redhat.com Thu Feb 3 17:03:13 2011 From: jortel at redhat.com (Jeff Ortel) Date: Thu, 03 Feb 2011 11:03:13 -0600 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D4ADA0E.70905@redhat.com> References: <4D49D001.3060509@redhat.com> <4D4AC915.4060005@redhat.com> <4D4AD25F.3090304@redhat.com> <4D4ADA0E.70905@redhat.com> Message-ID: <4D4ADFD1.10001@redhat.com> On 02/03/2011 10:38 AM, Jay Dobies wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > >>> Between writing this up and working on >>> https://bugzilla.redhat.com/show_bug.cgi?id=664040, I found that we >>> permit changes to the repo that really breaks the repo in many ways. >>> Assuming there are no valid use cases for changing the relative path >>> after the repo has been populated, I propose following restrictions on >>> 'repo update'. >> >>> If the repo has been populated (synced): >> >>> --relativepath (option rejected) >> >>> --symlinks (option rejected) >> >>> --feed (rejected if relative path (part) changed) >> >>> The main use case for changing the feed is that it was entered >>> incorrectly when the repo was created. User tries to sync and it >>> barfs. Then, the user corrects the feed URL, retries the sync and all >>> is good. > > I agree with the main use case and I like the idea of "you can make more > changes before you do the initial sync". The feed/relativepath > relationship you described may be confusing to users. We may want to > make it simpler and more coarse-grained by saying they have to make sure > to lock in all their values before synchronizing. If it really becomes > an issue in the future we can address it. > >>> A secondary use case is that the user wants to sync the repo from a >>> different mirror. To support this, we permit the --feed so long as the >>> relative path remains the same. > > Ahh, ok. That's a good use case and covered by your rules. Lock in the > relative path before synchronizing, but you can change the feed later. > > What about changing feed types? I can't think of any issues in the feed > being a whole new type on an update, but figured I'd throw it out there. > > >> - It says we can update the symlinks flag on a repo. What happens if >> the >> repo was already synchronized? Will we remove the packages from the repo >> and replace them with symlinks? Will future syncs on that repo result in >> a hybrid of full packages and symlinks? >> >>> Probably. > > Ok, that's bad, but will be covered by your rules above. > >> - When changing the relative path, is that something we need to >> broadcast out to all bound consumers so they can update their .repo >> files? If it is, that's a bug; we don't have any hooks in the update API >> method. >> >>> Yup, this is busted. > > Is there a bug on it? No. I originally thought this was covered by the rules (above) but I guess a user could: 1) create the repo 2) bind consumer(s) .... pulp.repo created on consumers ..... 3) update the feed or relative path 4) sync. So the pulp.repo file will be wrong. I'll add a bug. > >> - You can update a group ID with --groupid, but how do you remove it >> from a group? > > I'm going to guess this is a bug and I'll file it. > >> - When specifying to remove GPG keys, the docs say to specify the GPG >> key file. What if they don't have the file anymore, they can't remove >> that GPG key? >> >>> I can update the doc. Basically the user uses 'repo listkeys' and >>> specifies one of the file names listed. > > Cool, sounds good. I just double checked and the --help is correct, so > it should just be a wiki fix. > >> - When changing an existing --cacert, --cert, or --key is the old one >> deleted from the Pulp server? Looking at the code not only do we not >> delete the old ones, it doesn't look like we write the new ones to disk. >> Can someone more familiar with that area confirm this is a bug? > > I'm going to file this as a bug too. > > - -- > Jay Dobies > RHCE# 805008743336126 > Freenode: jdob > http://pulpproject.org > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJNStoOAAoJEOMmcTqOSQHCVzgIALOxA4MqVrPcjKWH/IqrN4uy > dCQR4CdkRS3O5hJ29PUKnFZHVgkmCwMZUwmuGCzDnmDvYxMSP3q8NrCEzpLwafo+ > 5jmyo9qoqkpQNQ7ZxVvEYVghilp+WEXZqteOu1Yh0lm+B4QwcXoRiImFT/vMnN+s > wmKlYkTFuzEJS99iJeD6+4kw8BH2/5vEvxOigtvwsJ0/b2k/V/9MwvcT9FHGKJNc > jsPNlJzeK7GmkYhht1CUvVK0isJruCYX9YtF6M8xLUrEEKav1VMFn8/9LRbCqQlk > 6z0Cow0918dzNbnyFKGxgDNXFHQvWNo7uMAqkYYftL8q67kPHQ6lj9qiQYsk2KA= > =aqfb > -----END PGP SIGNATURE----- > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Thu Feb 3 17:25:27 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 03 Feb 2011 12:25:27 -0500 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D4ADFD1.10001@redhat.com> References: <4D49D001.3060509@redhat.com> <4D4AC915.4060005@redhat.com> <4D4AD25F.3090304@redhat.com> <4D4ADA0E.70905@redhat.com> <4D4ADFD1.10001@redhat.com> Message-ID: <4D4AE507.5050501@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Just found another error. The update arguments table lists --key as a valid parameter, meant for updating the repo's private key. Running `pulp-admin repo update --help` doesn't reflect that --key is a valid argument. I'll file that one too; leave it in the docs since it should be there. On 02/03/2011 12:03 PM, Jeff Ortel wrote: > > > On 02/03/2011 10:38 AM, Jay Dobies wrote: >>>>> Between writing this up and working on >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=664040, I found that we >>>>> permit changes to the repo that really breaks the repo in many ways. >>>>> Assuming there are no valid use cases for changing the relative path >>>>> after the repo has been populated, I propose following restrictions on >>>>> 'repo update'. >>>> >>>>> If the repo has been populated (synced): >>>> >>>>> --relativepath (option rejected) >>>> >>>>> --symlinks (option rejected) >>>> >>>>> --feed (rejected if relative path (part) changed) >>>> >>>>> The main use case for changing the feed is that it was entered >>>>> incorrectly when the repo was created. User tries to sync and it >>>>> barfs. Then, the user corrects the feed URL, retries the sync and all >>>>> is good. > > I agree with the main use case and I like the idea of "you can make more > changes before you do the initial sync". The feed/relativepath > relationship you described may be confusing to users. We may want to > make it simpler and more coarse-grained by saying they have to make sure > to lock in all their values before synchronizing. If it really becomes > an issue in the future we can address it. > >>>>> A secondary use case is that the user wants to sync the repo from a >>>>> different mirror. To support this, we permit the --feed so long as the >>>>> relative path remains the same. > > Ahh, ok. That's a good use case and covered by your rules. Lock in the > relative path before synchronizing, but you can change the feed later. > > What about changing feed types? I can't think of any issues in the feed > being a whole new type on an update, but figured I'd throw it out there. > > >>>> - It says we can update the symlinks flag on a repo. What happens if >>>> the >>>> repo was already synchronized? Will we remove the packages from the repo >>>> and replace them with symlinks? Will future syncs on that repo result in >>>> a hybrid of full packages and symlinks? >>>> >>>>> Probably. > > Ok, that's bad, but will be covered by your rules above. > >>>> - When changing the relative path, is that something we need to >>>> broadcast out to all bound consumers so they can update their .repo >>>> files? If it is, that's a bug; we don't have any hooks in the update API >>>> method. >>>> >>>>> Yup, this is busted. > > Is there a bug on it? > >> No. I originally thought this was covered by the rules (above) but I >> guess a user could: >> 1) create the repo >> 2) bind consumer(s) >> .... pulp.repo created on consumers ..... >> 3) update the feed or relative path >> 4) sync. > >> So the pulp.repo file will be wrong. > >> I'll add a bug. > > >>>> - You can update a group ID with --groupid, but how do you remove it >>>> from a group? > > I'm going to guess this is a bug and I'll file it. > >>>> - When specifying to remove GPG keys, the docs say to specify the GPG >>>> key file. What if they don't have the file anymore, they can't remove >>>> that GPG key? >>>> >>>>> I can update the doc. Basically the user uses 'repo listkeys' and >>>>> specifies one of the file names listed. > > Cool, sounds good. I just double checked and the --help is correct, so > it should just be a wiki fix. > >>>> - When changing an existing --cacert, --cert, or --key is the old one >>>> deleted from the Pulp server? Looking at the code not only do we not >>>> delete the old ones, it doesn't look like we write the new ones to disk. >>>> Can someone more familiar with that area confirm this is a bug? > > I'm going to file this as a bug too. > >> _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNSuUHAAoJEOMmcTqOSQHCvy4IALedRtk6Ee+be9/WUfIGSGQs gsqBs+pN6pDgqGzLmcSgCh3xYsv8CGPzENHPLnUalBxL0KVMQU2SqPUoz4mYw0Av 7zxQR8FEueMXCB07vHc2LYvZnSx90mr6YGlgZZMVV1JWHWRG5SyC5BygsZeCe1nH k4xuSvmc7McLNSekdppO3Ahd1mcd3878ljYM/z8kLS/oMYvsHynxDb+CBtVuGQAU nB7CihKsHXKLckDRgqH9SEvFT9kHUXKXFDk5qK49Ao/KZPJ5VD8C3/X8falYVDj0 LbA9+zrtSzYPh6N3nDYmsdife9PblIXgSBLOFxngVx802nQYWZHO7pRsasuHpTI= =FyrA -----END PGP SIGNATURE----- From jason.dobies at redhat.com Thu Feb 3 17:30:24 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 03 Feb 2011 12:30:24 -0500 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D4AE507.5050501@redhat.com> References: <4D49D001.3060509@redhat.com> <4D4AC915.4060005@redhat.com> <4D4AD25F.3090304@redhat.com> <4D4ADA0E.70905@redhat.com> <4D4ADFD1.10001@redhat.com> <4D4AE507.5050501@redhat.com> Message-ID: <4D4AE630.7010203@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 To summarize my comments, here's a list of the bugs I added related to repo update: https://bugzilla.redhat.com/show_bug.cgi?id=674901 No way to remove a repo from a group https://bugzilla.redhat.com/show_bug.cgi?id=674902 When updating a repo with new cert/key/ca-cert, it is not saved to disk https://bugzilla.redhat.com/show_bug.cgi?id=674903 No way to remove the cert/key/ca-cert from a repo https://bugzilla.redhat.com/show_bug.cgi?id=674904 Repo update is missing --key option https://bugzilla.redhat.com/show_bug.cgi?id=674905 Repo update --help shows incorrect help string for --cert - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNSuYwAAoJEOMmcTqOSQHC3D0IAJvUgeHszEAd+3Z2QN1mpIpu CjjqkC1REqY9gUSQox4un8tal4qQh5lYLXqA5C3vX3idWEKbaUTZXaMTAP0n0GmP k7mpQwPw7e6h4yymB/k3453lxKmgn6AUSdpo6JJBUbd7/mhhkxOMkTCb9pz3Tr+s b8w2SuRjuqVMc4XV2ydb7rySjJhszonfAaLGx1bzfKAVSb5EajSWbE0+Vfi5J8rg XTsj/jrq8H1IDCJGawOlsRKZ/c+DYdEeU9ajyak4qOp75TMH5BU8+518KAOIRSWJ 91WnN2div4zqsMovnGj3e3guL2Y8B6oJRF0jZiAZYb0fLe/488KWSOpU5Y77Lxc= =j6cM -----END PGP SIGNATURE----- From jortel at redhat.com Thu Feb 3 17:42:21 2011 From: jortel at redhat.com (Jeff Ortel) Date: Thu, 03 Feb 2011 11:42:21 -0600 Subject: [Pulp-list] Proof UG Additions In-Reply-To: <4D4AE630.7010203@redhat.com> References: <4D49D001.3060509@redhat.com> <4D4AC915.4060005@redhat.com> <4D4AD25F.3090304@redhat.com> <4D4ADA0E.70905@redhat.com> <4D4ADFD1.10001@redhat.com> <4D4AE507.5050501@redhat.com> <4D4AE630.7010203@redhat.com> Message-ID: <4D4AE8FD.5040701@redhat.com> Thanks! On 02/03/2011 11:30 AM, Jay Dobies wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > To summarize my comments, here's a list of the bugs I added related to > repo update: > > https://bugzilla.redhat.com/show_bug.cgi?id=674901 > No way to remove a repo from a group > > https://bugzilla.redhat.com/show_bug.cgi?id=674902 > When updating a repo with new cert/key/ca-cert, it is not saved to disk > > https://bugzilla.redhat.com/show_bug.cgi?id=674903 > No way to remove the cert/key/ca-cert from a repo > > https://bugzilla.redhat.com/show_bug.cgi?id=674904 > Repo update is missing --key option > > https://bugzilla.redhat.com/show_bug.cgi?id=674905 > Repo update --help shows incorrect help string for --cert Also: https://bugzilla.redhat.com/show_bug.cgi?id=674908 Changes to pulp repo relative path or gpg keys not reflected in pulp.repo Should we have a tracker bug for this? > > - -- > Jay Dobies > RHCE# 805008743336126 > Freenode: jdob > http://pulpproject.org > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJNSuYwAAoJEOMmcTqOSQHC3D0IAJvUgeHszEAd+3Z2QN1mpIpu > CjjqkC1REqY9gUSQox4un8tal4qQh5lYLXqA5C3vX3idWEKbaUTZXaMTAP0n0GmP > k7mpQwPw7e6h4yymB/k3453lxKmgn6AUSdpo6JJBUbd7/mhhkxOMkTCb9pz3Tr+s > b8w2SuRjuqVMc4XV2ydb7rySjJhszonfAaLGx1bzfKAVSb5EajSWbE0+Vfi5J8rg > XTsj/jrq8H1IDCJGawOlsRKZ/c+DYdEeU9ajyak4qOp75TMH5BU8+518KAOIRSWJ > 91WnN2div4zqsMovnGj3e3guL2Y8B6oJRF0jZiAZYb0fLe/488KWSOpU5Y77Lxc= > =j6cM > -----END PGP SIGNATURE----- > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Fri Feb 4 19:24:25 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Fri, 04 Feb 2011 14:24:25 -0500 Subject: [Pulp-list] New Testing Build: 0.135 Message-ID: <4D4C5269.7070702@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNTFJpAAoJEOMmcTqOSQHCHqAH/3XngChY7j4hvAttdEEh/BHt +j4wNqWkAnae3oDezQ0UnMWFjI49+hMLrCCgupx21Bf6vz9/T4yH6UnBHKARAD6y 33Luy016r9qYkQjSMYu57ziIAuQjqWt8z6j1sC6hffZvYngUEATs6f0fmifEwFmr 05KGbLUsxaOATRtTNXarj04XTMBK7gq5MslCO4G1mAK4d936A1SprvpzPIDHogp3 FowVS1EJwk2q2AjE7ajJwEbAWuC10iQ2tDUiw5ysWy9Lh7JMJNsTNO+c7K/CQLB2 8jYCqb5a7ksFXeW2X8oLkT+X8QGbedneXmSgRr6QaN50HGw4T3WQCZxBIzElFg0= =TwJG -----END PGP SIGNATURE----- From jmatthew at redhat.com Fri Feb 4 20:20:29 2011 From: jmatthew at redhat.com (John Matthews) Date: Fri, 4 Feb 2011 15:20:29 -0500 (EST) Subject: [Pulp-list] Grinder Updated to 0.78 In-Reply-To: <379803185.107274.1296850787265.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <1819071665.107278.1296850829892.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> If you are running pulp from git, you will need to update grinder to 0.78 From pkilambi at redhat.com Mon Feb 7 19:20:32 2011 From: pkilambi at redhat.com (Pradeep Kilambi) Date: Mon, 7 Feb 2011 14:20:32 -0500 (EST) Subject: [Pulp-list] Grinder updated 0.0.79-1 Message-ID: <1393732096.332933.1297106432516.JavaMail.root@zmail01.collab.prod.int.phx2.redhat.com> I enhanced the yum syncs to pull the new metadata into a temporary location until the sync is complete before overriding the currently active location. This way we dont end up serving 404 if package is yet to be downloaded. This is not a mandatory update from pulp standpoint, but recommended. The syncs should work as usual from pulp stand point. Lemme know if you see any issues. ~ Prad From mmccune at redhat.com Mon Feb 7 22:24:39 2011 From: mmccune at redhat.com (Mike McCune) Date: Mon, 07 Feb 2011 14:24:39 -0800 Subject: [Pulp-list] Data model change: repository["packages"] to list of package ids only In-Reply-To: <743481889.12042.1296478148951.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> References: <743481889.12042.1296478148951.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <4D507127.2060503@redhat.com> On 01/31/2011 04:49 AM, John Matthews wrote: > I made changes to how we store packages under a repository document. If there are no major objections I plan to check this in later today. > > Pradeep and I have noticed a large performance issue when calling "_get_existing_repo()" from repo.py. For rhel-i386-server-5 this takes roughly 30 seconds to fetch information on 7k packages resulting in a 10MB repository document being returned. For Fedora 13 this is even larger and takes around ~90 seconds to fetch somewhere around 20k packages. > > The issue is that we store a dictionary of "packages" under the repository. The dictionary has a key of package id and a value of the full package object. (Technically in mongo a reference to the package object is stored, not the full object. When we fetch the repository through pymongo the AutoReference SON Manipulator fetches the contents of each package object). This results in large repos being very expensive. Further pulp relies on "_get_existing_repo()" in many places so this is a problem that will be seen often for large repos. > > Over the weekend I made changes to how we store "packages", it's no longer storing packages as a dictionary, now we only store the package id in a list. > > "_get_existing_repo()" is much quicker as you can see: > For rhel-i386-server-5: > Time: .2 seconds versus ~30 seconds > Size: 1.5MB versus 10MB > > For fedora 13: > Time: .3 seconds versus ~90 seconds > Size: 2.5 MB versus 24MB > > The result of fetching a repository object now will only yield "package ids" under "packages". > If we want to flesh out all of the package objects as the call was previously doing, we can make a second call to the PackageAPI. This is still much quicker than previous behavior. > For rhel-i386-server-5: > Time: ~3 seconds versus ~30 seconds > Size: 10MB and 10MB > > For fedora 13: > Time: ~7 seconds versus ~90 seconds > Size 24MB and 24MB > > > Developers need to be aware repo["packages"] will only contains package ids. It takes one extra call to flesh out the "packages" into their full objects, so if that's needed it's easy and not as expensive with the new approach. > > I've made most of the changes needed for this, if there are no major objections I plan to check this in today. > If we had python-pymongo-1.7 you could have fixed the above performance problem with a one line change in repo.py: diff --git a/src/pulp/server/api/repo.py b/src/pulp/server/api/repo.py index 2a46d6a..7ffec34 100644 --- a/src/pulp/server/api/repo.py +++ b/src/pulp/server/api/repo.py @@ -103,7 +103,8 @@ class RepoApi(BaseApi): Protected helper function to look up a repository by id and raise a PulpException if it is not found. """ - repo = self.repository(id, fields) + # Filter out the packages field because it is big + repo = self.repository(id, fields={"packages": 0}) if repo is None: raise PulpException("No Repo with id: %s found" % id) return repo http://dirolf.com/2010/06/17/pymongo-1.7-released.html Don't be afraid to make large documents in Mongo and don't feel that you need to restructure everything to fix a performance problem. Instead just make your queries filter out large subsets of a document unless they are needed. Mike -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650.254.4248 From jmatthew at redhat.com Tue Feb 8 14:13:37 2011 From: jmatthew at redhat.com (John Matthews) Date: Tue, 8 Feb 2011 09:13:37 -0500 (EST) Subject: [Pulp-list] Data model change: repository["packages"] to list of package ids only In-Reply-To: <4D507127.2060503@redhat.com> Message-ID: <2124046858.147186.1297174417150.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > On 01/31/2011 04:49 AM, John Matthews wrote: > > I made changes to how we store packages under a repository document. > > If there are no major objections I plan to check this in later > > today. > > > > Pradeep and I have noticed a large performance issue when calling > > "_get_existing_repo()" from repo.py. For rhel-i386-server-5 this > > takes roughly 30 seconds to fetch information on 7k packages > > resulting in a 10MB repository document being returned. For Fedora > > 13 this is even larger and takes around ~90 seconds to fetch > > somewhere around 20k packages. > > > > The issue is that we store a dictionary of "packages" under the > > repository. The dictionary has a key of package id and a value of > > the full package object. (Technically in mongo a reference to the > > package object is stored, not the full object. When we fetch the > > repository through pymongo the AutoReference SON Manipulator fetches > > the contents of each package object). This results in large repos > > being very expensive. Further pulp relies on "_get_existing_repo()" > > in many places so this is a problem that will be seen often for > > large repos. > > > > Over the weekend I made changes to how we store "packages", it's no > > longer storing packages as a dictionary, now we only store the > > package id in a list. > > > > "_get_existing_repo()" is much quicker as you can see: > > For rhel-i386-server-5: > > Time: .2 seconds versus ~30 seconds > > Size: 1.5MB versus 10MB > > > > For fedora 13: > > Time: .3 seconds versus ~90 seconds > > Size: 2.5 MB versus 24MB > > > > The result of fetching a repository object now will only yield > > "package ids" under "packages". > > If we want to flesh out all of the package objects as the call was > > previously doing, we can make a second call to the PackageAPI. This > > is still much quicker than previous behavior. > > For rhel-i386-server-5: > > Time: ~3 seconds versus ~30 seconds > > Size: 10MB and 10MB > > > > For fedora 13: > > Time: ~7 seconds versus ~90 seconds > > Size 24MB and 24MB > > > > > > Developers need to be aware repo["packages"] will only contains > > package ids. It takes one extra call to flesh out the "packages" > > into their full objects, so if that's needed it's easy and not as > > expensive with the new approach. > > > > I've made most of the changes needed for this, if there are no major > > objections I plan to check this in today. > > > > If we had python-pymongo-1.7 you could have fixed the above > performance > problem with a one line change in repo.py: > > diff --git a/src/pulp/server/api/repo.py b/src/pulp/server/api/repo.py > index 2a46d6a..7ffec34 100644 > --- a/src/pulp/server/api/repo.py > +++ b/src/pulp/server/api/repo.py > @@ -103,7 +103,8 @@ class RepoApi(BaseApi): > Protected helper function to look up a repository by id and > raise a > PulpException if it is not found. > """ > - repo = self.repository(id, fields) > + # Filter out the packages field because it is big > + repo = self.repository(id, fields={"packages": 0}) > if repo is None: > raise PulpException("No Repo with id: %s found" % id) > return repo > > http://dirolf.com/2010/06/17/pymongo-1.7-released.html > > Don't be afraid to make large documents in Mongo and don't feel that > you > need to restructure everything to fix a performance problem. > > Instead just make your queries filter out large subsets of a document > unless they are needed. > Hi Mike, Thanks for sharing the info. I'm in the process of upgrading to pymongo 1.9 right now, so pulp will be on it fairly soon. I'm not sure if dropping the packages field would have been sufficient. Here are my biggest concerns: 1) We rely on a full fetch of the object prior to an update 2) The big performance hit I saw was related to the AutoReference manipulation and not the actual fetching of raw data. 3) The actual usage of "packages" inside of API calls generally only required the package_id, the actual package contents weren't used. #1 If you look at the flow of synchronizing a repository you will see we change the repository object in several places and call repo update frequently. Our current usage of mongo does not do partial document updates, hence we need the full document available prior to an update. Jeff raised the issue yesterday we need to improve how we update documents in mongo, agreement was reached this area needs to be addressed. #2 Background: We were storing an AutoReference to each package under the repository document "packages" field. AutoReference is a driver level abstraction, as in it's a pymongo concept and not natively known by mongo. The AutoReference manipulator doesn't seem to do a batch fetch of IDs. I believe this is the main issue for the performance numbers we are seeing. For a fedora repo with 20k packages it was taking roughly 90 seconds to fetch a repo and the packages, yet if you fetch the ids then flesh out all of those ids it takes just a few seconds to return the same data. The AutoReference usage would be very useful if we were fetching package data under the repository document and modifying it, but that's not how we have been using the data. When we make changes to a package we do so through the package api. I felt we were paying a large performance price for a feature we weren't using. #3 When looking at the code inside of pulp I saw most of the usages of the repo["packages"] data was examining ids and wasn't concerned with the actual content. Granted there are some places where content is desired, but it wasn't the popular case. Very often we fetched the whole content and would iterate over it, ignoring everything but the id. Last thought on my mind was that with the current implementation of ids under repo["packages"] instead of AutoReference's I've been able to push more of the queries down to mongo and I do less in the python level. I like that. -John From jmatthew at redhat.com Tue Feb 8 22:49:43 2011 From: jmatthew at redhat.com (John Matthews) Date: Tue, 8 Feb 2011 17:49:43 -0500 (EST) Subject: [Pulp-list] Upgrade to pymongo 1.9.x [Resolution for conflict with python-pymongo] In-Reply-To: <1370210547.161156.1297204077263.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <129183826.161406.1297205383688.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> If running on a developer setup, please remove python-pymongo and upgrade to pymongo 1.9, this is in EPEL and F13/F14. We have upgraded pulp to use pymongo 1.9 opposed to python-pymongo 1.6.x. This will take effect in the next testing release. If you are doing an upgrade from built rpms you will see a conflict with python-pymongo and pymongo. Resolution requires a manual step of: "sudo rpm -e --nodeps python-pymongo" This note will be included with the Release Notes for the next community release. Background: python-pymongo was the pulp's team packaging of pymongo before it was in Fedora. From jortel at redhat.com Tue Feb 8 22:56:37 2011 From: jortel at redhat.com (Jeff Ortel) Date: Tue, 08 Feb 2011 16:56:37 -0600 Subject: [Pulp-list] Client configuration made easier Message-ID: <4D51CA25.6010706@redhat.com> After replacing the pulp agent with a gofer (agent) plugin, configuring the client to use a remote qpid server became less intuitive. The intent was for pulp's use of gofer to be an implementation detail not exposed to pulp users. But, ... Today, in fixing (669484 & 669498), I pushed changes to make client configuration more intuitive for pulp users again. The pulp (gofer) plugin descriptor new references the server.host in /etc/pulp/client.conf for the qpid broker url. This means editing the /etc/gofer/agent.conf is no longer necessary. Further, I symlinked 'pulp-agent' to 'goferd' in /etc/rc.d/init.d. This creates pseudo service like 'pulp-server' but for the agent. When configuring a client (consumer) to connect to a remote pulp server the steps are as follows: 1) edit /etc/pulp/client.conf and change the server.host 2) service pulp-agent restart Done. wiki/ug updates to follow. Hope this helps, Jeff From jconnor at redhat.com Wed Feb 9 00:21:18 2011 From: jconnor at redhat.com (Jason L Connor) Date: Tue, 08 Feb 2011 17:21:18 -0700 Subject: [Pulp-list] Client configuration made easier In-Reply-To: <4D51CA25.6010706@redhat.com> References: <4D51CA25.6010706@redhat.com> Message-ID: <1297210878.23899.20.camel@precision> On Tue, 2011-02-08 at 16:56 -0600, Jeff Ortel wrote: > After replacing the pulp agent with a gofer (agent) plugin, configuring the client to use > a remote qpid server became less intuitive. The intent was for pulp's use of gofer to be > an implementation detail not exposed to pulp users. But, ... > > Today, in fixing (669484 & 669498), I pushed changes to make client configuration more > intuitive for pulp users again. The pulp (gofer) plugin descriptor new references the > server.host in /etc/pulp/client.conf for the qpid broker url. This means editing the > /etc/gofer/agent.conf is no longer necessary. Further, I symlinked 'pulp-agent' to > 'goferd' in /etc/rc.d/init.d. This creates pseudo service like 'pulp-server' but for the > agent. When configuring a client (consumer) to connect to a remote pulp server the steps > are as follows: > > 1) edit /etc/pulp/client.conf and change the server.host > 2) service pulp-agent restart > > Done. > > wiki/ug updates to follow. > > Hope this helps, > > Jeff > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list Nice! This will make it much easier to configure clients. -- Jason L Connor Software Engineer Red Hat, Inc. RHCE: 805010912355231 Freenode: linear GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jconnor at redhat.com Wed Feb 9 01:49:05 2011 From: jconnor at redhat.com (Jason L Connor) Date: Tue, 08 Feb 2011 18:49:05 -0700 Subject: [Pulp-list] YOUR PULP INSTALLATION IS BROKEN! Message-ID: <1297216145.23899.77.camel@precision> All, If you not are using the developer deployment, you can ignore this. A lot changed today. Jeff and Pradeep both had large pushes and here two more: 1. The default database name has changed. I have renamed it to: pulp_database. The old name, _database, is a relic from our prototyping days and not really suitable for a serious deployment. The name is changed in both the default config and the config file: pulp.conf. If you want to continue using the old database, just change it back in pulp.conf. 2. More importantly: the new authorization is pushed. This will very likely break your existing database as the users and roles needed for authorization have changed. I have tried to incorporate the changes into the latest version of pulp-migrate, so if you run into trouble, run pulp-migrate again, with the --force flag if necessary and restart pulp. Or you could just use the new database created by the name change and not worry about it, your choice. Possible trouble includes: no administrative users, admin user without proper permissions, consumers that do not have permissions. Good luck! -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jason.dobies at redhat.com Wed Feb 9 13:49:10 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 09 Feb 2011 08:49:10 -0500 Subject: [Pulp-list] Client configuration made easier In-Reply-To: <4D51CA25.6010706@redhat.com> References: <4D51CA25.6010706@redhat.com> Message-ID: <4D529B56.80701@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/08/2011 05:56 PM, Jeff Ortel wrote: > After replacing the pulp agent with a gofer (agent) plugin, configuring > the client to use a remote qpid server became less intuitive. The > intent was for pulp's use of gofer to be an implementation detail not > exposed to pulp users. But, ... > > Today, in fixing (669484 & 669498), I pushed changes to make client > configuration more intuitive for pulp users again. The pulp (gofer) > plugin descriptor new references the server.host in > /etc/pulp/client.conf for the qpid broker url. This means editing the > /etc/gofer/agent.conf is no longer necessary. Further, I symlinked > 'pulp-agent' to 'goferd' in /etc/rc.d/init.d. This creates pseudo > service like 'pulp-server' but for the agent. When configuring a client > (consumer) to connect to a remote pulp server the steps are as follows: > > 1) edit /etc/pulp/client.conf and change the server.host > 2) service pulp-agent restart > > Done. > > wiki/ug updates to follow. Last night I swear I read an e-mail that said something about waiting to update the UG when the CR was published, but this is the only reference to the UG still in my inbox. But I'll make my comment anyway just to make sure it's clear. Just a reminder, the wiki UG is intended to be the current master/testing build install instructions. So feel free to update those when you make any changes to how pulp runs. They won't be published to pulpproject.org until the CR, so there's no risk of being in a situation where the instructions you add/change don't apply to the CR. > Hope this helps, > > Jeff > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNUptWAAoJEOMmcTqOSQHC3dQIALAdxdOwHN0go85M3DoY3IGQ LwS8MaZ2UywxAfG1KvDuYSdnF4kKGJZ4ob/vXvvji55z6/DEjuf7hyIMwL4JHJ7p vIVqEc9tw16s9HY7Z/KCrikREi64XdImMx7OOT11UUzmYyIcVC8l2rAdU6yXFfX+ sQdj1xWBVxek1EjmfMoLDy1OTtLOtPtNw02ctXbUPnzgsrjuK/jni2VixT4EC1RT VbUJL/Ho4jwar1sZZc00r4E6yLGPJ34HSaNglaGB1s5CHNLtw53izBIPyC+YIx5s Be6IMKm/07KZ+6CiT3jNMPwYAhtjeZwR2kgyUbjDWrgwEDmDSz8cg5gCdLSTNgM= =4lxi -----END PGP SIGNATURE----- From jason.dobies at redhat.com Wed Feb 9 13:50:05 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 09 Feb 2011 08:50:05 -0500 Subject: [Pulp-list] Client configuration made easier In-Reply-To: <4D529B56.80701@redhat.com> References: <4D51CA25.6010706@redhat.com> <4D529B56.80701@redhat.com> Message-ID: <4D529B8D.40707@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/09/2011 08:49 AM, Jay Dobies wrote: > On 02/08/2011 05:56 PM, Jeff Ortel wrote: >> After replacing the pulp agent with a gofer (agent) plugin, configuring >> the client to use a remote qpid server became less intuitive. The >> intent was for pulp's use of gofer to be an implementation detail not >> exposed to pulp users. But, ... > >> Today, in fixing (669484 & 669498), I pushed changes to make client >> configuration more intuitive for pulp users again. The pulp (gofer) >> plugin descriptor new references the server.host in >> /etc/pulp/client.conf for the qpid broker url. This means editing the >> /etc/gofer/agent.conf is no longer necessary. Further, I symlinked >> 'pulp-agent' to 'goferd' in /etc/rc.d/init.d. This creates pseudo >> service like 'pulp-server' but for the agent. When configuring a client >> (consumer) to connect to a remote pulp server the steps are as follows: > >> 1) edit /etc/pulp/client.conf and change the server.host >> 2) service pulp-agent restart > >> Done. > >> wiki/ug updates to follow. > > Last night I swear I read an e-mail that said something about waiting to > update the UG when the CR was published, but this is the only reference > to the UG still in my inbox. But I'll make my comment anyway just to > make sure it's clear. > > Just a reminder, the wiki UG is intended to be the current > master/testing build install instructions. So feel free to update those > when you make any changes to how pulp runs. They won't be published to > pulpproject.org until the CR, so there's no risk of being in a situation > where the instructions you add/change don't apply to the CR. P.S. Very cool changes, glad we got these done this sprint. >> Hope this helps, > >> Jeff > >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNUpuNAAoJEOMmcTqOSQHCVrEIAK6VCtwxyeRkQwHJ8xxjZkKY ALDhwlxKWWRj9Ht0uhSL47IEF4bT8NKCsxCHUdS1hwNtHXWHC9wDBNZq3oftBpmp /G1QNlLsCSAHWz88MpMpHtfDwZvTLol+1O/G0PUSmTNk/U3HKi8rzgJLldIUxqt5 4xfqUOThKfr2dIRBRCze0ukOUCs+7C5MTwOal/zLE3KGK9BWuG8fQeD+m59fvRK8 1L2oraucYsmY82wdycyTU3Ps3P4LAD9+DnoynNRJxUHanLWmgFqSPhKFyb5KgEGP +UTEgbNRGIaKpaNWsWAzaBGwWEMlN3si8ZyfnLP7XXpNbADD2As4gvBJbNBTSAU= =fG3T -----END PGP SIGNATURE----- From jortel at redhat.com Wed Feb 9 14:04:10 2011 From: jortel at redhat.com (Jeff Ortel) Date: Wed, 09 Feb 2011 08:04:10 -0600 Subject: [Pulp-list] Client configuration made easier In-Reply-To: <4D529B56.80701@redhat.com> References: <4D51CA25.6010706@redhat.com> <4D529B56.80701@redhat.com> Message-ID: <4D529EDA.3020900@redhat.com> On 02/09/2011 07:49 AM, Jay Dobies wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 02/08/2011 05:56 PM, Jeff Ortel wrote: >> After replacing the pulp agent with a gofer (agent) plugin, configuring >> the client to use a remote qpid server became less intuitive. The >> intent was for pulp's use of gofer to be an implementation detail not >> exposed to pulp users. But, ... >> >> Today, in fixing (669484& 669498), I pushed changes to make client >> configuration more intuitive for pulp users again. The pulp (gofer) >> plugin descriptor new references the server.host in >> /etc/pulp/client.conf for the qpid broker url. This means editing the >> /etc/gofer/agent.conf is no longer necessary. Further, I symlinked >> 'pulp-agent' to 'goferd' in /etc/rc.d/init.d. This creates pseudo >> service like 'pulp-server' but for the agent. When configuring a client >> (consumer) to connect to a remote pulp server the steps are as follows: >> >> 1) edit /etc/pulp/client.conf and change the server.host >> 2) service pulp-agent restart >> >> Done. >> >> wiki/ug updates to follow. > > Last night I swear I read an e-mail that said something about waiting to > update the UG when the CR was published, but this is the only reference > to the UG still in my inbox. But I'll make my comment anyway just to > make sure it's clear. > > Just a reminder, the wiki UG is intended to be the current > master/testing build install instructions. So feel free to update those > when you make any changes to how pulp runs. They won't be published to > pulpproject.org until the CR, so there's no risk of being in a situation > where the instructions you add/change don't apply to the CR. Thanks for the reminder! Updating the docs first thing this morning. > >> Hope this helps, >> >> Jeff >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > > - -- > Jay Dobies > RHCE# 805008743336126 > Freenode: jdob > http://pulpproject.org > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJNUptWAAoJEOMmcTqOSQHC3dQIALAdxdOwHN0go85M3DoY3IGQ > LwS8MaZ2UywxAfG1KvDuYSdnF4kKGJZ4ob/vXvvji55z6/DEjuf7hyIMwL4JHJ7p > vIVqEc9tw16s9HY7Z/KCrikREi64XdImMx7OOT11UUzmYyIcVC8l2rAdU6yXFfX+ > sQdj1xWBVxek1EjmfMoLDy1OTtLOtPtNw02ctXbUPnzgsrjuK/jni2VixT4EC1RT > VbUJL/Ho4jwar1sZZc00r4E6yLGPJ34HSaNglaGB1s5CHNLtw53izBIPyC+YIx5s > Be6IMKm/07KZ+6CiT3jNMPwYAhtjeZwR2kgyUbjDWrgwEDmDSz8cg5gCdLSTNgM= > =4lxi > -----END PGP SIGNATURE----- > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Wed Feb 9 15:37:17 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 09 Feb 2011 10:37:17 -0500 Subject: [Pulp-list] QE Build: 0.136 Message-ID: <4D52B4AD.9020009@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This build contains the changes to pulp-agent, pymongo, and the database as outlined in recent e-mails. I did a spot check and things ran, but it should be noted that this build was created despite failing unit tests. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNUrStAAoJEOMmcTqOSQHC0HEIAMNlMOAyqHCAaC6d66OdEOo0 ABqNknqWeejuZN6J23YwI+MPZuYTewGtjrXbNY0A5myjOs982dDQA+7uohR81MLn nGjCSHM+Qud1pRdOPFMYgC/tpWCYzPosSHmx5iigX6RS2zeYsDhx28rqod8Hbzeh kEV6/YVY3G1sRn3gcyBnArSf/OWtceF8WGS3vVqVMr9/7HvtDf5W51rWuUiezH/5 qg3If/zt/g/35qYlobHA/mru7R7wimhs1+ReFI5W9pj2jsbzo2YpSxIYmHjr+fb1 +2az/mvgScaEy7FUCiqOAEqcwjNgGS/G5myy5XpMSvUicG5wLuePVhHyacEqdmI= =pwjP -----END PGP SIGNATURE----- From pkilambi at redhat.com Wed Feb 9 15:49:46 2011 From: pkilambi at redhat.com (Pradeep Kilambi) Date: Wed, 09 Feb 2011 10:49:46 -0500 Subject: [Pulp-list] YOUR PULP INSTALLATION IS BROKEN! In-Reply-To: <1297216145.23899.77.camel@precision> References: <1297216145.23899.77.camel@precision> Message-ID: <4D52B79A.4080604@redhat.com> On 02/08/2011 08:49 PM, Jason L Connor wrote: > All, > > If you not are using the developer deployment, you can ignore this. > > A lot changed today. Jeff and Pradeep both had large pushes and here two > more: > > 1. The default database name has changed. I have renamed it to: > pulp_database. The old name, _database, is a relic from our prototyping > days and not really suitable for a serious deployment. The name is > changed in both the default config and the config file: pulp.conf. If > you want to continue using the old database, just change it back in > pulp.conf. > > 2. More importantly: the new authorization is pushed. This will very > likely break your existing database as the users and roles needed for > authorization have changed. > > I have tried to incorporate the changes into the latest version of > pulp-migrate, so if you run into trouble, run pulp-migrate again, with > the --force flag if necessary and restart pulp. Or you could just use > the new database created by the name change and not worry about it, your > choice. > > Possible trouble includes: no administrative users, admin user without > proper permissions, consumers that do not have permissions. > > Good luck! > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list If you wish to export your existing content in _database, try this: $ mongo > use _database > db.copyDatabase("_database", "pulp_database", ) then run pulp-migrate. This worked for me. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jconnor at redhat.com Wed Feb 9 16:01:33 2011 From: jconnor at redhat.com (Jason L Connor) Date: Wed, 09 Feb 2011 09:01:33 -0700 Subject: [Pulp-list] YOUR PULP INSTALLATION IS BROKEN! In-Reply-To: <4D52B79A.4080604@redhat.com> References: <1297216145.23899.77.camel@precision> <4D52B79A.4080604@redhat.com> Message-ID: <1297267293.30695.0.camel@precision> On Wed, 2011-02-09 at 10:49 -0500, Pradeep Kilambi wrote: > > If you wish to export your existing content in _database, try this: > > $ mongo > > use _database > > db.copyDatabase("_database", "pulp_database", ) > > then run pulp-migrate. This worked for me. > Sweet! Thank you Pradeep! -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From pkilambi at redhat.com Wed Feb 9 17:15:24 2011 From: pkilambi at redhat.com (Pradeep Kilambi) Date: Wed, 09 Feb 2011 12:15:24 -0500 Subject: [Pulp-list] QE Build: 0.137 Message-ID: <4D52CBAC.1020801@redhat.com> This build contains the fixes to unittests failures. I validated to make sure the basic operations work as expected. Please lemme know if you see any issues. ~ Prad From jconnor at redhat.com Wed Feb 9 18:37:24 2011 From: jconnor at redhat.com (Jason L Connor) Date: Wed, 09 Feb 2011 11:37:24 -0700 Subject: [Pulp-list] Roles and Permissions docs Message-ID: <1297276644.30695.3.camel@precision> Hi All, I'm in the process of putting together documentation for the roles and permissions portions of authorization that was pushed yesterday. I'm thinking that roles should go under the current user documentation and permissions should be its own new section in the user guide. However, I'm looking for your thoughts on this. Any other suggestions or even simple confirmations are welcome. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jason.dobies at redhat.com Wed Feb 9 18:39:54 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 09 Feb 2011 13:39:54 -0500 Subject: [Pulp-list] Roles and Permissions docs In-Reply-To: <1297276644.30695.3.camel@precision> References: <1297276644.30695.3.camel@precision> Message-ID: <4D52DF7A.7020509@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 IMO, they should both go under "Pulp Users" since they are typically thought of as user-related things. Personally I tend to think of users/roles as the same concept and I'd be inclined to bundle permissions in there as well. What's the rationale for it being its own section? On 02/09/2011 01:37 PM, Jason L Connor wrote: > Hi All, > > I'm in the process of putting together documentation for the roles and > permissions portions of authorization that was pushed yesterday. > > I'm thinking that roles should go under the current user documentation > and permissions should be its own new section in the user guide. > However, I'm looking for your thoughts on this. Any other suggestions or > even simple confirmations are welcome. > > > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNUt96AAoJEOMmcTqOSQHC5PwIAJrdFK/XsFhK/rx+iGCY7SNB it0luEw88HQZOpJO8x6P6yr2LKjCXCLrUNxLlPndd5vR8hRFwapSEwREaN/Wjprw /tWNCX4FoSYPjB3EMzTezEfjuZZKcdtGaGz9GcHftDc7/0XqqYn18U/6gEaRwW+b zZ4wVRTDJUs0tHBgDemvsGq3NSvCzHy6sZft5RFl8hJFCSteGdf3344aISKGFIcz zjRXYFFsnT6qQb6SV2vIPednj8LJ+K0XYXQ8myZTASHEDu7dZ+sniytvncmL/nkT 71THwJ1q5Cro+1zyIZG8FfmvPT5Rl6mF/0mCZ8O+UOPfiD8C56rXKTrIkau1w0Y= =wJkx -----END PGP SIGNATURE----- From jason.dobies at redhat.com Wed Feb 9 18:41:42 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 09 Feb 2011 13:41:42 -0500 Subject: [Pulp-list] Roles and Permissions docs In-Reply-To: <1297276644.30695.3.camel@precision> References: <1297276644.30695.3.camel@precision> Message-ID: <4D52DFE6.6070804@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Looks good. On 02/09/2011 01:37 PM, Jason L Connor wrote: > Hi All, > > I'm in the process of putting together documentation for the roles and > permissions portions of authorization that was pushed yesterday. > > I'm thinking that roles should go under the current user documentation > and permissions should be its own new section in the user guide. > However, I'm looking for your thoughts on this. Any other suggestions or > even simple confirmations are welcome. > > > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNUt/mAAoJEOMmcTqOSQHC1zQH/2KLVtuhAxgdHLcrlumtfhh8 jdNDqL1/sCqpmxbw3TzHhdnQnF+uqv4Mzj7vWxLSMzsh6LTeZnTTzr0EUM3wM7Th KTWmXtWsUxrrRcOUYWEau9oDNbmRDyBfRT80hbdsJrWevAzc3aueqAgFqCL/tfOX 60vj63hxlEONh3bGxuplybUHWBw7g4dyJWVGO1UnJond5CfsAl02VTioEzuz/RIb YAkWYxW866z9PNx7utUaE6Yhm1v2Gz/ot/Kf4dJQzpMpM1lbnrxVDuEXY26VDlUh QmwIxH7qclcDWLfasnU9nAAkufY+tvnh0c0OV66wA50UkRgRwWapfkaMReAqNNw= =rO6J -----END PGP SIGNATURE----- From jason.dobies at redhat.com Wed Feb 9 18:49:42 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 09 Feb 2011 13:49:42 -0500 Subject: [Pulp-list] Roles and Permissions docs In-Reply-To: <4D52DFE6.6070804@redhat.com> References: <1297276644.30695.3.camel@precision> <4D52DFE6.6070804@redhat.com> Message-ID: <4D52E1C6.5030903@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Looks good. Igore that, it was to another e-mail. > On 02/09/2011 01:37 PM, Jason L Connor wrote: >> Hi All, > >> I'm in the process of putting together documentation for the roles and >> permissions portions of authorization that was pushed yesterday. > >> I'm thinking that roles should go under the current user documentation >> and permissions should be its own new section in the user guide. >> However, I'm looking for your thoughts on this. Any other suggestions or >> even simple confirmations are welcome. > > > > >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNUuHGAAoJEOMmcTqOSQHCQ0YH/3Lj80G7rUO0Usltb/ax+6Hj bmP94S0V2l0SgeKbmuMIFH4+tLOz/TutchyeTucpKiR7zBriZ9KvPvOXDNOC4t6w 8yJoMo0kvzgXpkACaNhUFN17m1rjuODTx1uZFH+qSVRDL2er/rk28cQrXGjBJd9g 3WS3SB2QFNcupvpdLEjnvwO+5c6P4gVvbSSNSkX/xRj0mckB4rLQtPQXxSulDNK0 JoafhqlW79oBzo2OLbTB2XP+2rHdjB9Bb8/PFFbcEhcaZbLYtKhm58px0/uhBx7J kndyTQzB3jWRGaNOsBsqeBXlnCx5hIQ2tcna/w0numZ47DBkfv73MS6iVjk+ZWA= =j5xV -----END PGP SIGNATURE----- From jconnor at redhat.com Wed Feb 9 18:52:01 2011 From: jconnor at redhat.com (Jason L Connor) Date: Wed, 09 Feb 2011 11:52:01 -0700 Subject: [Pulp-list] Roles and Permissions docs In-Reply-To: <4D52DF7A.7020509@redhat.com> References: <1297276644.30695.3.camel@precision> <4D52DF7A.7020509@redhat.com> Message-ID: <1297277521.30695.6.camel@precision> On Wed, 2011-02-09 at 13:39 -0500, Jay Dobies wrote: > > What's the rationale for it being its own section? > > I was thinking that the permissions also touch more or less everything else in the system, but I can also see them going under the users section. It was actually this conflict the prompted me to request some feedback. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jason.dobies at redhat.com Thu Feb 10 15:06:01 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 10 Feb 2011 10:06:01 -0500 Subject: [Pulp-list] CLI Consistency Message-ID: <4D53FED9.2030909@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Take the following two examples: $ pulp-admin user create --username hal $ pulp-admin role add --role new-role --user user This may just be my OCD, but I think we should stay consistent in the names of the flags (--username v. --user). I didn't file a bug because I think this is more of just my opinion than something that is actually wrong, but wanted to throw it out there to let people think about it. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNU/7ZAAoJEOMmcTqOSQHCfysIAITpp04mEdz4PfbYNOQsBhNR 6zfYc8Y0uJFAAHb9WC3e8bRT87guoDMgg0i2wc0PRj0yv73naIW8bZ5Qxcwymu2b eeMS5PTM2NNdM1O6WKb+52/IBBSrQVLWp26ivUnf8QoC8K97bKjOo5NvAF2WLXpp JVSKkgUyCCdDexXnBTlCEybYyMMCyjLyjJBu8tF4UaYAgtpIzYtWhBx5cVmOiNfp mx+jRntyycPoDMQa6dHQrsqP6h/o+YlO6YByoLUv8w27KySr7amO4Qi7lw9nN+0K lWrJFkdApo67FWEVjMD80aPIxPnTQ86MEUfnNc5foKV+pnEV/nHJlz18pNJonEM= =tEQc -----END PGP SIGNATURE----- From jason.dobies at redhat.com Thu Feb 10 15:29:19 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 10 Feb 2011 10:29:19 -0500 Subject: [Pulp-list] User Guide Conventions Message-ID: <4D54044F.9030902@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Since there are additions being made I wanted to give a few notes on the conventions used throughout the User Guide in case people aren't completely familiar with the whole guide. Most of these are a result of proofreading the permissions section of the guide and I've made changes where necessary (i.e. correcting all of the typos). - - The guide isn't the most formal thing in the world, but it is still a guide and not a chat conversation. Avoid cases where you're talking to the user: Instead of: "You can grant, revoke, and show users permissions on various parts of the REST API with the `permission` command." Try: "Permissions can be granted, revoked, and displayed through the `permission` command." It's not supposed to be Shakespeare, but it should be professional. - - Command arguments are very important; one of the biggest reasons for the guide that we can provide more details than --help does. Instead of blowing through the possible arguments in a single run on sentence with no description of what they are, use a table. The format used throughout the guide is to note if the arguments are required or optional and then include a table: All arguments are considered required unless otherwise specified: || '''Name''' || '''Flag''' || '''Description''' || || Foo || `--foo` || (optional) Does foo. || Don't forget to indicate the restrictions on an argument. For instance, the `permission grant` command accepts `-o` parameters to list the operations. There is a list in the guide that indicates valid values for that. Same for repo feed types. It's not just enumerated values, keep in mind things like numbers and units too. - - All CLI snippets are started with $. For example: $ pulp-admin repo list This is just a consistency thing to make it appear as a uniform guide instead of a bunch of disparate sections. It doesn't mean anything other than the fact that it's what was used throughout the rest of the guide. - - When talking about a command, use the full path to it. Hierarchically they may fall under the same section, but don't expect the user to remember that after scrolling for 2 pages. So instead of: "The `info` command retrieves..." Use the full path: "The `role info` command retrieves..." - - Typically across the guide we use the term "display" instead of "show". So something like "Displays details of a user" instead of "Shows details of a user." This is minor, but I figured I'd point it out. Personally I'm also a fan of "Retrieves the value of..." instead of "Gets the value of...". - - Pulp starts with a capital letter when referring to the project or product. It starts with a lower case when referring to a command (i.e. pulp-admin). - - Proofread! I know some people don't like working in trac directly on firefox, which is fine. But one benefit of it is that firefox underlines mispelled words. So when you finally do paste in your changes, take a glance over it to make sure you haven't spelled anything incorrectly before hitting submit. It's just embarrassing to have 10 typos on a single page. It'll take like 2 minutes, I promise. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNVARPAAoJEOMmcTqOSQHC1fEH/iqj9KgjhqVHiAjJiXGeDd1B Nw3gFWsMEcsn95ONGBhFrwZX4fs+8QY88xCU+HpoSNB6OZvG3qojn4Db6fd8XodN tMbN59pYQ75HLMOX9Ph244kh5N+8uwLCIKv8TC9tzV2ucuGRzabPtnzQ6ryK/1ic ZCDpcgxVW7vgdSp8HnV2HdE34j/MN7MN9b3FOv2GDzw2u5atLiGGzO/Id1jd9e7n nDGgXfGU0hMMxAoqwr44tYL7XLaN8Zz2aire1FCXIMv1tFw/uDrxIwiRQ+f/k+PJ mUiCG0rTU+aD8XzBbGb4/JtQhwQHWaNG01hgLTQwuaRn3ABLI9pOyNM/JYKfujI= =YW7S -----END PGP SIGNATURE----- From jconnor at redhat.com Thu Feb 10 16:41:17 2011 From: jconnor at redhat.com (Jason L Connor) Date: Thu, 10 Feb 2011 09:41:17 -0700 Subject: [Pulp-list] CLI Consistency In-Reply-To: <4D53FED9.2030909@redhat.com> References: <4D53FED9.2030909@redhat.com> Message-ID: <1297356077.1033.6.camel@precision> On Thu, 2011-02-10 at 10:06 -0500, Jay Dobies wrote: > Take the following two examples: > > $ pulp-admin user create --username hal > $ pulp-admin role add --role new-role --user user > > This may just be my OCD, but I think we should stay consistent in the > names of the flags (--username v. --user). I didn't file a bug because > I > think this is more of > just my opinion than something that is actually wrong, but wanted to > throw it out there to let people think about it. A while back I actually put together a proposal to overcome this, which was pushed off for various reasons, but it may be time to refresh our collective memories. Here is my original proposal in its entirety: (Please realize this proposal was sent to this list and there were follow up emails that I'm not recapping here) ---- Hi All, I've been digging around in the cli a bit lately, and I wanted to propose some common options and option names for our cli. Please note that this is semantic nit-picking. :) We use a --id option for most of our actions that pertain to which ever collection of resources the command is meant to manipulate. E.G in "pulp-admin repo sync --id my-repo", the 'my-repo' is a repository id. And it'll mean something else for the consumer command, the package command, etc. Instead of using options called 'id' or even 'consumerid', 'repoid', etc. We should use options that associate themselves with the resources and provide short names for all of them. We should use this convention on all commands. This is what I propose: -c, --consumer : consumer id -C, --consumer-group : consumer group id -e, --errata : errata id -p, --package : package name -P, --package-group : package group id -r, --repo : repository id -u, --username : username I think --id is ambiguous and using concatenated --conumserid or --repoid is probably unnecessary. These proposed changes will make the cli more consistent and intuitive for our end users. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jason.dobies at redhat.com Thu Feb 10 16:42:43 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 10 Feb 2011 11:42:43 -0500 Subject: [Pulp-list] CLI Consistency In-Reply-To: <1297356077.1033.6.camel@precision> References: <4D53FED9.2030909@redhat.com> <1297356077.1033.6.camel@precision> Message-ID: <4D541583.5090801@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I remember this. I actually think about it every time I type --repoid. I'm still for it if others are. It'll be annoying to update the docs, but not impossible. On 02/10/2011 11:41 AM, Jason L Connor wrote: > On Thu, 2011-02-10 at 10:06 -0500, Jay Dobies wrote: >> Take the following two examples: >> >> $ pulp-admin user create --username hal >> $ pulp-admin role add --role new-role --user user >> >> This may just be my OCD, but I think we should stay consistent in the >> names of the flags (--username v. --user). I didn't file a bug because >> I >> think this is more of >> just my opinion than something that is actually wrong, but wanted to >> throw it out there to let people think about it. > > > A while back I actually put together a proposal to overcome this, which > was pushed off for various reasons, but it may be time to refresh our > collective memories. Here is my original proposal in its entirety: > > (Please realize this proposal was sent to this list and there were > follow up emails that I'm not recapping here) > > ---- > > Hi All, > > I've been digging around in the cli a bit lately, and I wanted to > propose some common options and option names for our cli. Please note > that this is semantic nit-picking. :) > > We use a --id option for most of our actions that pertain to which ever > collection of resources the command is meant to manipulate. > E.G in "pulp-admin repo sync --id my-repo", the 'my-repo' is a > repository id. And it'll mean something else for the consumer command, > the package command, etc. > > Instead of using options called 'id' or even 'consumerid', 'repoid', > etc. We should use options that associate themselves with the resources > and provide short names for all of them. We should use this convention > on all commands. This is what I propose: > -c, --consumer : consumer id > -C, --consumer-group : consumer group id > -e, --errata : errata id > -p, --package : package name > -P, --package-group : package group id > -r, --repo : repository id > -u, --username : username > > I think --id is ambiguous and using concatenated --conumserid or > --repoid is probably unnecessary. These proposed changes will make the > cli more consistent and intuitive for our end users. > - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNVBWDAAoJEOMmcTqOSQHC3e4H/2jglRjjB+GXIZC/7YKKTEfC 3QcGmkyYr0jW5fkm2l8S0xsufFxWmviyd8pBhP9CtS7mYGvjIMSz6TlX9rtJp32w mk4cLLB2MV+yusUYoshXkTljQO2IubWPeExZYHtlI8gyWEEkyfZF6ot93MI0ewTj k7RiNXs97+w0VSHgy60sHsde2PnL0XXp27IOrQzNZXC3f21dONXcDpVE7Ug5G4ry anLCvV+1gHJ3AHKc9D4rr0Atv+7ZGPvvOiZ4Ed4amrQYPjRF0xQoopvZO8OyyQzC j9C4VXnstuc+fX8/Bqqy9Rd4koH6JB/9t/mtSzJb4xtBtVyLY1RRdmYGWAzQHFg= =8dRa -----END PGP SIGNATURE----- From jortel at redhat.com Mon Feb 14 17:21:08 2011 From: jortel at redhat.com (Jeff Ortel) Date: Mon, 14 Feb 2011 11:21:08 -0600 Subject: [Pulp-list] QE Build: 0.138 Message-ID: <4D596484.9070301@redhat.com> The pulp 0.138 QE build released. Looks like mostly bug fixes. I'm in the process of verifying the RPMs on a fresh VM but in the interest of time, thought I'd go ahead and get the QE folks looking at this. From jortel at redhat.com Mon Feb 14 23:45:33 2011 From: jortel at redhat.com (Jeff Ortel) Date: Mon, 14 Feb 2011 17:45:33 -0600 Subject: [Pulp-list] Community Release 7 Released Message-ID: <4D59BE9D.6090207@redhat.com> Pulp Community Release 7 is available. This release features ~43 bugs fixed and several new features/enhancements as follows: * Role Based Authorization * Selective Repository Synchronization with Dependency Resolution * Delta-RPM Local Repository Synchronization * Enhanced Feedback During Repository Synchronization in Foreground * Simpler Client Configuration (see: https://fedorahosted.org/pulp/wiki/UGInstallation) * Database Schema Version 2. Builds: * Pulp: 0.138 * Grinder: 0.79 * Gofer: 0.19 Notes: * pymongo upgrade to 1.9 Upgrades of pulp require a manual step to address a name change to "pymongo". Pulp had a dependency on a custom packaged version of pymongo called python-pymongo. We have changed to using pymongo from Fedora/EPEL. The manual step required prior to upgrade is: "sudo rpm -e --nodeps python-pymongo" * /etc/rc.d/init.d/pulp-agent added (on client) and replaces users working with the goferd service directly Users should do: # service pulp-admin (start|stop|status|restart) instead of: # service goferd (start|stop|status|restart) * On the client, the /etc/gofer/agent.conf is no longer updated when configuring the pulp agent to connect to a remote QPID broker. Changing the host in /etc/pulp/client.conf [server] is sufficient. * The mongo database on the pulp server has changed from a default name of _database to pulp_database. You can continue to use the existing database by editing /etc/pulp/pulp.conf file to look like: [database] ... name: _database Or you can export your data to the new database using the mongo client: $ mongo > use _database > db.copyDatabase("_database", "pulp_database", ) From jortel at redhat.com Wed Feb 16 19:37:16 2011 From: jortel at redhat.com (Jeff Ortel) Date: Wed, 16 Feb 2011 13:37:16 -0600 Subject: [Pulp-list] QE Build: 0.139 In-Reply-To: <4D596484.9070301@redhat.com> References: <4D596484.9070301@redhat.com> Message-ID: <4D5C276C.3000300@redhat.com> The pulp 0.139 QE build released. From jconnor at redhat.com Wed Feb 16 20:31:31 2011 From: jconnor at redhat.com (Jason L Connor) Date: Wed, 16 Feb 2011 13:31:31 -0700 Subject: [Pulp-list] pulp client changes Message-ID: <1297888291.21343.16.camel@precision> Hi All, I've just pushed some disruptive changes to the client. In order to facilitate unittests on the client, I refactored pulp.client.connection. I abstracted out the restlib stuff into a "server" class representing the server that the client talks to and moved all of this into a new pulp.client.server module. We have retained the ability to specify a pulp server on the command line. However we now use --host instead of --server, along with --port, --scheme, and --path to match the entries from the server section of the client.conf file. I also broke out the rest of the 'connections' into a client-side api package: pulp.client.api.. This connections module was 1000 lines long, which is just plain silly in python. The new api classes seamlessly use the new server classes. The biggest changes here are that the request methods of the server return both the http status and the body of the response instead of just the body now. And on errors, it raises a ServerRequestError that takes the status and the body as its arguments and can be accessed via the fields: args[0] and args[1]. The server does *not* attempt to automatically handle any of these errors. It has been left to the api authors to figure out what to do. If the error is allowed to propigate, the action class will catch it, print out the body (potentially a server-side traceback) and exit. I'm happy to give a deep dive tomorrow to further go into the changes. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jortel at redhat.com Thu Feb 17 02:20:10 2011 From: jortel at redhat.com (Jeff Ortel) Date: Wed, 16 Feb 2011 20:20:10 -0600 Subject: [Pulp-list] Community Release 8 In-Reply-To: <4D59BE9D.6090207@redhat.com> References: <4D59BE9D.6090207@redhat.com> Message-ID: <4D5C85DA.6060700@redhat.com> Pulp Community Release 8 is available. This is a quick follow up to Release 7 needed to address a critical bug. From lzap+fed at redhat.com Thu Feb 17 13:35:41 2011 From: lzap+fed at redhat.com (Lukas Zapletal) Date: Thu, 17 Feb 2011 14:35:41 +0100 Subject: [Pulp-list] Devsetup certificates Message-ID: <4D5D242D.4000506@redhat.com> Hello, I am a little bit confused about certificates in my developer setup which I did according to the wiki pages. If I search for SSL related configuration in my httpd I see this: # grep -R crt /etc/httpd /etc/httpd/conf/httpd.conf:AddType application/x-x509-ca-cert .crt /etc/httpd/conf.d/pulp.conf:AddType application/x-x509-ca-cert .crt /etc/httpd/conf.d/pulp.conf:SSLCACertificateFile /etc/pki/pulp/ca.crt /etc/httpd/conf.d/ssl.conf:SSLCertificateFile /etc/pki/tls/certs/localhost.crt /etc/httpd/conf.d/ssl.conf:#SSLCertificateChainFile /etc/pki/tls/certs/server-chain.crt /etc/httpd/conf.d/ssl.conf:#SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt The server cert is configured twice. I can see my httpd uses the one configured in ssl.conf (/etc/pki/tls/certs/localhost.crt): # openssl s_client -connect myhost:443 What is the /etc/pki/pulp/ca.crt for? Thanks -- Later, Lukas "lzap" Zapletal From lzap+fed at redhat.com Thu Feb 17 14:19:39 2011 From: lzap+fed at redhat.com (Lukas Zapletal) Date: Thu, 17 Feb 2011 15:19:39 +0100 Subject: [Pulp-list] Simple ping service Message-ID: <4D5D2E7B.5040705@redhat.com> Hello, I have created a very simple ping service that just respond with current timestamp. I use it for my "alive" script. I prefer such a simple service because other services could likely pull some data from database. The ping service should be fast. It also does not require any authentication. It could return the pulp version rather than current time but I did not find a way to retrieve version. I guess its not stored in the codebase yet. What you think? -- Later, Lukas "lzap" Zapletal -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-New-dummy-webservice-ping.patch URL: From mwagner at redhat.com Thu Feb 17 14:39:33 2011 From: mwagner at redhat.com (Mark Wagner) Date: Thu, 17 Feb 2011 09:39:33 -0500 Subject: [Pulp-list] Simple ping service In-Reply-To: <4D5D2E7B.5040705@redhat.com> References: <4D5D2E7B.5040705@redhat.com> Message-ID: <4D5D3325.8050804@redhat.com> On 02/17/2011 09:19 AM, Lukas Zapletal wrote: > Hello, > > I have created a very simple ping service that just respond with current timestamp. I use it for my "alive" script. I prefer such a simple service because other services could likely pull some data > from database. The ping service should be fast. It also does not require any authentication. > > It could return the pulp version rather than current time but I did not find a way to retrieve version. I guess its not stored in the codebase yet. > > What you think? > > Hmmm, seems like that would say that the server is up but gives no indication that the connection to the database is still alive, etc. Perhaps storing the version or even updating a counter in the DB would ensure that the entire path is alive. You could also expand the scope and measure the latency for a db write to get an idea of the load / response times. Disclaimer, not sure of the initial scope of your work so this may be a bit off... -mark > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From lzap+fed at redhat.com Thu Feb 17 15:57:16 2011 From: lzap+fed at redhat.com (Lukas Zapletal) Date: Thu, 17 Feb 2011 16:57:16 +0100 Subject: [Pulp-list] Simple ping service In-Reply-To: <4D5D3325.8050804@redhat.com> References: <4D5D2E7B.5040705@redhat.com> <4D5D3325.8050804@redhat.com> Message-ID: <4D5D455C.1050104@redhat.com> On 02/17/2011 03:39 PM, Mark Wagner wrote: > Hmmm, seems like that would say that the server is up but gives no > indication that the connection to the database is still alive, etc. > Perhaps storing the version or even updating a counter in the DB > would ensure that the entire path is alive. You could also expand > the scope and measure the latency for a db write to get an idea of > the load / response times. > > Disclaimer, not sure of the initial scope of your work so this may > be a bit off... Good idea, I can do that. -- Later, Lukas "lzap" Zapletal From thomas.vonsteiger at bluewin.ch Fri Feb 18 07:01:42 2011 From: thomas.vonsteiger at bluewin.ch (Thomas von Steiger) Date: Fri, 18 Feb 2011 08:01:42 +0100 Subject: [Pulp-list] Installl pulp on centos5 failed because of missing qpid-cpp-server and co Message-ID: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> Hi, I try to install pulp on centos5 with the information from http://pulpproject.org/ug/UGInstallCentOS.html This failed because of missing packet qpid-cpp-server and co. Where i can find the right srpm to solve this ? pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems --> Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch (fedora-pulp) pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems --> Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch (fedora-pulp) pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems --> Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch (fedora-pulp) Error: Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch (fedora-pulp) Error: Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch (fedora-pulp) Error: Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch (fedora-pulp) Thanks, Thomas From lzap+fed at redhat.com Fri Feb 18 10:12:05 2011 From: lzap+fed at redhat.com (Lukas Zapletal) Date: Fri, 18 Feb 2011 11:12:05 +0100 Subject: [Pulp-list] Simple ping service In-Reply-To: <4D5D3325.8050804@redhat.com> References: <4D5D2E7B.5040705@redhat.com> <4D5D3325.8050804@redhat.com> Message-ID: <4D5E45F5.5060503@redhat.com> On 02/17/2011 03:39 PM, Mark Wagner wrote: > > Hmmm, seems like that would say that the server is up but gives no > indication that the connection to the database is still alive, etc. > Perhaps storing the version or even updating a counter in the DB > would ensure that the entire path is alive. You could also expand > the scope and measure the latency for a db write to get an idea of > the load / response times. > > Disclaimer, not sure of the initial scope of your work so this may > be a bit off... Before I start working I would like to discuss the possibilities. I would create new document in Mongo called "properties" (?) with a key-value structure. And two initial values, something like: { key: "version", value: "0.138" } { key: "ping_count", value: "0" } The ping service would return this (increasing the counter every time): { result: "ok", version: "0.138" } During the database migration the version number would be set. Is this acceptable? Thanks -- Later, Lukas "lzap" Zapletal From jortel at redhat.com Fri Feb 18 13:53:53 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 18 Feb 2011 07:53:53 -0600 Subject: [Pulp-list] Installl pulp on centos5 failed because of missing qpid-cpp-server and co In-Reply-To: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> References: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> Message-ID: <4D5E79F1.3000608@redhat.com> On 02/18/2011 01:01 AM, Thomas von Steiger wrote: > Hi, > > I try to install pulp on centos5 with the information from http://pulpproject.org/ug/UGInstallCentOS.html > This failed because of missing packet qpid-cpp-server and co. > Where i can find the right srpm to solve this ? The qpidc-0.5.752581-17.el5.src.rpm builds all the QPID packages you need. Let me know if you have any problems building. > > pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems > --> Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch (fedora-pulp) > pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems > --> Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch (fedora-pulp) > pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems > --> Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch (fedora-pulp) > Error: Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch (fedora-pulp) > Error: Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch (fedora-pulp) > Error: Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch (fedora-pulp) > > > Thanks, > Thomas > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jortel at redhat.com Fri Feb 18 14:38:49 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 18 Feb 2011 08:38:49 -0600 Subject: [Pulp-list] Installl pulp on centos5 failed because of missing qpid-cpp-server and co In-Reply-To: <4D5E79F1.3000608@redhat.com> References: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> <4D5E79F1.3000608@redhat.com> Message-ID: <4D5E8479.7060301@redhat.com> I'll update the wiki and User Guide to define the RPMs produced by building the specified SRPMS. And, instructions to install them. On 02/18/2011 07:53 AM, Jeff Ortel wrote: > > On 02/18/2011 01:01 AM, Thomas von Steiger wrote: >> Hi, >> >> I try to install pulp on centos5 with the information from >> http://pulpproject.org/ug/UGInstallCentOS.html >> This failed because of missing packet qpid-cpp-server and co. >> Where i can find the right srpm to solve this ? > > The qpidc-0.5.752581-17.el5.src.rpm builds all the QPID packages you need. Let me know if > you have any problems building. > >> >> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >> --> Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch >> (fedora-pulp) >> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >> --> Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch >> (fedora-pulp) >> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >> --> Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >> (fedora-pulp) >> Error: Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >> (fedora-pulp) >> Error: Missing Dependency: qpid-cpp-server-ssl is needed by package >> pulp-0.0.139-1.noarch (fedora-pulp) >> Error: Missing Dependency: qpid-cpp-server-store is needed by package >> pulp-0.0.139-1.noarch (fedora-pulp) >> >> >> Thanks, >> Thomas >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From pkilambi at redhat.com Fri Feb 18 14:57:37 2011 From: pkilambi at redhat.com (Pradeep Kilambi) Date: Fri, 18 Feb 2011 09:57:37 -0500 Subject: [Pulp-list] Cert based requests Broken? Message-ID: <4D5E88E1.9090004@redhat.com> Few of us already ran into this. But looks like the client is not able to handle any cert based requests. If you do a client command you get: making connection with: /root/.pulp/user-cert.pem, /root/.pulp/user-key.pem Traceback (most recent call last): File "/usr/bin/pulp-admin", line 7, in execfile(__file__) File "/home/pkilambi/code/pulp/bin/pulp-admin", line 149, in admin.main() File "/home/pkilambi/code/pulp/src/pulp/client/cli/base.py", line 137, in main command.main(args[1:]) File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line 118, in main action.main(args[1:]) File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line 224, in main self.run() File "/home/pkilambi/code/pulp/src/pulp/client/core/repo.py", line 1051, in run checksum=checksum) File "/home/pkilambi/code/pulp/src/pulp/client/api/file.py", line 33, in search_file return self.server.POST(path, params)[1] File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 281, in POST return self._request('POST', path, body=body) File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 238, in _request response = connection.getresponse() File "/usr/lib64/python2.6/httplib.py", line 974, in getresponse response.begin() File "/usr/lib64/python2.6/httplib.py", line 391, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.6/httplib.py", line 355, in _read_status raise BadStatusLine(line) httplib.BadStatusLine If we use -u admin -p admin it fixes it for few commands. So my guess is the certificate based requests are broken. -u/-p uses Basic Auth which works for few commands. We should make this a priority to fix it before todays build. ~ Prad From jason.dobies at redhat.com Fri Feb 18 15:19:36 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Fri, 18 Feb 2011 10:19:36 -0500 Subject: [Pulp-list] Devsetup certificates In-Reply-To: <4D5D242D.4000506@redhat.com> References: <4D5D242D.4000506@redhat.com> Message-ID: <4D5E8E08.7070703@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/17/2011 08:35 AM, Lukas Zapletal wrote: > Hello, > > I am a little bit confused about certificates in my developer setup > which I did according to the wiki pages. > > If I search for SSL related configuration in my httpd I see this: > > # grep -R crt /etc/httpd > /etc/httpd/conf/httpd.conf:AddType application/x-x509-ca-cert .crt > /etc/httpd/conf.d/pulp.conf:AddType application/x-x509-ca-cert .crt > /etc/httpd/conf.d/pulp.conf:SSLCACertificateFile /etc/pki/pulp/ca.crt > /etc/httpd/conf.d/ssl.conf:SSLCertificateFile > /etc/pki/tls/certs/localhost.crt > /etc/httpd/conf.d/ssl.conf:#SSLCertificateChainFile > /etc/pki/tls/certs/server-chain.crt > /etc/httpd/conf.d/ssl.conf:#SSLCACertificateFile > /etc/pki/tls/certs/ca-bundle.crt > > The server cert is configured twice. I can see my httpd uses the one > configured in ssl.conf (/etc/pki/tls/certs/localhost.crt): > > # openssl s_client -connect myhost:443 > > What is the /etc/pki/pulp/ca.crt for? Pulp creates its own certificates for identification/authorization. This is the CA pulp uses to sign them. > Thanks > - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNXo4IAAoJEOMmcTqOSQHCDrUH/iky+m0D07n6yJLjJYL7Jq9j oA8yStf5wpbhu72RgTP6nJDfr+/AdPLwX9ChU3jZvaVCTIqQ5b2UW/oHwQ0Oyyhq MQw4CXZ3WVA60gF9aDLko9vHo0GTsGrB010qi4jbHmRJOVLuKyDp3l2VyQnEPaB0 wVY4Gem7grQjLw8JzEDnZgebBs7WzUL3bIQx7qETmVP/0eFfWetJ4Ym8fbhrzZVM C5xi+PuUhjeuYfdGoC3Zvj3u/OYNCQBX84YpvNrNzsL6EPxXXbod9HfvJtzo2jYr I01BTnPXx8YggiuQ9zqBCVd8EapTmmFf07w2iN2x0gFE3Ylgtaqye08tXJfpf1o= =zXvm -----END PGP SIGNATURE----- From jconnor at redhat.com Fri Feb 18 15:46:12 2011 From: jconnor at redhat.com (Jason L Connor) Date: Fri, 18 Feb 2011 08:46:12 -0700 Subject: [Pulp-list] Cert based requests Broken? In-Reply-To: <4D5E88E1.9090004@redhat.com> References: <4D5E88E1.9090004@redhat.com> Message-ID: <1298043972.21338.1.camel@precision> On Fri, 2011-02-18 at 09:57 -0500, Pradeep Kilambi wrote: > Few of us already ran into this. But looks like the client is not able > to handle any cert based requests. If you do a client command you get: > > making connection with: /root/.pulp/user-cert.pem, /root/.pulp/user-key.pem > Traceback (most recent call last): > File "/usr/bin/pulp-admin", line 7, in > execfile(__file__) > File "/home/pkilambi/code/pulp/bin/pulp-admin", line 149, in > admin.main() > File "/home/pkilambi/code/pulp/src/pulp/client/cli/base.py", line > 137, in main > command.main(args[1:]) > File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line > 118, in main > action.main(args[1:]) > File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line > 224, in main > self.run() > File "/home/pkilambi/code/pulp/src/pulp/client/core/repo.py", line > 1051, in run > checksum=checksum) > File "/home/pkilambi/code/pulp/src/pulp/client/api/file.py", line 33, > in search_file > return self.server.POST(path, params)[1] > File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 281, > in POST > return self._request('POST', path, body=body) > File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 238, > in _request > response = connection.getresponse() > File "/usr/lib64/python2.6/httplib.py", line 974, in getresponse > response.begin() > File "/usr/lib64/python2.6/httplib.py", line 391, in begin > version, status, reason = self._read_status() > File "/usr/lib64/python2.6/httplib.py", line 355, in _read_status > raise BadStatusLine(line) > httplib.BadStatusLine > > > If we use -u admin -p admin it fixes it for few commands. So my guess is > the certificate based requests are broken. -u/-p uses Basic Auth which > works for few commands. > > We should make this a priority to fix it before todays build. > > ~ Prad > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list Hey Pradep, Can you tell me what commands you're using seeing this on. Jeff was seeing this last night on package upload. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From pkilambi at redhat.com Fri Feb 18 15:51:30 2011 From: pkilambi at redhat.com (Pradeep Kilambi) Date: Fri, 18 Feb 2011 10:51:30 -0500 Subject: [Pulp-list] Cert based requests Broken? In-Reply-To: <1298043972.21338.1.camel@precision> References: <4D5E88E1.9090004@redhat.com> <1298043972.21338.1.camel@precision> Message-ID: <4D5E9582.8090304@redhat.com> On 02/18/2011 10:46 AM, Jason L Connor wrote: > On Fri, 2011-02-18 at 09:57 -0500, Pradeep Kilambi wrote: >> Few of us already ran into this. But looks like the client is not able >> to handle any cert based requests. If you do a client command you get: >> >> making connection with: /root/.pulp/user-cert.pem, /root/.pulp/user-key.pem >> Traceback (most recent call last): >> File "/usr/bin/pulp-admin", line 7, in >> execfile(__file__) >> File "/home/pkilambi/code/pulp/bin/pulp-admin", line 149, in >> admin.main() >> File "/home/pkilambi/code/pulp/src/pulp/client/cli/base.py", line >> 137, in main >> command.main(args[1:]) >> File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line >> 118, in main >> action.main(args[1:]) >> File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line >> 224, in main >> self.run() >> File "/home/pkilambi/code/pulp/src/pulp/client/core/repo.py", line >> 1051, in run >> checksum=checksum) >> File "/home/pkilambi/code/pulp/src/pulp/client/api/file.py", line 33, >> in search_file >> return self.server.POST(path, params)[1] >> File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 281, >> in POST >> return self._request('POST', path, body=body) >> File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 238, >> in _request >> response = connection.getresponse() >> File "/usr/lib64/python2.6/httplib.py", line 974, in getresponse >> response.begin() >> File "/usr/lib64/python2.6/httplib.py", line 391, in begin >> version, status, reason = self._read_status() >> File "/usr/lib64/python2.6/httplib.py", line 355, in _read_status >> raise BadStatusLine(line) >> httplib.BadStatusLine >> >> >> If we use -u admin -p admin it fixes it for few commands. So my guess is >> the certificate based requests are broken. -u/-p uses Basic Auth which >> works for few commands. >> >> We should make this a priority to fix it before todays build. >> >> ~ Prad >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > Hey Pradep, > > Can you tell me what commands you're using seeing this on. Jeff was > seeing this last night on package upload. I get this for few commands, one of which is: $ sudo pulp-admin repo sync -F --id=test_local I think any POST on repo actions is giving this from what I can tell. ~ Prad > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From jortel at redhat.com Fri Feb 18 15:59:12 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 18 Feb 2011 09:59:12 -0600 Subject: [Pulp-list] Installl pulp on centos5 failed because of missing qpid-cpp-server and co In-Reply-To: <4D5E8479.7060301@redhat.com> References: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> <4D5E79F1.3000608@redhat.com> <4D5E8479.7060301@redhat.com> Message-ID: <4D5E9750.6090506@redhat.com> Looks like http://pulpproject.org/ug/UGInstallCentOS.html references the older SRPMs. I'll update it. Thomas -- thanks for bringing this to my attention. On 02/18/2011 08:38 AM, Jeff Ortel wrote: > I'll update the wiki and User Guide to define the RPMs produced by building the specified > SRPMS. And, instructions to install them. > > On 02/18/2011 07:53 AM, Jeff Ortel wrote: >> >> On 02/18/2011 01:01 AM, Thomas von Steiger wrote: >>> Hi, >>> >>> I try to install pulp on centos5 with the information from >>> http://pulpproject.org/ug/UGInstallCentOS.html >>> This failed because of missing packet qpid-cpp-server and co. >>> Where i can find the right srpm to solve this ? >> >> The qpidc-0.5.752581-17.el5.src.rpm builds all the QPID packages you need. Let me know if >> you have any problems building. >> >>> >>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>> --> Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch >>> (fedora-pulp) >>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>> --> Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch >>> (fedora-pulp) >>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>> --> Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >>> (fedora-pulp) >>> Error: Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >>> (fedora-pulp) >>> Error: Missing Dependency: qpid-cpp-server-ssl is needed by package >>> pulp-0.0.139-1.noarch (fedora-pulp) >>> Error: Missing Dependency: qpid-cpp-server-store is needed by package >>> pulp-0.0.139-1.noarch (fedora-pulp) >>> >>> >>> Thanks, >>> Thomas >>> >>> _______________________________________________ >>> Pulp-list mailing list >>> Pulp-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jortel at redhat.com Fri Feb 18 16:15:46 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 18 Feb 2011 10:15:46 -0600 Subject: [Pulp-list] Cert based requests Broken? In-Reply-To: <4D5E9582.8090304@redhat.com> References: <4D5E88E1.9090004@redhat.com> <1298043972.21338.1.camel@precision> <4D5E9582.8090304@redhat.com> Message-ID: <4D5E9B32.8030101@redhat.com> On 02/18/2011 09:51 AM, Pradeep Kilambi wrote: > On 02/18/2011 10:46 AM, Jason L Connor wrote: >> On Fri, 2011-02-18 at 09:57 -0500, Pradeep Kilambi wrote: >>> Few of us already ran into this. But looks like the client is not able >>> to handle any cert based requests. If you do a client command you get: >>> >>> making connection with: /root/.pulp/user-cert.pem, /root/.pulp/user-key.pem >>> Traceback (most recent call last): >>> File "/usr/bin/pulp-admin", line 7, in >>> execfile(__file__) >>> File "/home/pkilambi/code/pulp/bin/pulp-admin", line 149, in >>> admin.main() >>> File "/home/pkilambi/code/pulp/src/pulp/client/cli/base.py", line >>> 137, in main >>> command.main(args[1:]) >>> File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line >>> 118, in main >>> action.main(args[1:]) >>> File "/home/pkilambi/code/pulp/src/pulp/client/core/base.py", line >>> 224, in main >>> self.run() >>> File "/home/pkilambi/code/pulp/src/pulp/client/core/repo.py", line >>> 1051, in run >>> checksum=checksum) >>> File "/home/pkilambi/code/pulp/src/pulp/client/api/file.py", line 33, >>> in search_file >>> return self.server.POST(path, params)[1] >>> File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 281, >>> in POST >>> return self._request('POST', path, body=body) >>> File "/home/pkilambi/code/pulp/src/pulp/client/server.py", line 238, >>> in _request >>> response = connection.getresponse() >>> File "/usr/lib64/python2.6/httplib.py", line 974, in getresponse >>> response.begin() >>> File "/usr/lib64/python2.6/httplib.py", line 391, in begin >>> version, status, reason = self._read_status() >>> File "/usr/lib64/python2.6/httplib.py", line 355, in _read_status >>> raise BadStatusLine(line) >>> httplib.BadStatusLine >>> >>> >>> If we use -u admin -p admin it fixes it for few commands. So my guess is >>> the certificate based requests are broken. -u/-p uses Basic Auth which >>> works for few commands. >>> >>> We should make this a priority to fix it before todays build. >>> >>> ~ Prad >>> >>> _______________________________________________ >>> Pulp-list mailing list >>> Pulp-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> Hey Pradep, >> >> Can you tell me what commands you're using seeing this on. Jeff was >> seeing this last night on package upload. FYI, the issue I saw last night is fixed when using username/password instead of cert authentication. Seems like the same root cause. > > I get this for few commands, one of which is: > > $ sudo pulp-admin repo sync -F --id=test_local > > I think any POST on repo actions is giving this from what I can tell. > > ~ Prad > >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jconnor at redhat.com Fri Feb 18 16:23:26 2011 From: jconnor at redhat.com (Jason L Connor) Date: Fri, 18 Feb 2011 09:23:26 -0700 Subject: [Pulp-list] Cert based requests Broken? In-Reply-To: <4D5E9B32.8030101@redhat.com> References: <4D5E88E1.9090004@redhat.com> <1298043972.21338.1.camel@precision> <4D5E9582.8090304@redhat.com> <4D5E9B32.8030101@redhat.com> Message-ID: <1298046206.21338.6.camel@precision> Hi All, As of hash 48fa65d8ee188775ed94b05cf5b5bf3243c85aa9, this problem is fixed. This was caused by the fact that I was caching and reusing the connection objects, instead of creating a new one on each request. This technique works just fine for the connection objects that are from the standard python httplib, which we use for http and https + basic auth connections. However, the object from the m2crypto module, which we use for ssl credentials, cannot be reused. I have forgone the caching and now create a new connection object for every single request. One more reason to love m2crypto... -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From thomas.vonsteiger at bluewin.ch Fri Feb 18 19:17:48 2011 From: thomas.vonsteiger at bluewin.ch (Thomas von Steiger) Date: Fri, 18 Feb 2011 20:17:48 +0100 Subject: [Pulp-list] Installl pulp on centos5 failed because of missing qpid-cpp-server and co In-Reply-To: <4D5E9750.6090506@redhat.com> References: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> <4D5E79F1.3000608@redhat.com> <4D5E8479.7060301@redhat.com> <4D5E9750.6090506@redhat.com> Message-ID: Thank you for your help. Looks interesting in future this project. We need to learn more about qpid rollout. Thomas On 18.02.2011, at 16:59, Jeff Ortel wrote: > Looks like http://pulpproject.org/ug/UGInstallCentOS.html references the older SRPMs. I'll update it. > > Thomas -- thanks for bringing this to my attention. > > On 02/18/2011 08:38 AM, Jeff Ortel wrote: >> I'll update the wiki and User Guide to define the RPMs produced by building the specified >> SRPMS. And, instructions to install them. >> >> On 02/18/2011 07:53 AM, Jeff Ortel wrote: >>> >>> On 02/18/2011 01:01 AM, Thomas von Steiger wrote: >>>> Hi, >>>> >>>> I try to install pulp on centos5 with the information from >>>> http://pulpproject.org/ug/UGInstallCentOS.html >>>> This failed because of missing packet qpid-cpp-server and co. >>>> Where i can find the right srpm to solve this ? >>> >>> The qpidc-0.5.752581-17.el5.src.rpm builds all the QPID packages you need. Let me know if >>> you have any problems building. >>> >>>> >>>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>>> --> Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch >>>> (fedora-pulp) >>>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>>> --> Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch >>>> (fedora-pulp) >>>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>>> --> Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >>>> (fedora-pulp) >>>> Error: Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >>>> (fedora-pulp) >>>> Error: Missing Dependency: qpid-cpp-server-ssl is needed by package >>>> pulp-0.0.139-1.noarch (fedora-pulp) >>>> Error: Missing Dependency: qpid-cpp-server-store is needed by package >>>> pulp-0.0.139-1.noarch (fedora-pulp) >>>> >>>> >>>> Thanks, >>>> Thomas >>>> >>>> _______________________________________________ >>>> Pulp-list mailing list >>>> Pulp-list at redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-list >>> >>> _______________________________________________ >>> Pulp-list mailing list >>> Pulp-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jortel at redhat.com Fri Feb 18 22:37:18 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 18 Feb 2011 16:37:18 -0600 Subject: [Pulp-list] QE build postponed to Monday Message-ID: <4D5EF49E.5080508@redhat.com> General consensus is that we have too many know issues for QE build. Most of them were addressed today but we still have enough outstanding to postpone the build. From jortel at redhat.com Fri Feb 18 22:46:20 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 18 Feb 2011 16:46:20 -0600 Subject: [Pulp-list] Installl pulp on centos5 failed because of missing qpid-cpp-server and co In-Reply-To: References: <9C424C65-88DC-4008-9DF8-586FAA790AAC@bluewin.ch> <4D5E79F1.3000608@redhat.com> <4D5E8479.7060301@redhat.com> <4D5E9750.6090506@redhat.com> Message-ID: <4D5EF6BC.6040602@redhat.com> Hey Thomas, I've updated the centos guide but did not get it posted to the web yet. He's an updated wiki page that describes: https://fedorahosted.org/pulp/wiki/UGInstallCentOS. You will need to uninstall python-qpid and install the following (additional) build deps: ruby-devel swig db4-devel libaio-devel The main problem is that the qpidc SRPM should have been: qpid-cpp-mrg which will build the qpid-cpp-* rpms. As I mentioned: You must uninstall python-qpid before building qpid-cpp-mrg or you will get patching. I'd suggest you start over from step #3. Hope this helps. Regards, Jeff On 02/18/2011 01:17 PM, Thomas von Steiger wrote: > Thank you for your help. > Looks interesting in future this project. > We need to learn more about qpid rollout. > > Thomas > > On 18.02.2011, at 16:59, Jeff Ortel wrote: > >> Looks like http://pulpproject.org/ug/UGInstallCentOS.html references the older SRPMs. I'll update it. >> >> Thomas -- thanks for bringing this to my attention. >> >> On 02/18/2011 08:38 AM, Jeff Ortel wrote: >>> I'll update the wiki and User Guide to define the RPMs produced by building the specified >>> SRPMS. And, instructions to install them. >>> >>> On 02/18/2011 07:53 AM, Jeff Ortel wrote: >>>> >>>> On 02/18/2011 01:01 AM, Thomas von Steiger wrote: >>>>> Hi, >>>>> >>>>> I try to install pulp on centos5 with the information from >>>>> http://pulpproject.org/ug/UGInstallCentOS.html >>>>> This failed because of missing packet qpid-cpp-server and co. >>>>> Where i can find the right srpm to solve this ? >>>> >>>> The qpidc-0.5.752581-17.el5.src.rpm builds all the QPID packages you need. Let me know if >>>> you have any problems building. >>>> >>>>> >>>>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>>>> --> Missing Dependency: qpid-cpp-server-ssl is needed by package pulp-0.0.139-1.noarch >>>>> (fedora-pulp) >>>>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>>>> --> Missing Dependency: qpid-cpp-server-store is needed by package pulp-0.0.139-1.noarch >>>>> (fedora-pulp) >>>>> pulp-0.0.139-1.noarch from fedora-pulp has depsolving problems >>>>> --> Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >>>>> (fedora-pulp) >>>>> Error: Missing Dependency: qpid-cpp-server is needed by package pulp-0.0.139-1.noarch >>>>> (fedora-pulp) >>>>> Error: Missing Dependency: qpid-cpp-server-ssl is needed by package >>>>> pulp-0.0.139-1.noarch (fedora-pulp) >>>>> Error: Missing Dependency: qpid-cpp-server-store is needed by package >>>>> pulp-0.0.139-1.noarch (fedora-pulp) >>>>> >>>>> >>>>> Thanks, >>>>> Thomas >>>>> >>>>> _______________________________________________ >>>>> Pulp-list mailing list >>>>> Pulp-list at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/pulp-list >>>> >>>> _______________________________________________ >>>> Pulp-list mailing list >>>> Pulp-list at redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-list >>> >>> _______________________________________________ >>> Pulp-list mailing list >>> Pulp-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > From jason.dobies at redhat.com Mon Feb 21 15:39:35 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 21 Feb 2011 10:39:35 -0500 Subject: [Pulp-list] Anyone ever seen this? Message-ID: <4D628737.5090206@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm seeing this off of master, does it look familiar to anyone? pulp-admin auth login -u admin -p admin Traceback (most recent call last): File "/usr/bin/pulp-admin", line 7, in execfile(__file__) File "/home/jdob/code/pulp/bin/pulp-admin", line 149, in admin.main() File "/home/jdob/code/pulp/src/pulp/client/cli/base.py", line 137, in main command.main(args[1:]) File "/home/jdob/code/pulp/src/pulp/client/core/base.py", line 118, in main action.main(args[1:]) File "/home/jdob/code/pulp/src/pulp/client/core/base.py", line 224, in main self.run() File "/home/jdob/code/pulp/src/pulp/client/core/auth.py", line 46, in run cert_dict = self.user_api.admin_certificate() File "/home/jdob/code/pulp/src/pulp/client/api/user.py", line 53, in admin_certificate return self.server.GET(path)[1] File "/home/jdob/code/pulp/src/pulp/client/server.py", line 269, in GET return self._request('GET', path, queries) File "/home/jdob/code/pulp/src/pulp/client/server.py", line 226, in _request url = self._build_url(path, queries) File "/home/jdob/code/pulp/src/pulp/client/server.py", line 214, in _build_url if not path.startswith(self.path_prefix): TypeError: expected a character buffer object - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNYoc3AAoJEOMmcTqOSQHCzVYIAMPWgzYrvoWA6ySKfVkRkFWc VvooThog5BpxxHyxb4dhD6GIauA4AopyObM27DamfaRgEWqRqPhHHoqrE92qowj0 mvmR/eZqy47xYj2e9WX8ISgxjLXYKsZmw6bXyNE5H69LBR0JEeGPSnEQhztuxJth VILGKMBB1w349451VAsvvpymg7nloyyuBUZRjslCy8hv3abFZXMEsaK7Ow2ViqHw F0ZuEaWai9O78K+t1Wp3wVtPsXtLjgFMSljAQyNXPq6qG/u7BQlRE7eetfoyCyn7 aCLloDA5Kmch/xan7Jipf/mIYk8Hf6F2mA6yZfpBiGzw3m+/FiQbxx9E+Ird7b4= =0zMy -----END PGP SIGNATURE----- From jason.dobies at redhat.com Mon Feb 21 15:51:43 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 21 Feb 2011 10:51:43 -0500 Subject: [Pulp-list] Anyone ever seen this? In-Reply-To: <4D628737.5090206@redhat.com> References: <4D628737.5090206@redhat.com> Message-ID: <4D628A0F.5030604@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I figured out what it was, my /etc/pulp/client.conf was out of date. In the future, please try to drop a note to the mailing list when either /etc/pulp/client.conf or /etc/pulp/pulp.conf change. I know the developer setup script symlinks these and it works in most cases, but if we need to change them locally for whatever reason (in my case, for the CDS development), we need to copy them instead so that we don't accidentally commit our changes to git. So unless we know to manually go in and update these files, we're going to run into these sorts of issues that no one else sees, which is what bit me here. Thanks :) On 02/21/2011 10:39 AM, Jay Dobies wrote: > I'm seeing this off of master, does it look familiar to anyone? > > pulp-admin auth login -u admin -p admin > Traceback (most recent call last): > File "/usr/bin/pulp-admin", line 7, in > execfile(__file__) > File "/home/jdob/code/pulp/bin/pulp-admin", line 149, in > admin.main() > File "/home/jdob/code/pulp/src/pulp/client/cli/base.py", line 137, in main > command.main(args[1:]) > File "/home/jdob/code/pulp/src/pulp/client/core/base.py", line 118, in > main > action.main(args[1:]) > File "/home/jdob/code/pulp/src/pulp/client/core/base.py", line 224, in > main > self.run() > File "/home/jdob/code/pulp/src/pulp/client/core/auth.py", line 46, in run > cert_dict = self.user_api.admin_certificate() > File "/home/jdob/code/pulp/src/pulp/client/api/user.py", line 53, in > admin_certificate > return self.server.GET(path)[1] > File "/home/jdob/code/pulp/src/pulp/client/server.py", line 269, in GET > return self._request('GET', path, queries) > File "/home/jdob/code/pulp/src/pulp/client/server.py", line 226, in > _request > url = self._build_url(path, queries) > File "/home/jdob/code/pulp/src/pulp/client/server.py", line 214, in > _build_url > if not path.startswith(self.path_prefix): > TypeError: expected a character buffer object > > _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNYooPAAoJEOMmcTqOSQHC4jcH/i38RTmnMBlwY/UJdiRyvLZs TLNg4+kOlI995x0fIGt0jHaOWkYAjstL+nmlwq2u7dXQt19o4M4g6IxOyhoA22Mi 1pzMepej0eynZaYNLpxNEbOm9UPPiTAz9vw1Tq6Z8Qa9hjD9/+JEn699TQagUW0u 7bJj51LHZ51tHmUHHcinszIhrS2Y6QDeSoNT1ss+QAFML2YPpv6QhyRG2PmK2/ba xg5+LI7XwrtOwpJ5Mgu8+3JxnJpTP7jFxbzDEcy4e/kK05Y22BY8ZKGMVDVISB8e F1YBi0fE27AJ04mweERkqoGBQ8Wc8fvma62EBrmtRrEeTgWqpFKRkJallsVQH28= =xPkY -----END PGP SIGNATURE----- From jason.dobies at redhat.com Tue Feb 22 14:47:48 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Tue, 22 Feb 2011 09:47:48 -0500 Subject: [Pulp-list] CDS Binding Approaches Message-ID: <4D63CC94.7000904@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 https://fedorahosted.org/pulp/wiki/ExternalCDSBind I put together some thoughts on how we can handle binding consumers to CDS instances instead of/in addition to the Pulp server. I apologize if it's a bit tricky to read; I'm still struggling to come up with some terms to describe the different pieces. Please try to take a look before the 1pm meeting, but this is what we'll be talking about if you don't get a chance to look ahead of time. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNY8yUAAoJEOMmcTqOSQHCC+4IAKeC4m9NwDNqTasC9sEgnHFk IVAsdAIC4rk2riKXzUhiLhPbu+/s7fQjS9QFCfwNtihCopW5BVuI9TkiHiVQgQh2 8owLDtIjqzsxTucjFKj81nVlph7/RJKZoTUQc0OYUT2W4YO7b4frOPAtk3pYS/mF sZX8GhFOCsr6ehJW9FKgSCEzCMVTOqRkGXTgQHhugo3geOq6y0McToZgaxgPuDhj wpVVldkbMMiySjRGoEmVEGEd2rPIFQphexDvEcm7tQqUg2JD9zED+8Rr+LOassUI OmE6kZZIC7qCALjW4BGkmvvZIA5J8Q//qYmoX2Irrp+G/XmNrR741Tlqr5USGJA= =gLkB -----END PGP SIGNATURE----- From lzap+fed at redhat.com Tue Feb 22 15:00:05 2011 From: lzap+fed at redhat.com (Lukas Zapletal) Date: Tue, 22 Feb 2011 16:00:05 +0100 Subject: [Pulp-list] Simple ping service In-Reply-To: <4D5E45F5.5060503@redhat.com> References: <4D5D2E7B.5040705@redhat.com> <4D5D3325.8050804@redhat.com> <4D5E45F5.5060503@redhat.com> Message-ID: <4D63CF75.4060306@redhat.com> On 02/18/2011 11:12 AM, Lukas Zapletal wrote: > I would create new document in Mongo called "properties" (?) with a > key-value structure. And two initial values, something like: > > { key: "version", value: "0.138" } > { key: "ping_count", value: "0" } > > The ping service would return this (increasing the counter every time): > > { result: "ok", version: "0.138" } > > During the database migration the version number would be set. > > Is this acceptable? No comments at all? You even like my document name "properties"? :-) I wonder if its ok to create new document from the service itself. I guess it should be somewhere in the model but I am not familiar with it yet. -- Later, Lukas "lzap" Zapletal From jortel at redhat.com Tue Feb 22 15:15:42 2011 From: jortel at redhat.com (Jeff Ortel) Date: Tue, 22 Feb 2011 09:15:42 -0600 Subject: [Pulp-list] QE Build: 0.140 Message-ID: <4D63D31E.2040209@redhat.com> QE Build: 0.140 released. Contains: * File upload enhancements: - chunked - support non-rpm uploads * Repo: allow_upload option removed * Added 'repo remove_file' command * Added 'repo add_file' command * Added: --orphaned, --repoid options to 'package list' command * Added: --csv option to 'repo add_package' & 'repo remove_package' commands * Added: --csv option to 'repo add_file' & 'repo remove_file' commands * Added --purge-files option to 'repo remove_file' command * Added: --nosig to 'package upload' command. force unsigned packages * Added: --chunksize to 'package upload' command. * Renamed: 'repo delete_package' --> 'repo remove_package' * Added Package filtering. * Client "connections" refactoring into API classes mirroring server.api package. * Many bug fixes Pulp: 0.140 Grinder: 0.80 Gofer: 0.20 From jason.dobies at redhat.com Tue Feb 22 15:46:09 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Tue, 22 Feb 2011 10:46:09 -0500 Subject: [Pulp-list] Potential big bug in QE build Message-ID: <4D63DA41.1030102@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Has anyone run the QE build with pulp-admin/pulp-client on a different machine than the server? Take a look at the following bug I just filed: https://bugzilla.redhat.com/show_bug.cgi?id=679455 In short, it doesn't look like the CLI is using the host parameter of the client.conf. I worry that means it's defaulting internally to localhost somewhere and not telling the user. So it appears to use host but isn't really. Also related bugs (but not as urgent): https://bugzilla.redhat.com/show_bug.cgi?id=679450 New command line features are not shown in help (--host, --port, - --scheme, etc) https://bugzilla.redhat.com/show_bug.cgi?id=679443 Missing credentials shows misleading error message (it says "Invalid SSL Certificate" instead of informing the user they need to either login or pass user/pass). - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNY9pBAAoJEOMmcTqOSQHCypwH/R3gnX19edxPyDCqyIW4wD3/ T34cfNnekdV6V34a94zl/UCa9g9wMP/ypTq7zYm9k9Ss26sAaR4ghSzDxcjr9MaD QcdiLgwRyR5RQcv/1ILMfPF3X4Z7xryOHwCyCIywVLVK6KsDMf9B9ViFAC38m7aI fTOX9+ki1Dzk0CUDLWimSujRl/2BC/3Dab1B5bI00KPN3AiroK3YmJb8xkt2n01S Ckr7QmvUdiunfgsNIDauHtRQPmEjXmJzC040697in3KhhTSL8GuaxsGM+72efxrF t5ZUG68haF5QQNXkWupZGDcVaX5slKZNXdglSWnWJiwy0K14Aiaw1Bhv1cFM9ao= =7Twc -----END PGP SIGNATURE----- From lzap+fed at redhat.com Tue Feb 22 16:35:54 2011 From: lzap+fed at redhat.com (Lukas Zapletal) Date: Tue, 22 Feb 2011 17:35:54 +0100 Subject: [Pulp-list] Simple ping service - solution In-Reply-To: <4D63CF75.4060306@redhat.com> References: <4D5D2E7B.5040705@redhat.com> <4D5D3325.8050804@redhat.com> <4D5E45F5.5060503@redhat.com> <4D63CF75.4060306@redhat.com> Message-ID: <4D63E5EA.8070503@redhat.com> Hello here comes my solution. It pings the database, increments the counter, measures the lag and also shows the version number. Example output: # curl -k -u admin:admin https://xxx/pulp/api/services/ping/ { "db_version": 2, "ping_duration_ms": "1.68", "ping_count": 35 } The code: class PingAction(JSONController): @JSONController.error_handler def GET(self): """ Dummy call that just prints time. @return: db_version - current DB version number """ _props_db = get_object_db('properties', ['key']) # insert the first entry if there is no such document if _props_db.find_one({"key": "ping_service_count"}) == None: _props_db.insert({"key": "ping_service_count", "value": 0}) # increment the counter and return _start_time = time.time() _props_db.update({"key": "ping_service_count"}, {"$inc": {"value": 1}}) count = _props_db.find_one( {"key": "ping_service_count"})['value'] # return the response return self.ok({ "db_version": VERSION, "ping_count": count, "ping_duration_ms": str(round((time.time() - _start_time) * 1000, 2)), }) Attaching the patch. Comments? -- Later, Lukas "lzap" Zapletal -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-New-dummy-webservice-ping.patch URL: From jconnor at redhat.com Tue Feb 22 16:44:04 2011 From: jconnor at redhat.com (Jason L Connor) Date: Tue, 22 Feb 2011 09:44:04 -0700 Subject: [Pulp-list] Simple ping service - solution In-Reply-To: <4D63E5EA.8070503@redhat.com> References: <4D5D2E7B.5040705@redhat.com> <4D5D3325.8050804@redhat.com> <4D5E45F5.5060503@redhat.com> <4D63CF75.4060306@redhat.com> <4D63E5EA.8070503@redhat.com> Message-ID: <1298393044.24374.4.camel@precision> On Tue, 2011-02-22 at 17:35 +0100, Lukas Zapletal wrote: > Hello > > here comes my solution. It pings the database, increments the counter, > measures the lag and also shows the version number. > > Example output: > > # curl -k -u admin:admin https://xxx/pulp/api/services/ping/ > { > "db_version": 2, > "ping_duration_ms": "1.68", > "ping_count": 35 > } > > The code: > > class PingAction(JSONController): > > @JSONController.error_handler > def GET(self): > """ > Dummy call that just prints time. > @return: db_version - current DB version number > """ > _props_db = get_object_db('properties', ['key']) > > # insert the first entry if there is no such document > if _props_db.find_one({"key": "ping_service_count"}) == None: > _props_db.insert({"key": "ping_service_count", "value": 0}) > > # increment the counter and return > _start_time = time.time() > _props_db.update({"key": "ping_service_count"}, > {"$inc": {"value": 1}}) > count = _props_db.find_one( > {"key": "ping_service_count"})['value'] > > # return the response > return self.ok({ > "db_version": VERSION, > "ping_count": count, > "ping_duration_ms": str(round((time.time() - > _start_time) * 1000, 2)), > }) > > Attaching the patch. > > Comments? > Hi Lukas, Sorry about our lack of responsiveness as of late. We've been pretty busy dousing fires in the code base. I'll be happy to look at this today and give you some feedback. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jmatthew at redhat.com Tue Feb 22 17:53:09 2011 From: jmatthew at redhat.com (John Matthews) Date: Tue, 22 Feb 2011 12:53:09 -0500 (EST) Subject: [Pulp-list] CDS Binding Approaches In-Reply-To: <4D63CC94.7000904@redhat.com> Message-ID: <1867113116.166227.1298397189727.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > https://fedorahosted.org/pulp/wiki/ExternalCDSBind > > I put together some thoughts on how we can handle binding consumers to > CDS instances instead of/in addition to the Pulp server. I apologize > if > it's a bit tricky to read; I'm still struggling to come up with some > terms to describe the different pieces. > > Please try to take a look before the 1pm meeting, but this is what > we'll > be talking about if you don't get a chance to look ahead of time. > > - -- While reading/thinking about this I felt there might be similarities in what we want to accomplish and what Fedora Infrastructure is doing with handling their mirrorlists. I only have a high level understanding, so I can't expand much just yet but I added some notes to the wiki. I was thinking pulp could provide a mirrorlist in metalink format, this could have extra information on CDS's available that could be used by the clients to pick which 'host' to communicate to. We'd need to learn more about what is currently supported in yum, maybe we could write a plugin so the clients could use this mirror info to determine for themselves which 'hosts' to communicate with? From jason.dobies at redhat.com Tue Feb 22 22:01:53 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Tue, 22 Feb 2011 17:01:53 -0500 Subject: [Pulp-list] repolib.py Message-ID: <4D643251.8070304@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Please try to avoid making any changes to this for the next day or so. I'm refactoring it so I can add unit tests and the necessary changes to allow the hostname(s) to come from the server. If you do need to change it for some reason, please let me know. There's no way git will resolve the diff correctly for me and I don't want to lose any new changes while I'm doing this. I'll fire off another e-mail when my changes are in. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZDJRAAoJEOMmcTqOSQHCY6cIAIsmScpizBGocAZwpZMmtIdt Qh/DCpQfLsyzC8PSXMJLEFSzGHM/z6+1kgz8o4aff3ehLPrqTZUlMd1wTE7DlNbL pI2aASiFAQzVAmJ4fmY9ruNLCSVGcHsqKway8h/1pK/UMmdlnpm6dMmcuh9OOHnj CMKXH/Wj/yk5YZfPaKS7Yyg+XXUtqXzJ1yz0OYNI/HZBsGVkXbHAo2zvD6AghNRg PXgh58f87SJu/F6aami+f/mxfKDy/1xK1+gOs50tHzDCwlLRhhMQhunsrcuUoUg4 c9vkGdzZOAO8en2N6QMWd+Ozlp9bqVFXGJTCdu4EYSGWXmmK/MaWzZgSVyTKqrk= =G8BA -----END PGP SIGNATURE----- From tsanders at redhat.com Wed Feb 23 13:52:56 2011 From: tsanders at redhat.com (Todd B Sanders) Date: Wed, 23 Feb 2011 08:52:56 -0500 Subject: [Pulp-list] CDS Binding Approaches In-Reply-To: <1867113116.166227.1298397189727.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> References: <1867113116.166227.1298397189727.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <4D651138.6010003@redhat.com> On 02/22/2011 12:53 PM, John Matthews wrote: > > ----- Original Message ----- >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> https://fedorahosted.org/pulp/wiki/ExternalCDSBind >> >> I put together some thoughts on how we can handle binding consumers to >> CDS instances instead of/in addition to the Pulp server. I apologize >> if >> it's a bit tricky to read; I'm still struggling to come up with some >> terms to describe the different pieces. >> >> Please try to take a look before the 1pm meeting, but this is what >> we'll >> be talking about if you don't get a chance to look ahead of time. >> >> - -- > While reading/thinking about this I felt there might be similarities in what we want to accomplish and what Fedora Infrastructure is doing with handling their mirrorlists. I only have a high level understanding, so I can't expand much just yet but I added some notes to the wiki. > > I was thinking pulp could provide a mirrorlist in metalink format, this could have extra information on CDS's available that could be used by the clients to pick which 'host' to communicate to. We'd need to learn more about what is currently supported in yum, maybe we could write a plugin so the clients could use this mirror info to determine for themselves which 'hosts' to communicate with? We have to be careful here. Please keep in mind that we are going to be serving more than just yum clients as we are going to handle multiple content types in the long-run; so choosing a mechanism that relies on yum for determining CDS might not be the right approach. Also, we are going to proxy httpd requests (i.e. pulp-admin) via the CDS to the Pulp Server, as well as command and control (amp events) through a broker on the CDS. -Todd > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jortel at redhat.com Wed Feb 23 18:22:14 2011 From: jortel at redhat.com (Jeff Ortel) Date: Wed, 23 Feb 2011 12:22:14 -0600 Subject: [Pulp-list] QE Build: 0.141 Message-ID: <4D655056.2060300@redhat.com> QE Build: 0.141 released. Contains: * File upload enhancements: - binary (not base64 encoded) - better support for concurrent (parallel) uploads * CLI changes: - Added: 'pulp-admin content upload' - Added: 'pulp-admin content list' command - Added: 'pulp-admin content delete' command - Removed: 'pulp-admin package upload' -- see 'content upload' * Enhanced 'pulp-amdin repo delete' to remove applicable rpms and files so long as the content is not associated to other repos * Bug fixes Pulp: 0.141 Grinder: 0.80 Gofer: 0.20 From mmccune at redhat.com Wed Feb 23 18:30:31 2011 From: mmccune at redhat.com (Mike McCune) Date: Wed, 23 Feb 2011 10:30:31 -0800 Subject: [Pulp-list] QE Build: 0.141 In-Reply-To: <4D655056.2060300@redhat.com> References: <4D655056.2060300@redhat.com> Message-ID: <4D655247.1030503@redhat.com> On 02/23/2011 10:22 AM, Jeff Ortel wrote: > QE Build: 0.141 released. > > * Enhanced 'pulp-amdin repo delete' to remove applicable rpms and files > so long as the content is not associated to other repos sweet! -- Mike McCune mmccune AT redhat.com Red Hat Engineering | Portland, OR Systems Management | 650.254.4248 From jason.dobies at redhat.com Wed Feb 23 18:43:04 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 23 Feb 2011 13:43:04 -0500 Subject: [Pulp-list] Reworking bind Message-ID: <4D655538.5050407@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A bit of an expansion on yesterday's repolib e-mail. There are some bugs and limitations in the current implementation of bind that I'm going to be addressing. I'll be working in a branch, but I wanted to give a heads up to not bother with any bind-related bugs/changes for a bit since the API/agent messages themselves are changing. Same for unbind. I'll fire off another e-mail when I merge in the branch. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZVU4AAoJEOMmcTqOSQHC5KYH/A+FFo10CLyi1Jo+4aDCPxu5 wCl7dGosSyp7v1nwrpcoCEkizd7jXqP8JZmg1o145450OEbaieYw18VrklByE2v3 Q9moSYN7bUcXOsDFELhtbKve2rTwIL2ptbc4y0Z8PZGDvADTebCk7jNyrsCzotMZ SHdjR3lsDCVUm7VXbxN64wGJKtr3b8ctPHHw6sraq/8LxFsXNRsvOCDjpY1aIWJd ZI7tmYvaM69srPwuy17SW6wdCTfDjmqeJigEBC1iolFBF7BjplMgCoJk2rvAGcrv s7Ixug1kioAI+mw9jaEb2/JDGcYFwG5VfQ7WfGvSE8A8B9CuSMudKtm13fQXS3Q= =Ypxw -----END PGP SIGNATURE----- From jason.dobies at redhat.com Wed Feb 23 18:45:03 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Wed, 23 Feb 2011 13:45:03 -0500 Subject: [Pulp-list] Speaking of branches Message-ID: <4D6555AF.9010807@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We still have a bunch of remote branches hanging around that are no longer being used. If you've made one in the past and didn't delete it when you finished merging it into master, please take a look. Either look on gitweb or run `git branch -r`. Thanks :) - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZVWvAAoJEOMmcTqOSQHCqdwIAJdflX4pAL7/DD3bj+z4hYdf 1qx/HUKlkjd64YIruO+yNr3Je0y1Qa0EDcRuLGt/CxGb66GTCT1SP6qNucVcxXAw rMlTR3Gv62NmEtE8YeV6V/z4ZiBk0nsimPwU4xG7ert5WY3u4ANLqMzEWWrm5eBl ClknA4fnjSflx8sHnHntCk7W+TQQ3AXKhBf3YI+qfY13dfrLy+8t8l9MZ0Jv661B 7sWque2iWWvBV/mc68CxUq6mRcMzEZbqszLO4jP9dX2Zyt0yYM3ztrpufhJHmGG+ 1TLqlC2CkoJmjPL0lWiKn69ZSweEsXp7588/ul8t9heprtRHl9EAYxs7aX9qccI= =sMgT -----END PGP SIGNATURE----- From jconnor at redhat.com Wed Feb 23 22:03:06 2011 From: jconnor at redhat.com (Jason L Connor) Date: Wed, 23 Feb 2011 22:03:06 +0000 Subject: [Pulp-list] Speaking of branches In-Reply-To: <4D6555AF.9010807@redhat.com> References: <4D6555AF.9010807@redhat.com> Message-ID: <1298498586.27478.3.camel@precision> On Wed, 2011-02-23 at 13:45 -0500, Jay Dobies wrote: > We still have a bunch of remote branches hanging around that are no > longer being used. If you've made one in the past and didn't delete it > when you finished merging it into master, please take a look. > > Either look on gitweb or run `git branch -r`. > > Thanks :) BTW, I deleted 15 branches that were either mine or mmccune's There's a couple left though... -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jortel at redhat.com Thu Feb 24 01:06:41 2011 From: jortel at redhat.com (Jeff Ortel) Date: Wed, 23 Feb 2011 19:06:41 -0600 Subject: [Pulp-list] QE Build: 0.142 In-Reply-To: <4D655056.2060300@redhat.com> References: <4D655056.2060300@redhat.com> Message-ID: <4D65AF21.5030200@redhat.com> QE Build: 0.142 released. Basically the same features and bug fixes as 0.141 but with additional critical bug fixes. > QE Build: 0.141 released. > > Contains: > * File upload enhancements: > - binary (not base64 encoded) > - better support for concurrent (parallel) uploads > * CLI changes: > - Added: 'pulp-admin content upload' > - Added: 'pulp-admin content list' command > - Added: 'pulp-admin content delete' command > - Removed: 'pulp-admin package upload' -- see 'content upload' > * Enhanced 'pulp-amdin repo delete' to remove applicable rpms and files > so long as the content is not associated to other repos > * Bug fixes > > Pulp: 0.141 > Grinder: 0.80 > Gofer: 0.20 > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Thu Feb 24 19:37:58 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 24 Feb 2011 14:37:58 -0500 Subject: [Pulp-list] Agent decoupling Message-ID: <4D66B396.4000401@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 As part of the bind refactor, I'm adding test cases for the bind API call. Currently, the agent call is tightly coupled into the API call: def bind(self, id, repoid): [snip] agent = Agent(id, async=True) repolib = agent.Repo() repolib.update() That doesn't let us mock out the qpidd call, so if we had unit tests for any non-async agent calls they would fail because the receiver wasn't present. It's best to just mock out something like a message bus connection in a unit test (the tests for the Agent itself can be handled separately), so I added a factory method to agent.py. Going forward, instead of instantiating an Agent object directly in the API, use the factory method (agent.retrieve_agent()). That way we can monkey patch that method in the API unit test to return a mock so we can track that the API methods are invoking the correct methods on an agent, just not doing them over a live bus. That factory method currently exists in the bind-refactor branch and should be in master in the next few days. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZrOWAAoJEOMmcTqOSQHCx0YIAJFEw0ru9LdeWgWhc7j+n/wz HBKNHHgt1Dcj8WTXjaZyqaGjPIsS/R5tg7ufXi72VkAvQvu7JKaO4OfFHMICGJff fEi1Nv6oDh9BetwNuJKJbjWfRE2J8tK698I+bvCGZdxYiVCj4sjmtpIYQAMvIv4c l07GXhRVheSZ7L3ElvYesdg4ESSLn+tecsU/3GqjiZDf4qDF97ERsKVEv8u+gaxY 1zfwoHV+IS1z7Eu5ddm7nCcNOd+h5ijFW1l1egyD3uuMZPnV0V14rYBXN+ge4UUu sErmtQ/Tg+EhSNuuOsaV43Z3/RcjJOHxNGXfXelOzs9JzuwuT6Zr84l3mSF06yk= =b+4y -----END PGP SIGNATURE----- From jason.dobies at redhat.com Thu Feb 24 20:23:56 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Thu, 24 Feb 2011 15:23:56 -0500 Subject: [Pulp-list] Agent decoupling In-Reply-To: <4D66B396.4000401@redhat.com> References: <4D66B396.4000401@redhat.com> Message-ID: <4D66BE5C.8040908@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Followup: I had to take it a step further. The usage of gofer is to call Agent().Repo().method. So using the approach I suggested still doesn't work, since you can never get a reference to the actual object created by your mock agent since it's a new object only used in a single invocation of the API call and then thrown away. I'm changing the abstraction to have factory methods for the consumer proxies themselves: def retrieve_repo_proxy(uuid, **options): ''' Utility factory method for retrieving the repo proxy to a consumer. @param uuid: uuid of the consumer @type uuid: string @param options: options to the underlying message bus @type options: dict ''' agent = retrieve_agent(uuid, **options) return agent.Repo() That way, in your unit test you make an instance of a mock with whatever testing needs you have and monkey patch the individual factory method to return that mock: class MockRepoProxy(object): def bind(self): print('Calling bind') MOCK_REPO_PROXY = MockRepoProxy() def retrieve_mock_repo_proxy(uuid, **options): return MOCK_REPO_PROXY pulp.server.agent.retrieve_repo_proxy = retrieve_mock_repo_proxy Then in the API, when bind retrieves the repo proxy it'll get my mock and I can control it's behavior to simulate error conditions and log invocations in the test. On 02/24/2011 02:37 PM, Jay Dobies wrote: > As part of the bind refactor, I'm adding test cases for the bind API > call. Currently, the agent call is tightly coupled into the API call: > > def bind(self, id, repoid): > [snip] > agent = Agent(id, async=True) > repolib = agent.Repo() > repolib.update() > > That doesn't let us mock out the qpidd call, so if we had unit tests for > any non-async agent calls they would fail because the receiver wasn't > present. > > It's best to just mock out something like a message bus connection in a > unit test (the tests for the Agent itself can be handled separately), so > I added a factory method to agent.py. > > Going forward, instead of instantiating an Agent object directly in the > API, use the factory method (agent.retrieve_agent()). That way we can > monkey patch that method in the API unit test to return a mock so we can > track that the API methods are invoking the correct methods on an agent, > just not doing them over a live bus. > > That factory method currently exists in the bind-refactor branch and > should be in master in the next few days. > _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZr5cAAoJEOMmcTqOSQHC2/oH/jwiUYINUgJ5JVOdzjDudfsh hkVGW9zaRsx0LThjpSiOvc0YqqkEqoHHCxUJ1nvVXZqOg01Yd4aviQzGG/Cqht99 fjP5sXoCKI0QuGTKs86k5kvTYg2UuwhXIBri+xnYWDPsaUsuv5nacC33Y3A0UqLx exxeFiH+5pZsJPQ6Hje60JV+DWYwsyx9Y7XQc6ZIhMkXv3zW3CMu7Emr8Xnq29AB oBcynyYZkEELz4mwjrPdAvGvDr7axFmXb8Xo29sUMCJa6gFY6I8vtBxl43i7O4GL SgclcjGUGYvAELr3oANSTW0p3j4oVhKk+5ZgcsXbzhTtyGxBl8PcGG0cmgEv3QI= =KT/q -----END PGP SIGNATURE----- From jortel at redhat.com Fri Feb 25 14:30:30 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 25 Feb 2011 08:30:30 -0600 Subject: [Pulp-list] Api.update() semantics change Message-ID: <4D67BD06.4010401@redhat.com> All, Be aware that I'm in the process of changing the xxApi.update() semantics as described here: https://engineering.redhat.com/trac/mgmt-integrated/wiki/PulpUpdateSemantics. This involves changes to the Api.update() itself and the corresponding WS controller. Also, we have a lot of unit tests that hack on model objects then call update() expecting it to just be stored instead of calling the proper API method (which was probably added after the test was written). As I work through the unit tests, I'm replacing calls to Api.update() with the proper call if it exists. But, in some cases there is none so I'm replacing with Model.get_collection().save(). I'm starting with the RepoApi because it's the poster child for this and probably doing ConsumerApi next. I'll push after each API is done to minimize merge conflicts. I'm also converting to using Model.get_collection() instead of BaseApi CRUD methods while I'm there. -jeff From jason.dobies at redhat.com Fri Feb 25 14:31:44 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Fri, 25 Feb 2011 09:31:44 -0500 Subject: [Pulp-list] Api.update() semantics change In-Reply-To: <4D67BD06.4010401@redhat.com> References: <4D67BD06.4010401@redhat.com> Message-ID: <4D67BD50.3040509@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/25/2011 09:30 AM, Jeff Ortel wrote: > All, > > Be aware that I'm in the process of changing the xxApi.update() > semantics as described here: > https://engineering.redhat.com/trac/mgmt-integrated/wiki/PulpUpdateSemantics. > This involves changes to the Api.update() itself and the corresponding > WS controller. Also, we have a lot of unit tests that hack on model > objects then call update() expecting it to just be stored instead of > calling the proper API method (which was probably added after the test > was written). As I work through the unit tests, I'm replacing calls to > Api.update() with the proper call if it exists. But, in some cases > there is none so I'm replacing with Model.get_collection().save(). +1, this is a really good call to replace calls to update to the collection calls themselves now that we have them. That'll keep things clean in terms of side effects. > I'm starting with the RepoApi because it's the poster child for this and > probably doing ConsumerApi next. I'll push after each API is done to > minimize merge conflicts. > > I'm also converting to using Model.get_collection() instead of BaseApi > CRUD methods while I'm there. > > -jeff > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZ71QAAoJEOMmcTqOSQHCMaMIAKXWG1kibNPhVfxGVjSgLqOo 1MSgojxUHkUJ0edhtrLUeC1bJfYS5s4PupwbB6sT14fYMiwDNznGGoKMpqpDW/yA JgKIYtDNWwIDCGddctwJz15uQGTCmdaQzI+du+x08bt6dnfPk8gkdx5eEuJM7DSY mmWErrSnu4vpWrWlpOV+eN4TEb2AG2WVc3PDZxY79IutmLyp0Yf9FcKBVf30iVde aAo4sna9vaqJl0amHNFjpBjz7vKwxuGvo3yjQLKecutNGbYZcyh7wSv3FnSYMQZI mvjKsNIdfgYrcZtdxKMK8aHLU7p2LtEgb3swRGPVb1EFtdbWbfj+LpTUhwmRCGw= =+rrv -----END PGP SIGNATURE----- From jason.dobies at redhat.com Fri Feb 25 14:35:13 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Fri, 25 Feb 2011 09:35:13 -0500 Subject: [Pulp-list] Default inconsistency Message-ID: <4D67BE21.3030200@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 In pulp.conf, we have: relative_url: /pulp/repos But in config.py, the default is set to: 'relative_url': '/pub' Is that just the config.py being out of date or is there a reason it's defaulted to /pub? - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNZ74hAAoJEOMmcTqOSQHCkOMIAIMxoskaDms94SP9EzfSA0QA tLs1ExHROoUxXnxw0GtY+QJrjkNPn9UanSordnek4vGwUzhELm1r9kvoAo1PNUyc uwjhknS0fSrV7mRsdkzTWp3+TRm8nYW2j/fnhKVQcDk7IwJoslWV0sMiDlderykP KyOhUnI0oU17YsG0svbjS00vE6VaenFdMODxxAFbHMRhoKNZ1awuqEo/t9ws/DyS TYB8vF6ptVARTQu1NiAgydPSWZUoJr5bTGGzwh28mZatvJytMPW+4sTnk74TEJW0 JS0i85m8Hdgj5wmN11KcVaKGnoIefFnuIJMSc6HqPE4ZiaMc+hTG9p3Aay+XgWw= =MlZO -----END PGP SIGNATURE----- From jortel at redhat.com Fri Feb 25 20:39:18 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 25 Feb 2011 14:39:18 -0600 Subject: [Pulp-list] QE Build: 0.143 Message-ID: <4D681376.9050108@redhat.com> QE Build: 0.143 released. Notes: * Seems at upgrading the pulp-client rpm does not reinstall the /etc/init.d/pulp-agent symlink correctly. Bug: 680427. Pulp: 0.143 Grinder: 0.81 Gofer: 0.20 From jortel at redhat.com Fri Feb 25 21:46:37 2011 From: jortel at redhat.com (Jeff Ortel) Date: Fri, 25 Feb 2011 15:46:37 -0600 Subject: [Pulp-list] Api.update() semantics change In-Reply-To: <4D67BD06.4010401@redhat.com> References: <4D67BD06.4010401@redhat.com> Message-ID: <4D68233D.1060501@redhat.com> RepoApi and associated unit tests converted and pushed to master. All nosetests are clean. Review and feedback appreciated. On 02/25/2011 08:30 AM, Jeff Ortel wrote: > All, > > Be aware that I'm in the process of changing the xxApi.update() semantics as described > here: https://engineering.redhat.com/trac/mgmt-integrated/wiki/PulpUpdateSemantics. This > involves changes to the Api.update() itself and the corresponding WS controller. Also, we > have a lot of unit tests that hack on model objects then call update() expecting it to > just be stored instead of calling the proper API method (which was probably added after > the test was written). As I work through the unit tests, I'm replacing calls to > Api.update() with the proper call if it exists. But, in some cases there is none so I'm > replacing with Model.get_collection().save(). > > I'm starting with the RepoApi because it's the poster child for this and probably doing > ConsumerApi next. I'll push after each API is done to minimize merge conflicts. > > I'm also converting to using Model.get_collection() instead of BaseApi CRUD methods while > I'm there. > > -jeff > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Mon Feb 28 13:40:38 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 28 Feb 2011 08:40:38 -0500 Subject: [Pulp-list] pulp-migrate failed Message-ID: <4D6BA5D6.3060803@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I tried running migrate on a new DB from master today and it failed, showing this in the logs: migration to data model version 1 starting migration to data model version 1 complete migration to data model version 1 started migration to data model version 1 complete migration to data model version 2 started RepoApi instance has no attribute '_getcollection' Traceback (most recent call last): File "/home/jdob/code/pulp/src/pulp/server/db/migrate/script.py", line 87, in datamodel_migration mod.migrate() File "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", line 108, in migrate _migrate_repo_model() File "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", line 63, in _migrate_repo_model collection = RepoApi()._getcollection() AttributeError: RepoApi instance has no attribute '_getcollection' migration to data model version 2 failed Is this just because we changed the model/collection handling and didn't update the migrate scripts? Is there a way we can have unit tests check migrate so we get notified if any APIs used by migrate in the future change? It'll probably be rare, but still a possibility that things used by migrate will change. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNa6XWAAoJEOMmcTqOSQHCUCkH/A7eIgeZThfnaDtOL2I8Ngfe Ix0UpcVQr4fbYqd3vAOjVi99w8iel5cE5Pk7BG3cSWRMdArd5Nh3HcGOrzfVRJIU PkTslk93I/fNlreuUvWE7ULK7ff1QJg7UFFRfx7NRpHmoLVG7IC9rp9qDurajL3D vO6Qhlp2mKyOiEIj43qbACLe+Knhj/MqlNY584zWCLK/D/5Nc4hQmRIbg4azEW0W oDrpvw0MLcjt1eH+MMWcey/m0fjUAPq4Is+qpVujV2m8am/Q8l8H+yBzwzm9C4M1 60atLw5j2PA4smypHrfN06UD+5Qy5nbZGEKWfmmjXZPjCDpNyxzh2qowCjxAbuY= =ixOX -----END PGP SIGNATURE----- From jason.dobies at redhat.com Mon Feb 28 14:29:26 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 28 Feb 2011 09:29:26 -0500 Subject: [Pulp-list] pulp-migrate failed In-Reply-To: <4D6BA5D6.3060803@redhat.com> References: <4D6BA5D6.3060803@redhat.com> Message-ID: <4D6BB146.7080806@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 FYI, I updated two.py and the validate scripts to use the new collection APIs. On 02/28/2011 08:40 AM, Jay Dobies wrote: > I tried running migrate on a new DB from master today and it failed, > showing this in the logs: > > migration to data model version 1 starting > migration to data model version 1 complete > migration to data model version 1 started > migration to data model version 1 complete > migration to data model version 2 started > RepoApi instance has no attribute '_getcollection' > Traceback (most recent call last): > File "/home/jdob/code/pulp/src/pulp/server/db/migrate/script.py", line > 87, in datamodel_migration > mod.migrate() > File > "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", line > 108, in migrate > _migrate_repo_model() > File > "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", line > 63, in _migrate_repo_model > collection = RepoApi()._getcollection() > AttributeError: RepoApi instance has no attribute '_getcollection' > > migration to data model version 2 failed > > > Is this just because we changed the model/collection handling and didn't > update the migrate scripts? Is there a way we can have unit tests check > migrate so we get notified if any APIs used by migrate in the future > change? It'll probably be rare, but still a possibility that things used > by migrate will change. > > _______________________________________________ Pulp-list mailing list Pulp-list at redhat.com https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNa7FGAAoJEOMmcTqOSQHCBSAH/0sajMu5HSyLrMnEgo/46eoe CzDinf6Zw/7AdsYuFLg4tkAnWnKw7IACi9BSKN0EytNPvE6VS7PnHqJtUFp0U5I9 4xP6V2F6jAUdAr9w1FLOr2V+trkbm4FwjZBuvbJHrXnPFF86aF7zBG/K2tKPiAGe 3a0nd++PFI7A34+YG5SlGuNn+uan8J+eBTWySxdnvgUcoXg06n7WN0A4M+wdwzQg qIVzAUS0ToKazQMn2411sG1DIeY0janCyunN0iBb90GrS/Ss4Q7ZzPsGKjYDLHv8 e7Z9nn6Oic9wxYdRZhiXPOgZAkD5ajjDyakVq84zNATEWz1X2mzNaJAKA+Olyuo= =YX8x -----END PGP SIGNATURE----- From jason.dobies at redhat.com Mon Feb 28 14:42:56 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 28 Feb 2011 09:42:56 -0500 Subject: [Pulp-list] New bind Message-ID: <4D6BB470.2060306@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I merged the bind refactor into master. If you don't symlink client.conf, you'll need to update it to pick up a few new properties (repo_file and mirror_list_dir to be exact). Let me know if you run into any issues, I'll be writing up some docs and what not on it shortly. - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNa7RwAAoJEOMmcTqOSQHC0YcIAIUaaIxKp8HDhEcCBPi4NcbC q7sQCdNBUBi3kuIoUDMjyHlDl0XLgI/TR0uy9nKHRTlL+atSoXisdYqDwRfydwls wtUjoQ/ndu0o1VG2Gh1+dw75nlEP7O9GCrSDHPZLa2MNWYoPLB5jDTsYbDTXSDw7 cBMxihHsfiJLNfbrWkyawF241tdyqJDQPJTBPkZSxO1sWSMJA6uAKL+jW+3tE/Vm 4QhJSgR7qdBgc/6qgooy9iV8GCq3IFBvwL9Mn+rgtpzUGiy10IsIjRxFJ4zc9Ts2 1x93bxPnhcrak+z6EOxVkoOSTHeEgL9WfZ3mOVMHCyE0CISIhQ/+3EpP26njiuk= =QFQI -----END PGP SIGNATURE----- From jconnor at redhat.com Mon Feb 28 15:05:23 2011 From: jconnor at redhat.com (Jason L Connor) Date: Mon, 28 Feb 2011 08:05:23 -0700 Subject: [Pulp-list] Default inconsistency In-Reply-To: <4D67BE21.3030200@redhat.com> References: <4D67BE21.3030200@redhat.com> Message-ID: <1298905523.2467.1.camel@precision> On Fri, 2011-02-25 at 09:35 -0500, Jay Dobies wrote: > In pulp.conf, we have: > > relative_url: /pulp/repos > > But in config.py, the default is set to: > > 'relative_url': '/pub' > > > Is that just the config.py being out of date or is there a reason it's > defaulted to /pub? According to the apache config, the first one is correct. It looks like when we changed it the defaults did not get updated. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jconnor at redhat.com Mon Feb 28 15:10:38 2011 From: jconnor at redhat.com (Jason L Connor) Date: Mon, 28 Feb 2011 08:10:38 -0700 Subject: [Pulp-list] pulp-migrate failed In-Reply-To: <4D6BA5D6.3060803@redhat.com> References: <4D6BA5D6.3060803@redhat.com> Message-ID: <1298905838.2467.6.camel@precision> On Mon, 2011-02-28 at 08:40 -0500, Jay Dobies wrote: > I tried running migrate on a new DB from master today and it failed, > showing this in the logs: > > migration to data model version 1 starting > migration to data model version 1 complete > migration to data model version 1 started > migration to data model version 1 complete > migration to data model version 2 started > RepoApi instance has no attribute '_getcollection' > Traceback (most recent call last): > File "/home/jdob/code/pulp/src/pulp/server/db/migrate/script.py", > line > 87, in datamodel_migration > mod.migrate() > File > "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", > line > 108, in migrate > _migrate_repo_model() > File > "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", > line > 63, in _migrate_repo_model > collection = RepoApi()._getcollection() > AttributeError: RepoApi instance has no attribute '_getcollection' > > migration to data model version 2 failed > > > Is this just because we changed the model/collection handling and > didn't > update the migrate scripts? Is there a way we can have unit tests > check > migrate so we get notified if any APIs used by migrate in the future > change? It'll probably be rare, but still a possibility that things > used > by migrate will change. When I switched over to the new Model.get_collection(), I left the existing PulpApi._getcollection() in place (just passing through to the previous call) for just this reason. If you want to remove an api call, I recommend using your ide's refactoring tools to rename it first, this will give you all of the locations that it's called. That said, the db migration stuff is notoriously hard to test because a new db will always pass. Ideas are welcome here. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jconnor at redhat.com Mon Feb 28 15:24:43 2011 From: jconnor at redhat.com (Jason L Connor) Date: Mon, 28 Feb 2011 08:24:43 -0700 Subject: [Pulp-list] Default inconsistency In-Reply-To: <1298905523.2467.1.camel@precision> References: <4D67BE21.3030200@redhat.com> <1298905523.2467.1.camel@precision> Message-ID: <1298906683.2467.7.camel@precision> On Mon, 2011-02-28 at 08:05 -0700, Jason L Connor wrote: > According to the apache config, the first one is correct. It looks > like > when we changed it the defaults did not get updated. I changed the default and pushed it up in e520a97..adff5a1 -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From jason.dobies at redhat.com Mon Feb 28 15:25:28 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 28 Feb 2011 10:25:28 -0500 Subject: [Pulp-list] pulp-migrate failed In-Reply-To: <1298905838.2467.6.camel@precision> References: <4D6BA5D6.3060803@redhat.com> <1298905838.2467.6.camel@precision> Message-ID: <4D6BBE68.1070204@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/28/2011 10:10 AM, Jason L Connor wrote: > On Mon, 2011-02-28 at 08:40 -0500, Jay Dobies wrote: >> I tried running migrate on a new DB from master today and it failed, >> showing this in the logs: >> >> migration to data model version 1 starting >> migration to data model version 1 complete >> migration to data model version 1 started >> migration to data model version 1 complete >> migration to data model version 2 started >> RepoApi instance has no attribute '_getcollection' >> Traceback (most recent call last): >> File "/home/jdob/code/pulp/src/pulp/server/db/migrate/script.py", >> line >> 87, in datamodel_migration >> mod.migrate() >> File >> "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", >> line >> 108, in migrate >> _migrate_repo_model() >> File >> "/home/jdob/code/pulp/src/pulp/server/db/migrate/versions/two.py", >> line >> 63, in _migrate_repo_model >> collection = RepoApi()._getcollection() >> AttributeError: RepoApi instance has no attribute '_getcollection' >> >> migration to data model version 2 failed >> >> >> Is this just because we changed the model/collection handling and >> didn't >> update the migrate scripts? Is there a way we can have unit tests >> check >> migrate so we get notified if any APIs used by migrate in the future >> change? It'll probably be rare, but still a possibility that things >> used >> by migrate will change. > > When I switched over to the new Model.get_collection(), I left the > existing PulpApi._getcollection() in place (just passing through to the > previous call) for just this reason. If you want to remove an api call, > I recommend using your ide's refactoring tools to rename it first, this > will give you all of the locations that it's called. > > That said, the db migration stuff is notoriously hard to test because a > new db will always pass. Ideas are welcome here. I don't have any solid ideas, but this was on a new DB. Some coverage is probably better than none at all. > > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNa75oAAoJEOMmcTqOSQHCW48IAJe9ux6pVB+FY59/RzpFlbaz OnreRSlYdsoW/NqXZJgDcTbb9TEaaOaWAN546tMOLNtUuChbL8bUYvmaz9YVAyQA byTcD1i6wQkVKq1ZirOi+pQ3qLmpqz0e1VoEBbHRbRCjXB1BaZc+a8x4ENXvtJ/6 9j4GeYYKjr7dNUFddZ4Zglqt32CywMgffVWk5fUOLQ1BXrT8i3A1akQz46wu5NEl cDY/Js3c5vXyIKgyZ6FjAgycRIkjIJM83YCaL/6N2bR6/L4cyvPAkMCjjW/8epAA 9aOV9zfk2xQhwdGpSRwYa9NR6+ram/fz3m2499XAyzx7k4EN+4UjHKATemwIVzk= =jtHT -----END PGP SIGNATURE----- From jconnor at redhat.com Mon Feb 28 18:15:26 2011 From: jconnor at redhat.com (Jason L Connor) Date: Mon, 28 Feb 2011 11:15:26 -0700 Subject: [Pulp-list] Api.update() semantics change In-Reply-To: <4D68233D.1060501@redhat.com> References: <4D67BD06.4010401@redhat.com> <4D68233D.1060501@redhat.com> Message-ID: <1298916926.22789.10.camel@precision> On Fri, 2011-02-25 at 15:46 -0600, Jeff Ortel wrote: > RepoApi and associated unit tests converted and pushed to master. All > nosetests are > clean. Review and feedback appreciated. Hi Jeff, The code looks much cleaner now, thank you for taking on this task. I do have one comment, in the RepoApi constructor are the following fields: self.localStoragePath = constants.LOCAL_STORAGE self.published_path = os.path.join(self.localStoragePath, "published", "repos") self.distro_path = os.path.join(self.localStoragePath, "published", "ks") I believe that all of these could live in the constants module. The first is just a new reference to something already in the constants module. The second two are read-only constants based on the first, and should probably live with the rest of the constants. Keeping everything that's hard-coded in pulp in on place seems desirable from a programming stand point. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From thomas.vonsteiger at bluewin.ch Mon Feb 28 20:33:46 2011 From: thomas.vonsteiger at bluewin.ch (Thomas von Steiger) Date: Mon, 28 Feb 2011 21:33:46 +0100 Subject: [Pulp-list] pulp-admin -u admin -p admin auth login failed Message-ID: <0F62A697-9E45-4157-A7FC-4D325D8CFF3C@bluewin.ch> Hi, I have installed pulp with self compiled rpm's from here https://fedorahosted.org/pulp/wiki/UGInstallCentOS This documentation is working fine for centos5! After some mongodb startup problems i think everything is running for pulp now. >From here: https://fedorahosted.org/pulp/wiki/UGQuickStart I try to change the password with: pulp-admin -u admin -p admin user update --username admin --password test --name "pulptest" or to write the credentials file with: pulp-admin -u admin -p admin auth login Both commands are waiting for something read if i use strace: brk(0xf28b000) = 0xf28b000 read(3, "\26\3\1\0000", 5) = 5 read(3, "0\205\371v>~\322\344N\v\7\271\3p\345\300\331\6,@\6\316qN1\1>\205\277\261\1q"..., 48) = 48 read(3, [root at cent5 ~]# rpm -qa| grep mongo pymongo-1.9-8.el5 mongodb-1.6.4-1.el5 mongodb-server-1.6.4-1.el5 [root at cent5 ~]# rpm -qa| grep pulp pulp-common-0.0.139-1 pulp-client-0.0.139-1 pulp-0.0.139-1 [root at cent5 ~]# rpm -qa| grep qpid qpid-cpp-server-store-0.7.946106-28 qpid-cpp-server-ssl-0.7.946106-28 qpid-cpp-server-0.7.946106-28 qpid-cpp-client-ssl-0.7.946106-28 python-qpid-0.7.946106-15 qpid-cpp-client-0.7.946106-28 [root at cent5 ~]# service mongod status mongod (pid 9483) is running... [root at enclouder-cent5 ~]# service qpidd status qpidd (pid 9503) is running... Do i need a new pulp version? thanks, Thomas From jortel at redhat.com Mon Feb 28 21:36:38 2011 From: jortel at redhat.com (Jeff Ortel) Date: Mon, 28 Feb 2011 15:36:38 -0600 Subject: [Pulp-list] Api.update() semantics change In-Reply-To: <1298916926.22789.10.camel@precision> References: <4D67BD06.4010401@redhat.com> <4D68233D.1060501@redhat.com> <1298916926.22789.10.camel@precision> Message-ID: <4D6C1566.20407@redhat.com> Thanks for taking a look. On 02/28/2011 12:15 PM, Jason L Connor wrote: > On Fri, 2011-02-25 at 15:46 -0600, Jeff Ortel wrote: >> RepoApi and associated unit tests converted and pushed to master. All >> nosetests are >> clean. Review and feedback appreciated. > > Hi Jeff, > > The code looks much cleaner now, thank you for taking on this task. > > I do have one comment, in the RepoApi constructor are the following > fields: > > self.localStoragePath = constants.LOCAL_STORAGE > self.published_path = os.path.join(self.localStoragePath, "published", "repos") > self.distro_path = os.path.join(self.localStoragePath, "published", "ks") > > I believe that all of these could live in the constants module. The > first is just a new reference to something already in the constants > module. The second two are read-only constants based on the first, and > should probably live with the rest of the constants. Keeping everything > that's hard-coded in pulp in on place seems desirable from a programming > stand point. Agreed. > > > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list From jason.dobies at redhat.com Mon Feb 28 21:36:29 2011 From: jason.dobies at redhat.com (Jay Dobies) Date: Mon, 28 Feb 2011 16:36:29 -0500 Subject: [Pulp-list] Api.update() semantics change In-Reply-To: <1298916926.22789.10.camel@precision> References: <4D67BD06.4010401@redhat.com> <4D68233D.1060501@redhat.com> <1298916926.22789.10.camel@precision> Message-ID: <4D6C155D.1040400@redhat.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 One small note, there are now two methods in test_api called test_repo_update :) On 02/28/2011 01:15 PM, Jason L Connor wrote: > On Fri, 2011-02-25 at 15:46 -0600, Jeff Ortel wrote: >> RepoApi and associated unit tests converted and pushed to master. All >> nosetests are >> clean. Review and feedback appreciated. > > Hi Jeff, > > The code looks much cleaner now, thank you for taking on this task. > > I do have one comment, in the RepoApi constructor are the following > fields: > > self.localStoragePath = constants.LOCAL_STORAGE > self.published_path = os.path.join(self.localStoragePath, "published", "repos") > self.distro_path = os.path.join(self.localStoragePath, "published", "ks") > > I believe that all of these could live in the constants module. The > first is just a new reference to something already in the constants > module. The second two are read-only constants based on the first, and > should probably live with the rest of the constants. Keeping everything > that's hard-coded in pulp in on place seems desirable from a programming > stand point. > > > > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list - -- Jay Dobies RHCE# 805008743336126 Freenode: jdob http://pulpproject.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNbBVcAAoJEOMmcTqOSQHCvwAIAIYkdB/Gs+X+Lby4lBR+vGY1 e2V6nxIlagy+3XhpzM9GavZDa7BqrB0+hJ7N1S48uGqAwfroDk4j8hWrvjrn/ZCm yRrfQoptvRqOSiBZsZZB1l6wkqvIP1l4ajj+nN+OuKFQGZp7U6FwgMpdHFuiaRlV SoR+z2C0vCAUwH4N2fV/l8K56dWf+X3gRe1buZnuP6ZzqZjub3fTOFivYSySjBi+ kh8HKmRb3wr//C+47B+6vbnoWNN6A5xKz4FrFQt76AUqHM73ItzidokcADQA6j0g PCp4G2xfGGm1Rsd7nrgj2HAR1wF4VSfDT0kNzzKZeP3XKAK1sKVmuY+wOvuPLmw= =tYWX -----END PGP SIGNATURE----- From jortel at redhat.com Mon Feb 28 23:04:40 2011 From: jortel at redhat.com (Jeff Ortel) Date: Mon, 28 Feb 2011 17:04:40 -0600 Subject: [Pulp-list] Api.update() semantics change In-Reply-To: <4D6C155D.1040400@redhat.com> References: <4D67BD06.4010401@redhat.com> <4D68233D.1060501@redhat.com> <1298916926.22789.10.camel@precision> <4D6C155D.1040400@redhat.com> Message-ID: <4D6C2A08.5070600@redhat.com> Thanks for looking. On 02/28/2011 03:36 PM, Jay Dobies wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > One small note, there are now two methods in test_api called > test_repo_update :) I think the 2nd one got added last sprint. I'll consolidate them. > > On 02/28/2011 01:15 PM, Jason L Connor wrote: >> On Fri, 2011-02-25 at 15:46 -0600, Jeff Ortel wrote: >>> RepoApi and associated unit tests converted and pushed to master. All >>> nosetests are >>> clean. Review and feedback appreciated. >> >> Hi Jeff, >> >> The code looks much cleaner now, thank you for taking on this task. >> >> I do have one comment, in the RepoApi constructor are the following >> fields: >> >> self.localStoragePath = constants.LOCAL_STORAGE >> self.published_path = os.path.join(self.localStoragePath, "published", "repos") >> self.distro_path = os.path.join(self.localStoragePath, "published", "ks") >> >> I believe that all of these could live in the constants module. The >> first is just a new reference to something already in the constants >> module. The second two are read-only constants based on the first, and >> should probably live with the rest of the constants. Keeping everything >> that's hard-coded in pulp in on place seems desirable from a programming >> stand point. >> >> >> >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list at redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > > - -- > Jay Dobies > RHCE# 805008743336126 > Freenode: jdob > http://pulpproject.org > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJNbBVcAAoJEOMmcTqOSQHCvwAIAIYkdB/Gs+X+Lby4lBR+vGY1 > e2V6nxIlagy+3XhpzM9GavZDa7BqrB0+hJ7N1S48uGqAwfroDk4j8hWrvjrn/ZCm > yRrfQoptvRqOSiBZsZZB1l6wkqvIP1l4ajj+nN+OuKFQGZp7U6FwgMpdHFuiaRlV > SoR+z2C0vCAUwH4N2fV/l8K56dWf+X3gRe1buZnuP6ZzqZjub3fTOFivYSySjBi+ > kh8HKmRb3wr//C+47B+6vbnoWNN6A5xKz4FrFQt76AUqHM73ItzidokcADQA6j0g > PCp4G2xfGGm1Rsd7nrgj2HAR1wF4VSfDT0kNzzKZeP3XKAK1sKVmuY+wOvuPLmw= > =tYWX > -----END PGP SIGNATURE----- > > _______________________________________________ > Pulp-list mailing list > Pulp-list at redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list