From rudd-o at rudd-o.com Wed Mar 11 03:34:48 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Tue, 10 Mar 2009 22:34:48 -0500 Subject: SOLVED: bdist_rpm and pre-release python packages / eggs (was: [Distutils] pre-release versioning problems with sdist, bdist_rpm, bdist_debian) Message-ID: <200903102234.50856.rudd-o@rudd-o.com> Hello, guys, I have fixed distutils (and setuptools remains working) with the attached patch. Now, RPMs will be built according to the Fedora Package Naming Guidelines: http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non- Numeric_Version_in_Release which I understand to be the most useful reference in terms of naming pre- release packages. This should work correctly in at least: - Fedora - RHEL - SUSE I urge you patch your python 2.4s and 2.5s and 2.6s and push this update to distributions. I have done that myself at my own repository. Now we can enjoy one more reason to build RPMs (and eggs! ... according to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel free to pick its brains) DIRECTLY from the cheese shop, especially if you're using pip. Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). See attached patch. I will log bugs where it corresponds too. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Aqua - Cartoon heroes Windows 95 is not a virus. Viruses actually do something. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: distutils-bdist_rpm-prereleaseversion.patch Type: text/x-patch Size: 1596 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Wed Mar 11 04:32:23 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Tue, 10 Mar 2009 23:32:23 -0500 Subject: Addendum SOLVED: bdist_rpm and pre-release python packages / eggs (was: [Distutils] pre-release versioning problems with sdist, bdist_rpm, bdist_debian) Message-ID: <200903102332.24052.rudd-o@rudd-o.com> Hello, guys, The mail I sent earlier had a patch which seems to depend upon a RHEL- distributed patch which I am now attaching (along with the original patch too). I have logged bugs in the pertinent bug trackers. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Aqua - Cartoon heroes Windows 95 is not a virus. Viruses actually do something. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: distutils-bdist_rpm-prereleaseversion.patch Type: text/x-patch Size: 1596 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.4-distutils-bdist-rpm.patch (prerequisite for the patch I wrote which is the prereleaseversion.patch) Type: text/x-patch Size: 2935 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Wed Mar 11 12:28:40 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 08:28:40 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903102234.50856.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> Message-ID: <49B7AE78.1060208@verizon.net> Manuel Amador (Rudd-O) wrote: > Hello, guys, > > > I have fixed distutils (and setuptools remains working) with the > attached patch. Now, RPMs will be built according to the Fedora > Package Naming Guidelines: > > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Version_in_Release > > > which I understand to be the most useful reference in terms of naming > pre-release packages. This should work correctly in at least: > > > - Fedora > - RHEL > - SUSE > > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this > update to distributions. I have done that myself at my own repository. > > > Now we can enjoy one more reason to build RPMs (and eggs! ... > according to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel > free to pick its brains) DIRECTLY from the cheese shop, especially if > you're using pip. > > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > > See attached patch. I will log bugs where it corresponds too. > -- > > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > Windows 95 is not a virus. Viruses actually do something. > > Hi Manuel, You worked on my problem! Great. So today what we have been doing to deal with the pre-release and lexical ordering problem involving pre-releases is this: We impose a restriction on how the pre-release is identified. So for example if you intend to end up with a final version-release of 5.0.0-1 and you want to first put out some betas or release candidates then we have to name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is so that the lexical ordering for RPM will be correct. In other words you must put the pre-release designation into the 'release' part of VERSION-RELEASE. What we had seen developers doing previously was to name these as 5.0.0_beta1 or 5.0.0_rc1 (making the pre-release designation part of the 'version' string) which then did not work for the lexical ordering of the final release of 5.0.0-1 because 5.0.0 (version) was not lexically superior to 5.0.0_rc1. So we were able to solve this problem without any code changes to distutils. But this also presented a challenge for the other distribution targets such as 'sdist' because they were totally unaware of this 'version-release' combination and only knew about 'version'. So as a workaround we were doing this: # WORKAROUND # define both version AND release version='5.0.0' release='1' # combine them for all targets except 'bdist_rpm' if sys.argv[1] != 'bdist_rpm': version = version+'-'+release So this wasn't perfect but it actually worked quite well and we could get 'sdist' to work properly in conjunction with 'bdist_rpm'. So now with your patch all the targets should be able to set and use both 'version' and 'release' and we don't need our workaround and that will be great. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Wed Mar 11 12:33:41 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 11 Mar 2009 13:33:41 +0100 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B7AE78.1060208@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> Message-ID: <94bdd2610903110533g38ea70ccg73fa8648451ce73e@mail.gmail.com> Hi, Great work ! FYI it's on my pile in the bug tracker in Python. I'll try to work on these before Pycon Also, note that I am planning to release Distutils as a standalone package before Pycon; since the current trunk targets Python 2.3 to 3.1 Regards Tarek On Wed, Mar 11, 2009 at 1:28 PM, Gerry Reno wrote: > Manuel Amador (Rudd-O) wrote: > > Hello, guys, > > I have fixed distutils (and setuptools remains working) with the attached > patch. Now, RPMs will be built according to the Fedora Package Naming > Guidelines: > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Version_in_Release > > which I understand to be the most useful reference in terms of naming > pre-release packages. This should work correctly in at least: > > - Fedora > - RHEL > - SUSE > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this update to > distributions. I have done that myself at my own repository. > > Now we can enjoy one more reason to build RPMs (and eggs! ... according to > my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel free to pick its > brains) DIRECTLY from the cheese shop, especially if you're using pip. > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > See attached patch. I will log bugs where it corresponds too. > -- > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > Windows 95 is not a virus. Viruses actually do something. > > Hi Manuel, > ? You worked on my problem!? Great. > ? So today what we have been doing to deal with the pre-release and lexical > ordering problem involving pre-releases is this: > ??? We impose a restriction on how the pre-release is identified.? So for > example if you intend to end up with a final version-release of 5.0.0-1 and > you want to first put out some betas or release candidates then we have to > name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is so that the lexical > ordering for RPM will be correct.? In other words you must put the > pre-release designation into the 'release' part of VERSION-RELEASE.? What we > had seen developers doing previously was to name these as 5.0.0_beta1 or > 5.0.0_rc1 (making the pre-release designation part of the 'version' string) > which then did not work for the lexical ordering of the final release of > 5.0.0-1 because 5.0.0 (version) was not lexically superior to 5.0.0_rc1.? So > we were able to solve this problem without any code changes to distutils. > But this also presented a challenge for the other distribution targets such > as 'sdist' because they were totally unaware of this 'version-release' > combination and only knew about 'version'.? So as a workaround we were doing > this: > # WORKAROUND > # define both version AND release > version='5.0.0' > release='1' > # combine them for all targets except 'bdist_rpm' > if sys.argv[1] != 'bdist_rpm': > ??? version = version+'-'+release > So this wasn't perfect but it actually worked quite well and we could get > 'sdist' to work properly in conjunction with 'bdist_rpm'. > So now with your patch all the targets should be able to set and use both > 'version' and 'release' and we don't need our workaround and that will be > great. > > Regards, > Gerry > > > > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From rudd-o at rudd-o.com Wed Mar 11 13:29:21 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 08:29:21 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <94bdd2610903110533g38ea70ccg73fa8648451ce73e@mail.gmail.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> <94bdd2610903110533g38ea70ccg73fa8648451ce73e@mail.gmail.com> Message-ID: <200903110829.24944.rudd-o@rudd-o.com> Great. I just pushed onto the various trackers tracking the issue -- among which is the python tracker -- support for prerelease packages (...dev and ...dev-r77655 versioned packages) so they do not overwrite alphas, betas, or final -1 and above releases. El Mi?rcoles 11 Marzo 2009, Tarek Ziad? escribi?: > Hi, > > Great work ! > > FYI it's on my pile in the bug tracker in Python. I'll try to work on > these before Pycon > > Also, note that I am planning to release Distutils as a standalone > package before Pycon; > since the current trunk targets Python 2.3 to 3.1 > > Regards > Tarek > > On Wed, Mar 11, 2009 at 1:28 PM, Gerry Reno wrote: > > Manuel Amador (Rudd-O) wrote: > > > > Hello, guys, > > > > I have fixed distutils (and setuptools remains working) with the attached > > patch. Now, RPMs will be built according to the Fedora Package Naming > > Guidelines: > > > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Vers > >ion_in_Release > > > > which I understand to be the most useful reference in terms of naming > > pre-release packages. This should work correctly in at least: > > > > - Fedora > > - RHEL > > - SUSE > > > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this update > > to distributions. I have done that myself at my own repository. > > > > Now we can enjoy one more reason to build RPMs (and eggs! ... according > > to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel free to pick > > its brains) DIRECTLY from the cheese shop, especially if you're using > > pip. > > > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > > > See attached patch. I will log bugs where it corresponds too. > > -- > > > > Manuel Amador (Rudd-O) > > Rudd-O.com - http://rudd-o.com/ > > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > > Windows 95 is not a virus. Viruses actually do something. > > > > Hi Manuel, > > ? You worked on my problem!? Great. > > ? So today what we have been doing to deal with the pre-release and > > lexical ordering problem involving pre-releases is this: > > ??? We impose a restriction on how the pre-release is identified.? So for > > example if you intend to end up with a final version-release of 5.0.0-1 > > and you want to first put out some betas or release candidates then we > > have to name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is so that > > the lexical ordering for RPM will be correct.? In other words you must > > put the pre-release designation into the 'release' part of > > VERSION-RELEASE.? What we had seen developers doing previously was to > > name these as 5.0.0_beta1 or 5.0.0_rc1 (making the pre-release > > designation part of the 'version' string) which then did not work for the > > lexical ordering of the final release of 5.0.0-1 because 5.0.0 (version) > > was not lexically superior to 5.0.0_rc1.? So we were able to solve this > > problem without any code changes to distutils. But this also presented a > > challenge for the other distribution targets such as 'sdist' because they > > were totally unaware of this 'version-release' combination and only knew > > about 'version'.? So as a workaround we were doing this: > > # WORKAROUND > > # define both version AND release > > version='5.0.0' > > release='1' > > # combine them for all targets except 'bdist_rpm' > > if sys.argv[1] != 'bdist_rpm': > > ??? version = version+'-'+release > > So this wasn't perfect but it actually worked quite well and we could get > > 'sdist' to work properly in conjunction with 'bdist_rpm'. > > So now with your patch all the targets should be able to set and use both > > 'version' and 'release' and we don't need our workaround and that will be > > great. > > > > Regards, > > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ A copy of the universe is not what is required of art; one of the damned things is ample. -- Rebecca West -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Wed Mar 11 14:21:45 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 09:21:45 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm , bdist_debian) In-Reply-To: <49B7AE78.1060208@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> Message-ID: <200903110921.48576.rudd-o@rudd-o.com> My solution tackles it from another direction, using observed egg packaging practices. This is what I have observed in setup.pys, and eggs' requires.txts so far: 1.2dev: refers to unspecified not-even-alpha quality checkout or source drop 1.2dev-r5667: refers to the checkout of revision 5667, not-even-alpha 1.2a1: refers to Alpha 1 of intended version 1.2 1.2a2: refers to Alpha 2 of intended version 1.2 1.2b1: refers to Beta 1 of intended version 1.2 1.2: refers to official 1.2 release Note NO UNDERSCORES. In order to satisfy the fedora package naming guidelines and RPM's lexicographical sort, this is what happens within my patches: 1.2dev: version 1.2 release 0.0.0dev REGARDLESS of user-specified release 1.2dev-r56: version 1.2 release 0.0.56devr REGARDLESS of user-specified rel 1.2a1: version 1.2 release 0..a1 1.2a2: version 1.2 release 0..a2 1.2b1: version 1.2 release 0..b1 (note in the last three it is the responsibility of the builder to bump the manually, there is really nothing we can do about it) 1.2: version 1.2 release In this manner we can accurately map existing practice (what people are actually DOING at least in a significant amount of eggs -- the plone ones) in python egg versioning numbers to RPM lexicographical sort in a manner that lets package managers "do the right thing". Also, we need a standard way of trolling the requires.txt for the egg dependencies and transforming them into RPM dependencies, otherwise your RPMs won't work in concert. Code for that in my workbench yum.rudd-o.com subdir SCRIPTS, it is documented step by step, but this should really be integrated within the setuptools bdist_rpm kludge. Really. I am exhausted. El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > Manuel Amador (Rudd-O) wrote: > > Hello, guys, > > > > > > I have fixed distutils (and setuptools remains working) with the > > attached patch. Now, RPMs will be built according to the Fedora > > Package Naming Guidelines: > > > > > > http://fedoraproject.org/wiki/Packaging/NamingGuidelines#Non-Numeric_Vers > >ion_in_Release > > > > > > which I understand to be the most useful reference in terms of naming > > pre-release packages. This should work correctly in at least: > > > > > > - Fedora > > - RHEL > > - SUSE > > > > > > I urge you patch your python 2.4s and 2.5s and 2.6s and push this > > update to distributions. I have done that myself at my own repository. > > > > > > Now we can enjoy one more reason to build RPMs (and eggs! ... > > according to my workbench at http://yum.rudd-o.com/SCRIPTS/ -- feel > > free to pick its brains) DIRECTLY from the cheese shop, especially if > > you're using pip. > > > > > > Oh, I also have pip at my repo (cd ../RPMS/noarch in my workbench). > > > > > > See attached patch. I will log bugs where it corresponds too. > > -- > > > > > > Manuel Amador (Rudd-O) > > Rudd-O.com - http://rudd-o.com/ > > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > > > > Now playing, courtesy of Amarok: Aqua - Cartoon heroes > > Windows 95 is not a virus. Viruses actually do something. > > Hi Manuel, > You worked on my problem! Great. > So today what we have been doing to deal with the pre-release and > lexical ordering problem involving pre-releases is this: > We impose a restriction on how the pre-release is identified. So > for example if you intend to end up with a final version-release of > 5.0.0-1 and you want to first put out some betas or release candidates > then we have to name them as, 5.0.0-0_beta1, or 5.0.0-0_rc1 and this is > so that the lexical ordering for RPM will be correct. In other words > you must put the pre-release designation into the 'release' part of > VERSION-RELEASE. What we had seen developers doing previously was to > name these as 5.0.0_beta1 or 5.0.0_rc1 (making the pre-release > designation part of the 'version' string) which then did not work for > the lexical ordering of the final release of 5.0.0-1 because 5.0.0 > (version) was not lexically superior to 5.0.0_rc1. So we were able to > solve this problem without any code changes to distutils. But this also > presented a challenge for the other distribution targets such as 'sdist' > because they were totally unaware of this 'version-release' combination > and only knew about 'version'. So as a workaround we were doing this: > # WORKAROUND > # define both version AND release > version='5.0.0' > release='1' > # combine them for all targets except 'bdist_rpm' > if sys.argv[1] != 'bdist_rpm': > version = version+'-'+release > So this wasn't perfect but it actually worked quite well and we could > get 'sdist' to work properly in conjunction with 'bdist_rpm'. > So now with your patch all the targets should be able to set and use > both 'version' and 'release' and we don't need our workaround and that > will be great. > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Example is not the main thing in influencing others. It is the only thing. -- Albert Schweitzer -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Wed Mar 11 14:51:47 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 10:51:47 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903110921.48576.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B7AE78.1060208@verizon.net> <200903110921.48576.rudd-o@rudd-o.com> Message-ID: <49B7D003.80305@verizon.net> Manuel Amador (Rudd-O) wrote: > My solution tackles it from another direction, using observed egg > packaging practices. This is what I have observed in setup.pys, and > eggs' requires.txts so far: > > > 1.2dev: refers to unspecified not-even-alpha quality checkout or > source drop > 1.2dev-r5667: refers to the checkout of revision 5667, not-even-alpha > 1.2a1: refers to Alpha 1 of intended version 1.2 > 1.2a2: refers to Alpha 2 of intended version 1.2 > 1.2b1: refers to Beta 1 of intended version 1.2 > 1.2: refers to official 1.2 release > > > Note NO UNDERSCORES. In order to satisfy the fedora package naming > guidelines and RPM's lexicographical sort, this is what happens within > my patches: > > > 1.2dev: version 1.2 release 0.0.0dev REGARDLESS of user-specified release > 1.2dev-r56: version 1.2 release 0.0.56devr REGARDLESS of > user-specified rel > 1.2a1: version 1.2 release 0..a1 > 1.2a2: version 1.2 release 0..a2 > 1.2b1: version 1.2 release 0..b1 > (note in the last three it is the responsibility of the builder to > bump the manually, there is really nothing we can do about it) > 1.2: version 1.2 release > > > In this manner we can accurately map existing practice (what people > are actually DOING at least in a significant amount of eggs -- the > plone ones) in python egg versioning numbers to RPM lexicographical > sort in a manner that lets package managers "do the right thing". > > > Also, we need a standard way of trolling the requires.txt for the egg > dependencies and transforming them into RPM dependencies, otherwise > your RPMs won't work in concert. Code for that in my workbench > yum.rudd-o.com subdir SCRIPTS, it is documented step by step, but this > should really be integrated within the setuptools bdist_rpm kludge. > Really. > > > I am exhausted. Hi Manuel, Let me share with you what we had implemented and the POLICY that we put in place with regards to VERSION-RELEASE strings. We did not look at this from egg standpoint as we just distribute RPMS. I sent this to Tarek a while ago: What I had implemented seems to have the VERSION-RELEASE string issue solved using a 'workaround' and a manual policy for the moment. It would of course be better if distutils just did this automatically but this is at least working. Tarballs, rpms have the same designations and all the lexical ordering is correct so that even for pre-release versioning the rpms update each other correctly. #WORKAROUND: What I did was in setup.py added: ======================== version = '5.0.0' release = '0_rc3_00001681' if sys.argv[1] != 'bdist_rpm': version = version + '-' + release ======================== In setup.cfg added: ======================== [bdist_rpm] # release must exactly match 'release' as set in setup.py release=0_rc3_00001681 ================== # END WORKAROUND And then I setup the following policy regarding VERSION-RELEASE naming: (this example uses bzr as source code control system but you can substitute svn, rcs, or any other type of sccs) Development code: version="5.0.0" release="0_00012345" (where 00012345 is the revision number from bzr in ZERO-PADDED FIELD OF 8) tarball: foo-5.0.0-0_00012345.tar.gz rpm: foo-5.0.0-0_00012345.noarch.rpm Alpha code: version="5.0.0" release="0_alpha1_00123456" tarball: foo-5.0.0-0_alpha1_00123456.tar.gz rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm Beta code: version="5.0.0" release="0_beta1_00234567" tarball: foo-5.0.0-0_beta1_00234567.tar.gz rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm Release Candidate code: version="5.0.0" release="0_rc1" tarball: foo-5.0.0-0_rc1.tar.gz rpm: foo-5.0.0-0_rc1.noarch.rpm Release Candidate code fix: version="5.0.0" release="0_rc1_00345678" tarball: foo-5.0.0-0_rc1_00345678.tar.gz rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm Final Release code: version="5.0.0" release="1" tarball: foo-5.0.0-1.tar.gz rpm: foo-5.0.0-1.noarch.rpm Notice that lexical ordering is proper in all cases. Even where the alpha, beta, rc releases may be followed by a bzr fix revision. The ensures that both tarballs and packages such as rpm will contain the same VERSION-RELEASE designations. Also, the use of UNDERSCORES in the 'release' string assures that there is no ambiguity with regard to parsing VERSION-RELEASE string combination or full package names which have hyphens in specific positions. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Wed Mar 11 21:17:50 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 16:17:50 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B7D003.80305@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903110921.48576.rudd-o@rudd-o.com> <49B7D003.80305@verizon.net> Message-ID: <200903111617.50953.rudd-o@rudd-o.com> Gerry, What you have said is interesting. Note that you're using underscores in the release number (against F packaging policy), and the general scheme for devising the release number is also against Fedora packaging policy too (which is highly likely to cause problems in RHEL down the road too). Sure, it works too, but it'd be good if we could stick to the published policies which are also known to work and everybody adheres to them already. > Hi Manuel, > Let me share with you what we had implemented and the POLICY that we > put in place with regards to VERSION-RELEASE strings. We did not look > at this from egg standpoint as we just distribute RPMS. > > I sent this to Tarek a while ago: > What I had implemented seems to have the VERSION-RELEASE string issue > solved using a 'workaround' and a manual policy for the moment. It > would of course be better if distutils just did this automatically but > this is at least working. Tarballs, rpms have the same designations and > all the lexical ordering is correct so that even for pre-release > versioning the rpms update each other correctly. > > #WORKAROUND: > What I did was in setup.py added: > ======================== > version = '5.0.0' > release = '0_rc3_00001681' > > if sys.argv[1] != 'bdist_rpm': > version = version + '-' + release > ======================== > > In setup.cfg added: > ======================== > [bdist_rpm] > # release must exactly match 'release' as set in setup.py > release=0_rc3_00001681 > ================== > # END WORKAROUND > > > > And then I setup the following policy regarding VERSION-RELEASE naming: > (this example uses bzr as source code control system but you can > substitute svn, rcs, or any other type of sccs) > > Development code: > version="5.0.0" > release="0_00012345" (where 00012345 is the revision number from > bzr in ZERO-PADDED FIELD OF 8) > tarball: foo-5.0.0-0_00012345.tar.gz > rpm: foo-5.0.0-0_00012345.noarch.rpm > > Alpha code: > version="5.0.0" > release="0_alpha1_00123456" > tarball: foo-5.0.0-0_alpha1_00123456.tar.gz > rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm > > Beta code: > version="5.0.0" > release="0_beta1_00234567" > tarball: foo-5.0.0-0_beta1_00234567.tar.gz > rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm > > Release Candidate code: > version="5.0.0" release="0_rc1" > tarball: foo-5.0.0-0_rc1.tar.gz > rpm: foo-5.0.0-0_rc1.noarch.rpm > > Release Candidate code fix: > version="5.0.0" > release="0_rc1_00345678" > tarball: foo-5.0.0-0_rc1_00345678.tar.gz > rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm > > Final Release code: > version="5.0.0" > release="1" > tarball: foo-5.0.0-1.tar.gz > rpm: foo-5.0.0-1.noarch.rpm > > Notice that lexical ordering is proper in all cases. Even where the > alpha, beta, rc releases may be followed by a bzr fix revision. > The ensures that both tarballs and packages such as rpm will contain the > same VERSION-RELEASE designations. > > Also, the use of UNDERSCORES in the 'release' string assures that there > is no ambiguity with regard to parsing VERSION-RELEASE string > combination or full package names which have hyphens in specific positions. > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Reality always seems harsher in the early morning. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Thu Mar 12 00:13:33 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 20:13:33 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903111617.50953.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903110921.48576.rudd-o@rudd-o.com> <49B7D003.80305@verizon.net> <200903111617.50953.rudd-o@rudd-o.com> Message-ID: <49B853AD.5090807@verizon.net> Manuel Amador (Rudd-O) wrote: > Gerry, > > > What you have said is interesting. Note that you're using underscores > in the release number (against F packaging policy), and the general > scheme for devising the release number is also against Fedora > packaging policy too (which is highly likely to cause problems in RHEL > down the road too). Sure, it works too, but it'd be good if we could > stick to the published policies which are also known to work and > everybody adheres to them already. > > > > > Hi Manuel, > > Let me share with you what we had implemented and the POLICY that we > > put in place with regards to VERSION-RELEASE strings. We did not look > > at this from egg standpoint as we just distribute RPMS. > > > > I sent this to Tarek a while ago: > > What I had implemented seems to have the VERSION-RELEASE string issue > > solved using a 'workaround' and a manual policy for the moment. It > > would of course be better if distutils just did this automatically but > > this is at least working. Tarballs, rpms have the same designations and > > all the lexical ordering is correct so that even for pre-release > > versioning the rpms update each other correctly. > > > > #WORKAROUND: > > What I did was in setup.py added: > > ======================== > > version = '5.0.0' > > release = '0_rc3_00001681' > > > > if sys.argv[1] != 'bdist_rpm': > > version = version + '-' + release > > ======================== > > > > In setup.cfg added: > > ======================== > > [bdist_rpm] > > # release must exactly match 'release' as set in setup.py > > release=0_rc3_00001681 > > ================== > > # END WORKAROUND > > > > > > > > And then I setup the following policy regarding VERSION-RELEASE naming: > > (this example uses bzr as source code control system but you can > > substitute svn, rcs, or any other type of sccs) > > > > Development code: > > version="5.0.0" > > release="0_00012345" (where 00012345 is the revision number from > > bzr in ZERO-PADDED FIELD OF 8) > > tarball: foo-5.0.0-0_00012345.tar.gz > > rpm: foo-5.0.0-0_00012345.noarch.rpm > > > > Alpha code: > > version="5.0.0" > > release="0_alpha1_00123456" > > tarball: foo-5.0.0-0_alpha1_00123456.tar.gz > > rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm > > > > Beta code: > > version="5.0.0" > > release="0_beta1_00234567" > > tarball: foo-5.0.0-0_beta1_00234567.tar.gz > > rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm > > > > Release Candidate code: > > version="5.0.0" release="0_rc1" > > tarball: foo-5.0.0-0_rc1.tar.gz > > rpm: foo-5.0.0-0_rc1.noarch.rpm > > > > Release Candidate code fix: > > version="5.0.0" > > release="0_rc1_00345678" > > tarball: foo-5.0.0-0_rc1_00345678.tar.gz > > rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm > > > > Final Release code: > > version="5.0.0" > > release="1" > > tarball: foo-5.0.0-1.tar.gz > > rpm: foo-5.0.0-1.noarch.rpm > > > > Notice that lexical ordering is proper in all cases. Even where the > > alpha, beta, rc releases may be followed by a bzr fix revision. > > The ensures that both tarballs and packages such as rpm will contain the > > same VERSION-RELEASE designations. > > > > Also, the use of UNDERSCORES in the 'release' string assures that there > > is no ambiguity with regard to parsing VERSION-RELEASE string > > combination or full package names which have hyphens in specific > positions. > > > > > > Regards, > > Gerry > > > > -- > > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > Reality always seems harsher in the early morning. > > Manuel, Are you on the FPP committee? I looked at the fedora packaging policy and it is rather similar except it has periods where we have underscores. It would be nice if fedora could expand the policy to allow the underscore style as well. Both approaches work just as well. And to me the underscore form is a lot more readable. Also, there isn't a lot of examples in the FPP regarding adding the vcs revision number into the release string. I like the way we do it because it's lexically correct always. Maybe this could be added to FPP? Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From greno at verizon.net Thu Mar 12 00:54:26 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 20:54:26 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B853AD.5090807@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903110921.48576.rudd-o@rudd-o.com> <49B7D003.80305@verizon.net> <200903111617.50953.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> Message-ID: <49B85D42.9020501@verizon.net> Gerry Reno wrote: > Manuel Amador (Rudd-O) wrote: >> Gerry, >> >> >> What you have said is interesting. Note that you're using underscores >> in the release number (against F packaging policy), and the general >> scheme for devising the release number is also against Fedora >> packaging policy too (which is highly likely to cause problems in >> RHEL down the road too). Sure, it works too, but it'd be good if we >> could stick to the published policies which are also known to work >> and everybody adheres to them already. >> >> >> >> > Hi Manuel, >> > Let me share with you what we had implemented and the POLICY that we >> > put in place with regards to VERSION-RELEASE strings. We did not look >> > at this from egg standpoint as we just distribute RPMS. >> > >> > I sent this to Tarek a while ago: >> > What I had implemented seems to have the VERSION-RELEASE string issue >> > solved using a 'workaround' and a manual policy for the moment. It >> > would of course be better if distutils just did this automatically but >> > this is at least working. Tarballs, rpms have the same designations and >> > all the lexical ordering is correct so that even for pre-release >> > versioning the rpms update each other correctly. >> > >> > #WORKAROUND: >> > What I did was in setup.py added: >> > ======================== >> > version = '5.0.0' >> > release = '0_rc3_00001681' >> > >> > if sys.argv[1] != 'bdist_rpm': >> > version = version + '-' + release >> > ======================== >> > >> > In setup.cfg added: >> > ======================== >> > [bdist_rpm] >> > # release must exactly match 'release' as set in setup.py >> > release=0_rc3_00001681 >> > ================== >> > # END WORKAROUND >> > >> > >> > >> > And then I setup the following policy regarding VERSION-RELEASE naming: >> > (this example uses bzr as source code control system but you can >> > substitute svn, rcs, or any other type of sccs) >> > >> > Development code: >> > version="5.0.0" >> > release="0_00012345" (where 00012345 is the revision number from >> > bzr in ZERO-PADDED FIELD OF 8) >> > tarball: foo-5.0.0-0_00012345.tar.gz >> > rpm: foo-5.0.0-0_00012345.noarch.rpm >> > >> > Alpha code: >> > version="5.0.0" >> > release="0_alpha1_00123456" >> > tarball: foo-5.0.0-0_alpha1_00123456.tar.gz >> > rpm: foo-5.0.0-0_alpha1_00123456.noarch.rpm >> > >> > Beta code: >> > version="5.0.0" >> > release="0_beta1_00234567" >> > tarball: foo-5.0.0-0_beta1_00234567.tar.gz >> > rpm: foo-5.0.0-0_beta1_00234567.noarch.rpm >> > >> > Release Candidate code: >> > version="5.0.0" release="0_rc1" >> > tarball: foo-5.0.0-0_rc1.tar.gz >> > rpm: foo-5.0.0-0_rc1.noarch.rpm >> > >> > Release Candidate code fix: >> > version="5.0.0" >> > release="0_rc1_00345678" >> > tarball: foo-5.0.0-0_rc1_00345678.tar.gz >> > rpm: foo-5.0.0-0_rc1_00345678.noarch.rpm >> > >> > Final Release code: >> > version="5.0.0" >> > release="1" >> > tarball: foo-5.0.0-1.tar.gz >> > rpm: foo-5.0.0-1.noarch.rpm >> > >> > Notice that lexical ordering is proper in all cases. Even where the >> > alpha, beta, rc releases may be followed by a bzr fix revision. >> > The ensures that both tarballs and packages such as rpm will >> contain the >> > same VERSION-RELEASE designations. >> > >> > Also, the use of UNDERSCORES in the 'release' string assures that there >> > is no ambiguity with regard to parsing VERSION-RELEASE string >> > combination or full package names which have hyphens in specific >> positions. >> > >> > >> > Regards, >> > Gerry >> >> >> >> -- >> >> >> Manuel Amador (Rudd-O) >> Rudd-O.com - http://rudd-o.com/ >> GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ >> >> >> Reality always seems harsher in the early morning. >> >> > Manuel, > Are you on the FPP committee? I looked at the fedora packaging > policy and it is rather similar except it has periods where we have > underscores. It would be nice if fedora could expand the policy to > allow the underscore style as well. Both approaches work just as > well. And to me the underscore form is a lot more readable. > Also, there isn't a lot of examples in the FPP regarding adding the > vcs revision number into the release string. I like the way we do it > because it's lexically correct always. Maybe this could be added to FPP? > > Manuel, Also examples in the fedora packaging policy don't even make sense. It's no wonder no one every gets this right. alsa-lib-0.9.2-0.1.beta1 (add a new patch on top of 0.9.2beta1) alsa-lib-0.9.2-0.2.beta1 (upgrade to 0.9.2beta2) alsa-lib-0.9.2-0.3.beta2 (upgrade to 0.9.2beta3) ^^^^^^^^^^^^^^^^^^And this ridiculous construction is from the Department of Redundancy Department This would be far better and make a lot more sense like: alsa-lib-0.9.2-0_beta1_00001727 # where 1727 is the vcs revision, using numbers like patch 1 is absolutely meanless to most people, they want to know what tagged release (beta1 tag) and then what revision contains the update patches. OR alsa-lib-0.9.2-0_beta1_p0 # where p0 means patch 0 Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Thu Mar 12 01:19:17 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 20:19:17 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B85D42.9020501@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> Message-ID: <200903112019.20391.rudd-o@rudd-o.com> 1) No, I'm not on the committee. 2) The policy is similar. 3) Both approaches cannot work in parallel because the period is lexicographically distinct to the underscore 5) The policy has been tested and takes advantage of the already-deployed RPM code in tens of millions of machines out there. We do not know what sorts of malfunctions using underscores might cause. 4) The policy *does* make sense especially in the place you quoted it. The release begins with 0.1 and increments to 0.2 0.3 0.4 whenever you change the package (add patches, modify specfile) but continue to use the same source package, which is the purpose of the release tag in the first place -- to mark a new release of the SAME source package. Unfortunately, it requires that when a new beta source package (even one that is lexicographically later than the existing package) is compiled, the release must be bumped to upgrade it. My distutils packages explicitly punt this release bump to the packager so that he has control over which beta packages to push out. And the reason that the release tag always begins with 0. in nonstable releases is pretty dead simple: so that the release package with version 0.9.8-1 (the first official stable release) can be lexicographically higher and is automatically updated with package managers like yum. Guys, amirite? > > Manuel, > > Are you on the FPP committee? I looked at the fedora packaging > > policy and it is rather similar except it has periods where we have > > underscores. It would be nice if fedora could expand the policy to > > allow the underscore style as well. Both approaches work just as > > well. And to me the underscore form is a lot more readable. > > Also, there isn't a lot of examples in the FPP regarding adding the > > vcs revision number into the release string. I like the way we do it > > because it's lexically correct always. Maybe this could be added to FPP? > > Manuel, > Also examples in the fedora packaging policy don't even make sense. > It's no wonder no one every gets this right. > > alsa-lib-0.9.2-0.1.beta1 (add a new patch on top of 0.9.2beta1) > alsa-lib-0.9.2-0.2.beta1 (upgrade to 0.9.2beta2) > alsa-lib-0.9.2-0.3.beta2 (upgrade to 0.9.2beta3) > ^^^^^^^^^^^^^^^^^^And this ridiculous construction is from the > Department of Redundancy Department > > This would be far better and make a lot more sense like: > alsa-lib-0.9.2-0_beta1_00001727 # where 1727 is the vcs revision, > using numbers like patch 1 is absolutely meanless to most people, they > want to know what tagged release (beta1 tag) and then what revision > contains the update patches. > OR > alsa-lib-0.9.2-0_beta1_p0 # where p0 means patch 0 > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ If you are what you eat, does that mean Euell Gibbons really was a nut? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Thu Mar 12 01:43:58 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 21:43:58 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903112019.20391.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> <200903112019.20391.rudd-o@rudd-o.com> Message-ID: <49B868DE.9010108@verizon.net> Manuel Amador (Rudd-O) wrote: > 1) No, I'm not on the committee. > 2) The policy is similar. > 3) Both approaches cannot work in parallel because the period is > lexicographically distinct to the underscore No. Within a project you would have to choose, either the underscore form or the dot form within the 'release'. You could not mix them. And then either form would work just fine. > 5) The policy has been tested and takes advantage of the > already-deployed RPM code in tens of millions of machines out there. > We do not know what sorts of malfunctions using underscores might cause. Yes, and there are plenty of examples of packages that don't follow the policy as well. Probably more than are actually following it. > 4) The policy *does* make sense especially in the place you quoted it. > The release begins with 0.1 and increments to 0.2 0.3 0.4 whenever you > change the package (add patches, modify specfile) but continue to use > the same source package, which is the purpose of the release tag in > the first place -- to mark a new release of the SAME source package. > Unfortunately, it requires that when a new beta source package (even > one that is lexicographically later than the existing package) is > compiled, the release must be bumped to upgrade it. My distutils > packages explicitly punt this release bump to the packager so that he > has control over which beta packages to push out. And the reason that > the release tag always begins with 0. in nonstable releases is pretty > dead simple: so that the release package with version 0.9.8-1 (the > first official stable release) can be lexicographically higher and is > automatically updated with package managers like yum. The policy doesn't make sense in the example I quoted if you read the comments. Look where it says upgrade to 0.9.2beta3 yet the example shows beta2. I can see that what is trying to be done is to add extra numerics just to make the numerics significant and then basically you can ignore the beta, alpha, etc. But this is more confusing than helpful and its not necessary to make things lexically ordered. The goal is proper lexical ordering so that rpm/yum is happy and packages will upgrade correctly. And either form accomplishes this. Regards, Gerry > > Guys, amirite? > > > > > Manuel, > > > Are you on the FPP committee? I looked at the fedora packaging > > > policy and it is rather similar except it has periods where we have > > > underscores. It would be nice if fedora could expand the policy to > > > allow the underscore style as well. Both approaches work just as > > > well. And to me the underscore form is a lot more readable. > > > Also, there isn't a lot of examples in the FPP regarding adding the > > > vcs revision number into the release string. I like the way we do it > > > because it's lexically correct always. Maybe this could be added > to FPP? > > > > Manuel, > > Also examples in the fedora packaging policy don't even make sense. > > It's no wonder no one every gets this right. > > > > alsa-lib-0.9.2-0.1.beta1 (add a new patch on top of 0.9.2beta1) > > alsa-lib-0.9.2-0.2.beta1 (upgrade to 0.9.2beta2) > > alsa-lib-0.9.2-0.3.beta2 (upgrade to 0.9.2beta3) > > ^^^^^^^^^^^^^^^^^^And this ridiculous construction is from the > > Department of Redundancy Department > > > > This would be far better and make a lot more sense like: > > alsa-lib-0.9.2-0_beta1_00001727 # where 1727 is the vcs revision, > > using numbers like patch 1 is absolutely meanless to most people, they > > want to know what tagged release (beta1 tag) and then what revision > > contains the update patches. > > OR > > alsa-lib-0.9.2-0_beta1_p0 # where p0 means patch 0 > > > > > > Regards, > > Gerry > > > > -- > > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > > If you are what you eat, does that mean Euell Gibbons really was a nut? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greno at verizon.net Thu Mar 12 01:52:34 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 21:52:34 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B868DE.9010108@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> <200903112019.20391.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> Message-ID: <49B86AE2.4030608@verizon.net> Manuel, Additionally, from a Python Distutils perspective, Distutils should not be enforcing one distro's policy. There are RPM-based distro's that DO NOT follow fedora's packaging policy. The software should only be concerned with providing for the use of both the 'version' and 'release' strings in all distribution targets. It should not concern itself with HOW those fields are formatted. Regards, Gerry From ivazqueznet at gmail.com Thu Mar 12 03:28:09 2009 From: ivazqueznet at gmail.com (Ignacio Vazquez-Abrams) Date: Wed, 11 Mar 2009 23:28:09 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B86AE2.4030608@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B853AD.5090807@verizon.net> <49B85D42.9020501@verizon.net> <200903112019.20391.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> <49B86AE2.4030608@verizon.net> Message-ID: <1236828489.31110.108.camel@ignacio.lan> On Wed, 2009-03-11 at 21:52 -0400, Gerry Reno wrote: > Manuel, > Additionally, from a Python Distutils perspective, Distutils should > not be enforcing one distro's policy. There are RPM-based distro's that > DO NOT follow fedora's packaging policy. Actually, Fedora people have been approached by other distros because they want to adopt most or all of the policies due to its comprehensiveness. But you didn't hear this from me. -- Ignacio Vazquez-Abrams -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From rudd-o at rudd-o.com Thu Mar 12 03:18:55 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 22:18:55 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B868DE.9010108@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903112019.20391.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> Message-ID: <200903112218.58341.rudd-o@rudd-o.com> > The policy doesn't make sense in the example I quoted if you read the > comments. Look where it says upgrade to 0.9.2beta3 yet the example > shows beta2. I already told you that you need to bump the release to if you want beta3 to upgrade beta2. This is for good reason. > I can see that what is trying to be done is to add extra numerics just > to make the numerics significant and then basically you can ignore the > beta, alpha, etc. But this is more confusing than helpful and its not > necessary to make things lexically ordered. Actually not totally true, b comes after a, so an alpha package with the same release integer as a beta package would be upgraded by the beta package. > > The goal is proper lexical ordering so that rpm/yum is happy and > packages will upgrade correctly. And either form accomplishes this. Yet the policy says dots and the policy applies to over 10000 packages already built in the Fedora and RHEL / CentOS repos. >-- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ And we heard him exclaim As he started to roam: "I'm a hologram, kids, please don't try this at home!'" -- Bob Violence -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Thu Mar 12 03:26:42 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Wed, 11 Mar 2009 22:26:42 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B86AE2.4030608@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> <49B86AE2.4030608@verizon.net> Message-ID: <200903112226.43210.rudd-o@rudd-o.com> El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > Manuel, > Additionally, from a Python Distutils perspective, Distutils should > not be enforcing one distro's policy. There are RPM-based distro's that > DO NOT follow fedora's packaging policy. Well then, give me one packaging policy that is discordant or incompatible with Fedora's packaging policy. I'm sure you can show us that. Besides, we're not enforcing a distro policy here over all platforms. My patches merely reformat the existing information to make it lexicographically compatible with RPM package managers (which are the SAME across all bdist_rpm distro targets), and that's it. You still get to specify your release numbers if you want to, and the version numbers are intelligently determined based on heuristics but in the case of stable releases, they are just as they used to be before. > The software should only be > concerned with providing for the use of both the 'version' and 'release' > strings in all distribution targets. It should not concern itself with > HOW those fields are formatted. If distutils did only that and we were to reject the distutils patches I wrote, then I would simply be technically unable to build Plone for Fedora or RHEL. In fact, with the current non-patched distutils, nobody could build eggs using bdist_rpm, because many packages present in the cheese shop rely heavily on beta packages and dependencies that specify python-style version numbers. The mission of the patches is to provide a reasonable way to build eggs and other python source drops ensuring a maximum of compatibility with RPM package manager upgrade paths. The patches accomplish this. And the next hurdle -- something I still need to write into setuptools, but I have already scripted in my workbench -- is the ability to auto-detect egg dependencies and take those dependencies into account for the purpose of proper dependency management of auto-built eggs. > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Every path has its puddle. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Thu Mar 12 03:52:21 2009 From: greno at verizon.net (Gerry Reno) Date: Wed, 11 Mar 2009 23:52:21 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903112226.43210.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49B868DE.9010108@verizon.net> <49B86AE2.4030608@verizon.net> <200903112226.43210.rudd-o@rudd-o.com> Message-ID: <49B886F5.9090500@verizon.net> Manuel Amador (Rudd-O) wrote: > El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > > Manuel, > > Additionally, from a Python Distutils perspective, Distutils should > > not be enforcing one distro's policy. There are RPM-based distro's that > > DO NOT follow fedora's packaging policy. > > > Well then, give me one packaging policy that is discordant or > incompatible with Fedora's packaging policy. I'm sure you can show us > that. Ok, Mandriva comes to mind. It's policy is different and allows more flexibility than Fedora's. Besides, policies are meant for humans. And policies change. We don't need distro policies hard-coded into Distutils. That makes for brittle code. > > Besides, we're not enforcing a distro policy here over all platforms. > My patches merely reformat the existing information to make it > lexicographically compatible with RPM package managers (which are the > SAME across all bdist_rpm distro targets), and that's it. You still > get to specify your release numbers if you want to, and the version > numbers are intelligently determined based on heuristics but in the > case of stable releases, they are just as they used to be before. > > > > The software should only be > > concerned with providing for the use of both the 'version' and 'release' > > strings in all distribution targets. It should not concern itself with > > HOW those fields are formatted. > > > If distutils did only that and we were to reject the distutils patches > I wrote, then I would simply be technically unable to build Plone for > Fedora or RHEL. Why? A distro's policy is defined for 'human' behavior for users of that distro. Users of distro X should not be hostage to policies of distro Y. 'version' and 'release' should be formatted by humans for whatever distro is the target. There should be no enforcement of any distro's 'packaging policy' over the fields. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From greno at verizon.net Thu Mar 12 22:48:24 2009 From: greno at verizon.net (Gerry Reno) Date: Thu, 12 Mar 2009 18:48:24 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903121611.21771.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903112226.43210.rudd-o@rudd-o.com> <49B886F5.9090500@verizon.net> <200903121611.21771.rudd-o@rudd-o.com> Message-ID: <49B99138.1010202@verizon.net> Manuel Amador (Rudd-O) wrote: > El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > > > > Ok, Mandriva comes to mind. It's policy is different and allows more > > flexibility than Fedora's. > > > Well, would you be so kind to link the Mandriva policy for us to read > about it? Mandriva link: http://wiki.mandriva.com/en/Policies/RpmSpecProposal You're failing to keep in mind that Fedora is not the only distro out there that is RPM-based. Mandriva is just one of many other RPM-based distros that have packaging policies different from Fedora. Again, the core software should not enforce one distros packaging policies. If you want to perform some type of check for a particular distro then that should be done through an extension module and there should be a special command line option to trigger it --enforce-fedora-packaging-policy. > > > > > Besides, policies are meant for humans. And policies change. We don't > > need distro policies hard-coded into Distutils. That makes for brittle > > code. > > > So what you're saying, in effect, translates to "sorry, man, you > cannot build your Plone packages for RPM distributions, nor can > anybody else build any sort of pre-release eggs either". Well, sorry, > man, that's not acceptable. Why do you keep saying this? What is preventing you (the human) from filling in the version and release fields with "fedora-compliant strings"? The core software does not have to know anything about fedora. But you do. > > > Why? A distro's policy is defined for 'human' behavior for users of > > that distro. Users of distro X should not be hostage to policies of > > distro Y. 'version' and 'release' should be formatted by humans for > > whatever distro is the target. There should be no enforcement of any > > distro's 'packaging policy' over the fields. > > > Policy schmolicy, the current distutils behavior builds pre-release > RPMs that UPGRADE final release packages, which is broken and stupid > because distutils ought to be taking care of the impedance mismatch > between python-style versioning and RPM-style lexicographical > versioning, but it doesn't. That's a lexical ordering problem. That should be fixed by the human making sure that he declares the version and release with proper strings. > I have a solution that works in fedora, rhel and centos, and likely > works just as well on other RPM distros including Mandriva and SUSE. Your solution LIMITS the version and release strings to ONLY fedora packaging style. Mandriva users don't want that. They want to build RPM according to their own packaging policy for Mandriva but your patch will not let them because it enforces fedora policy. THAT'S WRONG. > Do you have an alternative solution? Yes, I do. Remove all artificial restriction on formatting on the 'version' and 'release' strings. All that's necessary is for both the 'version' and 'release' strings to be available to all the distribution command which it is not at the moment. That's it. Nothing else is necessary. If you want to do some policy enforcing thing, then put it in some type of optional extension called from a special command line option. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From greno at verizon.net Fri Mar 13 01:52:48 2009 From: greno at verizon.net (Gerry Reno) Date: Thu, 12 Mar 2009 21:52:48 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903122001.41744.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903121611.21771.rudd-o@rudd-o.com> <49B99138.1010202@verizon.net> <200903122001.41744.rudd-o@rudd-o.com> Message-ID: <49B9BC70.4020307@verizon.net> Manuel Amador (Rudd-O) wrote: > > Mandriva link: http://wiki.mandriva.com/en/Policies/RpmSpecProposal > > > Thanks for providing the link. Which is a PROPOSAL, not a polcy. On a > user-editable WIKI. And on top of that, what you say about versioning > style is in NO WAY backed by that document. That is the ONLY policy there is for Mandriva. So, if you like, then you can say the policy is whatever 'version' and 'release' naming that will work in an RPM spec file. And that's way more flexible than Fedora's policy. And Mandriva is just one example. You have other distros that do not use Fedora naming policy and you have all sorts of derivatives of these distros, none of which are using Fedora packaging policy. > > Now we know for a fact that my patch is neither in conflict with > Mandriva policy, nor will fail to work there. This is the kind of > fact-based exchange I enjoy. It does conflict. See above. > > > Again, the core software should not enforce one distros packaging > > policies. > > > We already established that the Mandriva policy is not in conflict > with Mandriva policy or any other policy that you have quoted so far > (none, really), so the prescriptive argument you are making has no > basis in fact -- It's just dogma. Again, your patch makes changes to 'version' and 'release' based on Fedora naming policy. Not every distro follows Fedora package naming policy. > > > Why do you keep saying this? What is preventing you (the human) from > > filling in the version and release fields with "fedora-compliant > > strings"? The core software does not have to know anything about > > fedora. But you do. > > > If you need to know: cheese shop eggs come using a particular python > policy for naming them, a policy that was defined by setuptools' > dependency handling. This python policy is incompatible with RPM > lexicographical order. This is, in a nutshell, the source of the problem. If that's the problem then fix it where the problem lies. In the egg naming policy in setuptools. > > Therefore, the bdist_rpm patch needs to adapt the version to an RPM > lexicographically valid form. My patches do exactly this -- they > merely adapt the bdist_rpm version and release in very specific cases > (pre-release packages) which ought not to matter for a distributor > making a stable release. In addition to that, my code works across > distributions and is not in contradiction to any policy. Manuel, please put this policy stuff in an extension that will override the core. Not everybody uses eggs or cares about eggs. Or what problems egg people have with RPM. > > Without my patches, there needs to be a way to override the version of > the package in setup.cfg, and in addition to that, each package in the > cheese shop would need to be modified in order to be buildable. That's > thousands of eggs. Again, put this logic in an extension so it doesn't affect the core and call it from a commandline option. > > You're invited to talk to all the python egg developers yourself if > you disagree with me. Me? I prefer practical solutions, thus the patch. > > > > That's a lexical ordering problem. That should be fixed by the human > > making sure that he declares the version and release with proper > strings. > > > Well, I'm sorry but the version is not overridable in setup.cfg. So > you either use my patch, or chao eggs. Again, put the patch logic in an extension called from a commandline option and leave the core alone. > > > > I have a solution that works in fedora, rhel and centos, and likely > > > works just as well on other RPM distros including Mandriva and SUSE. > > > > Your solution LIMITS the version and release strings to ONLY fedora > > packaging style. Mandriva users don't want that. > > > FYI: There is nothing in the Mandriva policy supporting this statement. Yes, there is. Mandriva naming is whatever will work in a spec file. Much less restrictive than Fedora. > > > They want to build RPM according to their own packaging policy for > Mandriva but your patch will not let them because it enforces fedora > policy. THAT'S WRONG. > > > Again, the policy link you gave provides no basis for your argument. See above. > > > > > > Do you have an alternative solution? > > > > Yes, I do. Remove all artificial restriction on formatting on the > > 'version' and 'release' strings. > > > I have not introduced any artificial restrictions at all. This is just > a lie. I'd appreciate it very much if, in the future, you refrain from > telling lies about my work. Manuel, here is part of your patch: + version = self.distribution.get_version() + release = self.release.replace('-','_') + import re ; regex = '[^0-9\.]' + splits = re.split(regex,version,maxsplit=1) + firstnonalnumchar = re.findall(regex,version) + if len(splits) == 1: + pass + else: + version = splits[0] + release = "0." + release + "." + firstnonalnumchar[0] + splits[1] spec_file = [ '%define name ' + self.distribution.get_name(), '%define version ' + self.distribution.get_version().replace('-','_'), - '%define release ' + self.release.replace('-','_'), + '%define version ' + version, + '%define release ' + release, Your patch is enforcing restrictions on 'version' and 'release' that are directly from Fedora. If some other distro user creates 'version' string that happens to match your pattern, his strings will be changed. That's why I said that the better way to do what you want is to leave the core code alone and put your policy logic in an extension that overrides the core and call it from a commandline option. This would be more community friendly and that way you have what you want and you don't impact other distro users. > > > All that's necessary is for both the > > 'version' and 'release' strings to be available to all the distribution > > command which it is not at the moment. That's it. Nothing else is > > necessary. > > > You are encouraged to write that patch, then make every single egg > developer put stuff in their eggs' setup.cfg files. Have fun draining > the ocean. That patch needs written and if necessary I will write it. > > > If you want to do some policy enforcing thing, then put > > it in some type of optional extension called from a special command line > > option. > > Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Thu Mar 12 21:11:21 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 16:11:21 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B886F5.9090500@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903112226.43210.rudd-o@rudd-o.com> <49B886F5.9090500@verizon.net> Message-ID: <200903121611.21771.rudd-o@rudd-o.com> El Mi?rcoles 11 Marzo 2009, Gerry Reno escribi?: > Ok, Mandriva comes to mind. It's policy is different and allows more > flexibility than Fedora's. Well, would you be so kind to link the Mandriva policy for us to read about it? > > Besides, policies are meant for humans. And policies change. We don't > need distro policies hard-coded into Distutils. That makes for brittle > code. So what you're saying, in effect, translates to "sorry, man, you cannot build your Plone packages for RPM distributions, nor can anybody else build any sort of pre-release eggs either". Well, sorry, man, that's not acceptable. > Why? A distro's policy is defined for 'human' behavior for users of > that distro. Users of distro X should not be hostage to policies of > distro Y. 'version' and 'release' should be formatted by humans for > whatever distro is the target. There should be no enforcement of any > distro's 'packaging policy' over the fields. Policy schmolicy, the current distutils behavior builds pre-release RPMs that UPGRADE final release packages, which is broken and stupid because distutils ought to be taking care of the impedance mismatch between python-style versioning and RPM-style lexicographical versioning, but it doesn't. I have a solution that works in fedora, rhel and centos, and likely works just as well on other RPM distros including Mandriva and SUSE. Do you have an alternative solution? No? Then let's stop discussing this because what you're saying sounds like a baseless complaint on the grounds of abstract pureness instead of practical results. > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Excellent day to have a rotten day. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Fri Mar 13 01:01:38 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 20:01:38 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B99138.1010202@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903121611.21771.rudd-o@rudd-o.com> <49B99138.1010202@verizon.net> Message-ID: <200903122001.41744.rudd-o@rudd-o.com> > Mandriva link: http://wiki.mandriva.com/en/Policies/RpmSpecProposal Thanks for providing the link. Which is a PROPOSAL, not a polcy. On a user- editable WIKI. And on top of that, what you say about versioning style is in NO WAY backed by that document. Now we know for a fact that my patch is neither in conflict with Mandriva policy, nor will fail to work there. This is the kind of fact-based exchange I enjoy. > Again, the core software should not enforce one distros packaging > policies. We already established that the Mandriva policy is not in conflict with Mandriva policy or any other policy that you have quoted so far (none, really), so the prescriptive argument you are making has no basis in fact -- It's just dogma. > Why do you keep saying this? What is preventing you (the human) from > filling in the version and release fields with "fedora-compliant > strings"? The core software does not have to know anything about > fedora. But you do. If you need to know: cheese shop eggs come using a particular python policy for naming them, a policy that was defined by setuptools' dependency handling. This python policy is incompatible with RPM lexicographical order. This is, in a nutshell, the source of the problem. Therefore, the bdist_rpm patch needs to adapt the version to an RPM lexicographically valid form. My patches do exactly this -- they merely adapt the bdist_rpm version and release in very specific cases (pre-release packages) which ought not to matter for a distributor making a stable release. In addition to that, my code works across distributions and is not in contradiction to any policy. Without my patches, there needs to be a way to override the version of the package in setup.cfg, and in addition to that, each package in the cheese shop would need to be modified in order to be buildable. That's thousands of eggs. You're invited to talk to all the python egg developers yourself if you disagree with me. Me? I prefer practical solutions, thus the patch. > That's a lexical ordering problem. That should be fixed by the human > making sure that he declares the version and release with proper strings. Well, I'm sorry but the version is not overridable in setup.cfg. So you either use my patch, or chao eggs. > > I have a solution that works in fedora, rhel and centos, and likely > > works just as well on other RPM distros including Mandriva and SUSE. > Your solution LIMITS the version and release strings to ONLY fedora > packaging style. Mandriva users don't want that. FYI: There is nothing in the Mandriva policy supporting this statement. > They want to build RPM according to their own packaging policy for Mandriva but your patch will not let them because it enforces fedora policy. THAT'S WRONG. Again, the policy link you gave provides no basis for your argument. > > > Do you have an alternative solution? > > Yes, I do. Remove all artificial restriction on formatting on the > 'version' and 'release' strings. I have not introduced any artificial restrictions at all. This is just a lie. I'd appreciate it very much if, in the future, you refrain from telling lies about my work. > All that's necessary is for both the > 'version' and 'release' strings to be available to all the distribution > command which it is not at the moment. That's it. Nothing else is > necessary. You are encouraged to write that patch, then make every single egg developer put stuff in their eggs' setup.cfg files. Have fun draining the ocean. > If you want to do some policy enforcing thing, then put > it in some type of optional extension called from a special command line > option. Look, this back-and-forth is pointless. * If you in some capacity have some control on what goes into the distutils repo, then reject the patches. The Fedora/RH people will probably pick them up anyway, because it has value for tens of millions of users out there. * If you are a Mandriva developer, and there is real basis for your argument (despite the policy document you quoted which does not support it), then you are welcome to include a patch to the patch in your Python sources (you already include several patches, so it should not be a problem) which should be a change of a few characters in my patch to confirm to Mandriva policy. * If not, then either produce code that has the same net results as mine, or let's stop this argument. This bickering can be reduced to the following: I have working code that will work on Mandriva too. You have a baseless complaint with a proposal that would require thousands of people to do extra work, and no code to boot. luck, -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: Eiffel 65 - Sopra un palco per tutto il mondo Blessed is the man who, having nothing to say, abstains from giving wordy evidence of the fact. -- George Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Fri Mar 13 02:29:19 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 21:29:19 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B9BC70.4020307@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122001.41744.rudd-o@rudd-o.com> <49B9BC70.4020307@verizon.net> Message-ID: <200903122129.19895.rudd-o@rudd-o.com> > That is the ONLY policy there is for Mandriva. So, if you like, then > you can say the policy is whatever 'version' and 'release' naming that > will work in an RPM spec file. And that's way more flexible than > Fedora's policy. If the policy is "whatever works", then my patches comply with the Mandriva policy (in other words, they cause bdist_rpm to generate specfiles pursuant to the policy) and there is no need to discuss this any further. > And Mandriva is just one example. You have other distros that do not > use Fedora naming policy and you have all sorts of derivatives of these > distros, none of which are using Fedora packaging policy. Show them to me and I will rewrite my patches to accomodate them. > Again, put the patch logic in an extension called from a commandline > option and leave the core alone. My patches do not touch core distutils. Stop lying about my work. > Yes, there is. Mandriva naming is whatever will work in a spec file. > Much less restrictive than Fedora. ------------------ The core of the issue is simple: * As of today, bdist_rpm generates invalid spec files when supplied with a pre-release python package (egg or standard distutils one). What I mean is that it generates RPMs that would UPGRADE final releases. Which is broken behavior. * My patch fixes this bug in bdist_rpm, and the fix works in mandriva, fedora, centos, and other RPM distributions, due to the fact that the RPM version algorithm is the same in librpm.so everywhere. Now, onto this discussion itself -- let's go meta: I have had enough of your lies and misrepresentations. Now, you had your chance to make a case as to why my patches break anything in Mandriva, and I have conclusively shown they do not break anything there. You have had plenty of chances to make your case, present examples, show code, improve the patches, one-up my work in MANY WAYS. Alas, you haven't done any of that -- all you did was complain about "policy" -- and, frankly, I have worked very hard to make this work and I do not appreciate you chiming in with pointless bitching that has no basis in engineering reasons. So further mails from you will be directed to /dev/null. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Now playing, courtesy of Amarok: DJ J?rgen presenting Alice Deejay - Better off alone (club vocal mix) One meets his destiny often on the road he takes to avoid it. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 03:51:15 2009 From: greno at verizon.net (Gerry Reno) Date: Thu, 12 Mar 2009 23:51:15 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903122129.19895.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122001.41744.rudd-o@rudd-o.com> <49B9BC70.4020307@verizon.net> <200903122129.19895.rudd-o@rudd-o.com> Message-ID: <49B9D833.3050703@verizon.net> Manuel, In RPM the only restriction on 'version' is that it must be made of alphanumeric or period. Ref: http://www.rpm.org/max-rpm-snapshot/s1-rpm-inside-tags.html Just for a small test I ran a perfectly legal version pattern, that is allowable in distros other than Fedora, through your patch logic. version = A2 release = 0_rc1 version-release = A2-0_rc1 # this is what we expected to see version-release = -0.0_rc1.A2 # this is what we get instead as strings are modified by patch logic This is why I suggest that you make an extension module triggered by a commandline option so there are no restrictions or unexpected side-effects introduced to non-Fedora distro users. Right now, I'm a big -1 on the patch. If it would be made as an extension triggered by commandline option then I would support it. Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 04:00:35 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Thu, 12 Mar 2009 23:00:35 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B9D833.3050703@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> Message-ID: <200903122300.37492.rudd-o@rudd-o.com> OK, now we're talking. You have given me a test case that makes my patch fail and I will address it after How I Met Your Mother. El Jueves 12 Marzo 2009, Gerry Reno escribi?: > Manuel, > In RPM the only restriction on 'version' is that it must be made of > alphanumeric or period. Ref: > http://www.rpm.org/max-rpm-snapshot/s1-rpm-inside-tags.html > Just for a small test I ran a perfectly legal version pattern, that is > allowable in distros other than Fedora, through your patch logic. > > version = A2 > release = 0_rc1 > version-release = A2-0_rc1 # this is what we expected to see > version-release = -0.0_rc1.A2 # this is what we get instead as strings > are modified by patch logic > > > This is why I suggest that you make an extension module triggered by a > commandline option so there are no restrictions or unexpected > side-effects introduced to non-Fedora distro users. > Right now, I'm a big -1 on the patch. If it would be made as an > extension triggered by commandline option then I would support it. > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Mix's Law: There is nothing more permanent than a temporary building. There is nothing more permanent than a temporary tax. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Fri Mar 13 05:45:44 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 00:45:44 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49B9D833.3050703@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> Message-ID: <200903130045.46626.rudd-o@rudd-o.com> OK, Gerry, I have addressed your concerns. 1) Note that the release number is still squelched in the case of RPMs compiled from SVN checkouts of distutils packages. This is so that alpha, beta or release candidate packages can upgrade SVN checkouts without extra intervention from the packager. Alpha, beta and rc packages do obey the release variable just fine. The whole intention here is to put a sensible release variable and a sensible version variable when no release variable was specified. 2) I have used a relsep variable that you can tweak to use a dot or an elephant if your package policy wants that (note that the Mandrake policy says nothing about that). 3) If anything non-numeric is specified in the release, then you as a packager are indicating that you do not want this automatic behavior and the modifications are automatically skipped. Note that the whole point of the patch is to AVOID this extra work of specifying the release for prerelease packages. 4) The cases you spoke about are addressed. Attached is the patch. Here is a map table with the default release (which happens to be "1": ----------------------- [rudd-o at tobey ~]$ for a in `cat versions` ; do python bdist_rpm.py rewrite $a 1 ; done 1.0-1 1.0-1 1.1-1 1.1-1 1.1a-1 1.1-0.1.a 1.1a2-1 1.1-0.1.a2 1.1b1-1 1.1-0.1.b1 1.1dev-1 1.1-0.0.dev 1.1svn-1 1.1-0.0.svn 1.1dev-r56554-1 1.1-0.0.56554dev 1.1svn-r400-1 1.1-0.0.400svn 1.1svn-r401-1 1.1-0.0.401svn 1.2svn-r34-1 1.2-0.0.34svn 5.5rc2-1 5.5-0.1.rc2 0.9-1 0.9-1 0.9.8-1 0.9.8-1 20080706-1 20080706-1 A2005-1 A2005-1 ABCDEF15-1 ABCDEF15-1 abc45-1 abc45-1 0.9alpha23-1 0.9-0.1.alpha23 A2-1 A2-1 -------------------- Now here is a map table with a release with a non-integer release: ---------------- [rudd-o at tobey ~]$ for a in `cat versions` ; do python bdist_rpm.py rewrite $a 0_1 ; done 1.0-0_1 1.0-0_1 1.1-0_1 1.1-0_1 1.1a-0_1 1.1a-0_1 1.1a2-0_1 1.1a2-0_1 1.1b1-0_1 1.1b1-0_1 1.1dev-0_1 1.1dev-0_1 1.1svn-0_1 1.1svn-0_1 1.1dev-r56554-0_1 1.1dev_r56554-0_1 1.1svn-r400-0_1 1.1svn_r400-0_1 1.1svn-r401-0_1 1.1svn_r401-0_1 1.2svn-r34-0_1 1.2svn_r34-0_1 5.5rc2-0_1 5.5rc2-0_1 0.9-0_1 0.9-0_1 0.9.8-0_1 0.9.8-0_1 20080706-0_1 20080706-0_1 A2005-0_1 A2005-0_1 ABCDEF15-0_1 ABCDEF15-0_1 abc45-0_1 abc45-0_1 0.9alpha23-0_1 0.9alpha23-0_1 A2-0_1 A2-0_1 ------------------ I hope the next note you have about this patch is as constructive as the last one. The intention here is to solve a problem, not to bikeshed. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ If puns were deli meat, this would be the wurst. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.4-distutils-bdist_rpm-rpmversion-lexicalorder.patch Type: text/x-patch Size: 2511 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From rudd-o at rudd-o.com Fri Mar 13 08:39:48 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 03:39:48 -0500 Subject: [Distutils] Automatic RPM dependencies based on egg metadata In-Reply-To: <49B9D833.3050703@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> Message-ID: <200903130339.51154.rudd-o@rudd-o.com> Based on my earlier work, it brings me much pleasure to contribute the attached patch (which requires the earlier, lexicographical-order patch to be applied first). The following attached file makes distutils automatically add dependencies to the Requires: field in the RPM spec file, if the egg metadata specifies any primary dependency. With this patch, it is no longer needed to manually specify your dependencies in both setup.cfg and requires.txt. DRY FTW! regards, -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Some people have a way about them that seems to say: "If I have only one life to live, let me live it as a jerk." -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.4-distutils-bdist_rpm-autodeps.patch Type: text/x-patch Size: 3152 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 13:28:10 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 09:28:10 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903130045.46626.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903122129.19895.rudd-o@rudd-o.com> <49B9D833.3050703@verizon.net> <200903130045.46626.rudd-o@rudd-o.com> Message-ID: <49BA5F6A.8030309@verizon.net> Manuel, Here's what I get with this patch: $ python test2.py rewrite 3.3beta1 4333 3.3beta1-4333 3.3-0.4333.beta1 $ python test2.py rewrite 3.3rc1 4333 3.3rc1-4333 3.3-0.4333.rc1 It is REWRITING the strings when we don't want them rewritten. And that was the point about putting this in an extension called by a commandline arg. Instead of going through all these callesthenics writing obscure code in bdist_rpm.py just write your simpler code into an override module and call it via a commandline option. That way people who work on bdist_rpm in the future don't have to decipher all this Fedora-specific code. Why do you resist doing it this way? Why do you feel you have the right to come in and take over bdist_rpm with Fedora-specific modifications? Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 14:28:36 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 09:28:36 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49BA5F6A.8030309@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130045.46626.rudd-o@rudd-o.com> <49BA5F6A.8030309@verizon.net> Message-ID: <200903130928.42700.rudd-o@rudd-o.com> OK, I am at the limit of my patience, so this will be the last mail I will write about this to you, ever. --------- First of all, my work has already been given good reviews by people involved with distutils. --------- Second, I will absolutely not build an extension that must be enabled explicitly by the packager. This is intended to be default behavior. There is good reason for that decision. The current default behavior *breaks RPM package managers like RPM, Yum and Smart*. Let me reiterate that: the current distutils is BRO-KEN. My patches merely fixing this breakage, which is common to ALL distributions. When this gets into distutils, casual packagers won't have to troll the documentation to figure this out, or be baffled when their "beta" RPMs upgrade their stable releases because "Oh, I didn't know I had to pass this --unbreak- me option to bdist_rpm!" There is no engineering reason to make a bug fix optional. This is a bug fix. It must replace the broken behavior. Heeding your baseless advice would defeat the purpose of my work. --------- Third: Any future people looking into distutils have *nothing* to decipher -- the code is documented and the rationale is explained a bit in the code and well in the bug reports and across the Internet in terms of the RPM versioning algorithm. -------- Fourth: The modifications aren't Fedora-specific. They work correctly across all RPM distributions, whether you want to admit it or not. I am not barging in and taking over distutils for Fedora, and for you to frame the discussion in terms of "Fedora vs. The Others" is just corrupt and dishonest. IOW: I REALLY REALLY do not want to hear you lie "But this is Fedoraism" again. And finally, even if some wacko distributor wants to put a BIG ASCII PENIS on the release tag (which, by the way, is a valid string), they can always patch my work out of distutils just as all distributions have been doing all along. Enjoy your broken behavior -- the rest of us actually want good RPM packages. --------- Fifth: you have been given a mechanism to disable the rewriting which you so loathe. No rewriting (thus reverting to the old, BROKEN bdist_rpm behavior) happens if: 1) your package is a final stable release 2) you put a non-numeric character in your setup.cfg's release= config item, such as 2mdk or something like that. This ought to cover any hypothetical policy that is ass-backwards and likes to generate broken prerelease packages that upgrade stable ones. See big ASCII penis comment above, it fits perfectly here. --------- Sixth: you complain about the rewriting: $ python test2.py rewrite 3.3beta1 4333 3.3beta1-4333 ? ? ? ? ? 3.3-0.4333.beta1 How deficient are you that you can't understand how without the rewriting, 3.3beta1 RPM would upgrade 3.3-1 first final release? TRY IT. DO THE LEGWORK. Do what I did, build two packages of the same code, one with beta in the version name, another without it, and see which one upgrades which other. --------- In summary: * bdist_rpm generates broken RPM packages if supplied with pre-release modules. It has always been broken. * My patches FIX this breakage in a manner that is compatible with all RPM distributions out there. This is not a new feature but a BUG FIX. * They do not require extra action by the packager because making a bug fix require manual intervention from the operator would be the epitome of imbecility. * They have been reviewed positively and I have been thanked personally for making them. The prognosis for their inclusion looks rather positive. * I have addressed every single of your complaints to the limit of my abilities, both with code and with detailed, reasoned justifications for the code. * Yet you object to them fiercely, with absolutely NO BASIS in engineering, and only a made-up distro rivalry that is simply false. In addition, on the basis of this patch, I have created new distutils code that auto-detects dependencies based on egg metadata and adds them to the RPM. These patches have already been submitted to the various bug trackers, and without this patch you so much object to, it would have been technically impossible to deliver on that. Before this patch, people had to specify their dependencies twice, once in the requires.txt and another in the setup.cfg file -- a monumental, error-prone task to be done for tens of thousands of eggs -- and now, we have this. If you had your way, we'd be back in the stone age of manually preparing spec files for each and every Python module. OK. You have had repeated chances to make your case, and all you have come up with is this complaint that stinks of crab mentality, evidences a severe lack of technical ability to understand the underlying problem and its importance, and a false allegation that the patch engages in distribution favoritism just because it uses a fucking dot (which is now configurable, but doesn't really impact any distros negatively) for a separator. I have had enough of this. From now on, you will be redacted from the recipient list in future conversations about this topic. I think it's time for you to visit http://www.sadtrombone.com/ . Now have some respect for my time and stop barging in on my inbox. El Viernes 13 Marzo 2009, Gerry Reno escribi?: > Manuel, > Here's what I get with this patch: > $ python test2.py rewrite 3.3beta1 4333 > 3.3beta1-4333 3.3-0.4333.beta1 > $ python test2.py rewrite 3.3rc1 4333 > 3.3rc1-4333 3.3-0.4333.rc1 > > It is REWRITING the strings when we don't want them rewritten. And that > was the point about putting this in an extension called by a commandline > arg. Instead of going through all these callesthenics writing obscure > code in bdist_rpm.py just write your simpler code into an override > module and call it via a commandline option. That way people who work > on bdist_rpm in the future don't have to decipher all this > Fedora-specific code. Why do you resist doing it this way? Why do you > feel you have the right to come in and take over bdist_rpm with > Fedora-specific modifications? > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ ether leak -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 15:27:20 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 11:27:20 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903130928.42700.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130045.46626.rudd-o@rudd-o.com> <49BA5F6A.8030309@verizon.net> <200903130928.42700.rudd-o@rudd-o.com> Message-ID: <49BA7B58.70303@verizon.net> Manuel, 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the 'version' and 'release' strings. They do stupid things like put version='3.0' and release='rc1' and then wonder why their final release cannot update the release candidate. THIS IS A TRAINING ISSUE. Their poorly planned sequence of release designations is not in lexical order. They should be putting version='3.0' and release='0.rc1' or '0_rc1' and then their final release of version='3.0' and release='1' WILL update their release candidate. AGAIN, this is not something that is broken with 'bdist_rpm' IT IS A TRAINING ISSUE. Packagers need to understand the importance of lexical ordering when using RPM. You cannot make a patch that outlaws stupidity. I vote a big -1 on this patch because it interjects Fedora-specific modifications into bdist_rpm. Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 15:56:05 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 10:56:05 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49BA7B58.70303@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130928.42700.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> Message-ID: <200903131056.08524.rudd-o@rudd-o.com> 1) The Python StrictVersion and LooseVersion classes in Setuptools used to validate and compare versions are incompatible with RPM's lexicographical order. Even the setuptools metadata documentation, which is in full force and implemented, says alpha and beta and rc are expressly permitted (and ENCOURAGED) in the version name, and setuptools' dependency resolution actually uses that information. It is clear that the only solution that unbreaks bdist_rpm is version rewriting of prerelease packages. 2) We already know this "fedora-specific" blahblah is false. My patches work fine in other distros and can be disabled easily. This guy can't tell his ass from his head, and he has the audacity to come here with this ignorant chicanery, trying to convince us that the world is flat that the PEPs and the deployed setuptools code do not work the way they are implemented, that "training" is the solution (see above for technical reasons why "training" is not only draining the ocean but technically impossible) and that we're committing a crime against nature by including these patches. I strongly urge those of you with the knowledge and the power, to do the right thing, and ignore this charlatan. El Viernes 13 Marzo 2009, Gerry Reno escribi?: > Manuel, > 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the > 'version' and 'release' strings. They do stupid things like put > version='3.0' and release='rc1' and then wonder why their final release > cannot update the release candidate. THIS IS A TRAINING ISSUE. Their > poorly planned sequence of release designations is not in lexical > order. They should be putting version='3.0' and release='0.rc1' or > '0_rc1' and then their final release of version='3.0' and release='1' > WILL update their release candidate. AGAIN, this is not something that > is broken with 'bdist_rpm' IT IS A TRAINING ISSUE. Packagers need to > understand the importance of lexical ordering when using RPM. You > cannot make a patch that outlaws stupidity. > > > I vote a big -1 on this patch because it interjects Fedora-specific > modifications into bdist_rpm. > > > Regards, > Gerry -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Repartee is something we think of twenty-four hours too late. -- Mark Twain -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 16:08:49 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 12:08:49 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903131056.08524.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130928.42700.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> <200903131056.08524.rudd-o@rudd-o.com> Message-ID: <49BA8511.9080604@verizon.net> Manuel Amador (Rudd-O) wrote: > 1) The Python StrictVersion and LooseVersion classes in Setuptools > used to validate and compare versions are incompatible with RPM's > lexicographical order. Even the setuptools metadata documentation, > which is in full force and implemented, says alpha and beta and rc are > expressly permitted (and ENCOURAGED) in the version name, and > setuptools' dependency resolution actually uses that information. Then this is a 'setuptools' ERROR. Go fix it in setuptools. If setuptools creates lexigraphically stupid version and release strings that are incompatible with RPM then that is a 'setuptools' problem, NOT a bdist_rpm problem. Go fix 'setuptools'. Again, I'm a big -1 on this patch. There is nothing broken in 'bdist_rpm'. But there is obviously something broken in 'setuptools'. Regards, Gerry -------------- next part -------------- An HTML attachment was scrubbed... URL: From rudd-o at rudd-o.com Fri Mar 13 16:41:43 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 11:41:43 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm , bdist_debian) In-Reply-To: <49BA8511.9080604@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903131056.08524.rudd-o@rudd-o.com> <49BA8511.9080604@verizon.net> Message-ID: <200903131141.47844.rudd-o@rudd-o.com> I told you to STOP WRITING TO ME. Setuptools IS NOT the cause of the invalid prerelease RPMs, it is the DEVELOPERS OF PYTHON PACKAGES, which are merely following Setuptools policy which is ALREADY DOCUMENTED, CODED AND DEPLOYED ON HUNDREDS OF MILLIONS OF MACHINES AROUND THE GLOBE. "Education" is not a solution when the problem has already acquired these proportions. And we already know you're a big -1. You don't need to keep repeating it, you have made it abundantly transparent that you do not understand the issue at hand, nor have you provided a single line of code or even a workable counterproposal, so it's pretty clear you're full of shit and you vote -1 for bigotry rather than for valid engineering reasons. Now go away. To whoever has had the endurance to resist this continuous crapflood that this Reno dude has been flooding us with: you have my admiration, but I won't continue this pointless discussion anymore. Do whatever you want with the patches, they're yours to butcher, reject or accept, and I will continue building more features (I have already proposed a new one in the python bug tracker) -- I am not gonna let this troll break my stride. Peace out. -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ The famous politician was trying to save both his faces. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From greno at verizon.net Fri Mar 13 17:36:46 2009 From: greno at verizon.net (Gerry Reno) Date: Fri, 13 Mar 2009 13:36:46 -0400 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <200903131141.47844.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903131056.08524.rudd-o@rudd-o.com> <49BA8511.9080604@verizon.net> <200903131141.47844.rudd-o@rudd-o.com> Message-ID: <49BA99AE.0@verizon.net> Manuel, For years I see developers trying to export problems from the source domain over to some other domain because they don't want to fix the problem at its source. And this ends up creating unnecessary dependencies in software which makes it brittle and difficult to evolve the software. What you are proposing is an example of people "throwing things over the fence". This is a bad practice. In this particular instance the SOURCE of the problem is 'setuptools' naming conventions. And therefore the solution needs to be taken in setuptools. By taking problems like this and throwing them over the fence, you create more problems by making the software more tightly coupled, with more dependencies, and that makes software brittle and easily broken, hard to maintain, difficult of evolve. As far as offering alternative solutions to your patch. I've already done that but you want to ignore them. Here's the two solutions: 1. Go fix the problem in 'setuptools'. 2. Put your string rewriter into an extension that overrides bdist_rpm and call it from a special commandline option. This way it's in Distutils but does not impact every user. Those that need it can call it using the commandline option. Regards, Gerry From rudd-o at rudd-o.com Fri Mar 13 18:18:07 2009 From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O)) Date: Fri, 13 Mar 2009 13:18:07 -0500 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <20090313170544.GA15144@ayn.mi.celestial.com> References: <200903102234.50856.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> <20090313170544.GA15144@ayn.mi.celestial.com> Message-ID: <200903131318.20170.rudd-o@rudd-o.com> Bill, Thank you very much for sharing your code with us. I am pretty sure it works quite well because it's more-or-less what RPM does (bar general alphabetical sort of the non-numeric parts of the release). It'd be pretty great if this code had been the foundation for distutils from the start. Regrettably, we need to cope with the few thousand eggs that already follow the existing naming convention based on the Python setuptools implementation, and the widespread deployment of existing setuptools implementations as well. That makes it pretty clear that ascribing the cause of the problem to "bad training only" is misleading at best. Dynamically rewriting the version numbers at RPM biuld time -- exclusively for pre-release packages -- sort of patches over the issue with fairly good results and no need to change a single line of deployed code, a single line of existing eggs, the need for retraining or cumbersome per-package intervention. Now, the process isn't complete. We already have the correct version rewriting technology, and on top of that the auto-dependency technology as well, but there are a few loose ends that we still need to tie, and three things come to my mind now: 1. Pure python modules should be built to install in site-python, so they can be imported everywhere (regardless of architecture, available python interpreter version or distribution) without N*M rebuilds. I already volunteered to write this, which amounts to a two-liner patch. 2. (this is only Fedora-specific and I think it's fixed already) bdist_rpm should auto-create the .pyo files so they have selinux security contexts. 3. - the python compilation miniscript that bdist_rpm uses to install .pyc files fails in a number of packages that contain pseudo-python files (files that have the extension .py but do not really contain python code -- this is prevalent in Plone products) -- the INSTALLED_FILES contains the list of py and pyc files but does not take into account the failure to build some pyc files, thus the RPM build process bombs out because it does not find the auto- specified files at all. In my Plone build tree I have worked around all of these issues successfully by using a custom RPM %install shellscript, so I have real confidence that I can port these shellscripts directly into distutils (they should be at most a three four-liner patches, and of course the fedora-specific fix does not need to go into distutils at all). Oh, and the good news is, I have built Zope and Plone 3.2.2 for (at least) CentOS (and other RPM distros should just work, if not, msg me privately to report it as a bug and I will fix it). They are at: http://yum.rudd-o.com/ together with the scripted build process available there too. And thanks for breathing some common sense in this discussion. El Viernes 13 Marzo 2009, Bill Campbell escribi?: > On Fri, Mar 13, 2009, Gerry Reno wrote: > > Manuel, > > 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the > > 'version' and 'release' strings. They do stupid things like put > > version='3.0' and release='rc1' and then wonder why their final release > > cannot update the release candidate. THIS IS A TRAINING ISSUE... > > This is a pretty good description of the cause of the problem, > but doesn't address a solution that may be used in software. > > I deal mostly with building RPMs used under the OpenPKG portable > package management system which generally uses sane naming > conventions, at least as far as the release designations while > versions depend on the original package versions. > > The attached bit of code is basically what we use to determine > the most recent packages, and works by splitting the version and > release into tuples of numeric and non-numeric parts, then > comparing these tuples. This is based on a recipe in the > O'Reilly Python Cookbook to sort file names containing numerics. > > This seems to solve most of the problems of sorting RPM packages > on version and release, but might fail in places where there are > issues of case-sensitivity. > > Bill -- Manuel Amador (Rudd-O) Rudd-O.com - http://rudd-o.com/ GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ Women are more easily and more deeply terrified ... generating more sheer horror than the male of the species. -- Spock, "Wolf in the Fold", stardate 3615.4 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From distutils at celestial.com Fri Mar 13 17:05:44 2009 From: distutils at celestial.com (Bill Campbell) Date: Fri, 13 Mar 2009 10:05:44 -0700 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian) In-Reply-To: <49BA7B58.70303@verizon.net> References: <200903102234.50856.rudd-o@rudd-o.com> <200903130045.46626.rudd-o@rudd-o.com> <49BA5F6A.8030309@verizon.net> <200903130928.42700.rudd-o@rudd-o.com> <49BA7B58.70303@verizon.net> Message-ID: <20090313170544.GA15144@ayn.mi.celestial.com> On Fri, Mar 13, 2009, Gerry Reno wrote: > Manuel, > 'bdist_rpm' is NOT broken. What is broken is packagers misuse of the > 'version' and 'release' strings. They do stupid things like put > version='3.0' and release='rc1' and then wonder why their final release > cannot update the release candidate. THIS IS A TRAINING ISSUE... This is a pretty good description of the cause of the problem, but doesn't address a solution that may be used in software. I deal mostly with building RPMs used under the OpenPKG portable package management system which generally uses sane naming conventions, at least as far as the release designations while versions depend on the original package versions. The attached bit of code is basically what we use to determine the most recent packages, and works by splitting the version and release into tuples of numeric and non-numeric parts, then comparing these tuples. This is based on a recipe in the O'Reilly Python Cookbook to sort file names containing numerics. This seems to solve most of the problems of sorting RPM packages on version and release, but might fail in places where there are issues of case-sensitivity. Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 During times of universal deceit, telling the truth becomes a revolutionary act. --George Orwell -------------- next part -------------- # Class to facilitate sorting RPM packages based on version and # release by splitting the version and release into tuples with # numeric and non-numeric parts. # # This is based on code from the O'Reilly Python Cookbook, which # I don't have handy to cite the page number. # import os, re rpmcmd = '/bin/rpm' _fmt = r"--queryformat='%{NAME}:%{VERSION}:%{RELEASE}:%{ARCH}\n'" _digits = re.compile(r'(\d+)'); class RPM(object): def _ver(self, s): parts = [] for part in s.split('.'): r = _digits.split(part) r[1::2] = [int(p) for p in r[1::2]] parts.append(tuple(r)) return(tuple(parts)) def __init__(self, file=None, name=None, query=None): if not query: if file: cmd = '%s -qp %s %s' % (rpmcmd, _fmt, file) self.file = file else: cmd = '%s -q %s %s' % (rpmcmd, _fmt, name) self.file = 'installed' fh = os.popen(cmd) query = fh.readline().strip() fh.close() self.query = query (self.name, version, release, self.arch) = query.split(':') self._version = version self._release = release self.version = self._ver(version) self.release = self._ver(release) self.ver_rel = (self.version, self.release) def __cmp__(self, other): assert isinstance(other, RPM) return(cmp(self.ver_rel, other.ver_rel)) if __name__ == '__main__': print 'OK'; print rpmcmd print _fmt rpm = RPM(name='python') print rpm.query; print rpm.name print rpm.version print rpm.release print rpm.ver_rel print rpm.arch print rpm.__cmp__(rpm) From ziade.tarek at gmail.com Sat Mar 14 07:54:15 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 14 Mar 2009 08:54:15 +0100 Subject: [Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm , bdist_debian) In-Reply-To: <200903131141.47844.rudd-o@rudd-o.com> References: <200903102234.50856.rudd-o@rudd-o.com> <200903131056.08524.rudd-o@rudd-o.com> <49BA8511.9080604@verizon.net> <200903131141.47844.rudd-o@rudd-o.com> Message-ID: <94bdd2610903140054w4aef07a4q4ff44683a9ade5f9@mail.gmail.com> Please guys stop this now. While this is a complex topic, there's no need to be rude or impolite here. I am -1 on this kind of attitude. If the solutions were so obvious they would be applied for years. Thanks. On Fri, Mar 13, 2009 at 5:41 PM, Manuel Amador (Rudd-O) wrote: > I told you to STOP WRITING TO ME. > > Setuptools IS NOT the cause of the invalid prerelease RPMs, it is the > DEVELOPERS OF PYTHON PACKAGES, which are merely following Setuptools policy > which is ALREADY DOCUMENTED, CODED AND DEPLOYED ON HUNDREDS OF MILLIONS OF > MACHINES AROUND THE GLOBE. "Education" is not a solution when the problem > has already acquired these proportions. > > And we already know you're a big -1. You don't need to keep repeating it, > you have made it abundantly transparent that you do not understand the issue > at hand, nor have you provided a single line of code or even a workable > counterproposal, so it's pretty clear you're full of shit and you vote -1 > for bigotry rather than for valid engineering reasons. > > Now go away. > > To whoever has had the endurance to resist this continuous crapflood that > this Reno dude has been flooding us with: you have my admiration, but I > won't continue this pointless discussion anymore. Do whatever you want with > the patches, they're yours to butcher, reject or accept, and I will continue > building more features (I have already proposed a new one in the python bug > tracker) -- I am not gonna let this troll break my stride. > > Peace out. > -- > > Manuel Amador (Rudd-O) > Rudd-O.com - http://rudd-o.com/ > GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/ > > The famous politician was trying to save both his faces. > > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/