Proposal: Python 3 in Fedora 13
yersinia
yersinia.spiros at gmail.com
Sat Oct 3 20:09:17 UTC 2009
On Thu, Oct 1, 2009 at 7:15 PM, David Malcolm <dmalcolm at redhat.com> wrote:
> Proposal: Python 3 in Fedora 13
>
> "Evolutionary, not revolutionary": build a python 3 stack
> parallel-installable with the python 2 stack.
>
> = High-level summary =
> - Python 3.0 was released almost 10 months ago, on 2008-12-03, and the
> latest release of the 3.* branch is 3.1.1, released on 2009-08-17.
> - Other distros have python 3, though not necessarily with anything
> "on top" resembling the full python 2 stack.
> - We have a working, valuable python 2 stack, which is used by
> critical system components (yum and anaconda): we must not destabilize
> the python 2 stack.
> - Python 3 is sufficiently different from python 2 that we need them
> to be independent software stacks.
> - I plan to spend a large chunk of my $DAYJOB over the next few months
> trying to build a useful Python 3 stack for Fedora 13, for some
> definition of "useful" (help will be appreciated!)
> - I don't want to add extra work for package maintainers: if you
> maintain an SRPM of a python 2 module that's working for you, you
> shouldn't feel obligated to own a separate SRPM for python 3. If
> someone has a need for the module on python 3, they can take on that
> work.
>
> = Background =
> Python 3 is intended by upstream to be the future of Python, but we have
> many critical components that use Python 2. Python 2 and Python 3 are
> sufficiently different that we need both (try writing "print" in each).
> Python 2 will be around for a long time.
>
> An interesting summary of Python 3 adoption can be seen here:
>
> http://renesd.blogspot.com/2009/09/py3kpython3-more-than-one-year-on-096.html
>
> An earlier proposal about python 3 in Fedora is here:
> https://www.redhat.com/archives/fedora-devel-list/2008-May/msg02417.html
>
> Going forward, I will have plenty of time to spend, as part of my
> dayjob, on Python in Fedora [1]
>
> = Proposal =
> I want to get python 3 into Fedora, but I don't want to break yum or
> anaconda (or anything else, for that matter!)
>
> How to do this? I propose that Fedora shall have separate,
> parallel-installable Python 2 and Python 3 stacks. I believe we can get
> things to the point where on a Fedora box you'd be able to install both
> stacks, and have some processes running python 2 code, and some running
> python 3, simultaneously.
>
> Where I would draw the line is on having both python 2 and python 3
> running within the same _process_: the two libraries share most of their
> symbol names, but with differing implementations, and the result of
> trying to dynamically link the two into the same address space would be
> highly unstable.
>
> As an example, you'd be able to install both mod_python and mod_python3
> rpms, but you wouldn't be able to (sanely) configure httpd to have both
> running simultaneously (I guess we should add a run-time warning for
> this case)
>
> Scoping:
> - this work would target Fedora 13. I'd avoid pushing it into F12
> until it's proven safe to do so
> - the proposal is for python 2 vs python 3. It could be extended to
> support having multiple minor-versions of Python as well, but that's a
> big extension of the work involved and not something I'm planning to
> work on myself.
>
> = Details =
> We should split python 2 and python 3 at the source RPM level, where
> possible.
>
> The easy case is when upstream release separate tarballs for the python
> 2 and python 3 versions of code. For example, given package
> "python-foo" in packaging CVS, there would be a separate "python3-foo"
> for the python 3 version. There would be no expectation that the two
> would need to upgrade in lock-step. (The two SRPMS could have different
> maintainers within Fedora: the packager of a python 2 module might not
> yet have any interest in python 3)
>
> The more difficult case is when the python module is emitted as part of
> the build of a larger module. Some examples:
> - the build of "rpm" itself emits an "rpm-python" subpackage.
> - Another example is the "postgres" srpm, which emits a
> "postgresql-python" subpackage.
>
> In a quick attempt at seeing other examples, on my laptop (F11), here
> are the packages installed that provide python modules where the package
> name differs from the srpm name:
> $ rpm -qf /usr/lib/python2.*/site-packages/* | grep -v "is not owned" |
>
SIA, this is off of topic , i am sure. BUT, it is very strange that could be
exists, perhaps, some file or directory not owned by someone. Isn't it ?
> sort | uniq | xargs rpm -q --qf "%{NAME}-%{VERSION}-%{RELEASE}
> %{SOURCERPM}\n" | sed -e"s/.src.rpm//" | squeal -f table col0, col1 from
> - where col0 != col1
> col0| col1|
>
> ----------------------------------------+----------------------------------+
> at-spi-python-1.26.0-1.fc11| at-spi-1.26.0-1.fc11|
> audit-libs-python-1.7.13-1.fc11| audit-1.7.13-1.fc11|
> cracklib-python-2.8.13-4| cracklib-2.8.13-4|
> gamin-python-0.1.10-4.fc11| gamin-0.1.10-4.fc11|
> hplip-libs-3.9.8-12.fc11| hplip-3.9.8-12.fc11|
> libproxy-python-0.2.3-10.fc11| libproxy-0.2.3-10.fc11|
> libselinux-python-2.0.80-1.fc11| libselinux-2.0.80-1.fc11|
> libsemanage-python-2.0.31-4.fc11| libsemanage-2.0.31-4.fc11|
> libuser-python-0.56.9-3| libuser-0.56.9-3|
> libxml2-python-2.7.3-3.fc11| libxml2-2.7.3-3.fc11|
> newt-python-0.52.10-4.fc11| newt-0.52.10-4.fc11|
> plague-common-0.4.5.7-5.20090612cvs.fc11|
> plague-0.4.5.7-5.20090612cvs.fc11|
> policycoreutils-python-2.0.62-12.12.fc11|
> policycoreutils-2.0.62-12.12.fc11|
> python-magic-5.03-2.fc11| file-5.03-2.fc11|
> python-slip-dbus-0.1.15-3.fc11| python-slip-0.1.15-3.fc11|
> python-slip-gtk-0.1.15-3.fc11| python-slip-0.1.15-3.fc11|
> rpm-python-4.7.1-1.fc11| rpm-4.7.1-1.fc11|
> setroubleshoot-server-2.1.14-3.fc11| setroubleshoot-2.1.14-3.fc11|
>
> system-config-printer-libs-1.1.8-6.fc11|system-config-printer-1.1.8-6.fc11|
>
> Such SRPMS have a:
> BuildRequires: python-devel
> which is a subpackage from the python build (2.6)
>
> My plan here is that we should create a python3-devel subpackage as a
> subpackage of the python3 build, and have it parallel-installable with
> the python-devel package.
>
> We could then %prep the rpm build for each of the above so that the
> python 3 support is built as a parallel component of the build,
> independently of the python 2 support e.g. by copying the python2
> support into a separate dir, then applying a patch as necessary (and
> somehow wiring up the configuration/make so it builds both...) The
> caveat here is that I haven't tried actually doing this yet for any of
> these packages. Issues with this approach:
> - I don't yet know if autoconfiguration will work well with both
> -devel packages installed
> - It will probably involve actually doing the porting work for each
> package (yay, we get to be leaders!)
> - Whoever does this for a package needs to work closely with the
> upstream for that package
>
> I'll have a go at doing this for rpm when I get back from vacation.
> Arguably the existing python 2 binding of rpm isn't great, but it's
> probably best to go for close compatibility between python 2 and python
> 3 rather than try to overhaul the bindings as part of the port to python
> 3: one thing at a time!
>
>
> "Naming convention" proposal:
> How does this sound:
> - an rpm with a "python-" prefix means a python 2 rpm, of the
> "default" python 2 minor version (for Fedora this will be the most
> recent stable upstream minor release, for EPEL it will be the minor
> release of 2 that came with the distro, so 2.4 for EPEL5)
>
> - an rpm with a "python3-" prefix means a python 3 rpm, of the
> "default" python 3 minor version (for Fedora this will be the most
> recent stable upstream release)
>
> (we could extend this to have specific minor-releases (e.g. use
> "python24-" for a python 2.4 stack, in case some brave soul wants to get
> zope/plone running. But may be better to try to fix the zope/2.6
> incompatibility at that point (caveat: haven't looked at the details of
> the issue). I don't intend to work on such versions))
>
> What about packages without a "python-" prefix? Proposal: If upstream
> has a naming convention for python2 vs python3, use it. Otherwise, add
> a "python3-" prefix to make things clear. I'm not sure about the
> details here. Examples?
>
> There have been various discussions upstream about what to call the
> python 3 binary. My favorite quote on the subject is from Guido,
> http://mail.python.org/pipermail/python-3000/2008-March/012421.html :
> > During the next 3 years or so, installing Py3k as the default "python"
> > will be a deed of utter irresponsibility and is likely to break your
> > system in subtle ways (both OSX and Linux these days use Python for
> > certain system tasks). If you *really* want to shoot yourself in the
> > foot this way, go ahead and explicitly use "make altinstall
> > bininstall" or link it yourself.
>
> I propose that for Fedora we have "/usr/bin/python3" for the
> system/default version of python 3 and "/usr/bin/python" for the
> system/default version of python 2. Both would be symlinks to a binary
> with the minor-release embedded in the name ("/usr/bin/python3.1" and
> "/usr/bin/python/2.6").
>
> As I understand things, this should make us broadly in agreement with
> upstream; see e.g.:
> http://mail.python.org/pipermail/python-dev/2009-April/088862.html
> http://mail.python.org/pipermail/python-dev/2009-April/088884.html
>
> A rough plan for Fedora 13 might be:
> - get python3 packaged in a manner compatible with the above
> - (persuade /usr/lib/rpm/brp-python-bytecompile to use the correct
> python when building rpms containing .py files)
> - get rpm bindings working with python3
> - get some useful components working e.g. a web stack: Django,
> TurboGears etc (though e.g. Django's py3k support is a long way off
> IIRC); ideas?
> - solidify packaging guidelines for python 2 vs python 3 once we've
> got some experience with the above and hopefully proven the techniques
> - look at porting major components over to python3 (but probably don't
> actually do this for F13; leave python 2 as the critical component, I
> suspect):
> - yum (rpm)
> - anaconda
>
> However "no plan survives contact with the enemy", we'll see how things
> turn out in reality when trying to get a full integrated stack working.
>
> Future work (F14?) could involve cutting over the major components, so
> that the base install would bring in "python3", and "python" would
> become optional. Obviously there's a _lot_ to be done before that can
> be done sanely.
>
> = Progress so far =
> I've put together a somewhat-working python3 srpm, based on the python
> srpm (with lots of FIXMEs added...) It's not yet ready for a formal
> package review (I'm working through the various patches, and it's not
> yet fully installable in parallel with the python 2 stack), but you can
> see the specfile here:
> http://people.redhat.com/dmalcolm/python3/python3.spec
> and an SRPM here:
> http://people.redhat.com/dmalcolm/python3/python3-3.1.1-1.fc13.src.rpm
>
> After I did this work, I saw that a couple of other people have written
> Python 3 srpms for Fedora:
> http://ivazquez.fedorapeople.org/packages/python3000/
> and
> https://bugzilla.redhat.com/show_bug.cgi?id=526126
> and there was this proposal for doing Python 3 in F10:
>
> https://www.redhat.com/archives/fedora-devel-list/2008-May/msg02417.html
> similar to this one. Obviously I want to work with those people to come
> up with a working python 3 rpm in Fedora.
>
> There's also the merge-review for python:
> https://bugzilla.redhat.com/show_bug.cgi?id=226342 ; many of the
> comments there would thus also apply to my srpm, given that I used the
> former as my starting point.
>
> On a tangent: we currently have 2.6.2 in F12/rawhide. Upstream is
> preparing a 2.6.3 release, with many bugfixes. This seems to me like a
> candidate for F13. (the release notes for the 2.6.3 rc are long; I'd
> want it to have a _lot_ of testing before pushing it to F12))
>
>
> Thoughts?
> Dave
>
> [1] I'm being transferred at work, and will be able to spend a lot of
> time on Python within Fedora. Previously, my job involved keeping
> various internal RH systems working (with any Fedora work done
> afterhours or as a side project). Having said that, I'm about to go on
> vacation with no access to a computer for the period October 3rd-10th
>
>
> --
> fedora-devel-list mailing list
> fedora-devel-list at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-devel-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20091003/1c4b80d1/attachment.htm>
More information about the fedora-devel-list
mailing list