[scl.org] Python3 (always latest) community SCL

[Nick Coghlan]

On Fri, Apr 14, 2017 at 12:27 AM, Tomas Orsava <torsava at redhat.com> wrote:
> Hi!
>
> On 04/11/2017 10:24 AM, Nick Coghlan wrote:
>>
>> I've been mulling this idea over for the past couple of weeks, and I'm
>> wondering if it might make sense to create a rolling "sclo-python3"
>> SCL, that's initially forked from
>> https://www.softwarecollections.org/en/scls/rhscl/rh-python35/, but
>> explicitly promises to rebase to new Python feature releases when they
>> come out.
>>
>> So if people were happy to always run on the leading edge (even for
>> X.Y.0 releases), they could use "sclo-python3", but if they wanted to
>> stay on a particular X.Y release for a while, they would need to
>> switch to the downstream rh-pythonXY SCLs.
>>
>> Remi, if I wanted to do that, where would I start?
>> https://github.com/sclorg-distgit is useful as a reference for
>> submitting changes to existing community SCLs, but it doesn't provide
>> any guidance on how to start a new one (and that info is also missing
>> from the wiki).
>
> It would be great to have a state-of-the-art Python in Enterprise Linux, but
> I think it would be better using the existing (though lacking maintenance)
> Python 3 in EPEL [0] mechanism.
>
> While installing and using SCLs isn't hard, I think an RPM packaged version
> is still easier for both maintenance and usage and people can have EPEL
> packages built against it as well. In addition the transition mechanism is
> smoother, as two Python versions are coexisting during the  transition
> period, whereas the rolling SCL would just switch and everyone would have to
> immediately adjust.
>
> What would be the advantage of creating a rolling 'scl-python3' collection
> over the EPEL mechanism [1]?

The main practical issue I see with the EPEL parallel installation
design is the part about updating "/usr/bin/python3" to new "stable"
versions:

==============
In a situation when python3X is in EPEL and 3.X+1 is released
upstream, the following happens:

* python3X+1 package is created for EPEL ASAP and all extension
packages are built also for this new python3X+1 stack.
* When all packages are rebuilt for python3X+1, the old python3X stack
is retired after certain period. This period gives everyone enough
time to rebuild their packages while being as short as possible. There
is intentionally no hard limit here, we will approach this case by
case.
* "/usr/bin/python3" belongs to the "stable" python3X stack. Switching
/usr/bin/python3 from python3X to python3X+1 happens shortly before
the end of transitional period (== before obsoleting python3X) and it
is announced on epel-devel.
Usage of "/usr/bin/python3" is discouraged in favour of using
/usr/bin/python3.X explicitly.
==============

While I thought this sounded reasonable at the time, it turns out to
have a lot of problems in practice, as it makes it hard for developers
to say "I just want to run on a pre-built version of the latest
upstream Python". Instead, they have to choose between:

* referring to python3.x in their own code, and having to update all
those references to switch to a new version
* keeping their "python3" references, and trusting that the originally
planned EPEL transitions will happen in a timely fashion

And then regardless of the approach they choose, they have to rely on
either virtualenv or fiddling with the system level symlink to run
against an alternative Python 3 stack.

Accordingly, the main benefit I see to the rolling community SCL is
that it makes "python3" load the SCL version when the SCL is enabled,
which *does* make it easy for developers to choose between running
against either the latest Python 3.x (by building on top of either
Fedora or the proposed community SCL), or against a specific Python
3.x version (by using the Red Hat SCLs). If the native EPEL builds are
retained, then they would be a *downstream* of the Red Hat SCLs, so
the complete maintenance flow would be:

- upstream release happens
- Fedora system Python and rolling Python 3 community SCL are updated
- Python 3.x Red Hat SCLs are updated
- EPEL Python 3.x stacks are updated

That approach is then a lot closer to the traditional RPM update flow
from upstream->Fedora->RHEL->CentOS, just modified to be
upstream->SCLo->RHSCL->EPEL since there isn't a system Python 3 stack
in RHEL & CentOS.

Cheers,
Nick.




Cheers,
Nick.

-- 
Nick Coghlan
Red Hat Platform Engineering, Brisbane