[scl.org] rh-python36 SPEC file question (circular dependency)

Charalampos Stratakis cstratak at redhat.com
Fri Jun 15 09:00:42 UTC 2018 

I am thinking that at this point something else is missing from your system.

do you have the scl-utils and scl-utils-build packages installed?

On which system are you actually trying to build the collection?

----- Original Message -----
> From: "Joshua Kramer" <joskra42.list at gmail.com>
> To: "Charalampos Stratakis" <cstratak at redhat.com>, sclorg at redhat.com
> Sent: Thursday, June 14, 2018 3:29:31 AM
> Subject: Re: [scl.org] rh-python36 SPEC file question (circular dependency)
> 
> I'm still trying to figure out why some of the macros are throwing the
> error with RPM.  I notice that the file
> /etc/rpm/macros.python3.rh-python36 has several lines like the
> following:
> 
> %%{scl_no_vendor}_python_requires /usr/lib/rpm/pythondeps-scl-36.sh
> --requires %{?scl:rh-python36-}
> 
> If I comment out all of those lines, the "Macro % has illegal name..."
> errors go away.  If I uncomment one line, I get one instance of the
> error.
> 
> Looking in some of the other macro files, I did not see any instance
> of %%{macro_name}, it was always %{macro_name} ... so I removed one of
> the % at the beginning of the line, and I still get the illegal name
> error.
> 
> Does anyone know where I should look next, lest I compile RPM from
> source and run it in a debugger?
> 
> On Sun, Jun 10, 2018 at 1:44 PM, Joshua Kramer <joskra42.list at gmail.com>
> wrote:
> > With the packages installed, I did a strace while running RPM to see
> > what I could find.  Here is an excerpt of the results:
> >
> > open("/etc/rpm/macros.python3.rh-python36", O_RDONLY|O_LARGEFILE) = 3
> > fstat64(3, {st_mode=S_IFREG|0644, st_size=1879, ...}) = 0
> > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > 0) = 0x76efb000
> > read(3, "# define these macros in a way w"..., 4096) = 1879
> > open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
> > fstat64(4, {st_mode=S_IFREG|0644, st_size=2502, ...}) = 0
> > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > 0) = 0x76efa000
> > read(4, "# Locale name alias data base.\n#"..., 4096) = 2502
> > read(4, "", 4096)                       = 0
> > close(4)                                = 0
> > munmap(0x76efa000, 4096)                = 0
> > open("/usr/share/locale/en_US.utf8/LC_MESSAGES/rpm.mo", O_RDONLY) = -1
> > ENOENT (No such file or directory)
> > open("/usr/share/locale/en_US/LC_MESSAGES/rpm.mo", O_RDONLY) = -1
> > ENOENT (No such file or directory)
> > open("/usr/share/locale/en.utf8/LC_MESSAGES/rpm.mo", O_RDONLY) = -1
> > ENOENT (No such file or directory)
> > open("/usr/share/locale/en/LC_MESSAGES/rpm.mo", O_RDONLY) = -1 ENOENT
> > (No such file or directory)
> > write(2, "error: ", 7error: )                  = 7
> > write(2, "Macro % has illegal name (%defin"..., 35Macro % has illegal
> > name (%define)
> >
> > It appears that once RPM reads the line "# define these amcros in a
> > way..." it runs into something it doesn't like, then it attempts to
> > generate an error message using the correct locale and LC_MESSAGES.
> > Failing that it spits out the default message.
> >
> > I went to the offending file (macros.python3.rh-python36) and removed
> > the comment, with the same result.
> >
> > Where should I look next?
> >
> >
> > On Sat, Jun 9, 2018 at 2:33 PM, Joshua Kramer <joskra42.list at gmail.com>
> > wrote:
> >> I had actually attempted to do just that, and ran into another
> >> interesting problem.  When I install rh-python36-python-devel package,
> >> it breaks the RPM database in some way.  With that package installed,
> >> every time I do anything with RPM, I get several instances of this
> >> error message:  "error: Macro % has illegal name (%define)"
> >>
> >> This error does not interfere with the ability of RPM to install or
> >> remove packages.  However, as part of the rpmbuild process,
> >> dependencies of RPM packages are checked.  When this occurs, some
> >> program spits out several instances of that error message, and the
> >> program returns with an error.  RPMBUILD then aborts the build when it
> >> detects the error.
> >>
> >> I know the error is a bash shell error, but I have not yet determined
> >> what in the -devel package is broken in such a way to cause that error
> >> to occur whenever RPM is run.
> >>
> >> On Sat, Jun 9, 2018 at 12:03 PM, Charalampos Stratakis
> >> <cstratak at redhat.com> wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Joshua Kramer" <joskra42.list at gmail.com>
> >>>> To: "Charalampos Stratakis" <cstratak at redhat.com>, sclorg at redhat.com
> >>>> Sent: Saturday, June 9, 2018 4:47:43 PM
> >>>> Subject: Re: [scl.org] rh-python36 SPEC file question (circular
> >>>> dependency)
> >>>>
> >>>> > Long story short:
> >>>> > 1. Build python with rehweel set to 0.
> >>>> .....
> >>>>
> >>>> I got further with these steps, but there are some omissions.  I
> >>>> attempted to build python.spec as the first step (after setting
> >>>> with_rewheel to 0), but it complained about a missing dependency of
> >>>> rh-python36.  So I then built the rh-python36 metapackage first, and I
> >>>> had to install that as root.  The first pass of python.spec then built
> >>>> successfully.
> >>>>
> >>>> I then modified python-setuptools.spec to disable build_wheel, and
> >>>> attempted to build.  It failed because it requires rh-python36-devel.
> >>>> So I attempt to install that package as root, and there are additional
> >>>> dependency issues:
> >>>>         /opt/rh/rh-python36/root/usr/bin/python3.6m is needed by
> >>>> rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
> >>>>         libpython3.6m.so.rh-python36-1.0 is needed by
> >>>> rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
> >>>>         python(abi) = 3.6 is needed by
> >>>> rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
> >>>>         rh-python36-python(armv7hnl-32) = 3.6.3-3.el7.centos is needed
> >>>> by rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
> >>>>         rh-python36-python-libs(armv7hnl-32) = 3.6.3-3.el7.centos is
> >>>> needed by rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
> >>>>
> >>>> I've attempted various combinations of force-installing the packages I
> >>>> have, with various other issues.  So what is the correct next step in
> >>>> getting the setuptools package to build?
> >>>>
> >>>>
> >>>
> >>> Without something which contains a dependency solver you will have to
> >>> manually install the packages at your system.
> >>>
> >>> At this point it seems that you will have to install the python-devel,
> >>> python and python-libs.
> >>>
> >>> You can check the dependencies of each rpm by invoking 'rpm -qp
> >>> --requires <rpm>', or by examining the %package
> >>> directives at the SPEC file of python. Although without doing it in some
> >>> sort of build system (e.g. centos/fedora builds system),
> >>> you will have to manually inspect what the dependencies are each time.
> >>>
> >>> Please let me know if that worked for you.
> >>>
> >>>>
> >>>>
> >>>> On Fri, Jun 8, 2018 at 12:10 PM, Charalampos Stratakis
> >>>> <cstratak at redhat.com> wrote:
> >>>> >
> >>>> >
> >>>> > ----- Original Message -----
> >>>> >> From: "Joshua Kramer" <joskra42.list at gmail.com>
> >>>> >> To: "Neal Gompa" <ngompa13 at gmail.com>
> >>>> >> Cc: sclorg at redhat.com
> >>>> >> Sent: Thursday, June 7, 2018 2:22:56 PM
> >>>> >> Subject: Re: [scl.org] rh-python36 SPEC file question (circular
> >>>> >> dependency)
> >>>> >>
> >>>> >> Thanks, Neal!  That got me a little farther.  Per the comments at the
> >>>> >> top of they python.spec file, I set with_rewheel to 0, then I built
> >>>> >> the Python RPM.  That seems to have worked.
> >>>> >>
> >>>> >> Now, when I attempt to build setuptools, using this command:
> >>>> >> rpmbuild -ba ./python-setuptools.spec --define 'scl rh-python36' >
> >>>> >> python-setuptools-build.log 2>&1
> >>>> >>
> >>>> >> ...the build command fails with these errors:
> >>>> >> error: Failed build dependencies:
> >>>> >>         rh-python36-python-devel is needed by
> >>>> >> rh-python36-python-setuptools-36.5.0-1.el7.centos.noarch
> >>>> >>         rh-python36-python-pip is needed by
> >>>> >> rh-python36-python-setuptools-36.5.0-1.el7.centos.noarch
> >>>> >>         rh-python36-python-wheel is needed by
> >>>> >> rh-python36-python-setuptools-36.5.0-1.el7.centos.noarch
> >>>> >>
> >>>> >> Attempting to build the python-pip package results in similar errors.
> >>>> >>
> >>>> >> I have the following packages installed:
> >>>> >> rh-python36-2.0-1.el7.src.rpm
> >>>> >> rh-python36-python-3.6.3-3.el7.src.rpm
> >>>> >> rh-python36-python-pip-9.0.1-2.el7.src.rpm
> >>>> >> rh-python36-python-psycopg2-2.7.3-1.el7.src.rpm
> >>>> >> rh-python36-python-setuptools-36.5.0-1.el7.src.rpm
> >>>> >> rh-python36-python-virtualenv-15.1.0-2.el7.src.rpm
> >>>> >> rh-python36-python-wheel-0.30.0a0-1.el7.src.rpm
> >>>> >> rh-python36-scipy-0.19.1-2.el7.src.rpm
> >>>> >>
> >>>> >> What should I look for next?
> >>>> >>
> >>>> >> Thanks!
> >>>> >>
> >>>> >
> >>>> > Hello. I am one of the maintainers for that collection so feel free to
> >>>> > ping
> >>>> > me
> >>>> > directly, either on IRC or mail.
> >>>> >
> >>>> > Sorry for the complications you are facing, bootstrapping the
> >>>> > interpreter
> >>>> > to an rpm
> >>>> > is indeed a quite complex procedure and it could be documented better.
> >>>> >
> >>>> > I will try to outline the steps with some explanations.
> >>>> >
> >>>> > Firstly, python bundles pip and setuptools, so ideally we'd like
> >>>> > python to
> >>>> > utilize the system
> >>>> > installed packages instead of the bundled ones. What happens is that
> >>>> > we
> >>>> > first compile python
> >>>> > including the bundled wheels (that's when we set the 'rewheel' macro
> >>>> > to 0).
> >>>> >
> >>>> > Then we'd like to build pip and setuptools in the wheel [0] format so
> >>>> > they
> >>>> > can be utilized by python.
> >>>> > However we still do not have the 'wheel' package built in order to do
> >>>> > that,
> >>>> > so we'll have to (byte)compile
> >>>> > them the usual way, then compile wheel, and then build again python
> >>>> > and
> >>>> > setuptools as wheels.
> >>>> >
> >>>> > So you should set the build_wheel macro in python-pip's and
> >>>> > python-setuptools' SPEC file to 0, build them and then
> >>>> > build the python-wheel package.
> >>>> >
> >>>> > Then after you have built python-wheel, rebuild python-pip and
> >>>> > python-setuptools with the build_wheel macro macro set to 1,
> >>>> > as we can create the wheel format now since we have the wheel package
> >>>> > on
> >>>> > our system.
> >>>> >
> >>>> > Finally rebuild python with rewheel set to 1 in order for python to
> >>>> > strip
> >>>> > the bundled wheels and utilize the system installed
> >>>> > ones (with a little bit of magic [1].
> >>>> >
> >>>> > Long story short:
> >>>> > 1. Build python with rehweel set to 0.
> >>>> > 2. Build python-setuptools and python-pip with build_wheel set to 0.
> >>>> > 3. Build python-wheel.
> >>>> > 4. Build python-setuptools and python-pip with build_wheel set to 1.
> >>>> > 5. Build python with rehweel set to 1.
> >>>> >
> >>>> > I hope that helps (and I hope I didn't forget any steps :) ).
> >>>> >
> >>>> > [0] https://pythonwheels.com/
> >>>> > [1]
> >>>> > https://github.com/fedora-python/rewheel/blob/master/rewheel/__init__.py
> >>>> >
> >>>> >> On Mon, Jun 4, 2018 at 10:23 PM, Neal Gompa <ngompa13 at gmail.com>
> >>>> >> wrote:
> >>>> >> > On Mon, Jun 4, 2018 at 9:38 PM Joshua Kramer
> >>>> >> > <joskra42.list at gmail.com>
> >>>> >> > wrote:
> >>>> >> >>
> >>>> >> >> Hello,
> >>>> >> >>
> >>>> >> >> I am trying to compile rh-python36 to use with CentOS on a
> >>>> >> >> Raspberry
> >>>> >> >> Pi.
> >>>> >> >>
> >>>> >> >> On line 192 of the SPEC file are these lines:
> >>>> >> >> %if 0%{?with_rewheel}
> >>>> >> >> BuildRequires: %{?scl_prefix}python-setuptools
> >>>> >> >> BuildRequires: %{?scl_prefix}python-pip
> >>>> >> >> %endif
> >>>> >> >>
> >>>> >> >> What is this supposed to do?  It prevents the package from
> >>>> >> >> building,
> >>>> >> >> because there is a circular dependency- if I try to build
> >>>> >> >> python36, it
> >>>> >> >> complains that python36-setuptools and python36-pip are not
> >>>> >> >> installed... but I can't install those without having Python
> >>>> >> >> installed
> >>>> >> >> first.  If I comment those lines out I get no error and the build
> >>>> >> >> almost completes.  (It then fails the "multiprocessing" test, but
> >>>> >> >> that's beside the point.)  However, I don't want to break
> >>>> >> >> something by
> >>>> >> >> commenting them out... so is there a better or "right" way to get
> >>>> >> >> rid
> >>>> >> >> of the circular dependency?
> >>>> >> >>
> >>>> >> >
> >>>> >> > Python is a two-cycle build. You build the first time with rewheel
> >>>> >> > disabled (that removes those BRs), then after that's successfully
> >>>> >> > built, you use those artifacts to build again with rewheel enabled.
> >>>> >> >
> >>>> >> >
> >>>> >> > --
> >>>> >> > 真実はいつも一つ！/ Always, there's only one truth!
> >>>> >>
> >>>> >> _______________________________________________
> >>>> >> SCLorg mailing list
> >>>> >> SCLorg at redhat.com
> >>>> >> https://www.redhat.com/mailman/listinfo/sclorg
> >>>> >>
> >>>> >
> >>>> > --
> >>>> > Regards,
> >>>> >
> >>>> > Charalampos Stratakis
> >>>> > Software Engineer
> >>>> > Python Maintenance Team, Red Hat
> >>>>
> >>>
> >>> --
> >>> Regards,
> >>>
> >>> Charalampos Stratakis
> >>> Software Engineer
> >>> Python Maintenance Team, Red Hat
> 

-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat

More information about the SCLorg mailing list