[scl.org] rh-python36 SPEC file question (circular dependency)
Joshua Kramer
joskra42.list at gmail.com
Thu Jun 14 01:29:31 UTC 2018
I'm still trying to figure out why some of the macros are throwing the
error with RPM. I notice that the file
/etc/rpm/macros.python3.rh-python36 has several lines like the
following:
%%{scl_no_vendor}_python_requires /usr/lib/rpm/pythondeps-scl-36.sh
--requires %{?scl:rh-python36-}
If I comment out all of those lines, the "Macro % has illegal name..."
errors go away. If I uncomment one line, I get one instance of the
error.
Looking in some of the other macro files, I did not see any instance
of %%{macro_name}, it was always %{macro_name} ... so I removed one of
the % at the beginning of the line, and I still get the illegal name
error.
Does anyone know where I should look next, lest I compile RPM from
source and run it in a debugger?
On Sun, Jun 10, 2018 at 1:44 PM, Joshua Kramer <joskra42.list at gmail.com> wrote:
> With the packages installed, I did a strace while running RPM to see
> what I could find. Here is an excerpt of the results:
>
> open("/etc/rpm/macros.python3.rh-python36", O_RDONLY|O_LARGEFILE) = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=1879, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x76efb000
> read(3, "# define these macros in a way w"..., 4096) = 1879
> open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
> fstat64(4, {st_mode=S_IFREG|0644, st_size=2502, ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0x76efa000
> read(4, "# Locale name alias data base.\n#"..., 4096) = 2502
> read(4, "", 4096) = 0
> close(4) = 0
> munmap(0x76efa000, 4096) = 0
> open("/usr/share/locale/en_US.utf8/LC_MESSAGES/rpm.mo", O_RDONLY) = -1
> ENOENT (No such file or directory)
> open("/usr/share/locale/en_US/LC_MESSAGES/rpm.mo", O_RDONLY) = -1
> ENOENT (No such file or directory)
> open("/usr/share/locale/en.utf8/LC_MESSAGES/rpm.mo", O_RDONLY) = -1
> ENOENT (No such file or directory)
> open("/usr/share/locale/en/LC_MESSAGES/rpm.mo", O_RDONLY) = -1 ENOENT
> (No such file or directory)
> write(2, "error: ", 7error: ) = 7
> write(2, "Macro % has illegal name (%defin"..., 35Macro % has illegal
> name (%define)
>
> It appears that once RPM reads the line "# define these amcros in a
> way..." it runs into something it doesn't like, then it attempts to
> generate an error message using the correct locale and LC_MESSAGES.
> Failing that it spits out the default message.
>
> I went to the offending file (macros.python3.rh-python36) and removed
> the comment, with the same result.
>
> Where should I look next?
>
>
> On Sat, Jun 9, 2018 at 2:33 PM, Joshua Kramer <joskra42.list at gmail.com> wrote:
>> I had actually attempted to do just that, and ran into another
>> interesting problem. When I install rh-python36-python-devel package,
>> it breaks the RPM database in some way. With that package installed,
>> every time I do anything with RPM, I get several instances of this
>> error message: "error: Macro % has illegal name (%define)"
>>
>> This error does not interfere with the ability of RPM to install or
>> remove packages. However, as part of the rpmbuild process,
>> dependencies of RPM packages are checked. When this occurs, some
>> program spits out several instances of that error message, and the
>> program returns with an error. RPMBUILD then aborts the build when it
>> detects the error.
>>
>> I know the error is a bash shell error, but I have not yet determined
>> what in the -devel package is broken in such a way to cause that error
>> to occur whenever RPM is run.
>>
>> On Sat, Jun 9, 2018 at 12:03 PM, Charalampos Stratakis
>> <cstratak at redhat.com> wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Joshua Kramer" <joskra42.list at gmail.com>
>>>> To: "Charalampos Stratakis" <cstratak at redhat.com>, sclorg at redhat.com
>>>> Sent: Saturday, June 9, 2018 4:47:43 PM
>>>> Subject: Re: [scl.org] rh-python36 SPEC file question (circular dependency)
>>>>
>>>> > Long story short:
>>>> > 1. Build python with rehweel set to 0.
>>>> .....
>>>>
>>>> I got further with these steps, but there are some omissions. I
>>>> attempted to build python.spec as the first step (after setting
>>>> with_rewheel to 0), but it complained about a missing dependency of
>>>> rh-python36. So I then built the rh-python36 metapackage first, and I
>>>> had to install that as root. The first pass of python.spec then built
>>>> successfully.
>>>>
>>>> I then modified python-setuptools.spec to disable build_wheel, and
>>>> attempted to build. It failed because it requires rh-python36-devel.
>>>> So I attempt to install that package as root, and there are additional
>>>> dependency issues:
>>>> /opt/rh/rh-python36/root/usr/bin/python3.6m is needed by
>>>> rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
>>>> libpython3.6m.so.rh-python36-1.0 is needed by
>>>> rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
>>>> python(abi) = 3.6 is needed by
>>>> rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
>>>> rh-python36-python(armv7hnl-32) = 3.6.3-3.el7.centos is needed
>>>> by rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
>>>> rh-python36-python-libs(armv7hnl-32) = 3.6.3-3.el7.centos is
>>>> needed by rh-python36-python-devel-3.6.3-3.el7.centos.armv7hl
>>>>
>>>> I've attempted various combinations of force-installing the packages I
>>>> have, with various other issues. So what is the correct next step in
>>>> getting the setuptools package to build?
>>>>
>>>>
>>>
>>> Without something which contains a dependency solver you will have to manually install the packages at your system.
>>>
>>> At this point it seems that you will have to install the python-devel, python and python-libs.
>>>
>>> You can check the dependencies of each rpm by invoking 'rpm -qp --requires <rpm>', or by examining the %package
>>> directives at the SPEC file of python. Although without doing it in some sort of build system (e.g. centos/fedora builds system),
>>> you will have to manually inspect what the dependencies are each time.
>>>
>>> Please let me know if that worked for you.
>>>
>>>>
>>>>
>>>> On Fri, Jun 8, 2018 at 12:10 PM, Charalampos Stratakis
>>>> <cstratak at redhat.com> wrote:
>>>> >
>>>> >
>>>> > ----- Original Message -----
>>>> >> From: "Joshua Kramer" <joskra42.list at gmail.com>
>>>> >> To: "Neal Gompa" <ngompa13 at gmail.com>
>>>> >> Cc: sclorg at redhat.com
>>>> >> Sent: Thursday, June 7, 2018 2:22:56 PM
>>>> >> Subject: Re: [scl.org] rh-python36 SPEC file question (circular
>>>> >> dependency)
>>>> >>
>>>> >> Thanks, Neal! That got me a little farther. Per the comments at the
>>>> >> top of they python.spec file, I set with_rewheel to 0, then I built
>>>> >> the Python RPM. That seems to have worked.
>>>> >>
>>>> >> Now, when I attempt to build setuptools, using this command:
>>>> >> rpmbuild -ba ./python-setuptools.spec --define 'scl rh-python36' >
>>>> >> python-setuptools-build.log 2>&1
>>>> >>
>>>> >> ...the build command fails with these errors:
>>>> >> error: Failed build dependencies:
>>>> >> rh-python36-python-devel is needed by
>>>> >> rh-python36-python-setuptools-36.5.0-1.el7.centos.noarch
>>>> >> rh-python36-python-pip is needed by
>>>> >> rh-python36-python-setuptools-36.5.0-1.el7.centos.noarch
>>>> >> rh-python36-python-wheel is needed by
>>>> >> rh-python36-python-setuptools-36.5.0-1.el7.centos.noarch
>>>> >>
>>>> >> Attempting to build the python-pip package results in similar errors.
>>>> >>
>>>> >> I have the following packages installed:
>>>> >> rh-python36-2.0-1.el7.src.rpm
>>>> >> rh-python36-python-3.6.3-3.el7.src.rpm
>>>> >> rh-python36-python-pip-9.0.1-2.el7.src.rpm
>>>> >> rh-python36-python-psycopg2-2.7.3-1.el7.src.rpm
>>>> >> rh-python36-python-setuptools-36.5.0-1.el7.src.rpm
>>>> >> rh-python36-python-virtualenv-15.1.0-2.el7.src.rpm
>>>> >> rh-python36-python-wheel-0.30.0a0-1.el7.src.rpm
>>>> >> rh-python36-scipy-0.19.1-2.el7.src.rpm
>>>> >>
>>>> >> What should I look for next?
>>>> >>
>>>> >> Thanks!
>>>> >>
>>>> >
>>>> > Hello. I am one of the maintainers for that collection so feel free to ping
>>>> > me
>>>> > directly, either on IRC or mail.
>>>> >
>>>> > Sorry for the complications you are facing, bootstrapping the interpreter
>>>> > to an rpm
>>>> > is indeed a quite complex procedure and it could be documented better.
>>>> >
>>>> > I will try to outline the steps with some explanations.
>>>> >
>>>> > Firstly, python bundles pip and setuptools, so ideally we'd like python to
>>>> > utilize the system
>>>> > installed packages instead of the bundled ones. What happens is that we
>>>> > first compile python
>>>> > including the bundled wheels (that's when we set the 'rewheel' macro to 0).
>>>> >
>>>> > Then we'd like to build pip and setuptools in the wheel [0] format so they
>>>> > can be utilized by python.
>>>> > However we still do not have the 'wheel' package built in order to do that,
>>>> > so we'll have to (byte)compile
>>>> > them the usual way, then compile wheel, and then build again python and
>>>> > setuptools as wheels.
>>>> >
>>>> > So you should set the build_wheel macro in python-pip's and
>>>> > python-setuptools' SPEC file to 0, build them and then
>>>> > build the python-wheel package.
>>>> >
>>>> > Then after you have built python-wheel, rebuild python-pip and
>>>> > python-setuptools with the build_wheel macro macro set to 1,
>>>> > as we can create the wheel format now since we have the wheel package on
>>>> > our system.
>>>> >
>>>> > Finally rebuild python with rewheel set to 1 in order for python to strip
>>>> > the bundled wheels and utilize the system installed
>>>> > ones (with a little bit of magic [1].
>>>> >
>>>> > Long story short:
>>>> > 1. Build python with rehweel set to 0.
>>>> > 2. Build python-setuptools and python-pip with build_wheel set to 0.
>>>> > 3. Build python-wheel.
>>>> > 4. Build python-setuptools and python-pip with build_wheel set to 1.
>>>> > 5. Build python with rehweel set to 1.
>>>> >
>>>> > I hope that helps (and I hope I didn't forget any steps :) ).
>>>> >
>>>> > [0] https://pythonwheels.com/
>>>> > [1]
>>>> > https://github.com/fedora-python/rewheel/blob/master/rewheel/__init__.py
>>>> >
>>>> >> On Mon, Jun 4, 2018 at 10:23 PM, Neal Gompa <ngompa13 at gmail.com> wrote:
>>>> >> > On Mon, Jun 4, 2018 at 9:38 PM Joshua Kramer <joskra42.list at gmail.com>
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Hello,
>>>> >> >>
>>>> >> >> I am trying to compile rh-python36 to use with CentOS on a Raspberry
>>>> >> >> Pi.
>>>> >> >>
>>>> >> >> On line 192 of the SPEC file are these lines:
>>>> >> >> %if 0%{?with_rewheel}
>>>> >> >> BuildRequires: %{?scl_prefix}python-setuptools
>>>> >> >> BuildRequires: %{?scl_prefix}python-pip
>>>> >> >> %endif
>>>> >> >>
>>>> >> >> What is this supposed to do? It prevents the package from building,
>>>> >> >> because there is a circular dependency- if I try to build python36, it
>>>> >> >> complains that python36-setuptools and python36-pip are not
>>>> >> >> installed... but I can't install those without having Python installed
>>>> >> >> first. If I comment those lines out I get no error and the build
>>>> >> >> almost completes. (It then fails the "multiprocessing" test, but
>>>> >> >> that's beside the point.) However, I don't want to break something by
>>>> >> >> commenting them out... so is there a better or "right" way to get rid
>>>> >> >> of the circular dependency?
>>>> >> >>
>>>> >> >
>>>> >> > Python is a two-cycle build. You build the first time with rewheel
>>>> >> > disabled (that removes those BRs), then after that's successfully
>>>> >> > built, you use those artifacts to build again with rewheel enabled.
>>>> >> >
>>>> >> >
>>>> >> > --
>>>> >> > 真実はいつも一つ！/ Always, there's only one truth!
>>>> >>
>>>> >> _______________________________________________
>>>> >> SCLorg mailing list
>>>> >> SCLorg at redhat.com
>>>> >> https://www.redhat.com/mailman/listinfo/sclorg
>>>> >>
>>>> >
>>>> > --
>>>> > Regards,
>>>> >
>>>> > Charalampos Stratakis
>>>> > Software Engineer
>>>> > Python Maintenance Team, Red Hat
>>>>
>>>
>>> --
>>> Regards,
>>>
>>> Charalampos Stratakis
>>> Software Engineer
>>> Python Maintenance Team, Red Hat
More information about the SCLorg
mailing list