[Spacewalk-list] Spacewalk 2.10 Failing to sync repo (Michael Mraka)
Jody McIvor
JMcIvor at bclc.com
Fri Feb 14 17:03:25 UTC 2020
Hi Michael,
I ran your suggested command to peek at the header info of one of (most recent) the RPMs:
====================================================
[jmcivor@<MYREPOSERVER>]$ rpm -qip <PACKAGENAME>.rpm
Name : <PACKAGENAME>-v2.15.01
Epoch : 0
Version : 1
Release : 1
Architecture: x86_64
Install Date: (not installed)
Group : BCLC
Size : 8121032
License : 2017 BCLC
Signature : (none)
Source RPM : terminal-source
Build Date : Thu 06 Feb 2020 01:44:20 PM PST
Build Host : <BUILDHOST>
Relocations : (not relocatable)
Packager : <USERNAMEOFPACKAGER>
Vendor : BCLC
URL : n/a
Summary : terminal
Description :
<APPDESCRIPTION>
====================================================
And ran manual repo sync:
====================================================
[root@<SPACEWALKSERVER>]# spacewalk-repo-sync --channel <CHANNELNAME> --type yum
08:42:02 ======================================
08:42:02 | Channel: <CHANNELNAME>
08:42:02 ======================================
08:42:02 Sync of channel started.
08:42:02
08:42:02 Processing repository with URL: http://<REPOSERVER>/rhel/x86_64/<CHANNELNAME> /
08:42:03 Packages in repo: 111
08:42:03 Packages already synced: 65
08:42:03 Packages to sync: 14
08:42:03 New packages to download: 14
08:42:03 Downloading packages:
08:42:03 1/14 : <PACKAGENAME>.rpm
08:42:03 2/14 : <PACKAGENAME>.rpm
08:42:03 3/14 : <PACKAGENAME>.rpm
08:42:03 4/14 : <PACKAGENAME>.rpm
08:42:03 5/14 : <PACKAGENAME>.rpm
08:42:03 6/14 : <PACKAGENAME>.rpm
08:42:04 7/14 : <PACKAGENAME>.rpm
08:42:04 8/14 : <PACKAGENAME>.rpm
08:42:04 9/14 : <PACKAGENAME>.rpm
08:42:04 10/14 : <PACKAGENAME>.rpm
08:42:04 11/14 : <PACKAGENAME>.rpm
08:42:04 12/14 : <PACKAGENAME>.rpm
08:42:04 13/14 : <PACKAGENAME>.rpm
08:42:04 14/14 : <PACKAGENAME>.rpm
08:42:05
08:42:05 Importing packages to DB:
Importing packages: |##################################################| 100.0%
08:42:05
08:42:05 Errata in repo: 0.
08:42:05 Sync of channel completed in 0:00:03.
08:42:05 Total time: 0:00:03
====================================================
As usual, it reports completion, but digging into the actual log for this channels reposync (var/log/rhn/reposync/<CHANNELNAME>.log) gives us the details I provided previously:
====================================================
2020/02/14 08:42:05 -07:00 Sync of channel completed in 0:00:03.
2020/02/14 08:48:08 -07:00 Command: ['/bin/spacewalk-repo-sync', '--channel', '<CHANNELNAME>', '--type', 'yum']
2020/02/14 08:48:08 -07:00 Sync of channel started.
2020/02/14 08:48:08 -07:00
2020/02/14 08:48:08 -07:00 Processing repository with URL: http://<REPOSERVER>/rhel/x86_64/<CHANNELNAME>/
2020/02/14 08:48:08 -07:00 Packages in repo: 111
2020/02/14 08:48:08 -07:00 Packages already synced: 65
2020/02/14 08:48:08 -07:00 Packages to sync: 14
2020/02/14 08:48:08 -07:00 New packages to download: 14
2020/02/14 08:48:08 -07:00 Downloading packages:
2020/02/14 08:48:09 -07:00 1/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 2/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 3/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 4/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 5/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 6/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 7/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 8/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 9/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 10/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 11/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 12/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 13/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:09 -07:00 14/14 : <PACKAGENAME>.rpm
2020/02/14 08:48:10 -07:00 Importing packages started.
2020/02/14 08:48:10 -07:00
2020/02/14 08:48:10 -07:00 Importing packages to DB:
2020/02/14 08:48:10 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 unknown header tag
2020/02/14 08:48:11 -07:00 Importing packages finished.
2020/02/14 08:48:11 -07:00
2020/02/14 08:48:11 -07:00 Errata in repo: 0.
2020/02/14 08:48:11 -07:00 Sync of channel completed in 0:00:03.
====================================================
If I don't hear from you today, have a great V-day :)
Thanks,
Jody McIvor
Sr. Systems Administrator, Integration
Les Services D'intégration
[TEL] (250) 852-5202
jmcivor at bclc.com
BCLC.com
-----Original Message-----
From: spacewalk-list-bounces at redhat.com <spacewalk-list-bounces at redhat.com> On Behalf Of spacewalk-list-request at redhat.com
Sent: February 14, 2020 1:06 AM
To: spacewalk-list at redhat.com
Subject: Spacewalk-list Digest, Vol 141, Issue 16
Send Spacewalk-list mailing list submissions to
spacewalk-list at redhat.com
To subscribe or unsubscribe via the World Wide Web, visit
https://www.redhat.com/mailman/listinfo/spacewalk-list
or, via email, send a message with subject or body 'help' to
spacewalk-list-request at redhat.com
You can reach the person managing the list at
spacewalk-list-owner at redhat.com
When replying, please edit your Subject line so it is more specific than "Re: Contents of Spacewalk-list digest..."
Today's Topics:
1. "Invalid function call attempted (code 6)" (Paul Greene)
2. Re: "Invalid function call attempted (code 6)" (Paul Greene)
3. Re: "Invalid function call attempted (code 6)" (Ezequiel Sozzi)
4. Re: Spacewalk 2.10 Failing to sync repo (Michael Mraka)
----------------------------------------------------------------------
Message: 1
Date: Thu, 13 Feb 2020 17:25:30 -0500
From: Paul Greene <paul.greene.va at gmail.com>
To: spacewalk-list at redhat.com
Subject: [Spacewalk-list] "Invalid function call attempted (code 6)"
Message-ID:
<CAMAkkvPzCUBjJPo8JrphSsboxSUA4fkaw8T9oWL3X35qTT03yA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I have a spacewalk 2.9 server with CentOS 7 clients. When I run a scheduled remote command on 50 systems, usually about half of the systems will get marked as "failed" with the error "Invalid function call attempted (code 6)".
They all have the same configuration, and every line put in the remote command will run just fine from a command prompt. If I go into a system that has been marked "failed" and manually verify if the command did what it was supposed to do, many times it actually did succeed, but was still marked "failed". And there are some that did in fact fail.
How can I address this error to get rid of the false "failed" messages?
I looked in /var/log/up2date on the clients that failed and get just these messages at the time the scheduled task failed:
up2date updateLoginfo() login info
up2date logging into up2date server
up2date successfully retrieved authentication token from up2date server
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.redhat.com/archives/spacewalk-list/attachments/20200213/cee09dea/attachment.html>
------------------------------
Message: 2
Date: Thu, 13 Feb 2020 18:26:16 -0500
From: Paul Greene <paul.greene.va at gmail.com>
To: spacewalk-list at redhat.com
Subject: Re: [Spacewalk-list] "Invalid function call attempted (code
6)"
Message-ID:
<CAMAkkvORhbxcqe2E+U5JTc5HHyynT9wCYLO_EPwqYoDkonQyoQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Just to add some more info that might be relevant.
The spacewalk server itself is running CentOS 7.7, kernel 3.10.0-1062.9.1.el7.x86_64, and most of the clients that are failing are running the same kernel.
There's some machines running an older kernel - 3.10.0-862.14.4.el7.x86_64
- and on these machines they seem to reliably run the task without problems.
A small number of machines on the newer kernel succeed with the task, but most don't.
On Thu, Feb 13, 2020 at 5:25 PM Paul Greene <paul.greene.va at gmail.com>
wrote:
> I have a spacewalk 2.9 server with CentOS 7 clients. When I run a
> scheduled remote command on 50 systems, usually about half of the
> systems will get marked as "failed" with the error "Invalid function
> call attempted (code 6)".
>
> They all have the same configuration, and every line put in the remote
> command will run just fine from a command prompt. If I go into a
> system that has been marked "failed" and manually verify if the
> command did what it was supposed to do, many times it actually did
> succeed, but was still marked "failed". And there are some that did in fact fail.
>
> How can I address this error to get rid of the false "failed" messages?
>
> I looked in /var/log/up2date on the clients that failed and get just
> these messages at the time the scheduled task failed:
>
> up2date updateLoginfo() login info
> up2date logging into up2date server
> up2date successfully retrieved authentication token from up2date
> server
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.redhat.com/archives/spacewalk-list/attachments/20200213/d301b339/attachment.html>
------------------------------
Message: 3
Date: Thu, 13 Feb 2020 21:22:23 -0300
From: Ezequiel Sozzi <sozeze at gmail.com>
To: spacewalk-list at redhat.com
Subject: Re: [Spacewalk-list] "Invalid function call attempted (code
6)"
Message-ID:
<CALqkNvL0M+zPnuZW-FtuKYc9M82sQHtFTm4208NRaai25VhA_w at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Paul,
This a more often issue than everybody things, in order to fix this, what we do is to run the next commands on the client side:
Disable all the plugins to disable rhnplugin: sed -i 's/plugins=1/plugins=0/g' /etc/yum.conf
Disable all the external repositores: yum-config-manager --disable \*
Re-enable all the plugins to enable rhnplugin: sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf Update all the packages related to rpm, rhn, and yum: yum update rpm* rhn*
yum* -y
This fix the issue. At least that's my experience. Hope this helps.
BR,
Ezequiel
El jue., 13 de febrero de 2020 7:26 p. m., Paul Greene < paul.greene.va at gmail.com> escribi?:
> I have a spacewalk 2.9 server with CentOS 7 clients. When I run a
> scheduled remote command on 50 systems, usually about half of the
> systems will get marked as "failed" with the error "Invalid function
> call attempted (code 6)".
>
> They all have the same configuration, and every line put in the remote
> command will run just fine from a command prompt. If I go into a
> system that has been marked "failed" and manually verify if the
> command did what it was supposed to do, many times it actually did
> succeed, but was still marked "failed". And there are some that did in fact fail.
>
> How can I address this error to get rid of the false "failed" messages?
>
> I looked in /var/log/up2date on the clients that failed and get just
> these messages at the time the scheduled task failed:
>
> up2date updateLoginfo() login info
> up2date logging into up2date server
> up2date successfully retrieved authentication token from up2date
> server _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.redhat.com/archives/spacewalk-list/attachments/20200213/aa87e3fe/attachment.html>
------------------------------
Message: 4
Date: Fri, 14 Feb 2020 10:06:01 +0100
From: Michael Mraka <michael.mraka at redhat.com>
To: spacewalk-list at redhat.com
Subject: Re: [Spacewalk-list] Spacewalk 2.10 Failing to sync repo
Message-ID: <20200214090601.GD3694 at magni.z31.cz>
Content-Type: text/plain; charset=utf-8
Jody McIvor:
...
> 2020/02/13 08:09:02 -07:00 Command: ['/usr/bin/spacewalk-repo-sync',
> '--channel', <MY CHANNEL NAME>', '--type', 'yum']
> 2020/02/13 08:09:02 -07:00 Sync of channel started.
> 2020/02/13 08:09:02 -07:00
> 2020/02/13 08:09:02 -07:00 Processing repository with URL: http://<MYREPOSERVER>/rhel/x86_64/<MYREPOFOLDER/
> 2020/02/13 08:09:03 -07:00 Packages in repo: 111
> 2020/02/13 08:09:04 -07:00 Packages already synced: 65
> 2020/02/13 08:09:04 -07:00 Packages to sync: 14
> 2020/02/13 08:09:04 -07:00 New packages to download: 14
> 2020/02/13 08:09:04 -07:00 Downloading packages:
> 2020/02/13 08:09:04 -07:00 1/14 : <PACKAGENAME>.rpm
...
> 2020/02/13 08:09:08 -07:00 Importing packages to DB:
> 2020/02/13 08:09:08 -07:00 unknown header tag
Hello,
I guess rpm packages you try to sync contain a new header which is not recognized by rpm on your server.
Try to run from commandline (on server)
# rpm -qip <PACKAGENAME>.rpm
# /usr/bin/spacewalk-repo-sync --channel <MY CHANNEL NAME> and watch the errors.
Regards,
--
Michael Mr?ka
System Management Engineering, Red Hat
------------------------------
_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list
End of Spacewalk-list Digest, Vol 141, Issue 16
***********************************************
________________________________
This email is intended only for the addressee. It may contain confidential or proprietary information that cannot be disclosed without BCLC's permission. If you have received this email in error, please notify the sender immediately and delete the email.
More information about the Spacewalk-list
mailing list