[Spacewalk-list] Antwort: Re: rhnpush fails with uninformative error msg
Lachlan Musicman
datakid at gmail.com
Wed May 25 06:32:07 UTC 2016
Thanks for answering Robert, I appreciate it. I think that those log msgs
were me casting about wildly for *anything*. I don't think they point to
the problem.
The one thing that I see that stands out more than anything else, is the
lines
<ipaddress> - - [20/May/2016:13:58:00 +1000] "GET /rhn/errors/500.jsp
HTTP/1.1" 302 -
<ipaddress> - - [20/May/2016:13:58:00 +1000] "POST /rhn/errors/500.jsp
HTTP/1.1" 302 -
<ipaddress> - - [20/May/2016:13:58:00 +1000] "POST
/rhn/Login.do?url_bounce=/rhn/errors/500.jsp&request_method=POST HTTP/1.1"
200 5967
These are seen on the server that isn't accepting rhnpush, in
/var/log/tomcat/localhost_access_<date>.txt
I do not see these on the working server, at all.
Today I did some straces - on each server - and while it's verbose and hard
to pick apart, it looks like the error is similar to the above. ie, that
the two straces appear very similar until just after the password is
entered, and then the server that wont import files looks like this. Note
that the post returns HTTP code 302:
sendto(3, "POST /APP HTTP/1.1\r\nHost: localh"..., 581, 0, NULL, 0) = 581
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "H", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "T", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "T", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "P", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "/", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "1", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, ".", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "1", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, " ", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "3", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "0", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[254525]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "2", 1, 0, NULL, NULL) = 1
On the working server it looks like this. Note that it returns HTTP code
200:
sendto(3, "POST /APP HTTP/1.1\r\nHost: localh"..., 586, 0, NULL, 0) = 586
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "H", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "T", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "T", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "P", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "/", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "1", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, ".", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "1", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, " ", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "2", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "0", 1, 0, NULL, NULL) = 1
poll([{fd=3<socket:[964921]>, events=POLLIN}], 1, 120000) = 1 ([{fd=3,
revents=POLLIN}])
recvfrom(3, "0", 1, 0, NULL, NULL) = 1
Why is that? The configurations are as identical as possible. Idiot checked
for me by two others.
------
The most dangerous phrase in the language is, "We've always done it this
way."
- Grace Hopper
On 24 May 2016 at 00:25, <Paschedag.Netlution at swr.de> wrote:
> The library looks the same to me, but the URL is different.
>
> Regards,
> Robert
>
> Mit freundlichen Grüßen
>
> Robert Paschedag
> Netlution GmbH
> Landteilstr. 33
> 68163 Mannheim
>
> im Auftrag des
> SWR
> Südwestrundfunk
> Informations- und Kommunikationssysteme
> Neckarstraße 230
> 70190 Stuttgart
>
> Telefon +49 (0)711 /929-12654 oder
> Telefon +49 (0)711 /929-13714
> paschedag.netlution at swr.de
>
> swr.de
>
>
>
>
>
> Von: Lachlan Musicman <datakid at gmail.com>
> An: spacewalk-list at redhat.com
> Datum: 23.05.2016 04:53
> Betreff: Re: [Spacewalk-list] rhnpush fails with uninformative
> error msg
> Gesendet von: spacewalk-list-bounces at redhat.com
> ------------------------------
>
>
>
> Ignore that, I've got dispatcher working again. That wasn't the problem
> apparently.
>
> On the working server, when I successfully import via rhnpush, I see this
> in /var/log/httpd/access_log:
>
> 112.12.4.12 - - [23/May/2016:11:39:33 +1000] "POST /XMLRPC HTTP/1.1" 200
> 11932 "-" "*rhn.rpclib.py/2.5.77-1.el7*
> <http://rhn.rpclib.py/2.5.77-1.el7>"
>
> When I do it on the server that doesn't work, I get this:
>
> 10.124.28.172 - - [23/May/2016:12:31:58 +1000] "POST /APP HTTP/1.1" 302 -
> "-" "*rhn.rpclib.py/2.5.77-1.el7* <http://rhn.rpclib.py/2.5.77-1.el7>"
>
> So I'm getting a different HTTP codes in my apache server for this
> rhn.rpclib.py....
>
> L.
>
> ------
> The most dangerous phrase in the language is, "We've always done it this
> way."
>
> - Grace Hopper
>
> On 23 May 2016 at 10:53, Lachlan Musicman <*datakid at gmail.com*
> <datakid at gmail.com>> wrote:
> Hmm. It looks like my osa-dispatcher isn't starting. Where are the
> osa-dispatcher config files?
>
> cheers
> L.
>
> ------
> The most dangerous phrase in the language is, "We've always done it this
> way."
>
> - Grace Hopper
>
> On 23 May 2016 at 10:29, Lachlan Musicman <*datakid at gmail.com*
> <datakid at gmail.com>> wrote:
> No one has seen this error before? No one can even give me pointers on
> where to start looking for a solution?
>
> It almost certainly seems to be a tomcat issue - that either the rhnpush
> isn't getting through a proxy (??! wtf) or it is an is failing on the other
> side of the proxy because some other reason?
>
> I've check the tomcat conf on both servers - they are the same (as
> necessary). The rhn is set up as identically as possible, the server OS is
> the same revision...I don't understand. The only discernable difference is
> that one is a VMWare VM and one is a stand alone installation.
>
> Cheers
>
> L.
>
> ------
> The most dangerous phrase in the language is, "We've always done it this
> way."
>
> - Grace Hopper
>
> On 20 May 2016 at 16:32, Lachlan Musicman <*datakid at gmail.com*
> <datakid at gmail.com>> wrote:
> In centos 7.2, using
>
> [root at vmpr-res-utils 16.05-rc2]# yum whatprovides rhnpush
>
> rhnpush-5.5.89-1.el7.noarch : Package uploader for the Spacewalk or Red
> Hat Satellite Server
> Repo : spacewalk
>
> rhnpush-5.5.89-1.el7.noarch : Package uploader for the Spacewalk or Red
> Hat Satellite Server
> Repo : @spacewalk
>
>
> [root at vmpr-res-utils 16.05-rc2]# rhnpush --server localhost -u admin
> --channel slurm_16_05_rc2 *rpm --nosig
> Password:
>
> ERROR: unhandled exception occurred: (mismatched tag: line 164, column 2).
>
> No logs are created at all (I've deleted them all and checked).
>
> There is a log in /var/log/tomcat/localhost_access_{date}.txt
>
> ::1 - - [20/May/2016:13:29:49 +1000] "GET /rhn/errors/500.jsp HTTP/1.1"
> 302 -
> ::1 - - [20/May/2016:13:29:49 +1000] "POST /rhn/errors/500.jsp HTTP/1.1"
> 302 -
> ::1 - - [20/May/2016:13:29:50 +1000] "POST
> /rhn/Login.do?url_bounce=/rhn/errors/500.jsp&request_method=POST HTTP/1.1"
> 200 5967
>
>
> Note here that it's defaulting to IPv6 address.
>
> If I use the fqdn/hostname (same thing) I get it using the ipv4, same
> problem:
>
> 10.126.18.72 - - [20/May/2016:13:58:00 +1000] "GET /rhn/errors/500.jsp
> HTTP/1.1" 302 -
> 10.126.18.72 - - [20/May/2016:13:58:00 +1000] "POST /rhn/errors/500.jsp
> HTTP/1.1" 302 -
> 10.126.18.72 - - [20/May/2016:13:58:00 +1000] "POST
> /rhn/Login.do?url_bounce=/rhn/errors/500.jsp&request_method=POST HTTP/1.1"
> 200 5967
>
>
> Note that the username and password are correct - I've tested 100 times,
> and I'm also logged into the website, *using the same credentials*.
>
> Further frustration is caused by the fact that I have it working on
> another server (and another domain/ip range), which was set up identically,
> runs the same software, and it works.
>
> ....
>
> OK, I tried to add the server to itself as a system with
>
> rhnreg_ks --serverUrl=*https://server/XMLRPC* <https://server/XMLRPC>
> --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=1-x
>
> and that also failed.
>
> Then I went into /var/log/up2date and noticed that the error was the same:
> Traceback (most recent call last):
> File "/usr/sbin/rhnreg_ks", line 216, in <module>
> cli.run()
> File "/usr/share/rhn/up2date_client/rhncli.py", line 96, in run
> sys.exit(self.main() or 0)
> File "/usr/sbin/rhnreg_ks", line 91, in main
> rhnreg.getCaps()
> File "/usr/share/rhn/up2date_client/rhnreg.py", line 248, in getCaps
> s.capabilities.validate()
> File "/usr/share/rhn/up2date_client/rhnserver.py", line 178, in
> __get_capabilities
> self.registration.welcome_message()
> File "/usr/share/rhn/up2date_client/rhnserver.py", line 63, in __call__
> return rpcServer.doCall(method, *args, **kwargs)
> File "/usr/share/rhn/up2date_client/rpcServer.py", line 203, in doCall
> ret = method(*args, **kwargs)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
> return self.__send(self.__name, args)
> File "/usr/share/rhn/up2date_client/rpcServer.py", line 37, in _request1
> ret = self._request(methodname, params)
> File "/usr/lib/python2.7/site-packages/rhn/rpclib.py", line 384, in
> _request
> self._handler, request, verbose=self._verbose)
> File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 187, in
> request
> return self._process_response(fd, connection)
> File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 215, in
> _process_response
> return self.parse_response(fd)
> File "/usr/lib/python2.7/site-packages/rhn/transports.py", line 235, in
> parse_response
> p.feed(response)
> File "/usr/lib64/python2.7/xmlrpclib.py", line 558, in feed
> self._parser.Parse(data, 0)
> <class 'xml.parsers.expat.ExpatError'>: mismatched tag: line 164, column 2
>
>
> which lead me to this:
>
> *https://bugzilla.redhat.com/show_bug.cgi?id=1277623*
> <https://bugzilla.redhat.com/show_bug.cgi?id=1277623>
>
> but both of my systems have
>
> python-debian-0.1.27-3.el7.noarch and python-six-1.9.0-2.el7.noarch
>
> I'm reticent to downgrade since I don't know what else that will affect,
> and it's *working* on one of the servers without the downgrade?
>
> I saw this reddit thread:
>
>
> *https://www.reddit.com/r/spacewalk/comments/3vnquh/centos_71_spacewalk_24_no_go/*
> <https://www.reddit.com/r/spacewalk/comments/3vnquh/centos_71_spacewalk_24_no_go/>
>
> but I wont try the solution(s) in the last comment bc both files in both
> systems are currently the same, so they are obv not the issue.
>
>
> I tried the patch shown here:
>
>
> *https://www.redhat.com/archives/spacewalk-list/2015-November/msg00108.html*
> <https://www.redhat.com/archives/spacewalk-list/2015-November/msg00108.html>
>
> But it just output a whole bunch of blather - looks like something is
> being chomped though - either time out or buffer length or...?
>
> asdIncoming headers:
> content-length : 5966
> set-cookie : JSESSIONID=69C01EA51CB966ABC58B3188A956B119; Path=/rhn/;
> Secure; HttpOnly; HttpOnly
> server : Apache
> x-ua-compatible : IE=edge,chrome=1
> date : Fri, 20 May 2016 06:17:12 GMT
> content-type : text/html;charset=UTF-8
> ('body:', '\'<!DOCTYPE HTML>\\n<html>\\n <head>\\n <!-- enclosing head
> tags in layout_c.jsp -->\\n <meta http-equiv="X-UA-Compatible"
> content="IE=edge"/>\\n <!-- HTML5 shim and Respond.js IE8 support of
> HTML5 elements and media queries -->\\n <!--[if lt IE 9]>\\n
> <script type=\\\'text/javascript\\\'
> src="/javascript/html5.js"></script>\\n <script
> type=\\\'text/javascript\\\' src="/javascript/respond.js"></script>\\n
> <![endif]-->\\n \\n <meta http-equiv="content-type"
> content="text/html;charset=UTF-8"/>\\n <title>\\n Spacewalk\\n
> \\n \\n - Sign In\\n \\n \\n </title>\\n <link
> rel="shortcut icon" href="/img/favicon.ico" />\\n\\n <meta
> name="viewport" content="width=device-width, initial-scale=1.0" />\\n\\n
> \\n \\n <link rel="stylesheet" href="/css/spacewalk.css" />\\n
> \\n <link rel="stylesheet" href="/css/jquery.timepicker.css" />\\n
> <link rel="stylesheet" href="/css/bootstrap-datepicker.css" />\\n\\n
> <link rel="stylesheet" href="/javascript/select2/select2.css" />\\n
> <link rel="stylesheet" href="\'')
> ('body:', '\'/javascript/select2/select2-bootstrap.css" />\\n\\n <link
> rel="stylesheet" href="/fonts/font-awesome/css/font-awesome.css" />\\n
> <link rel="stylesheet" href="/fonts/font-spacewalk/css/spacewalk-font.css"
> />\\n\\n <script src="/javascript/jquery.js"></script>\\n <script
> src="/javascript/bootstrap.js"></script>\\n <script
> src="/javascript/select2/select2.js"></script>\\n <script
> src="/javascript/spacewalk-essentials.js"></script>\\n <script
> src="/javascript/spacewalk-checkall.js"></script>\\n\\n <script
> src="/rhn/dwr/engine.js"></script>\\n <script
> src="/rhn/dwr/util.js"></script>\\n <script
> src="/rhn/dwr/interface/DWRItemSelector.js"></script>\\n <script
> src="/javascript/jquery.timepicker.js"></script>\\n <script
> src="/javascript/bootstrap-datepicker.js"></script>\\n\\n <meta
> name="decorator" content="layout_c" />\\n <script
> src="/javascript/spacewalk-login.js"></script>\\n </head>\\n <body
> onload="">\\n <div class="spacewalk-top-wrap">\\n <nav
> class="navbar navbar-default navbar-pf" role="navigation">\\n \'')
> ('body:', '\' \\n\\n\\n\\n<!-- header.jsp -->\\n\\n\\n\\n<div
> class="navbar-header">\\n <button type="button" class="navbar-toggle"
> data-toggle="collapse" data-target=".navbar-collapse">\\n <span
> class="sr-only">Toggle navigation</span>\\n <span
> class="icon-bar"></span>\\n <span class="icon-bar"></span>\\n <span
> class="icon-bar"></span>\\n </button>\\n <a class="navbar-brand" href="
> *http://www.spacewalkproject.org/* <http://www.spacewalkproject.org/>"
> title="Visit *http://www.spacewalkproject.org/*
> <http://www.spacewalkproject.org/> for more information">\\n <img
> src="/img/logo_vendor.png" alt="Spacewalk project" id="rhLogo" />\\n
> </a>\\n <a class="navbar-brand" href="/" title="Spacewalk homepage">\\n
> <img src="/img/logo_product.png" alt="Spacewalk" id="rhnLogo"
> accesskey="2"/>\\n </a>\\n \\n</div>\\n\\n<div class="navbar-collapse
> collapse">\\n \\n\\n \\n \\n <ul class="nav navbar-nav
> navbar-primary"><li class="active"><a href="/rhn/Login.do">Sign
> In</a></li>\\n<li><a href="/rhn/help/about.do">About</a></li>\\n</ul>\\n
> \\n</div>\\n<!-- end header.jsp -->\\n\\n </nav>\\n <div
> class="spacewalk-main-column-layout">\\n \'')
> ('body:', '\' <aside id="spacewalk-aside" class="navbar-collapse
> collapse">\\n \\n\\n<nav id="sidenav">\\n \\n \\n \\n
> \\n</nav>\\n\\n \\n\\n\\n\\n\\n\\n \\n\\n\\n<!-- TODO
> make this more generic -->\\n\\n\\n\\n </aside>\\n <section
> id="spacewalk-content">\\n <!-- Alerts and messages
> -->\\n \\n \\n <div class="text-center">\\n
> \\n \\n \\n \\n <h1>Welcome to Spacewalk</h1>\\n
> <p><strong>Spacewalk</strong> is an open source (GPLv2) Linux systems
> management solution. Below are some links that should prove useful if you
> have any questions regarding your Spacewalk server. </p>\\n \\n
> \\n\\n <form name="loginForm" id="loginForm" method="post"
> action="/rhn/LoginSubmit.do;jsessionid=69C01EA51CB966ABC58B3188A956B119"
> class="form-horizontal col-md-6 col-md-offset-3 text-left">\\n <input
> type="hidden" name="csrf_token" value="7530790251516499651">\\n
> \\n\\n \\n \\n \\n \\n \\n \\n \\n\\n\\n<div
> class="form-group">\\n <label for="username-field" class="col-xs-5
> control-label">Spacewal\'')
> ('body:', '\'k Login: </label>\\n <div class="col-xs-7">\\n <input
> type="text" id="username-field" class="form-control" tabindex="1"
> name="username" maxlength="63" placeholder="Login" />\\n
> </div>\\n</div>\\n\\n<div class="form-group">\\n <label
> for="password-field" class="col-xs-5 control-label">Password: </label>\\n
> <div class="col-xs-7">\\n <input type="password" id="password-field"
> class="form-control" tabindex="2" name="password" maxlength="32"
> placeholder="Password" />\\n </div>\\n</div>\\n\\n<div
> class="text-right">\\n <input type="hidden" name="login_cb"
> value="login">\\n <input type="hidden" name="url_bounce"
> value="/rhn/errors/500.jsp">\\n <input type="hidden"
> name="request_method" value="GET">\\n <input type="submit" tabindex="3"
> value="Sign In" id="login" class="btn btn-primary">\\n</div>\\n\\n
> </form>\\n <div class="col-md-6 col-md-offset-3 text-left">\\n \\n
> \\n \\n \\n <p>For details on Spacewalk, please visit
> our web-site at <a href="*http://www.spacewalkproject.org/*
> <http://www.spacewalkproject.org/>" target="_blank">
> *http://www.spacewalkprojec* <http://www.spacewalkprojec/>\'')
>
>
>
>
> Would appreciate any ideas people might have?
>
>
>
> cheers
> L.
>
> ------
> The most dangerous phrase in the language is, "We've always done it this
> way."
>
> - Grace Hopper
>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20160525/e14ee0d5/attachment.htm>
More information about the Spacewalk-list
mailing list