[Spacewalk-list] SUSE tokens broken since spacewalk 2.7
Michael Calmer
mc at suse.de
Mon Jan 15 11:05:41 UTC 2018
Hi
I just checked our code and remembered that there was a change in upstream and they use no the ThreadedDownloader
and this cannot handle the tokens.
Let me see if I can open a PR to add this. Check the upstream PRs.
Am Donnerstag, 11. Januar 2018, 11:27:09 CET schrieb Michael Calmer:
> Hi
>
> hmm, than it is maybe still yum and/or python-urlgrabber. I provided patches for both some years ago on there mailing lists.
> I am not sure if they were included into the packages.
>
> Am Donnerstag, 11. Januar 2018, 10:34:49 CET schrieb Sadri, Wafa (BITBW):
> > Hi there,
> >
> > Unfortunately, nothing has changed at all, after commenting out the 5 lines. This is the output of /var/log/rhn/reposync/sles-12-sp3-updates.log
> >
> >
> > 2018/01/11 10:21:46 +02:00 Command: ['/bin/spacewalk-repo-sync', '--channel=sles-12-sp3-updates']
> > 2018/01/11 10:21:46 +02:00 Sync of channel started.
> > 2018/01/11 10:21:46 +02:00 Repo URL: https://updates.suse.com/SUSE/Updates/SLE-SERVER/12-SP3/x86_64/update/?VERYLONGTOKEN
> > 2018/01/11 10:21:50 +02:00 Packages in repo: 1342
> > 2018/01/11 10:21:52 +02:00 Packages already synced: 820
> > 2018/01/11 10:21:52 +02:00 Packages to sync: 243
> > 2018/01/11 10:21:52 +02:00 New packages to download: 243
> > 2018/01/11 10:21:52 +02:00 ERROR: Download failed: https://updates.suse.com/SUSE/Updates/SLE-SERVER/12-SP3/x86_64/update/x86_64/cpp48-4.8.5-31.6.1.x86_64.rpm - [Errno 14] HTTPS Error 403 - Forbidden.
> > 2018/01/11 10:21:52 +02:00 1/243 : cpp48-4.8.5-31.6.1.x86_64.rpm (failed)
> >
> > [...]
> >
> > 2018/01/11 10:21:55 +02:00 ERROR: Download failed: https://updates.suse.com/SUSE/Updates/SLE-SERVER/12-SP3/x86_64/update/noarch/zypper-log-1.13.39-21.13.7.noarch.rpm - [Errno 14] HTTPS Error 403 - Forbidden.
> > 2018/01/11 10:21:55 +02:00 243/243 : zypper-log-1.13.39-21.13.7.noarch.rpm (failed)
> > 2018/01/11 10:21:55 +02:00 Importing packages started.
> > 2018/01/11 10:21:55 +02:00 Importing packages finished.
> > 2018/01/11 10:21:55 +02:00 Repo https://updates.suse.com/SUSE/Updates/SLE-SERVER/12-SP3/x86_64/update/?VERYLONGTOKEN has 229 errata.
> > 2018/01/11 10:21:56 +02:00 No new errata to sync.
> > 2018/01/11 10:21:57 +02:00 Sync of channel completed in 0:00:10
> >
> >
> > Before commenting out the lines, the output was exactly the same. I'm not sure wether the log doesn't show, if the token is attached to the package url or i fit really isn't used by spacewalk and the shown url is accurate.
> >
> > Regards,
> > Wafa
> >
> >
> > -----Ursprüngliche Nachricht-----
> > Von: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-bounces at redhat.com] Im Auftrag von Michael Calmer
> > Gesendet: Donnerstag, 11. Januar 2018 10:14
> > An: spacewalk-list at redhat.com
> > Betreff: Re: [Spacewalk-list] SUSE tokens broken since spacewalk 2.7
> >
> > Hi
> >
> > Am Donnerstag, 11. Januar 2018, 09:47:45 CET schrieb Sadri, Wafa (BITBW):
> > > Hi there,
> > >
> > > in my version of the file, the line looks a little different:
> > >
> > > # if self.url is metalink it will be expanded into
> > > # real urls in self.repo.urls and also save this metalink
> > > # in begin of the url list ("for repolist -v ... or anything else wants to know the baseurl")
> > > # Remove it from the list, we don't need it to download content of repo
> > > real_urls = []
> > > for url in self.repo.urls:
> > > if '?' not in url:
> > > real_urls.append(url)
> > > self.repo.urls = real_urls
> > >
> > > Am I commenting out the whole thing, or just the for-loop?
> >
> > All 5 lines or at least "self.repo.urls = real_urls".
> >
> > > I might be in the wrong function alltogether, because this part of the code appears in the function "def __init__(self, url, name, yumsrc_conf=YUMSRC_CONF, org="1", channel_label="", no_mirrors=False, ca_cert_file=None, client_cert_file=None, client_key_file=None):"
> > >
> > > The exact line you mentioned doesn't appear in my yum_src.py (which is located exactly where you said it would be).
> >
> > I think this is the right place. I checked master branch which had already changed.
> >
> > > Regards,
> > > Wafa
> > >
> > > -----Ursprüngliche Nachricht-----
> > > Von: spacewalk-list-bounces at redhat.com
> > > [mailto:spacewalk-list-bounces at redhat.com] Im Auftrag von Michael
> > > Calmer
> > > Gesendet: Mittwoch, 10. Januar 2018 16:54
> > > An: spacewalk-list at redhat.com
> > > Betreff: Re: [Spacewalk-list] SUSE tokens broken since spacewalk 2.7
> > >
> > > Hi
> > >
> > > Am Mittwoch, 10. Januar 2018, 16:18:07 CET schrieb Sadri, Wafa (BITBW):
> > > > Hello everyone,
> > > >
> > > > We're having issues syncing SUSE repositories since we upgraded to 2.7 in mid-december.
> > > > We're using these type of URLs in our repositories:
> > > > https://updates.suse.com/SUSE/Products/SLE-Module-Web-Scripting/12/x
> > > > 86_64/product&NOWAVERYLONGTOKENOFYOURREGISTEREDSYSTEM
> > > >
> > > > Spacewalk is able to receive the repomd.xml file and starts to download the packages. But it fails with every single download. Looking at the log files, I can see that it does not request the resources using the token anymore. It will only request the repomd.xml file with the token. Therefor we're getting 403 errors for the actual package downloads. We've refreshed all the URLs, used different tokens... all with the same result. I've tried the SUSE support but they don't understand the problem and were not helpful at all!
> > > >
> > > > Any idea if something has changed in the way spacewalk syncs repositories? Is the python-urlgrabber no longer used?
> > >
> > > Guessing a bit. Maybe you want to try it out. I cannot tell you the exact path to the file in your installation But I hope you can find the right one:
> > >
> > > /usr/lib/python2.7/site-packages/spacewalk/satellite_tools/repo_plugin
> > > s/yum_src.py At the end of the function "def setup_repo(self, repo,
> > > no_mirrors, ca_cert_file, client_cert_file, client_key_file):"
> > >
> > > # if self.url is metalink it will be expanded into
> > > # real urls in repo.urls and also save this metalink
> > > # in begin of the url list ("for repolist -v ... or anything else wants to know the baseurl")
> > > # Remove it from the list, we don't need it to download content of repo
> > > repo.urls = [url for url in repo.urls if '?' not in url]
> > >
> > > Just try to comment out this last line. It seems at it is excluding all URLs with a '?' but the token is seperated by it as well.
> > > Not sure if it help but maybe you can try out and report back.
> > >
> > >
> >
> >
> > --
> > Regards
> >
> > Michael Calmer
> >
> > --------------------------------------------------------------------------
> > Michael Calmer
> > SUSE LINUX GmbH, Maxfeldstr. 5, D-90409 Nuernberg
> > T: +49 (0) 911 74053 0
> > F: +49 (0) 911 74053575 - e-mail: Michael.Calmer at suse.com
> > --------------------------------------------------------------------------
> > SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> > HRB 21284 (AG Nürnberg)
> >
> >
> > _______________________________________________
> > Spacewalk-list mailing list
> > Spacewalk-list at redhat.com
> > https://www.redhat.com/mailman/listinfo/spacewalk-list
> >
> > _______________________________________________
> > Spacewalk-list mailing list
> > Spacewalk-list at redhat.com
> > https://www.redhat.com/mailman/listinfo/spacewalk-list
>
>
>
--
Regards
Michael Calmer
--------------------------------------------------------------------------
Michael Calmer
SUSE LINUX GmbH, Maxfeldstr. 5, D-90409 Nuernberg
T: +49 (0) 911 74053 0
F: +49 (0) 911 74053575 - e-mail: Michael.Calmer at suse.com
--------------------------------------------------------------------------
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
More information about the Spacewalk-list
mailing list