[Spacewalk-list] Red Hat updates: migrating from mrepo to reposync wrapper

[Brian Long]

As many of you are probably aware, Red Hat recently had CDN issues
lasting almost two weeks.  During that time, my Spacewalk server (and
many Satellite servers) were unable to download RHEL updates.  For
years I have used mrepo that, under the hood, uses lftp with
certificate-based authentication to mirror the repos from
cdn.redhat.com.

During the CDN hiccup, mrepo deleted over 15,000 RPMs from my
Spacewalk server.  After Red Hat fixed their CDN issues, mrepo was
unable to re-download RPMs it had previously deleted.  I was unable to
coerce it into syncing the entire repository again, but I have
recently found a replacement for mrepo that re-downloaded the missing
RPMs and I wanted to share it with you.

Someone wrote a wrapper around reposync that generates temporary
yum.conf files per Red Hat repo and allows you to sync each RHEL
channel to which you have access.  The wrapper is located here:
https://github.com/pyther/upstream_sync

In order to preserve all the structure and RPMs mrepo/lftp had already
downloaded, I edited upstream_sync and changed mirror_dir to
/var/mrepo.  I also used configuration file statements that match the
mrepo default directory names.  For example, to mirror rhel 7 updates,
I have the following in /etc/upstream_sync/rhel7.conf:

[rhel-7-x86_64-os]
auth = rhel-server
url = https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os
path = 7Server-x86_64/updates
createrepo = true

The auth.conf contains the following:
[rhel-server]
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslcert = /usr/share/keys/mycert.pem
sslkey = /usr/share/keys/mycert-key.pem

The certificates are the same ones I used for mrepo (lftp).  If you
are unsure what certificates to use, login to your RHN account, find a
system with the appropriate subscription attached, select the
Subscriptions tab and then Download Certificates.  Inside the zip file
will be another zip named consumer_export.zip.  Extract the only
certificate from that archive and copy it into a known directory; name
it something like mycert.pem.  Make another copy of the certificate
and name it mycert-key.pem.  Edit mycert-key.pem and remove everything
before the section -----BEGIN RSA PRIVATE KEY----- (the last section
of the file).  You can then reference mycert.pem and mycert-key.pem in
the upstream_sync auth.conf file.

The last piece I changed was Spacewalk's notion of where to grab the
RHEL updates.  mrepo generates repo metadata in /var/www/mrepo with
all the RPMs being symlinks back to the main /var/mrepo directories.
I wanted to bypass that, so I used spacecmd to edit each of my repos
and point straight to the /var/mrepo repos that were created by
reposync.

spacecmd repo_updateurl 'External - RHEL 6 x86_64 Updates'
file:///var/mrepo/6Server-x86_64/updates/

To find all repos' URL info, you can run something like the following:
IFS=$'\n'; for repo in `spacecmd repo_list`; do spacecmd repo_details
"$repo" 2>/dev/null | egrep 'Label:|URL:'; done

Or just run "spacewalk-repo-sync -l | grep mrepo"

I hope this is helpful.

/Brian/