[Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again

Tue Oct 24 08:44:43 UTC 2017

-----Ursprüngliche Nachricht-----
Von: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-bounces at redhat.com] Im Auftrag von Paul-Andre Panon
Gesendet: Mittwoch, 18. Oktober 2017 05:38
An: spacewalk-list at redhat.com; Robert Paschedag <robert.paschedag at web.de>
Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again

On  Tue, 17 Oct 2017 5:36:50 ,  Robert Paschedag <paschedag.netlution at swr.de> wrote:
>
>Ok.
>
>Looks like I got it working now.
>
>After creating a very, very ugly script to add the missing headers to the Packages file, I discovered the deb822 module, already installed on the spacewalk-server (python-debian), which can easily parse a Packages file.
>
>With the information extracted with Steven Meiers "spacewalk-debian-sync" script (while syncing the Debian repo), modifying Packages was pretty easy.
>
>See a modified version of Steve Meiers script and the new script to add the missing "Multi-Arch" header here https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frpasche%2Fspacewalk-debian-sync%2Ftree%2Fdevel&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=Ii%2BQ3AZE%2FpM0jxefRMS4p6s%2Bn7CFkUrBloramIPkjRw%3D&reserved=0
>
>At least, this works pretty good on my test system, and after "apt-get upgrade", another "apt-get upgrade" shows "no packages to upgrade" ?
>
>Robert

I think going back to using a modified version of Steve Meier's script would be a step back that I would prefer to avoid. I would prefer to update the spacewalk-repo-sync command.

In addition to the rhn_deb.py file you indicated, there also appears to be a repo-specific plugin, deb_src.py, that appears to be pulling down the Packages file from the repo and determining what packages (and versions)  and corresponding URLs are available, https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspacewalkproject%2Fspacewalk%2Fblob%2F45553919f852c4dbc5fcfe1c24499b2f9e9f544b%2Fbackend%2Fsatellite_tools%2Frepo_plugins%2Fdeb_src.py&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=UR26BszhI8uyGO3zYrLV4bVKgT4%2FmlV7r5LoPQI5seM%3D&reserved=0
One interesting thing is that that file has an empty get_updates() method which may be for Errata. If you were going to add a hook to incorporate Ubuntu errata (from https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ubuntu.com%2Farchives%2Fubuntu-security-announce%2F&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=K5vt9T%2FhobkSX1GeFyVJUQGLymRTz7tOcM9ei4HB%2FNg%3D&reserved=0) and/or Debian errata, that may be where it would have to go but you would have to prepare a data/class structure similar to what the yum_src plugin uses (from yum.update_md import UpdateMetadata, UpdateNoticeException, UpdateNotice). You would want new classes to parse the external Errata sources (sub-classed for the distro, but you would need a new repo metadata field to determine the distro) and then would cross-correlate the results for what applies to that repo. Seems messy and an externally scheduled process may be somewhat cleaner.

Anyways, I think you're correct that rhn_deb is used to parse the deb packages. Instead of subclassing per repo, rhn_pkg fronts rhn_deb and the other package header parsers.
What populates the db tables (to add a multi-arch table row)? The sync() method in  spacewalk/backend/satellite_tools/reposync.py  probably invokes spacewalk/backend/satellite_tools/repo_plugins/__init__.py via the repo plugins (and load_checksum_from_header() calls the deb header parser), and the plugin __init__.py also has an upload_package() method which calls *push_package()* in  spacewalk/backend/server/rhnPackageUpload.py to populate the db with the parsed header values.

What I haven't yet figured out is 
a) what generates the Spacewalk Packages file from the DB tables or header data structures.

It's

https://github.com/spacewalkproject/spacewalk/blob/master/java/code/src/com/redhat/rhn/taskomatic/task/repomd/DebPackageWriter.java and
https://github.com/spacewalkproject/spacewalk/blob/master/java/code/src/com/redhat/rhn/taskomatic/task/repomd/DebRepositoryWriter.java

This might also be a good place to put creation of "Release" files in future?

Packages use the PackageDto class

https://github.com/spacewalkproject/spacewalk/blob/master/java/code/src/com/redhat/rhn/frontend/dto/PackageDto.java

which should then include some sort of 

private String multiArch;

with get/set methods

b) The Package class in importLib.py and packageImport.py in https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspacewalkproject%2Fspacewalk%2Ftree%2F45553919f852c4dbc5fcfe1c24499b2f9e9f544b%2Fbackend%2Fserver%2Fimportlib&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=N47qPNfWJQV7nmY7ue%2FUDKfgSL4gizTj5%2BrZVabq91g%3D&reserved=0 may be involved but I'm still figuring them out. I've seen one reference to rhn_pkg in packageImport.py, but it's in _import_signatures so they might not need to change.

So far, it looks like you would need to 
a) add the table with the DDL I gave earlier
b) change rhn_deb to include parsing the Multi-Arch header value
c) change push_package() in rhnPackageUpload.py to populate the new table
d) change get_info_for_package in spacewalk/backend/server/rhnPackage.py to download the multi-arch value from the new table.
e) change ? to add Multi-Arch to the Package file creation
f) it's non-profit

Cheers,

Paul-Andre Panon
Senior systems administrator

_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fspacewalk-list&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C97788f18a5b342344d0c08d515da0173%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438948434825640&sdata=Y3ZHSIs18RuxBfhUp2QI88tkpQIgxEgo0%2FX%2FjMzZIaI%3D&reserved=0