[Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
Paschedag, Robert
paschedag.netlution at swr.de
Tue Oct 17 13:36:50 UTC 2017
Ok.
Looks like I got it working now.
After creating a very, very ugly script to add the missing headers to the Packages file, I discovered the deb822 module,
already installed on the spacewalk-server (python-debian), which can easily parse a Packages file.
With the information extracted with Steven Meiers "spacewalk-debian-sync" script (while syncing the Debian repo),
modifying Packages was pretty easy.
See a modified version of Steve Meiers script and the new script to add the missing "Multi-Arch" header here https://github.com/rpasche/spacewalk-debian-sync/tree/devel
At least, this works pretty good on my test system, and after "apt-get upgrade", another "apt-get upgrade" shows "no packages to upgrade" 😊
root at stretch:~# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
root at stretch:~#
Robert
-----Ursprüngliche Nachricht-----
Von: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-bounces at redhat.com] Im Auftrag von Paschedag, Robert
Gesendet: Dienstag, 17. Oktober 2017 10:02
An: spacewalk-list at redhat.com; Robert Paschedag <robert.paschedag at web.de>
Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
-----Ursprüngliche Nachricht-----
Von: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-bounces at redhat.com] Im Auftrag von Paul-Andre Panon
Gesendet: Dienstag, 17. Oktober 2017 03:53
An: Robert Paschedag <robert.paschedag at web.de>; spacewalk-list at redhat.com
Betreff: Re: [Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again
On Monday, October 16, 2017 2:08 PM, Robert Paschedag [mailto:robert.paschedag at web.de] wrote:
>On 10/16/17 21:25, Paul-Andre Panon wrote:
>> On Saturday, October 14, 2017 12:02 AM, Robert Paschedag [mailto:robert.paschedag at web.de] wrote:
>>> Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon <paul-andre.panon at avigilon.com>:
>>> Hi Paul-Andre,
>>>
>>> the big problem is... "allowed" is not the only value for Multi-Arch. In Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" header. No way to do that by hand.
>> Agreed, but so far I have only had problems with packages where the missing Multi-Arch value is "allowed". Maybe I've just been lucky though. The annoying thing is that, even with the ~couple dozen cases of "Multi-Arch: allowed" in the Ubuntu main and universe repos (mostly from python and the gcc toolchain), every sync wipes out your last set of manual fixes. When I ran into the problem, I found the clue to a workaround in matus' Jan 30th 2015 comment at https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.devops-blog.net%2Fspacewalk%2Fregistering-ubuntu-and-debian-servers-with-spacewalk&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C9ed97f18f9e14fe781fa08d51501f142%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438020499979024&sdata=%2Bjf%2FUgIwxmI40%2FHPcdHSgqij6J4h98inzkmH35xLSuU%3D&reserved=0, so this isn't a new issue. I'm not happy with his kludgy "fix" though since it also winds up adding that property to many packages where it doesn't apply.
>Yeah. I'm also using that workaround from the bugzilla report about the missing Multi-Arch headers and in jessie, this really works quite good.
>Now, with "stretch", this is just a nightmare. I was so happy, that now
>in SW2.7, the debian version string parsing has been improved and now
>debian has "improved" "apt". But that improvement somehow throws a big
>stick between your legs while you're running at max speed, causing you
>to crash and break with your face on the ground :-P
>
>>> Thanks
>>> Robert
>> Paul-Andre Panon
>> Senior systems administrator
>>
>> Office: 604.629.5182 ext 2341 Mobile: 604.679.1617
>
>I found the "error". It is really because of the "missing" "Release"
>file. If this is found, and there is an "Architectures:" header, the new "apt" will download the Packages for this architecture. If the "Release" is missing, "apt" will download the Packages for the clients configured "architectures" + "all".
>
>This is a new behaviour.
Ah, that explains why I haven't been seeing the same problems then then. I've been building Release files using another of Philip's blog postings (and customized it to build InRelease files as well).
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.devops-blog.net%2Fspacewalk%2Fgpg-signing-apt-repository-in-spacewalk&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C9ed97f18f9e14fe781fa08d51501f142%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438020499979024&sdata=POm2n8ppgRctqjdYyw5kzWLrcz4CDIYHarcla2Mhmjg%3D&reserved=0
>
>But I think, that this is not my main error. Because I'm still using Steve Meier's "debian sync" tool to download and import the debian packages, I modified that to create a "temporary" file with the "package" name and possible "multi-arch" header, which I can insert, after SW has created his own "Packages" file.
>
>But this does not yet work 100%, because there are sometimes more than one package with exact same name. I have to improve that and keep trying.
>
>Really the best thing would be, if the "Multi-Arch" header would be added to the database. If I'm right, this header is already "present", while the .deb files gets parsed but it just gets ignored. And I don't know, how much work and modification is needed to fully implement this header to every .deb package.
It appears to be in the .deb package so if that's what is being used as a metadata source (and not the Package file from the mirror site) then it would still be valid.
~$ dpkg-deb -I /var/cache/apt/archives/python2.7-minimal_2.7.12-1ubuntu0~16.04.1_amd64.deb
new debian package, version 2.0.
size 1295108 bytes: control archive=2355 bytes.
826 bytes, 21 lines control
338 bytes, 5 lines md5sums
2365 bytes, 78 lines * postinst #!/bin/sh
416 bytes, 21 lines * postrm #!/bin/sh
900 bytes, 39 lines * preinst #!/bin/sh
813 bytes, 36 lines * prerm #!/bin/sh
Package: python2.7-minimal
Source: python2.7
Version: 2.7.12-1ubuntu0~16.04.1
Architecture: amd64
Maintainer: Ubuntu Core Developers <ubuntu-devel-discuss at lists.ubuntu.com>
Installed-Size: 3584
Pre-Depends: libc6 (>= 2.15)
Depends: libpython2.7-minimal (= 2.7.12-1ubuntu0~16.04.1), zlib1g (>= 1:1.2.0)
Recommends: python2.7
Suggests: binfmt-support
Conflicts: binfmt-support (<< 1.1.2)
Replaces: python2.7 (<< 2.7.8-7~)
Section: python
Priority: standard
Multi-Arch: allowed
Description: Minimal subset of the Python language (version 2.7)
This package contains the interpreter and some essential modules. It can
be used in the boot process for some basic tasks.
See /usr/share/doc/python2.7-minimal/README.Debian for a list of the modules
contained in this package.
Original-Maintainer: Matthias Klose <doko at debian.org>
I think it is here
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspacewalkproject%2Fspacewalk%2Fblob%2F45553919f852c4dbc5fcfe1c24499b2f9e9f544b%2Fbackend%2Fcommon%2Frhn_deb.py%23L76%3AL78&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C4845c788b4d6450d4a5208d5153587fd%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438242033156368&sdata=Um5bKsWhamt7QgppstORXFJthXRUigbEqkIcgv%2Bo9bk%3D&reserved=0
>From my understanding, "self.hdr" should contain "all" headers available within the control file.
I agree that adding it to the database was what I thought would be needed to do it right as well. However it's an optional field that isn't set on all packages. I looked at the database model. Rhn_package contains a lot of package attributes (mostly numeric attributes, and some string attributes that are nearly always package specific - like description or summary), and many other package attributes are kept as associative entities between rhnpackage and rhnpackagecapabilities (i.e. conflicts, depends, provides, requires, etc). Architecture is maintained as a foreign key to the rhnpackagearch table.
There are four Multi-Arch values currently in the Ubuntu repo Packages files I checked: allowed, foreign, no, and same. So that leads me to thinking the approach most consistent with existing package attributes is a reference table similar to rhnpackage and rhnpackagegroup. i.e.
CREATE TABLE public.rhnpackagemultiarch (
id NUMERIC NOT NULL,
value CHARACTER VARYING(64) NOT NULL,
created TIMESTAMP DEFAULT now() NOT NULL,
modified TIMESTAMP DEFAULT now() NOT NULL,
CONSTRAINT rhn_package_multiarch_id_pk PRIMARY KEY (id),
CONSTRAINT vn_rhnpackagemultiarch_name CHECK ((name) <> '') );
CREATE TRIGGER public.rhn_package_multiarch_mod_trig BEFORE INSERT OR UPDATE ON rhnpackagemultiarch FOR EACH ROW EXECUTE PROCEDURE rhn_package_multiarch_mod_trig_fun();
CREATE OR REPLACE FUNCTION rhn_package_multiarch_mod_trig_fun () RETURNS trigger AS begin
new.modified := current_timestamp;
return new;
end;
LANGUAGE 'plpgsql' VOLATILE;
ALTER TABLE public.rhnpackage ADD COLUMN package_multiarch_id NUMERIC NOT NULL;
ALTER TABLE public.rhnpackage ADD CONSTRAINT rhnpackagemultiarch_rhnpackage
FOREIGN KEY (package_multiarch)id) REFERENCES public.rhnpackagemultiarch(id);
And you could optionall pre-populate the current 4 possible values
INSERT INTO public.rhnpackagemultiarch ( id, value) VALUES ( 1, 'allowed'), ( 2, 'foreign'), ( 3, 'no'), ( 4, 'same');
You could then use whatever approach is being used to store package group or arch to populate the new package attribute. I just have not idea where in the code that is. You could probably step through /usr/bin/spacewalk-repo-sync to find that out but I haven't had time.
>
>Would be cool, if the main developers of SW could look into that issue and I'll really try to support them with as much information as possible to get this fixed.
Maybe the above helps get someone started?
Spacewalk isn't running on a system with dpkg-deb, so either they are doing their own parsing of the deb file (and adding one more field then could be quite tricky), or they are just parsing the Packages file on the mirror's repo directory tree, which would be a lot easier.
>As final information. This is not an error in SW. This is "an
>improvement" in Debian 9, which most of us (and I think most of SW
>developers) just did not get but its a major problem to "manage" debian
>9 with spacewalk.
>
>Robert
>
Yeah, I think some of that stuff was already in Ubuntu 16.04, which is why I wound up building the Release and InRelease files for our environment.
Paul-Andre Panon
Senior systems administrator
_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fspacewalk-list&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C9ed97f18f9e14fe781fa08d51501f142%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438020499979024&sdata=Cgz3%2FXZUN1vDLfblHbYDuJpYjm5iAlB5sQZUco8eWJ4%3D&reserved=0
_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fspacewalk-list&data=02%7C01%7CPaschedag.Netlution%40swr.de%7C4845c788b4d6450d4a5208d5153587fd%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636438242033156368&sdata=3%2FvAeqg9eipXTysjL1DrhXmUqmtN3xwjSLQ%2B3HkOGw8%3D&reserved=0
More information about the Spacewalk-list
mailing list