[Spacewalk-list] Problems with Debian Stretch on SW 2.7. Can reinstall packages over and over again

Tue Oct 17 01:53:03 UTC 2017

On Monday, October 16, 2017 2:08 PM, Robert Paschedag [mailto:robert.paschedag at web.de] wrote:
>On 10/16/17 21:25, Paul-Andre Panon wrote:
>> On Saturday, October 14, 2017 12:02 AM, Robert Paschedag [mailto:robert.paschedag at web.de] wrote:
>>> Am 14. Oktober 2017 01:38:59 MESZ schrieb Paul-Andre Panon <paul-andre.panon at avigilon.com>:
>>> Hi Paul-Andre,
>>>
>>> the big problem is... "allowed" is not the only value for Multi-Arch. In Debian stretch "main" repo, there are 8979 Packages with "Multi-Arch" header. No way to do that by hand.
>> Agreed, but so far I have only had problems with packages where the missing Multi-Arch value is "allowed". Maybe I've just been lucky though. The annoying thing is that, even with the ~couple dozen cases of "Multi-Arch: allowed" in the Ubuntu main and universe repos (mostly from python and the gcc toolchain), every sync wipes out your last set of manual fixes.  When I ran into the problem, I found the clue to a workaround in matus' Jan 30th 2015 comment at http://www.devops-blog.net/spacewalk/registering-ubuntu-and-debian-servers-with-spacewalk, so this isn't a new issue. I'm not happy with his kludgy "fix" though since it also winds up adding that property to many packages where it doesn't apply.
>Yeah. I'm also using that workaround from the bugzilla report about the missing Multi-Arch headers and in jessie, this really works quite good.
>Now, with "stretch", this is just a nightmare. I was so happy, that now in SW2.7, the debian version string parsing has been improved and now debian has "improved" "apt". But that improvement somehow throws a big stick between your legs while you're running at max speed, causing you to crash and break with your face on the ground :-P
>
>>> Thanks
>>> Robert
>> Paul-Andre Panon
>> Senior systems administrator
>>
>> Office: 604.629.5182 ext 2341 Mobile: 604.679.1617
>
>I found the "error". It is really because of the "missing" "Release" 
>file. If this is found, and there is an "Architectures:" header, the new "apt" will download the Packages for this architecture. If the "Release" is missing, "apt" will download the Packages for the clients configured "architectures" + "all".
>
>This is a new behaviour.
Ah, that explains why I haven't been seeing the same problems then then. I've been building Release files using another of Philip's blog postings (and customized it to build InRelease files as well). 
http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk

>
>But I think, that this is not my main error. Because I'm still using Steve Meier's "debian sync" tool to download and import the debian packages, I modified that to create a "temporary" file with the "package" name and possible "multi-arch" header, which I can insert, after SW has created his own "Packages" file.
>
>But this does not yet work 100%, because there are sometimes more than one package with exact same name. I have to improve that and keep trying.
>
>Really the best thing would be, if the "Multi-Arch" header would be added to the database. If I'm right, this header is already "present", while the .deb files gets parsed but it just gets ignored. And I don't know, how much work and modification is needed to fully implement this header to every .deb package.

It appears to be in the .deb package so if that's what is being used as a metadata source (and not the Package file from the mirror site) then it would still be valid.

~$ dpkg-deb -I /var/cache/apt/archives/python2.7-minimal_2.7.12-1ubuntu0~16.04.1_amd64.deb
 new debian package, version 2.0.
 size 1295108 bytes: control archive=2355 bytes.
     826 bytes,    21 lines      control
     338 bytes,     5 lines      md5sums
    2365 bytes,    78 lines   *  postinst             #!/bin/sh
     416 bytes,    21 lines   *  postrm               #!/bin/sh
     900 bytes,    39 lines   *  preinst              #!/bin/sh
     813 bytes,    36 lines   *  prerm                #!/bin/sh
 Package: python2.7-minimal
 Source: python2.7
 Version: 2.7.12-1ubuntu0~16.04.1
 Architecture: amd64
 Maintainer: Ubuntu Core Developers <ubuntu-devel-discuss at lists.ubuntu.com>
 Installed-Size: 3584
 Pre-Depends: libc6 (>= 2.15)
 Depends: libpython2.7-minimal (= 2.7.12-1ubuntu0~16.04.1), zlib1g (>= 1:1.2.0)
 Recommends: python2.7
 Suggests: binfmt-support
 Conflicts: binfmt-support (<< 1.1.2)
 Replaces: python2.7 (<< 2.7.8-7~)
 Section: python
 Priority: standard
 Multi-Arch: allowed
 Description: Minimal subset of the Python language (version 2.7)
  This package contains the interpreter and some essential modules.  It can
  be used in the boot process for some basic tasks.
  See /usr/share/doc/python2.7-minimal/README.Debian for a list of the modules
  contained in this package.
 Original-Maintainer: Matthias Klose <doko at debian.org>

I agree that adding it to the database was what I thought would be needed to do it right as well. However it's an optional field that isn't set on all packages. I looked at the database model. Rhn_package contains a lot of package attributes (mostly numeric attributes, and some string attributes that are nearly always package specific - like description or summary), and many other package attributes are kept as associative entities between rhnpackage and rhnpackagecapabilities (i.e. conflicts, depends, provides, requires, etc). Architecture is maintained as a foreign key to the rhnpackagearch table. 

There are four Multi-Arch values currently in the Ubuntu repo Packages files I checked: allowed, foreign, no, and same. So that leads me to thinking the approach most consistent with existing package attributes is a reference table similar to rhnpackage and rhnpackagegroup. i.e.

CREATE TABLE public.rhnpackagemultiarch (
    id NUMERIC  NOT NULL,
    value CHARACTER VARYING(64)  NOT NULL,
    created TIMESTAMP DEFAULT now()  NOT NULL,
    modified TIMESTAMP DEFAULT now()  NOT NULL,
    CONSTRAINT rhn_package_multiarch_id_pk PRIMARY KEY (id),
    CONSTRAINT vn_rhnpackagemultiarch_name CHECK ((name) <> '')
);

CREATE TRIGGER public.rhn_package_multiarch_mod_trig BEFORE INSERT OR UPDATE ON rhnpackagemultiarch FOR EACH ROW EXECUTE PROCEDURE rhn_package_multiarch_mod_trig_fun();

CREATE OR REPLACE FUNCTION rhn_package_multiarch_mod_trig_fun () RETURNS trigger AS
begin
	new.modified := current_timestamp;

	return new;
end;
LANGUAGE 'plpgsql' VOLATILE;

ALTER TABLE public.rhnpackage  ADD COLUMN package_multiarch_id NUMERIC NOT NULL;

ALTER TABLE public.rhnpackage ADD CONSTRAINT rhnpackagemultiarch_rhnpackage
   FOREIGN KEY (package_multiarch)id) REFERENCES public.rhnpackagemultiarch(id);

And you could optionall pre-populate the current 4 possible values

INSERT INTO public.rhnpackagemultiarch ( id, value) VALUES 
( 1, 'allowed'),
( 2,  'foreign'),
( 3,  'no'),
( 4, 'same');

You could then use whatever approach is being used to store package group or arch to populate the new package attribute. I just have not idea where in the code that is. You could probably step through /usr/bin/spacewalk-repo-sync to find that out but I haven't had time. 

>
>Would be cool, if the main developers of SW could look into that issue and I'll really try to support them with as much information as possible to get this fixed.
Maybe the above helps get someone started?
Spacewalk isn't running on a system with dpkg-deb, so either they are doing their own parsing of the deb file (and adding one more field then could be quite tricky), or they are just parsing the Packages file on the mirror's repo directory tree, which would be a lot easier.

>As final information. This is not an error in SW. This is "an improvement" in Debian 9, which most of us (and I think most of SW
>developers) just did not get but its a major problem to "manage" debian
>9 with spacewalk.
>
>Robert
>
Yeah, I think some of that stuff was already in Ubuntu 16.04, which is why I wound up building the Release and InRelease files for our environment.

Paul-Andre Panon
Senior systems administrator