[Spacewalk-list] spacewalk 2.9 && never ending issue with ubuntu/debian clients

[Robert Paschedag]

On 2/22/19 10:30 PM, Robert Paschedag wrote:
> On 2/22/19 7:53 PM, Robert Paschedag wrote:
>> Am 22. Februar 2019 13:11:00 MEZ schrieb Florin Portase <portase.florin at medianetork.ro>:
>>> On 2019-02-13 10:56, Florin Portase wrote:
>>>
>>>> Hello,
>>>>
>>>> I just have upgraded the spacewalk server from 2.7 => 2.9.
>>>>
>>>> I have applied also the sql patch from
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1661347
>>>>
>>>> + upgraded the clients from :
>>>>
>>>>
>>> http://download.opensuse.org/repositories/systemsmanagement:/spacewalk:/2.9:/debclients/
>>>
>>>>
>>>> Just to mention spacewalk 2.7 +  patches was working just fine for
>>> both debian & ubuntu.
>>>>
>>>> Now, for ubuntu 16.05 I have over 100 packages marked as up-gradable(
>>> over and over....)
>>>>
>>>> _______________________________________________
>>>>
>>>> Spacewalk-list mailing list
>>>> Spacewalk-list at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>
>>> So, after digging through  SPW archive Dec '18 til Feb '19 finally I
>>> come to something more acceptable:
>>>
>>> 1. sync script for Ubuntu channels
>>>
>>> 2. "spacewalk-add-debian-multiarch-header.py.NEW " took it from
>>> "https://www.redhat.com/archives/spacewalk-list/2018-December/msg00017.html"
>>>
>>>
>>> wget  -q
>>> http://de.archive.ubuntu.com/ubuntu/dists/xenial/main/binary-amd64/Packages.gz
>>> \
>>>        -O /tmp/Packages-xenial-main.gz && gunzip -f
>>> /tmp/Packages-xenial-main.gz
>>> wget  -q
>>> http://cz.archive.ubuntu.com/ubuntu/dists/xenial-updates/main/binary-amd64/Packages.gz
>>> \
>>>        -O /tmp/Packages-xenial-updates.gz  && gunzip -f
>>> /tmp/Packages-xenial-updates.gz
>>> wget  -q
>>> http://cz.archive.ubuntu.com/ubuntu/dists/xenial-security/main/binary-amd64/Packages.gz
>>> \
>>>        -O /tmp/Packages-xenial-security.gz && gunzip -f
>>> /tmp/Packages-xenial-security.gz
>>>
>>> s=180
>>> trap 'echo "Ctrl-C detected."' 2
>>> for (( i=$s ; i>0; i--));
>>>        do
>>>                #printf '\rFinishing sync in: %2d seconds' $i; sleep 1
>>>                echo -ne  "\rFinishing sync in: $i seconds\033[0K";
>>> sleep 1
>>>        done
>>> echo -e "\nSync completed!"
>>> trap 2
>>>               $_BIN_PATH/spacewalk-add-debian-multiarch-header.py.NEW
>>> $_PKG_MAIN/Packages    /tmp/Packages-xenial-main
>>>               $_BIN_PATH/spacewalk-add-debian-multiarch-header.py.NEW
>>> $_PKG_UPD/Packages      /tmp/Packages-xenial-updates
>>>               $_BIN_PATH/spacewalk-add-debian-multiarch-header.py.NEW
>>> $_PKG_SEC/Packages      /tmp/Packages-xenial-security
>>>               $_BIN_PATH/spacewalk-add-debian-multiarch-header.py.NEW
>>> $_PKG_UNIV/Packages    /tmp/Packages-xenial-universe
>>>
>
> Below is your error
>
> Packages.new is the "modified" Packages which you rename and
> *afterwards* use its "modified" timestamp. This is wrong.
>
> You have to use the "modified" timestamp of the "original" (the one
> generated by Spacewalk) packages file
>
> So when you have the original "Packages" file (by spacewalk), do
>
> - run the <add_header> script (which generates "Packages.new")
> - gzip -c Packages.new > Packages.gz
> - touch -r Packages Packages.new Packages.gz && mv Packages.new Packages
>
> So you have transferred the original timestamp to the new files and all
> set to "secure" your repo then.
>
> Robert
>

Oops....I was wrong. The modified time of "Packages.gz" is the one you
need to preserve

See this function, which checks, if the files has to be regenerated.

...
    public boolean isChannelRepodataStale(Channel channel) {
        File theFile = new File(mountPoint + File.separator + pathPrefix +
                File.separator + channel.getLabel() + File.separator +
                "Packages.gz");
        // Init Date objects without milliseconds
        Calendar cal = Calendar.getInstance();
        cal.setTime(new Date(theFile.lastModified()));
        cal.set(Calendar.MILLISECOND, 0);
        Date fileModifiedDate = cal.getTime();
        cal.setTime(channel.getLastModified());
        cal.set(Calendar.MILLISECOND, 0);
        Date channelModifiedDate = cal.getTime();

        // the file Modified date should be getting set when the file
        // is moved into the correct location.
        log.info("File Modified Date:" + LocalizationService.getInstance().
                formatCustomDate(fileModifiedDate));
        log.info("Channel Modified Date:" +
LocalizationService.getInstance().
                formatCustomDate(channelModifiedDate));
        return !fileModifiedDate.equals(channelModifiedDate);
    }

...

Robert

>>>                /bin/mv $_PKG_MAIN/Packages.new $_PKG_MAIN/Packages
>>>                /bin/mv $_PKG_SEC/Packages.new $_PKG_SEC/Packages
>>>                /bin/mv $_PKG_UPD/Packages.new $_PKG_UPD/Packages
>>>                /bin/mv $_PKG_UNIV/Packages.new $_PKG_UNIV/Packages
>>>
>>>                gzip < $_PKG_MAIN/Packages > $_PKG_MAIN/Packages.gz
>>>                gzip < $_PKG_UPD/Packages  > $_PKG_UPD/Packages.gz
>>>                gzip < $_PKG_SEC/Packages  > $_PKG_SEC/Packages.gz
>>>                gzip < $_PKG_UNIV/Packages > $_PKG_UNIV/Packages.gz
>>>
>>> cd $_PKG_MAIN
>>>        $_BIN_PATH/secureApt.sh  xenial xenial-main
>>>        touch -r Packages.gz  Packages
>>> cd $_PKG_UPD
>>>        $_BIN_PATH/secureApt.sh  xenial xenial-updates
>>>        touch -r Packages.gz  Packages
>>> cd $_PKG_SEC
>>>        $_BIN_PATH/secureApt.sh  xenial xenial-security
>>>        touch -r Packages.gz  Packages
>>>
>>> So just to resume, SYNC =>OK, applying ALL missing headers =>OK, now
>>> the
>>> packages that are showed as up-gradable dropped from ~120 to only 6 (
>>> base-files libbind9-140 libisc160 libisccc140 libisccfg140 liblwres141
>>> )
>>>
>>>
>>> ~~BUT~~
>>>
>>> Here I run into another problem, it seems taskomatic is generating
>>> Package files over and over ( touch -r Packages.gz  Packages seems to
>>> have no effect)
>>
>> This seems to be new. You might have to check within code, at which conditions a rebuild of the Packages file gets triggered
>>
>> I'm still on SW2.7 so I cannot test on my environment.
>>
>> Robert
>>
>>
>>
>
>
>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>