[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Spacewalk-list] Support for LZMA-compressed comps.xml



Thanks Stephen. That is exactly what I had done on my end. Sorry I
couldn't send the patch directly, but you made the identical changes.

Brian

On 02/13/2015 01:10 PM, Stephen Herr wrote:
> Thanks guys, hopefully this looks reasonable:
> https://github.com/spacewalkproject/spacewalk/commit/76883065ff72449726adc82d081ae681f2a5bb59
>
>
> RHEL 6 does not have pyliblzma installed by default but it's available
> in EPEL so I had to add a requirement to the spec file. RHEL 5 doesn't
> have it available but it should be fine since we shouldn't be
> releasing Spacewalk 2.3 for RHEL 5 anyway.
>
> -Stephen
>
> On 02/03/2015 01:04 PM, Michael Guidero wrote:
>> I have Spacewalk 2.2 and this does occur with it as well.  I have the
>> issue with the Scientific Linux repository.
>>
>> http://ftp.scientificlinux.org/linux/scientific/7x/x86_64/os/repodata/
>>
>> Spacewalk delivers a compressed comps.xml file to my client systems with
>> no .xz extension.
>>
>> [root swtest01 sl-7-x86_64-7x]# pwd
>> /var/cache/yum/x86_64/7/sl-7-x86_64-7x
>> [root swtest01 sl-7-x86_64-7x]# file comps.xml
>> comps.xml: XZ compressed data
>>
>> Yum complains:
>>
>> [root swtest01 sl-7-x86_64-7x]# yum group list
>> Loaded plugins: rhnplugin, versionlock
>> This system is receiving updates from RHN Classic or Red Hat Satellite.
>> There is no installed groups file.
>> Maybe run: yum groups mark convert (see man yum)
>> Failed to add groups file for repository: sl-7-x86_64-7x - comps file is
>> empty/damaged
>> Available environment groups:
>> ...
>>
>>
>>
>>
>>
>> On Fri, Jan 30, 2015 at 8:21 AM, Brian Buesker
>> <bbuesker qti qualcomm com <mailto:bbuesker qti qualcomm com>> wrote:
>>
>>     Since I don't have Spacewalk 2.2 or one of the nightly versions
>>     installed here, I can't say for sure that the issue does not
>> exist in
>>     them. Note though that I was able to sync the repository
>> successfully
>>     even before I made the change. The problem is that spacewalk is
>> still
>>     storing the compressed comps.xml but serves it to yum clients as
>> if it
>>     was not compressed.
>>
>>     Can you confirm that the comps.xml as stored in the
>> satellite/rhn/comps
>>     directory for the channel is the uncompressed XML file? I found
>> that it
>>     would always be the LZMA-compressed version and be named
>>    
>> 503597f15d6c693a96558f46fbd8ebb5853b92ef528de2a7d55a4dd021162873-comps-f21.xml.xz.
>>     However, the repomd.xml file indicates that the comps file is the
>>     uncompressed form. As best I can tell, spacewalk is serving the
>>     compressed comps.xml as if it were uncompressed (as far as yum is
>>     concerned). This causes yum to expect it to be an uncompressed XML
>>     document and it thus generates an error indicating it is
>> malformed. You
>>     would only notice this if you actually subscribed a system to the
>>     channel and tried to do some sort of yum operation. From the
>> perspective
>>     of Spacewalk itself, everything seems to work fine.
>>
>>     The fix I found was to make sure spacewalk stores the
>> uncompressed one,
>>     since that is what it seems to indicate is present in repomd.xml.
>> This
>>     appears to be consistent with previous distros where the
>> comps.xml is
>>     gzip compressed but is stored/served by spacewalk in uncompressed
>> form.
>>
>>     Brian
>>
>>     On 01/30/2015 05:20 AM, Tomas Lestach wrote:
>>      > I was able to sync fc21 repo on Spacewalk nightly together with
>>     the comps file.
>>      > Feel free to check [1] for more information.
>>      >
>>      > Even if  Brian installed pyliblzma, it's possible there were more
>>     changes done
>>      > since Spacewalk 2.1 that affect the issue.
>>      >
>>      > Regards,
>>      > --
>>      > Tomas Lestach
>>      > Red Hat Satellite Engineering, Red Hat
>>      >
>>      > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1144008
>>      >
>>      >
>>      > ----- Original Message -----
>>      >> From: "Stephen Herr" <sherr redhat com
>> <mailto:sherr redhat com>>
>>      >> To: spacewalk-list redhat com <mailto:spacewalk-list redhat com>
>>      >> Sent: Thursday, January 29, 2015 4:07:08 PM
>>      >> Subject: Re: [Spacewalk-list] Support for LZMA-compressed
>> comps.xml
>>      >>
>>      >> Here you go Tomas:
>>      >>
>>      >>
>>    
>> https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Everything/x86_64/os/repodata/
>>      >>
>>      >> Take a look at *-comps-f21.xml.xz
>>      >>
>>      >> -Stephen
>>      >>
>>      >> On 01/29/2015 06:23 AM, Tomas Lestach wrote:
>>      >>> Would you share a public repo with lzma compressed comps?
>>      >>>
>>      >>> Regards,
>>      >>> --
>>      >>> Tomas Lestach
>>      >>> Red Hat Satellite Engineering, Red Hat
>>      >>>
>>      >>>
>>      >>> ----- Original Message -----
>>      >>>> From: "Brian Buesker" <bbuesker qti qualcomm com
>>     <mailto:bbuesker qti qualcomm com>>
>>      >>>> To: spacewalk-list redhat com
>> <mailto:spacewalk-list redhat com>
>>      >>>> Sent: Wednesday, January 28, 2015 9:53:48 PM
>>      >>>> Subject: [Spacewalk-list] Support for LZMA-compressed
>> comps.xml
>>      >>>>
>>      >>>>
>>      >>>> I've started trying to bring online a Fedora 21
>> distro/kickstart
>>      >>>> profile against our Spacewalk 2.1 server. Unlike others who
>> have
>>      >>>> had
>>      >>>> trouble getting the fedora installer generation to work, it
>> mostly
>>      >>>> seems to be working for me. I'm not sure if that is because
>> I am
>>      >>>> using 2.1 instead of 2.2 or if there is some other
>> explanation.
>>      >>>>
>>      >>>> However, one thing I ran into is that when I created a
>> Fedora 21
>>      >>>> Everything channel, Spacewalk ended up not handling the
>> comps.xml
>>      >>>> properly. In the upstream yum repo, the comps.xml is
>> actually LZMA
>>      >>>> compressed. It does not look like Spacewalk decompresses
>> this when
>>      >>>> staging the comps in the local file system. Thus, when a
>> client
>>      >>>> fetches the repo metadata, it gets a file named comps.xml
>> that is
>>      >>>> actually compressed and thus it complains about the XML not
>> being
>>      >>>> well formed.
>>      >>>>
>>      >>>> I looked at the Spacewalk git repo and I see that an RPM
>>      >>>> dependency
>>      >>>> was added for pyliblzma. I already have that installed on my
>>      >>>> system,
>>      >>>> but despite that, Spacewalk still does not handle the
>> compressed
>>      >>>> comps properly. I dug further into this and found that two
>> python
>>      >>>> modules need to be updated, namely reposync.py and
>> fileutils.py.
>>      >>>>
>>      >>>> I would provide a patch with the fixes, although I have to go
>>      >>>> through
>>      >>>> a somewhat lengthy procedure before I can get permission
>> from my
>>      >>>> employer to release the patch. Therefore, I'll describe the
>>      >>>> necessary changes in case others want to make the same ones
>> (or
>>      >>>> even
>>      >>>> better, get this updated in git).
>>      >>>>
>>      >>>>
>>      >>>>      1. In reposync.py, the list of suffixes in
>> import_groups()
>>      >>>>      needs
>>      >>>>      to include '.xz'.
>>      >>>>      2. In fileutils.py, decompress_open() needs to be
>> updated to
>>      >>>>      use
>>      >>>>      the lzma.LZMAFile() function on files ending in '.xz'
>> (and of
>>      >>>>      course an import is needed as well).
>>      >>>>
>>      >>>>
>>      >>>> Let me know if you have any questions.
>>      >>>>
>>      >>>>
>>      >>>> Thanks,
>>      >>>> Brian Buesker
>>      >>>>
>>      >>>> _______________________________________________
>>      >>>> Spacewalk-list mailing list
>>      >>>> Spacewalk-list redhat com <mailto:Spacewalk-list redhat com>
>>      >>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>      >>> _______________________________________________
>>      >>> Spacewalk-list mailing list
>>      >>> Spacewalk-list redhat com <mailto:Spacewalk-list redhat com>
>>      >>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>      >>>
>>      >> _______________________________________________
>>      >> Spacewalk-list mailing list
>>      >> Spacewalk-list redhat com <mailto:Spacewalk-list redhat com>
>>      >> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>      >>
>>      > _______________________________________________
>>      > Spacewalk-list mailing list
>>      > Spacewalk-list redhat com <mailto:Spacewalk-list redhat com>
>>      > https://www.redhat.com/mailman/listinfo/spacewalk-list
>>      >
>>
>>
>>
>>     _______________________________________________
>>     Spacewalk-list mailing list
>>     Spacewalk-list redhat com <mailto:Spacewalk-list redhat com>
>>     https://www.redhat.com/mailman/listinfo/spacewalk-list
>>
>>
>>
>>
>> -- 
>>
>> Michael Guidero
>> Sococo IT
>> 650-265-7013 Ext 1000
>>
>>
>>
>> _______________________________________________
>> Spacewalk-list mailing list
>> Spacewalk-list redhat com
>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list redhat com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
>


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]