[Spacewalk-list] Spacewalk 2.6 Debian repo sync file handle leaking

[Robert Paschedag]

Am 31. Oktober 2017 19:49:11 MEZ schrieb Jay McCanta <J.McCanta at F5.com>:
>I think I found the issue.  Here's the patch I applied to get
>spacewalk-repo-sync working. It closes the files opened when they are
>no longer used.  This prevents the file-handle leakage. 
>
>diff -ruN spacewalk.orig/common/rhn_deb.py spacewalk/common/rhn_deb.py
>--- spacewalk.orig/common/rhn_deb.py	2017-05-19 08:11:17.000000000
>+0000
>+++ spacewalk/common/rhn_deb.py	2017-10-31 17:05:52.438187084 +0000
>@@ -88,6 +88,7 @@
>         except Exception:
>             e = sys.exc_info()[1]
>             raise_with_tb(InvalidPackageError(e), sys.exc_info()[2])
>+        self.deb = None
> 
>     @staticmethod
>     def checksum_type():
>@@ -138,3 +139,4 @@
>         if output_stream:
>             self.payload_stream = output_stream
>             self.payload_size = output_stream.tell() - output_start
>+        self.header_data.close()
>
>
>
>-----Original Message-----
>From: Robert Paschedag [mailto:robert.paschedag at web.de] 
>Sent: Tuesday, October 31, 2017 12:16 AM
>To: spacewalk-list at redhat.com; Jay McCanta <J.McCanta at F5.com>;
>spacewalk-list at redhat.com
>Subject: Re: [Spacewalk-list] Spacewalk 2.6 Debian repo sync file
>handle leaking
>
>EXTERNAL MAIL: robert.paschedag at web.de
>
>Am 31. Oktober 2017 00:17:56 MEZ schrieb Jay McCanta
><J.McCanta at f5.com>:
>>I am running spacewalk 2.6 on CentOS7 trying to an sync Ubuntu 
>>repository.  I have discovered that somewhere in 
>>/usr/bin/spacewalk-sync-repo file handles are leaked.  It seems to
>leak
>>2 handles for every file it needs to import.  At some point, I run out
>
>>of handles.  The files are all temp files '/tmp/tmpXXXXXX.  Once all 
>>the open handles are exhausted, I get 'ERROR:
>>requests.exceptions.RequestException occurred'  over and over and
>over.
>>
>>I have tried looking in
>>/usr/lib/python2.7/site-packages/spacewalk/common/rhn_deb.py
>>/usr/lib/python2.7/site-packages/spacewalk/satellite_tools/repo_plugins
>>/deb_src.py
>>
>>To see if I can find it, but I cannot.  The repo (xenial-universe) has
>>43567 packages in it.  Has anyone else encountered (and fixed) this?
>>
>>Jay McCanta
>>F5 Networks, Inc.
>>Seattle,  WA 98119
>>
>>We Make Apps GO
>
>Hi,
>
>because I just upgraded from SW 2.4 to 2.7 I'm still using Steve
>Meier's "spacewalk-debian-sync" script to import Debian packages .I
>also tried to import Debian repos with spacewalk-repo-sync but that
>often had errors importing many packages. Someone then suggested, to
>use the script from Steve for the first, large import and then switch
>to SW default script. That should work, but I did not yet test it.
>
>Robert

Hi Jay,

If the import works now, you should open a pull request on GitHub so this patch finds its way into the master branch.

Good work.

Robert