[Spacewalk-list] Spacewalk 2.6 Debian repo sync file handle leaking

[Jay McCanta]

I think I found the issue.  Here's the patch I applied to get spacewalk-repo-sync working. It closes the files opened when they are no longer used.  This prevents the file-handle leakage. 

diff -ruN spacewalk.orig/common/rhn_deb.py spacewalk/common/rhn_deb.py
--- spacewalk.orig/common/rhn_deb.py	2017-05-19 08:11:17.000000000 +0000
+++ spacewalk/common/rhn_deb.py	2017-10-31 17:05:52.438187084 +0000
@@ -88,6 +88,7 @@
         except Exception:
             e = sys.exc_info()[1]
             raise_with_tb(InvalidPackageError(e), sys.exc_info()[2])
+        self.deb = None
 
     @staticmethod
     def checksum_type():
@@ -138,3 +139,4 @@
         if output_stream:
             self.payload_stream = output_stream
             self.payload_size = output_stream.tell() - output_start
+        self.header_data.close()



-----Original Message-----
From: Robert Paschedag [mailto:robert.paschedag at web.de] 
Sent: Tuesday, October 31, 2017 12:16 AM
To: spacewalk-list at redhat.com; Jay McCanta <J.McCanta at F5.com>; spacewalk-list at redhat.com
Subject: Re: [Spacewalk-list] Spacewalk 2.6 Debian repo sync file handle leaking

EXTERNAL MAIL: robert.paschedag at web.de

Am 31. Oktober 2017 00:17:56 MEZ schrieb Jay McCanta <J.McCanta at f5.com>:
>I am running spacewalk 2.6 on CentOS7 trying to an sync Ubuntu 
>repository.  I have discovered that somewhere in 
>/usr/bin/spacewalk-sync-repo file handles are leaked.  It seems to leak
>2 handles for every file it needs to import.  At some point, I run out 
>of handles.  The files are all temp files '/tmp/tmpXXXXXX.  Once all 
>the open handles are exhausted, I get 'ERROR:
>requests.exceptions.RequestException occurred'  over and over and over.
>
>I have tried looking in
>/usr/lib/python2.7/site-packages/spacewalk/common/rhn_deb.py
>/usr/lib/python2.7/site-packages/spacewalk/satellite_tools/repo_plugins
>/deb_src.py
>
>To see if I can find it, but I cannot.  The repo (xenial-universe) has
>43567 packages in it.  Has anyone else encountered (and fixed) this?
>
>Jay McCanta
>F5 Networks, Inc.
>Seattle,  WA 98119
>
>We Make Apps GO

Hi,

because I just upgraded from SW 2.4 to 2.7 I'm still using Steve Meier's "spacewalk-debian-sync" script to import Debian packages .I also tried to import Debian repos with spacewalk-repo-sync but that often had errors importing many packages. Someone then suggested, to use the script from Steve for the first, large import and then switch to SW default script. That should work, but I did not yet test it.

Robert