[Spacewalk-list] IOError: [Errno 28] No space left on device

Wed Jul 15 15:07:07 UTC 2015

Can you give output of "df -h". I assume, spacewalk temporarily writes to /var or /tmp.

Am 15.07.2015 3:50 nachm. schrieb "Camp, Neil (NIH/NCI) [C]" <neil.camp at nih.gov>:
>
> Hello,
>
> I am trying to upload a deb file that is 1019MB. It keeps timing out:
>
> INFO: 1/4301 : ns3-doc_3.17+dfsg-1build1_all.deb
> Error pushing /var/tmp/ns3-doc_3.17+dfsg-1build1_all.deb:  (500)
> Waiting 1 seconds and trying again...
> Error pushing /var/tmp/ns3-doc_3.17+dfsg-1build1_all.deb:  (500)
> Waiting 4 seconds and trying again...
> Error pushing /var/tmp/ns3-doc_3.17+dfsg-1build1_all.deb:  (500)
> Waiting 4 seconds and trying again...
> Giving up after 3 attempts
> Internal server error 500 Internal Server Error
> Internal server error 500 Internal Server Error
> Internal server error 500 Internal Server Error
> ERROR: rhnpush failed
>
> And the apache error_log shows the following:
>
> [Wed Jul 15 09:37:27 2015] [error] Spacewalk 3027 2015/07/15 09:37:27 -04:00: ('Unhandled exception',)
> [Wed Jul 15 09:38:14 2015] [error] Exception reported from ---
> [Wed Jul 15 09:38:14 2015] [error] Time: Wed Jul 15 09:38:14 2015
> [Wed Jul 15 09:38:14 2015] [error] Exception type <type 'exceptions.IOError'>
> [Wed Jul 15 09:38:14 2015] [error] Exception while handling function server.apacheUploadServer._wrapper
> [Wed Jul 15 09:38:14 2015] [error] Request object information:
> [Wed Jul 15 09:38:14 2015] [error] URI: /PACKAGE-PUSH
> [Wed Jul 15 09:38:14 2015] [error] Remote Host: localhost
> [Wed Jul 15 09:38:14 2015] [error] Server Name: localhost:443
> [Wed Jul 15 09:38:14 2015] [error] Headers passed in:
> [Wed Jul 15 09:38:14 2015] [error] \tAccept-Encoding: identity
> [Wed Jul 15 09:38:14 2015] [error] \tCONTENT_LENGTH: 1067972176
> [Wed Jul 15 09:38:14 2015] [error] \tCONTENT_TYPE: application/x-rpm
> [Wed Jul 15 09:38:14 2015] [error] \tDOCUMENT_ROOT: /var/www/html
> [Wed Jul 15 09:38:14 2015] [error] \tGATEWAY_INTERFACE: CGI/1.1
> [Wed Jul 15 09:38:14 2015] [error] \tHTTPS: 1
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_ACCEPT_ENCODING: identity
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_HOST: localhost
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_USER_AGENT: rhnpush
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_AUTH_SESSION: 51612x7d6f79c28c87cebea3a6fa646c188c57778bff4d466dd03af8f45a70fafca30f
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_FILE_MD5SUM: 8bbfcd73ef4814742f5c7a0a6a900427
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_FORCE: 0
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_PACKAGE_ARCH: all-deb
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_PACKAGE_NAME: ns3-doc
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_PACKAGE_RELEASE: 1build1
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_PACKAGE_VERSION: 3.17+dfsg
> [Wed Jul 15 09:38:14 2015] [error] \tHTTP_X_RHN_UPLOAD_PACKAGING: deb
> [Wed Jul 15 09:38:14 2015] [error] \tHost: localhost
> [Wed Jul 15 09:38:14 2015] [error] \tPATH_INFO:
> [Wed Jul 15 09:38:14 2015] [error] \tQUERY_STRING:
> [Wed Jul 15 09:38:14 2015] [error] \tREMOTE_ADDR: ::1
> [Wed Jul 15 09:38:14 2015] [error] \tREMOTE_PORT: 51536
> [Wed Jul 15 09:38:14 2015] [error] \tREQUEST_METHOD: POST
> [Wed Jul 15 09:38:14 2015] [error] \tREQUEST_URI: /PACKAGE-PUSH
> [Wed Jul 15 09:38:14 2015] [error] \tSCRIPT_FILENAME: /usr/share/rhn/wsgi/package_push.py
> [Wed Jul 15 09:38:14 2015] [error] \tSCRIPT_NAME: /PACKAGE-PUSH
> [Wed Jul 15 09:38:14 2015] [error] \tSCRIPT_URI: https://localhost/PACKAGE-PUSH
> [Wed Jul 15 09:38:14 2015] [error] \tSCRIPT_URL: /PACKAGE-PUSH
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_ADDR: ::1
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_ADMIN: root at localhost
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_NAME: localhost
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_PORT: 443
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_PROTOCOL: HTTP/1.1
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_SIGNATURE: <address>Apache Server at localhost Port 443</address>
> [Wed Jul 15 09:38:14 2015] [error]
> [Wed Jul 15 09:38:14 2015] [error] \tSERVER_SOFTWARE: Apache
> [Wed Jul 15 09:38:14 2015] [error] \tUser-Agent: rhnpush
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Auth-Session: 51612x7d6f79c28c87cebea3a6fa646c188c57778bff4d466dd03af8f45a70fafca30f
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-File-Checksum: 8bbfcd73ef4814742f5c7a0a6a900427
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-File-Checksum-Type: md5
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-File-MD5sum: 8bbfcd73ef4814742f5c7a0a6a900427
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Force: 0
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Package-Arch: all-deb
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Package-Name: ns3-doc
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Package-Release: 1build1
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Package-Version: 3.17+dfsg
> [Wed Jul 15 09:38:14 2015] [error] \tX-RHN-Upload-Packaging: deb
> [Wed Jul 15 09:38:14 2015] [error] \tmod_ssl.is_https: <built-in method ssl_is_https of mod_wsgi.Adapter object at 0x7f4f2f38ab70>
> [Wed Jul 15 09:38:14 2015] [error] \tmod_ssl.var_lookup: <built-in method ssl_var_lookup of mod_wsgi.Adapter object at 0x7f4f2f38ab70>
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.application_group: ncias-d1480-v.nci.nih.gov|/package-push
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.callable_object: application
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.handler_script:
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.input_chunked: 0
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.listener_host:
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.listener_port: 443
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.process_group:
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.request_handler: wsgi-script
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.script_reloading: 1
> [Wed Jul 15 09:38:14 2015] [error] \tmod_wsgi.version: (3, 2)
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.errors: <mod_wsgi.Log object at 0x7f4f2f3bfbb0>
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.file_wrapper: <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7f4f2f38ab70>
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.input: <mod_wsgi.Input object at 0x7f4f2f3bfb70>
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.multiprocess: True
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.multithread: False
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.run_once: False
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.url_scheme: https
> [Wed Jul 15 09:38:14 2015] [error] \twsgi.version: (1, 1)
> [Wed Jul 15 09:38:14 2015] [error]
> [Wed Jul 15 09:38:14 2015] [error] Exception Handler Information
> [Wed Jul 15 09:38:14 2015] [error] Traceback (most recent call last):
> [Wed Jul 15 09:38:14 2015] [error]   File "/usr/lib/python2.6/site-packages/spacewalk/server/apacheUploadServer.py", line 102, in _wrapper
> [Wed Jul 15 09:38:14 2015] [error]     log_debug(5, "Calling", function)
> [Wed Jul 15 09:38:14 2015] [error]   File "/usr/share/rhn/upload_server/handlers/package_push/package_push.py", line 115, in handler
> [Wed Jul 15 09:38:14 2015] [error]     self.file_checksum_type, self.file_checksum)
> [Wed Jul 15 09:38:14 2015] [error]   File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnPackageUpload.py", line 272, in save_uploaded_package
> [Wed Jul 15 09:38:14 2015] [error]     a_pkg.read_header()
> [Wed Jul 15 09:38:14 2015] [error]   File "/usr/lib/python2.6/site-packages/spacewalk/common/rhn_deb.py", line 118, in read_header
> [Wed Jul 15 09:38:14 2015] [error]     self._stream_copy(self.input_stream, self.header_data)
> [Wed Jul 15 09:38:14 2015] [error]   File "/usr/lib/python2.6/site-packages/spacewalk/common/rhn_pkg.py", line 119, in _stream_copy
> [Wed Jul 15 09:38:14 2015] [error]     dest.write(buf)
> [Wed Jul 15 09:38:14 2015] [error] IOError: [Errno 28] No space left on device
> [Wed Jul 15 09:38:14 2015] [error]
> [Wed Jul 15 09:38:14 2015] [error] Spacewalk 3023 2015/07/15 09:38:14 -04:00: ('Unhandled exception’,)
>
> I hace seen similar issues but none of them explain this problem. I have plenty of space on the file system, 28GB free so the file system is not the problem. Is it the file size or some other issue? I appreciate any help, thank you.