[Pulp-list] Can't sync or upload to repos

[Tata, Joseph]

Greetings, I have a pulp server that has started giving me a range of problems just recently.  The history: this server had been running 2.3 for several months without issues.  Its been auto syncing yum repos from RHEL and other sources and serving them to internal servers that we don't allow to touch the internet.  About 2 weeks ago we noticed a problem that updates weren't showing up on clients when running yum list available.  I checked the server and sure enough they had not been sync'd.  I did a sync manually and got some error I now cannot remember but some reading made me think I was behind on updates.  So yum update -y was run, pulp was updated to 2.4 and all went to hell.  I was unable to get the manage-db script to run, errors about symlinks to non-existent files and other problems.  After much wrangling I decided to try to remove pulp and do a clean install of 2.4.1.  I uninstalled anything remotely related to pulp, dropped the pulp-database from mongo and rebooted several times.  Re-install appeared successful, pulp-manage-db ran and after some tweaking of qpidd.conf to turn of authentication and encryption (we don't have an internal cert authority that is trusted by all systems) I was able to connect the admin client.

I then recreated a repo, which appeared to go successfully.  However when I try to add anything to it, either by running a sync or by uploading nothing happens.  When I run a sync I see the message "Synchronizing repository" but nothing happens.  Uploads will show the file(s) uploaded but then hang on "Importing into the repository".  These can be left indefinitely and never time out, nor do they appear to log anything.  System log, HTTP error and ssl error, and pulp admin log all appear untouched.  If I ctrl+c during the sync I get:

^CTraceback (most recent call last):
  File "/usr/bin/pulp-admin", line 9, in <module>
    load_entry_point('pulp-client-admin==2.4.0', 'console_scripts', 'pulp-admin')()
  File "/usr/lib/python2.6/site-packages/pulp/client/admin/__init__.py", line 35, in main
    config_files, exception_handler_class=AdminExceptionHandler
  File "/usr/lib/python2.6/site-packages/pulp/client/launcher.py", line 109, in main
    code = cli.run(args)
  File "/usr/lib/python2.6/site-packages/pulp/client/extensions/core.py", line 478, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.6/site-packages/okaara/cli.py", line 974, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/usr/lib/python2.6/site-packages/pulp/client/extensions/extensions.py", line 224, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/sync_publish.py", line 135, in run
    response = self.context.server.repo_actions.sync(repo_id, None)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/repository.py", line 370, in sync
    return self.server.POST(path, data)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 99, in POST
    return self._request('POST', path, body=body, ensure_encoding=ensure_encoding)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 143, in _request
    response_code, response_body = self.server_wrapper.request(method, url, body)
  File "/usr/lib/python2.6/site-packages/pulp/bindings/server.py", line 317, in request
    response = connection.getresponse()
  File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse
    response.begin()
  File "/usr/lib64/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.6/httplib.py", line 349, in _read_status
    line = self.fp.readline()
  File "/usr/lib64/python2.6/socket.py", line 433, in readline
    data = recv(1)
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 228, in read
    return self._read_bio(size)
  File "/usr/lib64/python2.6/site-packages/M2Crypto/SSL/Connection.py", line 213, in _read_bio
    return m2.ssl_read(self.ssl, size, self._timeout)
KeyboardInterrupt


It definitely appears that the client is sending the request to the server, as I find this in .pulp/server-calls.log:
2014-09-25 11:45:31,902 - INFO - POST request to /pulp/api/v2/tasks/search/ with parameters {"criteria": {"filters": {"state": {"$nin": ["finished", "error", "canceled", "skipped"]}, "tags": {"$all": ["pulp:repository:rhel5-x64", "pulp:action:sync"]}}}}
2014-09-25 11:45:31,902 - INFO - Response status : 200

2014-09-25 11:45:31,902 - INFO - Response body :
[]

But the server never responds, never times out and never actually performs the requested task.

Any idea what's going on or how to correct this?






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20140925/54f4072c/attachment.htm>