[Pulp-list] Problem with relative URLs

Tue Jun 9 13:01:53 UTC 2015

Hello Salvatore,

the missing "=" is not a problem, Pulp can work with both notations (I tested this when I was testing Pulp in my PoC environment). I also tried all combinations of "/" sign at beggining and at the end of path, didn't help.

Strange part is relative urls used to work with no problems, you can see there are already few repositories included in path /rhel/6Server/x86_64/ in the folder listing in my original message. I can't add new repo to the same path. Another interesting point is, if I deliberately use already existing path on another repo, Pulp gives me full error message (with conflicting path and conflicting repo name), but all I'm getting is non specific 500 Forbidden error.
I looked inside the failing script (../configuration.py), but that did not help much. From the KeyError I'm inclinded to think that problem is probably somewhere in Mongo, because it seems my Pulp doesn't get valid value returned from database. I'm going to check the DB, but I never worked with Mongo, so I have to read up on it's syntax first.

It seems that everything works fine if I don't use the relative url parameter, so I gave up on realtive urls and I'm recreating new all repos without it. I'll keep original broken repos for troubleshooing though.

Best regards,
Petr

-----Original Message-----
From: pulp-list-bounces at redhat.com [mailto:pulp-list-bounces at redhat.com] On Behalf Of pulp-list-request at redhat.com
Sent: Tuesday, June 09, 2015 1:07 PM
To: pulp-list at redhat.com
Subject: Pulp-list Digest, Vol 67, Issue 7

Send Pulp-list mailing list submissions to
	pulp-list at redhat.com

To subscribe or unsubscribe via the World Wide Web, visit
	https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_mailman_listinfo_pulp-2Dlist&d=AwICAg&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=RjwFQ7jscawlkrhWOaKT8A&m=uLsvfDnzS7fR4AcSGK5oDEg1ZJxSQV-Ve4J0QsGESIc&s=5JeZXoOYw_WwlXRcqkA7pQjw6ppS7vJs1JXMlxLZLj8&e=
or, via email, send a message with subject or body 'help' to
	pulp-list-request at redhat.com

You can reach the person managing the list at
	pulp-list-owner at redhat.com

When replying, please edit your Subject line so it is more specific than "Re: Contents of Pulp-list digest..."

Today's Topics:

   1. Re: Problem with relative URLs (Salvatore Di Nardo)

----------------------------------------------------------------------

Message: 1
Date: Tue, 09 Jun 2015 12:04:50 +0100
From: Salvatore Di Nardo <sdinardo at ebi.ac.uk>
To: pulp-list at redhat.com
Subject: Re: [Pulp-list] Problem with relative URLs
Message-ID: <5576C852.1050702 at ebi.ac.uk>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"

Hello,
relative urls works for me...

        *Example:*
        pulp-admin rpm repo create --repo-id=rhel-6-server \
             --feed=https://urldefense.proofpoint.com/v2/url?u=https-3A__cdn.redhat.com_content_dist_rhel_server_6_6Server_x86-5F64_os&d=AwICAg&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=RjwFQ7jscawlkrhWOaKT8A&m=uLsvfDnzS7fR4AcSGK5oDEg1ZJxSQV-Ve4J0QsGESIc&s=pTM1FKZl4eZRBmS0XGiZruYF4cl29256idZLUee8MAg&e=  \
             --feed-ca-cert=/var/lib/pulp/certs/rhel6/redhat-uep.pem \
             --feed-cert=/var/lib/pulp/certs/rhel6/entitlement.pem \
             --feed-key=/var/lib/pulp/certs/rhel6/entitlement-key.pem \
             --relative-url=rhel-6-server

        *Another example:*
        pulp-admin rpm repo create --repo-id=rhel-6-server-$(date +%Y%m%d) \
        --relative-url=/frozen/rhel-6-server/$(date +%Y%m%d)

Note that i use "\" to make the input more readable.
As far i can see you put spaces instead of  "=".
Maybe its this the problem?

Salvatore

On 09/06/15 09:35, Tuma, Petr (GE Capital) wrote:
>
> Hello,
>
> we are using Pulp (2.6.1) to mirror sub set of RedHat repositories and 
> my setup is to have URL paths as close to CDN as possible. Everything 
> worked so far, I've been able to mirror base, supplementary and 
> oracle-java without any problem with relative-urls. Now I have to add 
> few more repos we'd like to mirror, but I'm getting realtive_url 
> error. This problem seems to show up on any repository I created right 
> after I installed and configured Pulp, I can add new repositories 
> under same forlder as long as I don't use any already existing folder.
> For example: I can use realtive url /extra/6Server/x86_64/rhevh/os and 
> /extra/6Server/x86_64/rhn-tools/os with no problem at all, but if I 
> use /rhel/6Server/x86_64/rhevh/os it fails.
>
> *Commands I use:*
>
> pulp-admin rpm repo create --repo-id rhel6_64-rhn-tools --feed 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__cdn.redhat.com_co
> ntent_dist_rhel_server_6_6Server_x86-5F64_rhn-2Dtools_os&d=AwICAg&c=IV
> _clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=RjwFQ7jscawlkrhWOaKT8A&m=u
> LsvfDnzS7fR4AcSGK5oDEg1ZJxSQV-Ve4J0QsGESIc&s=raynloLgDEB99rSn7-h-q9XCK
> KjAJSJ4eTg73ZfErMQ&e= --remove-missing true --relative-url 
> /rhel/6Server/x86_64/rhn-tools/os --serve-http false --serve-https 
> true --verify-feed-ssl false --feed-cert 
> /rhndata/entitlements/rhel6_64.pem --feed-key 
> /rhndata/entitlements/rhel6_64-key.pem
>
> An internal error occurred on the Pulp server:
>
> RequestException: POST request
>
> on /pulp/api/v2/repositories/ failed with 500 - 'relative_url'
>
> *Folder listing of given path shows no conflict at all:*
>
> ls -l pulp/published/yum/https/repos/rhel/6Server/x86_64/
>
> total 28
>
> drwxr-x--- 3 apache apache 4096 Jun  8 08:01 jbeap
>
> -rw-r--r-- 1 apache apache   49 Jun  8 13:44 listing
>
> drwxr-x--- 2 apache apache 4096 Jun  8 13:44 optional
>
> drwxr-x--- 2 apache apache 4096 Jun  8 13:20 oracle-java
>
> lrwxrwxrwx 1 apache apache   78 Jun  8 13:12 os -> 
> /var/lib/pulp/published/yum/master/yum_distributor/rhel6_64-base/14337
> 61935.73
>
> drwxr-x--- 5 apache apache 4096 Jun  8 08:30 rhevm
>
> drwxr-x--- 2 apache apache 4096 Jun  8 13:31 supplementary
>
> *From /var/log/messages (Pulp is running with DEBUG logging):*
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) Exception 
> adding distributor to repo [rhel6_64-rhn-tools]; the repo will be 
> deleted
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) Traceback 
> (most recent call last):
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/cud.py",
> line 187, in create_and_configure_repo
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) 
> distributor_id)
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/distributo
> r.py",
> line 170, in add_distributor
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) result = 
> distributor_instance.validate_config(transfer_repo, call_config,
> config_conduit)
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/di
> stributor.py",
> line 84, in validate_config
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) return 
> configuration.validate_config(repo, config, config_conduit)
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/co
> nfiguration.py",
> line 118, in validate_config
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) 
> _check_for_relative_path_conflicts(repo, config, config_conduit,
> error_messages)
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/co
> nfiguration.py", line 469, in _check_for_relative_path_conflicts
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) 
> conflicting_relative_url = distributor['config']['relative_url'] or 
> conflicting_repo_id
>
> pulp: pulp.server.managers.repo.cud:ERROR: (15478-47648) KeyError: 
> 'relative_url'
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: Unhandled 
> Exception
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648) 'relative_url'
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648) Traceback (most recent call last):
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/middleware/e
> xception.py",
> line 44, in __call__
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return self.app(environ, start_response)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/middleware/p
> ostponed.py",
> line 42, in __call__
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return self.app(environ, start_response)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 279, in 
> wsgi
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     result = self.handle_with_processors()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 29, in _handle_with_processors
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return process(self.processors)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in process
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 566, in 
> processor
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return handler()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in <lambda>
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in process
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 581, in 
> processor
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     result = handler()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in <lambda>
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 28, in process
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return self.handle()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 230, in 
> handle
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return self._delegate(fn, self.fvars, args)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 422, in 
> _delegate
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return f()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 430, in 
> <lambda>
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     f = lambda: self._delegate_sub_application(pat, what)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 455, in 
> _delegate_sub_application
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return app.handle_with_processors()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 29, in _handle_with_processors
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return process(self.processors)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in process
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 566, in 
> processor
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return handler()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in <lambda>
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in process
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 581, in 
> processor
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     result = handler()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 26, in <lambda>
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return p(lambda: process(processors))
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.
> py",
> line 28, in process
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return self.handle()
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 230, in 
> handle
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return self._delegate(fn, self.fvars, args)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 420, in 
> _delegate
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return handle_class(cls)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/web/application.py", line 396, in 
> handle_class
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return tocall(*args)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/controllers/
> decorators.py",
> line 203, in _auth_decorator
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     value = method(self, *args, **kwargs)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/webservices/controllers/
> repositories.py",
> line 180, in POST
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     repo = repo_manager.create_and_configure_repo(*args, 
> **kwargs)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/cud.py",
> line 187, in create_and_configure_repo
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     distributor_id)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/distributo
> r.py",
> line 170, in add_distributor
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     result = 
> distributor_instance.validate_config(transfer_repo, call_config,
> config_conduit)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/di
> stributor.py",
> line 84, in validate_config
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     return configuration.validate_config(repo, config, 
> config_conduit)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/co
> nfiguration.py",
> line 118, in validate_config
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     _check_for_relative_path_conflicts(repo, config, 
> config_conduit, error_messages)
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)   File 
> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/co
> nfiguration.py", line 469, in _check_for_relative_path_conflicts
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648)     conflicting_relative_url = 
> distributor['config']['relative_url'] or conflicting_repo_id
>
> pulp: pulp.server.webservices.middleware.exception:ERROR: 
> (15478-47648) KeyError: 'relative_url'
>
> Is there any way how to fix this? I would like to have all RHEL 
> repositories in one folder.
>
> Best regards,
>
> Petr Tuma
>
>
> Notice: This electronic mail transmission, including any attachments, 
> is confidential. It is intended only for the person(s) named, and it 
> may contain information, which is proprietary, personal data or trade 
> secrets, which are protected under the existing law. If you received 
> this transmission in error, please notify the sender immediately at 
> the above mentioned e-mail address or telephone number and delete this 
> e-mail from your computer. Thank you.
>
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_ma
> ilman_listinfo_pulp-2Dlist&d=AwICAg&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3
> wUrLrDQYWSI&r=RjwFQ7jscawlkrhWOaKT8A&m=uLsvfDnzS7fR4AcSGK5oDEg1ZJxSQV-
> Ve4J0QsGESIc&s=5JeZXoOYw_WwlXRcqkA7pQjw6ppS7vJs1JXMlxLZLj8&e=

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_archives_pulp-2Dlist_attachments_20150609_37305d32_attachment.html&d=AwICAg&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=RjwFQ7jscawlkrhWOaKT8A&m=uLsvfDnzS7fR4AcSGK5oDEg1ZJxSQV-Ve4J0QsGESIc&s=ofqjOPJJlZXnmzHH8UcTmHMj0AP8vbtRyk1H4P3Hla4&e= >

------------------------------

_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_mailman_listinfo_pulp-2Dlist&d=AwICAg&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=RjwFQ7jscawlkrhWOaKT8A&m=uLsvfDnzS7fR4AcSGK5oDEg1ZJxSQV-Ve4J0QsGESIc&s=5JeZXoOYw_WwlXRcqkA7pQjw6ppS7vJs1JXMlxLZLj8&e= 

End of Pulp-list Digest, Vol 67, Issue 7
****************************************