[Pulp-list] pulp-cli migration plan create FAIL

David Davis daviddavis at redhat.com
Thu May 13 12:20:55 UTC 2021 

The /pulp/content/ endpoint should list all your distributions. I suspect
you're running into this bug:

https://pulp.plan.io/issues/8636

It should be fixed in 3.13 which releases tentatively on May 18. If you
can't wait, you can file a backport request[0] and we can get this into
3.12 for you.

[0]
https://docs.pulpproject.org/pulpcore/en/master/nightly/bugs-features.html#for-backport-requests

David


On Wed, May 12, 2021 at 11:07 PM Ben Stanley <ben.stanley at gmail.com> wrote:

> Matthias,
>
> Thank you for this information. The output from those commands allowed me
> to identify where the file repositories are published.
> https://honeybee/pulp/content/file-repository-name/
>
> Using that information, I was able to load the repository page and see the
> contents in my web browser (provided I knew the repository name from the
> pulp file distribution list command).
>
> However, pulp2 allowed me to browse the list of published repositories
> directly. Can I do that with pulp3?
>
> Should I be using Katello as a front end to provide some kind of user
> interface for pulp3? This seems very hard, even by pulp2 standards.
>
> Ben.
> On 12/5/21 6:36 pm, Matthias Dellweg wrote:
>
> Can you try to see, whether any of the following commands show the
> expected entities?
> pulp file remote list
> pulp file repository list
> pulp file publication list
> pulp file distribution list
> pulp migration pulp2 repository list
>
> On Wed, May 12, 2021 at 9:43 AM Ben Stanley <ben.stanley at gmail.com> wrote:
>
>> Please see responses inline.
>> On 6/5/21 8:15 pm, Matthias Dellweg wrote:
>>
>>
>>
>> On Thu, May 6, 2021 at 2:32 AM Ben Stanley <ben.stanley at gmail.com> wrote:
>>
>>> I have changed my ~/.config/pulp/settings.toml file as follows:
>>>
>>> Old: base_url = "http://honeybee" <http://honeybee>
>>> New: base_url = "https://honeybee" <https://honeybee>
>>>
>>> The old setting was created in accordance with instructions at
>>> https://github.com/pulp/pulp-cli/blob/develop/docs/install.md . The
>>> instructions need to be updated.
>>>
>>> With the settings file updated, I have managed to create a migration
>>> plan:
>>>
>>> [root at honeybee plans]# pulp -vvv --no-verify-ssl --username USERNAME
>>> --password PASSWORD migration plan create --plan='{"plugins": [{"type":
>>> "iso"}]}'
>>> post https://honeybee/pulp/api/v3/migration-plans/
>>>   Content-Type: application/json
>>>   Content-Length: 48
>>> b'{"plan": "{\\"plugins\\": [{\\"type\\": \\"iso\\"}]}"}'
>>> Response: 201
>>>
>>> {"pulp_href":"/pulp/api/v3/migration-plans/UUID/","pulp_created":"2021-05-05T23:54:28.456994Z","plan":{"plugins":[{"type":"iso"}]}}
>>> {
>>>   "pulp_href": "/pulp/api/v3/migration-plans/UUID/",
>>>   "pulp_created": "2021-05-05T23:54:28.456994Z",
>>>   "plan": {
>>>     "plugins": [
>>>       {
>>>         "type": "iso"
>>>       }
>>>     ]
>>>   }
>>> }
>>>
>>> [root at honeybee plans]#
>>>
>>> After that I was able to reset the v3 repositories and run a migration.
>>>
>> That sounds great!
>>
>>> Now, is there any way that I can simplify these commands? Can I run pulp
>>> commands without having to specify the username and password and
>>> --no-verify-ssl on every command line?
>>>
>> Yes there is. You can add these parameters to the config file. See
>> https://github.com/pulp/pulp-cli/blob/develop/docs/install.md#configuration
>> You should be able to perform a "pulp status" command without errors.
>>
>> Thank you for this I can indeed to a "pulp status" and obtain sensible
>> output now, without needing all the extra flags.
>>
>> Also, I tried to load the URL https://honeybee/pulp/isos/ but the page
>>> timed out. Am I trying to load the correct URL for pulp3? I did not even
>>> get a https certificate error problem (I am using a self-signed
>>> certificate).
>>>
>> That looks to me like a Pulp2 address. I think, all Pulp3 content will be
>> served at https://honeybee/pulp/content/ . Also note, that "iso" content
>> will be migrated into the pulp_file plugin.
>>
>> I have migrated the iso content using the 3 steps outlined at
>> https://pulp-2to3-migration.readthedocs.io/en/latest/workflows.html .
>> However, https://honeybee/pulp/content/ loads an empty page. Is there
>> some secret path from that point, or do I have to publish all of the
>> migrated repositories?
>>
>> Thanks for the help!
>>>
>>> Ben.
>>> On 5/5/21 6:33 pm, Matthias Dellweg wrote:
>>>
>>> I think, you are hitting this error:
>>> https://github.com/pulp/pulp-cli/issues/221
>>>
>>> The problem is that Pulp adds a redirect from http to https, but
>>> requests turns all POST requests into GET requests upon redirect. So the
>>> server answers like you wanted to list the migration plans (http 200)
>>> instead of creating a new one (http 202).
>>> To fix your specific problem, please point the base-address to "https://"
>>> instead of "http://" in your CLI config file.
>>> We captured the problem earlier here:
>>> https://github.com/pulp/pulp-cli#known-issues
>>>
>>> And we will try to capture that case in the abovementioned issue and
>>> create a better error message.
>>>
>>> Thank you so much for not giving up. Your feedback is really helpful to
>>> us.
>>>
>>>   Matthias Dellweg
>>>
>>> On Wed, May 5, 2021 at 8:20 AM Ben Stanley <ben.stanley at gmail.com>
>>> wrote:
>>>
>>>> Hello Matthias,
>>>>
>>>> Regarding the software versions:
>>>>
>>>> [root at honeybee plans]# /usr/local/lib/pulp/bin/pip list
>>>> Package                            Version
>>>> -------------------------------------------------------
>>>> pulp-2to3-migration       0.11.0
>>>> pulpcore                           3.12.0
>>>> pulp-cli                              0.7.0
>>>>
>>>> Looks like I could do some upgrading. I tried to upgrade the
>>>> pulp.pulp_installer, but that bombed (see separate email).
>>>>
>>>> Regarding the cached api-spec, I ran pulp status. There is no mention
>>>> of any api being outdated or refreshing (see attached output
>>>> 20210505_pulp_status.txt
>>>>
>>>> I have also re-run the migration plan create with -vvv . The output is
>>>> attached as 20210505_pulp_migration_create_vvv.txt . I hope it means more
>>>> to you than it does to me.
>>>>
>>>> Ben Stanley.
>>>> On 4/5/21 6:58 pm, Matthias Dellweg wrote:
>>>>
>>>> Hello again.
>>>> The pulp cli is using requests to talk to your server. I seems like
>>>> pointing the environment variable REQUESTS_CA_BUNDLE to the full path
>>>> of a copy of the file you may find as '/etc/pulp/certs/root.crt' or
>>>> '/etc/pulp/certs/ca.crt' tells requests to use the provided root
>>>> certificate.
>>>>
>>>> On Tue, May 4, 2021 at 10:28 AM Matthias Dellweg <mdellweg at redhat.com>
>>>> wrote:
>>>>
>>>>> Hello Stan,
>>>>> thank you for reporting this. First of all, you are right about the
>>>>> `--plan`. Are you willing to send a PR for that mistake?
>>>>> Let me first ask you the typical support questions: Which version of
>>>>> the migration plugin and the CLI is this?
>>>>> The error looks to me like the CLI get's an unexpected answer from the
>>>>> server, and that can have multiple reasons.
>>>>> 1. the cached api-spec of the CLI is outdated. Please run `pulp <...>
>>>>> status` and look for "Notice: Cached api is outdated. Refreshing...". After
>>>>> that, try again.
>>>>> 2. the server does not reply in the way, it's api-spec promises
>>>>> 3. something in the reverse proxy garbles the result
>>>>> Can you please try to rerun your command with `pulp -vvv
>>>>> --no-verify-ssl --username USERNAME ...` to add some debugging output. It
>>>>> will show you the requests sent to the server as well as the responses. I
>>>>> hope this can give some insight.
>>>>>
>>>>> On Tue, May 4, 2021 at 2:18 AM Ben Stanley <ben.stanley at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello pulp-list,
>>>>>>
>>>>>> I am attempting to migrate my pulp2 to pulp3.
>>>>>>
>>>>>> I seem to be facing difficulties at every step.
>>>>>>
>>>>>> After overcoming the initial difficulties of installation, I am now
>>>>>> up to trying to migrate the data. Looking at the migration documentation:
>>>>>>
>>>>>> https://pulp-2to3-migration.readthedocs.io/en/latest/workflows.html
>>>>>>
>>>>>> I note that the command in the documentation is incorrect:
>>>>>>
>>>>>> [root at honeybee plans]# *pulp --no-verify-ssl --username **USERNAME**
>>>>>> --password **PASSWORD** migration plan create plan='{"plugins":
>>>>>> [{"type": "iso"}]}'*
>>>>>>
>>>>>> Usage: pulp migration plan create [OPTIONS]
>>>>>>
>>>>>> Try 'pulp migration plan create --help' for help.
>>>>>>
>>>>>>
>>>>>> Error: Missing option '--plan'.
>>>>>>
>>>>>> [root at honeybee plans]#
>>>>>>
>>>>>> The documentation should be corrected to use --plan=<stuff> instead
>>>>>> of plan=<stuff> .
>>>>>>
>>>>>> Even after I corrected the command to use --plan=<stuff>, I got a
>>>>>> huge error message.
>>>>>>
>>>>>> root at honeybee plans]# *pulp --no-verify-ssl --username **USERNAME**
>>>>>> --password **PASSWORD** migration plan create --plan='{"plugins":
>>>>>> [{"type": "iso"}]}'*
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/cli/common/openapi.py",
>>>>>> line 219, in parse_response
>>>>>>
>>>>>> response_spec = method_spec["responses"][str(response.status_code)]
>>>>>>
>>>>>> KeyError: '200'
>>>>>>
>>>>>>
>>>>>> During handling of the above exception, another exception occurred:
>>>>>>
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File "/usr/local/bin/pulp", line 8, in <module>
>>>>>>
>>>>>> sys.exit(main())
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 829, in __call__
>>>>>>
>>>>>> return self.main(*args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 782, in main
>>>>>>
>>>>>> rv = self.invoke(ctx)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 1259, in invoke
>>>>>>
>>>>>> return _process_result(sub_ctx.command.invoke(sub_ctx))
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 1259, in invoke
>>>>>>
>>>>>> return _process_result(sub_ctx.command.invoke(sub_ctx))
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 1259, in invoke
>>>>>>
>>>>>> return _process_result(sub_ctx.command.invoke(sub_ctx))
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 1066, in invoke
>>>>>>
>>>>>> return ctx.invoke(self.callback, **ctx.params)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 610, in invoke
>>>>>>
>>>>>> return callback(*args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/decorators.py",
>>>>>> line 73, in new_func
>>>>>>
>>>>>> return ctx.invoke(f, obj, *args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 610, in invoke
>>>>>>
>>>>>> return callback(*args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/decorators.py",
>>>>>> line 73, in new_func
>>>>>>
>>>>>> return ctx.invoke(f, obj, *args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/click/core.py", line
>>>>>> 610, in invoke
>>>>>>
>>>>>> return callback(*args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/cli/common/generic.py",
>>>>>> line 287, in callback
>>>>>>
>>>>>> result = entity_ctx.create(body=body)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/cli/common/context.py",
>>>>>> line 364, in create
>>>>>>
>>>>>> self.CREATE_ID, parameters=_parameters, body=body,
>>>>>> non_blocking=non_blocking
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/cli/common/context.py",
>>>>>> line 129, in call
>>>>>>
>>>>>> result = self.api.call(operation_id, *args, **kwargs)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/cli/common/openapi.py",
>>>>>> line 284, in call
>>>>>>
>>>>>> return self.parse_response(method_spec, response)
>>>>>>
>>>>>> File
>>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/cli/common/openapi.py",
>>>>>> line 222, in parse_response
>>>>>>
>>>>>> response_spec = method_spec["responses"][str(100 *
>>>>>> int(response.status_code / 100))]
>>>>>>
>>>>>> KeyError: '200'
>>>>>>
>>>>>> [root at honeybee plans]#
>>>>>>
>>>>>>
>>>>>>
>>>>>> What do I need to fix to proceed with my migration?
>>>>>>
>>>>>> And I'd also like to know how I can get it to work without having to
>>>>>> specify --no-verify-ssl . Note that I have a self-signed certificate. I
>>>>>> don't mind pointing the pulp command to the self-signed certificate file so
>>>>>> that it will accept it, but I don't know how to do that.
>>>>>>
>>>>>> Thanks,
>>>>>> Ben Stanley.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Pulp-list mailing list
>>>>>> Pulp-list at redhat.com
>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-list
>>>>>
>>>>> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210513/8e0158df/attachment.htm>

More information about the Pulp-list mailing list