[Pulp-dev] crane usage for disconnected container import

[Tom McKay]

If I understand the conversations (in email and #pulp-dev), the issue is
that there is a desire (requirement?) that repository formats are exported
and imported natively by the format handler.

For example, I may export yum repositories which I can in turn sync back
into a different pulp when those repos are served via http. Similarly for
container images, I need to export them in a way that crane can serve them
so crane can sync them.

What I'm really asking for, and I think this would apply to all repo types,
is a pulp-native export and import. Pulp has information about a repository
of type X and that the X files (*chuckle*) reside on disk. Is there no way
to just have pulp slurp these files in and process the info in a way that
the repo is recreated?

Why is it important that pulp be limited by the crane/docker api for
export/import? Is it a "it's hard and we don't want to support it"
argument, or is there a real technical aspect?

Inter-Server Sync (ISS) is very important for foreman, especially in
disconnected scenarios. All the repo types are equally important: yum,
container, ostree, iso, file, whatever.

On Fri, Mar 3, 2017 at 8:33 AM, Tom McKay <thomasmckay at redhat.com> wrote:

> The problem: Container images need to be delivered to another pulp in a
> disconnected manner. Specifically I need to "save" something to disk from
> pulp-1, walk that disk to another server, then import that saved content
> into pulp-2.
>
> Quoting @mhrivnak from another email:
>
> The supported way to move docker content around is via the registry API,
> so we will need to do a "sync" on the disconnected capsule of a local crane
> instance that is serving walked-in content. It probably shouldn't be the
> same crane instance that's already running on the capsule, to avoid repo
> name collisions.
>
> You also need to have the docker content (blobs, manifests, etc) served
> locally.
>
> httpd on the capsule could certainly be configured to run a second
> instance of crane on a different port, and to serve the walked-in content
> from some designated location.
>
> 1. Satellite publishes a docker repo. Its "redirect-url" needs to match
> whatever the URL will look like from the capsule when the content is being
> served there. Here are details on where the data gets written to disk by
> pulp: http://docs.pulpproject.org/plugins/pulp_docker/tech-r
> eference/distributor.html#web-distributor
>
> 2. the json blob, plus the rest of the repo data, gets walked to the
> disconnected capsule.
>
> 3. The walked-in content is put in the right place so that a local crane
> and web server makes it available.
>
> 4. Pulp on the capsule syncs the walked-in repos.
>
> This has me wondering a few things:
>
> Could a single crane support serving up multiple registries on different
> ports that are serving up different pulp repos? docker pull
> mycrane:5001/image:tag
>
> Could a single crane support serving up multiple registries on different
> paths? docker pull mycrane:5000/some/path/image:tag
>
> The context for my questions is foreman[1] where the disconnected use case
> is useful. Being able to also serve multiple registries for different
> organizations (scoping) would also be useful.
>
> Creative ideas welcome!
>
> [1] https://theforeman.org/plugins/katello/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170305/35a061ae/attachment.htm>