[Pulp-list] Do you remove docker content from the repositories?

[Mihai Ibanescu]

Replies inline.

On Mon, Jul 22, 2019 at 6:54 AM Simon Baatz <gmbnomis at gmail.com> wrote:
> On Fri, Jul 19, 2019 at 01:13:20PM +0200, Ina Panova wrote:
> Btw.  we have code in our automation to explicitly address this case,
> i.e.  we filter out manifest digests when the manifest digests are
> referenced by manifest lists that are still needed.  This code always
> felt out of place; Docker is the only artifact type that requires
> special treatment when removing artifacts.

Docker in itself is "special", in that multiple tags can (and
typically will) point to the same manifest.

While rpm packages can be shared across multiple repositories
seamlessly, docker is different in that the content (manifests) can be
shared within the same repository. So you need to play the refcount
dance at some point.

> (As said in my previous mail, I don't like the approach to delete
> recursively on tag removal. But it may be what most users expect(?))

I don't like it either, but the docker engine can get away with it,
because the local registry acts kind of like a cache of the upstream
registry.

If you implement the upstream registry (which is what pulp does), you
can't remove a manifest just because no tags point to it - the
manifest is still perfectly usable by pulling with its digest.

> >    As a possible solution we could add a 'force' flag that will not rely
> >    on the users' good will, but will ensure that if a user specifies it he
> >    is aware of implications. Otherwise if no 'force' flag is specified we
> >    go with behaviour:
> >    ---->
> >    You can remove an artifact if no
> >    artifact on a higher level references it. If it can be removed, all
> >    (otherwise unused) artifacts on lower levels are deleted recursively.
> >    For example, you cannot delete a manifest that is still referenced by
> >    a manifest list or tag.
> >    Thoughts?
>
> That sounds like a good idea. Would the recursive removal also be done
> on deletion of a tag?

I like the force flag, I just fear unexpected transactional problems:

* user Luke notes that the manifest has tags A and B
* Luke wants to remove the manifest, so he decides to "use the force"
* user Joy tags the same manifest with tag C
* Luke proceeds with his removal, except that the state of the repo
has changed since he decided to use the force
* Joy gets no joy pulling the image with tag C, because it's gone