[Pulp-list] Do you remove docker content from the repositories?

[Ina Panova]

Hi Simon,
these are good observations.

It is challenging to find balance between the freedom given to the user and
still keeping the repository in a sane state.

General thinking is:

If a docker image manifest is *explicitly* mentioned to be removed by its
digest - we perform down recursive removal even if it is referenced by a
manifest list.
The reason why we also remove  the tags because they are completely useless
if they point to something non-existent, meanwhile manifest list is still
usable for other remaining image manifests.

This exact case we leave on the user's good will, especially if he has
provided the digest, meaning 'he knows what he's doing' . There is other
side of the coin where the restriction "you cannot delete a manifest that
is still referenced by
a manifest list or tag" is undesirable. It would force the user to 1) try
to remove manifest X and see that you cannot because of tags A,B,C 2)
removed those tags , 3) only after you can remove manifest X
or 1) before head find the tags manifest X references 2) remove tags 3)
remove manifest X

As a possible solution we could add a 'force' flag that will not rely on
the users' good will, but will ensure that if a user specifies it he is
aware of implications. Otherwise if no 'force' flag is specified we go with
behaviour:
---->
You can remove an artifact if no
artifact on a higher level references it. If it can be removed, all
(otherwise unused) artifacts on lower levels are deleted recursively.

For example, you cannot delete a manifest that is still referenced by
a manifest list or tag.


Thoughts?

--------
Regards,

Ina Panova
Senior Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."


On Wed, Jul 17, 2019 at 2:26 PM Simon Baatz <gmbnomis at gmail.com> wrote:

> On Tue, Jul 16, 2019 at 09:28:38PM +0200, Ina Panova wrote:
> >    Hello community,
> >    we are trying to gather feedback on docker content removal behaviour
> in
> >    Pulp3.
> >    Here are the use cases we came up with, please share your thoughts and
> >    expectations.
> >    When it is desired to remove docker content from a repo, the
> >    expectations are that:
> >       * when a docker image manifest is removed, all its blobs( not
> >    referenced by other image manifests) are removed as well. Also tags
> >    that were referencing this manifest will be removed.
>
> What should happen to manifest lists that reference the manifest? Are
> those removed as well? (and by extension, the tags pointing to them)
>
> >       * When a docker manifest list is removed, all its manifests( not
> >    referenced by other manifests lists and not tagged) are removed as
> >    well. Furthermore, same story with the blobs. Also tags that were
> >    referencing this manifest list will be removed.
> >       *When a tag is removed , only tag is removed.
>
> Yes, that makes sense.  I know a repo manager that removes the
> manifest list/manifest/blobs when the last tag is removed.  I don't
> like this behavior because it causes problems when adopting a
> workflow without tags (i.e.  using content adresses only).
>
> However, users may expect this recursive behavior as the tag may be
> the only identifier they are aware of.
>
> >    Do you find this recursive removal behaviour useful and expect it work
> >    the way it got described above? Or simple removal makes more sense,
> >    e.g. just image manifest and its tag are removed?
>
> I find the recursive removal very useful. As a user I expect Pulp to
> know the details of the artifact structure and do the "right thing".
>
> I am wondering whether we should only remove recursively "down" the
> reference hierarchy, but not up. AFAIK, the references are:
>
> tag -> manifest list -> manifest -> blob
> tag ------------------>
>
> Removal would work as follows: You can remove an artifact if no
> artifact on a higher level references it. If it can be removed, all
> (otherwise unused) artifacts on lower levels are deleted recursively.
>
> For example, you cannot delete a manifest that is still referenced by
> a manifest list or tag.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190719/b7f32fa8/attachment.htm>