[Pulp-list] Do you remove docker content from the repositories?

[Ina Panova]

Hi,

I opened a story that describes docker content removal in pulp3
https://pulp.plan.io/issues/5179
Thank you for the participation in the discussion!


--------
Regards,

Ina Panova
Senior Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."


On Wed, Jul 24, 2019 at 11:05 AM Ina Panova <ipanova at redhat.com> wrote:

> Hey guys,
> thank you for the provided feedback!
>
> Mihai,
> i think this kind of unexpected problems can be faced even nowadays where:
> Luke uploads an rpm A, Joy removes rpm A and B. Luke distributes repo and
> tries to dnf install rpm A.
>
> Simon,
> In my personal opinion, i don't see it as a good practice to apply 'force'
> flag on the Tag. As Mihai pointed out, manifest is still perfectly pullable
> by digest.
>
> I think the reason to introduce the force flag is to be able to remove
> artifact that has higher level references.
>
> For example: Force remove manifest/manifest list with digest X, even if it
> has tags.
> Or:
> Force remove manifest with digest X, even if it is referenced in manifest
> lists.
> I will create a ticket that will describe what we have discussed here.
> Possibility of force removal of the Tag will still be opened for
> discussion, before we groom the story.
>
> Thank you!
>
> --------
> Regards,
>
> Ina Panova
> Senior Software Engineer| Pulp| Red Hat Inc.
>
> "Do not go where the path may lead,
>  go instead where there is no path and leave a trail."
>
>
> On Mon, Jul 22, 2019 at 2:52 PM Mihai Ibanescu <mihai.ibanescu at gmail.com>
> wrote:
>
>> Replies inline.
>>
>> On Mon, Jul 22, 2019 at 6:54 AM Simon Baatz <gmbnomis at gmail.com> wrote:
>> > On Fri, Jul 19, 2019 at 01:13:20PM +0200, Ina Panova wrote:
>> > Btw.  we have code in our automation to explicitly address this case,
>> > i.e.  we filter out manifest digests when the manifest digests are
>> > referenced by manifest lists that are still needed.  This code always
>> > felt out of place; Docker is the only artifact type that requires
>> > special treatment when removing artifacts.
>>
>> Docker in itself is "special", in that multiple tags can (and
>> typically will) point to the same manifest.
>>
>> While rpm packages can be shared across multiple repositories
>> seamlessly, docker is different in that the content (manifests) can be
>> shared within the same repository. So you need to play the refcount
>> dance at some point.
>>
>> > (As said in my previous mail, I don't like the approach to delete
>> > recursively on tag removal. But it may be what most users expect(?))
>>
>> I don't like it either, but the docker engine can get away with it,
>> because the local registry acts kind of like a cache of the upstream
>> registry.
>>
>> If you implement the upstream registry (which is what pulp does), you
>> can't remove a manifest just because no tags point to it - the
>> manifest is still perfectly usable by pulling with its digest.
>>
>> > >    As a possible solution we could add a 'force' flag that will not
>> rely
>> > >    on the users' good will, but will ensure that if a user specifies
>> it he
>> > >    is aware of implications. Otherwise if no 'force' flag is
>> specified we
>> > >    go with behaviour:
>> > >    ---->
>> > >    You can remove an artifact if no
>> > >    artifact on a higher level references it. If it can be removed, all
>> > >    (otherwise unused) artifacts on lower levels are deleted
>> recursively.
>> > >    For example, you cannot delete a manifest that is still referenced
>> by
>> > >    a manifest list or tag.
>> > >    Thoughts?
>> >
>> > That sounds like a good idea. Would the recursive removal also be done
>> > on deletion of a tag?
>>
>> I like the force flag, I just fear unexpected transactional problems:
>>
>> * user Luke notes that the manifest has tags A and B
>> * Luke wants to remove the manifest, so he decides to "use the force"
>> * user Joy tags the same manifest with tag C
>> * Luke proceeds with his removal, except that the state of the repo
>> has changed since he decided to use the force
>> * Joy gets no joy pulling the image with tag C, because it's gone
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190726/05dcd6fc/attachment.htm>