[Pulp-dev] problems with docker tags

[Michael Hrivnak]

On Thu, Mar 8, 2018 at 11:48 AM, Dennis Kliban <dkliban at redhat.com> wrote:

>
> On Tue, Mar 6, 2018 at 4:03 PM, Austin Macdonald <austin at redhat.com>
> wrote:
>
>> Concept:
>> ContentUnits are created with a POST request to /v3/content/<type>/.
>> Adding the Tag to a Repository is a separate call. This workflow is awkward
>> for Tags because we would need to set the "repository" field on the Tag in
>> the first call, and actually add it to the repository in a second call. The
>> first call won't have enough information to enforce the uniqueness
>> constraints due to repository versions. The second call won't work either,
>> because adding content units to a repository is strictly a pulpcore
>> endpoint, and does not offer plugins the ability to enforce uniqueness
>> constraints. In Pulp 2, we enforced "only 1 tag of name X, type Y,
>> repository Z" with unique together, but this cannot work anymore because of
>> RepositoryVersions, because Tags created for old RepositoryVersions still
>> exist after they are removed from a later version.
>>
>
Good observation. We found in the past that uniqueness constraints on
content models are very important to get right, but there are many use
cases they can't cover. I think there are two issues you're encountering
here.

1. In any container image registry (Pulp or not), tags are fundamentally a
property of a repository. A tag is a pointer to a manifest within a
repository, and it is meant to be updated. We did the best we could to make
them usable by modeling them as content units. It's not a perfect fit, but
it is effective at enabling users to work with tags. There are rules about
how their values get updated/overwritten in Pulp 2 during operations such
as sync, upload and associate. In Pulp 3 the domain of uniqueness for a tag
name has shifted from the repo to the repo version, which presents an
interesting modeling challenge. One option is to treat it the same way we
did in Pulp 2, and give the Tag model the same repository relationship as a
content association. Instead of a repo foreign key, it would have similar
version_added and version_removed fields with similar uniqueness logic and
enforcement. I don't know how likable that idea is, but at least for
discussion's sake it would be the continuation of the previous strategy.
Perhaps there is a better way to model it with repo versions.

In any case, that particular uniqueness problem is theoretically solvable
for tags. But even solving it won't help when it comes time to decide how
to handle a conflict. Should the addition be rejected? Should it
overwrite/replace the preexisting tag? Should the older one be modified
somehow to allow them to coexist? Which brings us to...

2. There are many use cases for a plugin needing to be involved in the
decision about whether and how to add content to a repository. Uniqueness
constraints can't cover everything. Duplicate nevra, duplicate puppet
module names, duplicate file names, signature verification, whether to
reject a duplicate unit or overwrite the existing one, how many units with
a given name to keep, etc. have all been cases where a plugin needed to
participate in the process of adding content. The plugin provides logic
related to identifying potential conflicts and resolving them, according to
the rules of its particular content family. Specifically for container
image tags, I'm not sure you'll be able to meet its use cases without
letting the plugin participate in adding content to the repo.

At least in the past, there was no way to facilitate the right behavior
without letting plugin-specific code participate. Perhaps you'll find a new
way to deal with this, but otherwise you may need to continue providing a
way for plugins to hook into the lifecycle of the relationship between
content and repositories.

-- 

Michael Hrivnak

Principal Software Engineer, RHCE

Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180308/5a8899f2/attachment.htm>