[Pulp-dev] Docker Schema version data model question

Fri May 5 12:02:37 UTC 2017

Partha,

schema_version field specified on Manifest model and on Tag model refers
*just* to the V2 api content.
There is no concept of schema_version and manifest in general for V1
content. For V1 content you have concept of image and parent ids.

So when you are querying collections like units_docker_tag or
units_docker_manifest these are related *just* to V2 content. Which
consists of schema version 1 and schema version 2.

Back to your question about what changes have been made for Pulp data model.

Manifest model has a new field called 'confg_layer' . This layer appears in
specs for  V2 content schema version 2.

Tag model has a new field 'schema_version'. Because there could be tags
referencing V2 content schema version 2 and V2 content schema version 1 and
we need to distinguish them.
This will also facilitate,for example, copy operation to avoid confusion.
Same for tag removal operation.

If you take a look at specs for V1 and V2 content I think it will help you
to understand more the concepts and clear the confusion.

V1 specs https://github.com/moby/moby/blob/master/image/spec/v1.md
V2 specs https://docs.docker.com/registry/spec/manifest-v2-1/
               https://docs.docker.com/registry/spec/manifest-v2-2/

Since you say Katello uses just V2, the collections and models  you should
care are - blobs, tags, manifests.

----> Yet when I synced busy box I see manifests with schema-verision = 1.
Shouldnt it be only showing stuff with schema-version=2 ??

With the support of schema version 2, when you sync busybox, Pulp will
query Registry for manifests with tags which reference both schema version
1 and 2 and save those to DB
Again, these above stated concepts are related just to V2 content.

Let me know if you have further questions, I will be happy to answer those.

--------
Regards,

Ina Panova
Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."

On Thu, May 4, 2017 at 11:58 PM, Partha Aji <paji at redhat.com> wrote:

>
> I was trying to figure out pulp's data model changes with respect to
> DockerV2 Schema2. Looking at https://github.com/pulp/pulp_docker/commit/
> 35dc19c8522f840d464bec2e93f45b4bb57b4f80#diff-
> 6c740284d714e1a0349c86bed5617b76 I only see schema-version column added
> to tags and no changes in the manifest collection. But in the database I
> see the following
> For manifests there is a schema_version column
>
> > db.units_docker_manifest.find()[0]
> {
>   "_id" : "582cb097-5c81-499a-adec-5d18470014f5",
>   "pulp_user_metadata" : {
>
>   },
>   "_last_updated" : 1487212925,
>   "_storage_path" : "/var/lib/pulp/content/units/docker_manifest/9e/
> 2aadfbd6d73d35bddbf2db2925503047fc84fdcba439f4069c80ceaca37f02/sha256:
> b5dd2db609f090a39b65a39489eb3eb670f559af40fc5d206b2a05451355ba72",
>   "downloaded" : true,
>   "digest" : "sha256:b5dd2db609f090a39b65a39489eb3e
> b670f559af40fc5d206b2a05451355ba72",
>   "name" : "library/busybox",
>   "tag" : "1-glibc",
>   "schema_version" : 1,
>   "fs_layers" : [
>     {
>       "blob_sum" : "sha256:a3ed95caeb02ffe68cdd9fd8440668
> 0ae93d633cb16422d00e8a7c22955b46d4"
>     },
>     {
>       "blob_sum" : "sha256:df74c0cea8ad6b25004db74b9829d5
> 13990ac7e219128594eb2b7df2d785f961"
>     }
>   ],
>   "_ns" : "units_docker_manifest",
>   "_content_type_id" : "docker_manifest"
> }
>
>
> And for tags there is one also.
>
> > db.units_docker_tag.find()[0]
>> {
>>   "_id" : "cc3d103d-f6ab-4efd-a936-e99dfc43691d",
>>   "repo_id" : "dd8fa453-9923-4fc1-99b6-8137a2bb5bf3",
>>   "manifest_digest" : "sha256:3e00695ae65afe08d3cc4e1c0bc4ef
>> b335e9c158c9b5a5f7c045c9ec380c731b",
>>   "_ns" : "units_docker_tag",
>>   "_last_updated" : 1487212925,
>>   "schema_version" : 1,
>>   "pulp_user_metadata" : {
>>
>>   },
>>   "_content_type_id" : "docker_tag",
>>   "name" : "1.24.2-glibc"
>> }
>>
>
>
> I am I assuming correctly when I say "schema-version" in
> "units_docker_manifest" refers to api version V1 vs V2 while
> "schema-version" in "units_docker_tag" refers to schema-version1 vs
> schema-version 2 ?
>
> If my assumption is correct then I have a diff issue. Katello typically
> says "use v2 only" when creating a repo. Yet when I synced busy box I see
> manifests with schema-verision = 1.  Shouldnt it be only showing stuff with
> schema-version=2 ??
>
>
>>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170505/35838d0d/attachment.htm>