[Pulp-list] Can't import containers created with Buildah
Konstantin M. Khankin
khankin.konstantin at gmail.com
Tue Sep 17 09:58:08 UTC 2019
Sorry for necroposting. I'm not sure if I still have the same issue, but I
have the same symptoms. Could you please have a look and advice? Thanks!
Whenever I try to create and upload another version of a container to the
same repo, I get an error message like this:
$ /usr/bin/pulp-admin docker repo uploads upload --repo-id gsk_postfix -f
/tmp/postfix.tar
...
Uploading: postfix.tar
[==================================================] 100%
283176960/283176960 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[-]
Running...
Task Failed
The importer docker_importer indicated a failed response when uploading
docker_manifest unit to repository gsk_postfix. Summary: 'Cannot resolve
field
"digest"'. Details: '{}'
I run pulp 2.20.1 and use buildah 1.9.0 to build container images. First
image in the repository uploads fine, but the second always fails.
Current repo status:
Id: gsk_postfix
Display Name: None
Description: None
Content Unit Counts:
Docker Blob: 4
Docker Manifest: 1
Docker Tag: 1
# pulp-admin docker repo search image --repo-id gsk_postfix
<returns nothing>
# pulp-admin docker repo search manifest --repo-id gsk_postfix
Created: 2019-08-20T22:58:50Z
Metadata:
Config Layer:
sha256:d2b497c83fe099624567046432468d84a5fdb49139b5192c9b332f0b10784695
Digest:
sha256:d17067c71cb229f703b0f43ecd8a85c55995f3007b99570f580f9f6ebc5dbccc
Downloaded: True
Fs Layers:
Blob Sum:
sha256:b05580fca2f9aabb2d8fa975b29146c9147c8418e559f197c54a4fac04babb95
Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
Size: 210204160
Blob Sum:
sha256:204cafbfdb1d31cb427478102377a99ceccfd7cc7be19cc191568fb9008fd4af
Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
Size: 12288
Blob Sum:
sha256:a85e1551e989545ca081c9ed0584e2db535a1484b91060eed88a17991e4b25cf
Layer Type: application/vnd.docker.image.rootfs.diff.tar.gzip
Size: 72946688
Pulp User Metadata:
Schema Version: 2
Repo Id: gsk_postfix
Unit Id: 000c0b4a-fb98-4ca3-a3ec-3b2a42521949
Unit Type Id: docker_manifest
Updated: 2019-08-20T22:58:50Z
# pulp-admin docker repo search tag --repo-id gsk_postfix
Created: 2019-08-20T22:58:55Z
Metadata:
Manifest Digest:
sha256:d17067c71cb229f703b0f43ecd8a85c55995f3007b99570f580f9f6ebc5dbccc
Manifest Type: image
Name: latest
Pulp User Metadata:
Repo Id: gsk_postfix
Schema Version: 2
Repo Id: gsk_postfix
Unit Id: 26c40bb1-4f3b-438f-8f50-cc5afe6baeb6
Unit Type Id: docker_tag
Updated: 2019-08-20T22:58:55Z
To build and upload containers, I wrote a simple script. Here is the
command sequence:
### STEP 1: Build an image from Dockerfile.
$ /usr/bin/buildah bud -t gsk_postfix .
...
Writing manifest to image destination
Storing signatures
db008cf80da01afe8ca11898e566279a7765e682bf56070857d40ffd469e2486
### STEP 2: Get the image digest.
$ /usr/bin/buildah images --quiet --digests --no-trunc
localhost/gsk_postfix:latest
sha256:db008cf80da01afe8ca11898e566279a7765e682bf56070857d40ffd469e2486
### STEP 3: Pull blobs from the storage.
$ /usr/bin/skopeo copy --format v2s2
containers-storage:localhost/gsk_postfix:latest dir:/tmp/tmp.NYhxiiOsOe
Getting image source signatures
Copying blob b05580fca2f9 done
Copying blob 8d136bb340b2 done
Copying blob 7a406e7184ab done
Copying config db008cf80d done
Writing manifest to image destination
Storing signatures
### STEP 4: Tar the archive.
$ tar cf /tmp/postfix.tar *
### STEP 5: Upload the archive to the repo.
$ /usr/bin/pulp-admin docker repo uploads upload --repo-id gsk_postfix -f
/tmp/postfix.tar
<failure here>
This happens no matter if I use v2s1 or v2s2. Resulting manifest.json
contains digests for all blobs (different image but built at the same time
with the same script and failed to get uploaded with the same error):
{'config': {'digest':
'sha256:fef5b5f9bb6fca5f1909303822ae34ae9d3ac1d88da84144fa1a7b83229652f3',
'mediaType': 'application/vnd.docker.container.image.v1+json',
'size': 2339},
'layers': [{'digest':
'sha256:b05580fca2f9aabb2d8fa975b29146c9147c8418e559f197c54a4fac04babb95',
'mediaType':
'application/vnd.docker.image.rootfs.diff.tar.gzip',
'size': 210204160},
{'digest':
'sha256:bbd20206b01e4bf5d521890bf5ffe2d5f565bbf70eda987b059b267c80f4f9a1',
'mediaType':
'application/vnd.docker.image.rootfs.diff.tar.gzip',
'size': 39936}],
'mediaType': 'application/vnd.docker.distribution.manifest.v2+json',
'schemaVersion': 2}
ср, 24 окт. 2018 г. в 16:05, Konstantin M. Khankin <
khankin.konstantin at gmail.com>:
> Thank you, https://pulp.plan.io/issues/4095 filed (why didn't I wait a
> bit to get 4096...)
>
> ср, 24 окт. 2018 г. в 15:35, Austin Macdonald <amacdona at redhat.com>:
>
>> The other projects are sometimes hidden.
>> https://pulp.plan.io/projects/pulp_docker
>>
>> On Wed, Oct 24, 2018 at 2:55 AM Konstantin M. Khankin <
>> khankin.konstantin at gmail.com> wrote:
>>
>>> Dennis,
>>>
>>> Which project I should use? I don't see a separate project for
>>> pulp_docker.
>>>
>>> Thanks!
>>>
>>> вт, 23 окт. 2018 г. в 23:58, Dennis Kliban <dkliban at redhat.com>:
>>>
>>>> Konstantin,
>>>>
>>>> Could you please file an issue in pulp.plan.io?
>>>>
>>>> Thanks,
>>>> Dennis
>>>>
>>>> On Sat, Oct 20, 2018 at 5:40 AM Konstantin M. Khankin <
>>>> khankin.konstantin at gmail.com> wrote:
>>>>
>>>>> *Forgot to mention versions:*
>>>>>
>>>>> pulp-admin-client-2.17.1-1.el7.noarch
>>>>> pulp-agent-2.17.1-1.el7.noarch
>>>>> pulp-consumer-client-2.17.1-1.el7.noarch
>>>>> pulp-deb-admin-extensions-1.8.0-1.el7.noarch
>>>>> pulp-deb-plugins-1.8.0-1.el7.noarch
>>>>> pulp-docker-admin-extensions-3.2.1-1.el7.noarch
>>>>> pulp-docker-plugins-3.2.1-1.el7.noarch
>>>>> pulp-puppet-consumer-extensions-2.17.1-1.el7.noarch
>>>>> pulp-puppet-handlers-2.17.1-1.el7.noarch
>>>>> pulp-python-admin-extensions-2.0.3-1.el7.noarch
>>>>> pulp-python-plugins-2.0.3-1.el7.noarch
>>>>> pulp-rpm-admin-extensions-2.17.1-1.el7.noarch
>>>>> pulp-rpm-consumer-extensions-2.17.1-1.el7.noarch
>>>>> pulp-rpm-handlers-2.17.1-1.el7.noarch
>>>>> pulp-rpm-plugins-2.17.1-1.el7.noarch
>>>>> pulp-rpm-yumplugins-2.17.1-1.el7.noarch
>>>>> pulp-selinux-2.17.1-1.el7.noarch
>>>>> pulp-server-2.17.1-1.el7.noarch
>>>>> python-pulp-agent-lib-2.17.1-1.el7.noarch
>>>>> python-pulp-bindings-2.17.1-1.el7.noarch
>>>>> python-pulp-client-lib-2.17.1-1.el7.noarch
>>>>> python-pulp-common-2.17.1-1.el7.noarch
>>>>> python-pulp-deb-common-1.8.0-1.el7.noarch
>>>>> python-pulp-docker-common-3.2.1-1.el7.noarch
>>>>> python-pulp-oid_validation-2.17.1-1.el7.noarch
>>>>> python-pulp-puppet-common-2.17.1-1.el7.noarch
>>>>> python-pulp-python-common-2.0.3-1.el7.noarch
>>>>> python-pulp-repoauth-2.17.1-1.el7.noarch
>>>>> python-pulp-rpm-common-2.17.1-1.el7.noarch
>>>>>
>>>>> buildah-1.2-2.gitbe87762.el7.x86_64
>>>>>
>>>>> *Manifest v2s1 from one of the images created by buildah:*
>>>>> {'architecture': 'amd64',
>>>>> 'fsLayers': [{'blobSum':
>>>>> 'sha256:4d85b6fa2344380743db8ccf4f0404603f56c8289db0824c13bfc309b2009be8'},
>>>>> {'blobSum':
>>>>> 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
>>>>> {'blobSum':
>>>>> 'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4'},
>>>>> {'blobSum':
>>>>> 'sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f'}],
>>>>> 'history': [{'v1Compatibility':
>>>>> '{"architecture":"amd64","config":{"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Entrypoint":["/bin/bash"],"WorkingDir":"/","Labels":{"org.label-schema.build-date":"20181006","org.label-schema.license":"GPLv2","
>>>>> org.label-schema.name":"CentOS Base
>>>>> Image","org.label-schema.schema-version":"1.0","org.label-schema.vendor":"CentOS"}},"created":"2018-10-20T07:49:56.084883454Z","id":"4591c91f4c223fa61c800a84bcf711647786f3fe71e589606f17b4d11b9c5807","os":"linux","parent":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904"}'},
>>>>> {'v1Compatibility':
>>>>> '{"id":"f871460fb057cfa08e7268df9b884eecedeed72bc6dbc3262deb19a25e66a904","parent":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","created":"2018-10-09T18:19:48.447478476Z","container_config":{"Cmd":["/bin/sh
>>>>> -c #(nop) CMD [\\"/bin/bash\\"]"]},"throwaway":true}'},
>>>>> {'v1Compatibility':
>>>>> '{"id":"8a4f409d848e6f693d5a4e6c260fbd8ec6a4dc99fc9915e3210bd5ed9c22e700","parent":"961164371d3af72a151063f656820bc835c695fa44af66c2845d0850f01ce0d9","created":"2018-10-09T18:19:48.258784779Z","container_config":{"Cmd":["/bin/sh
>>>>> -c #(nop) LABEL org.label-schema.schema-version=1.0
>>>>> org.label-schema.name=CentOS Base Image
>>>>> org.label-schema.vendor=CentOS org.label-schema.license=GPLv2
>>>>> org.label-schema.build-date=20181006"]},"throwaway":true}'},
>>>>> {'v1Compatibility':
>>>>> '{"id":"961164371d3af72a151063f656820bc835c695fa44af66c2845d0850f01ce0d9","created":"2018-10-09T18:19:47.884079751Z","container_config":{"Cmd":["/bin/sh
>>>>> -c #(nop) ADD
>>>>> file:fbe9badfd2790f0747a25fbe5c94a6daa78969511ca08c8d4ac654f3442570de in /
>>>>> "]}}'}],
>>>>> 'name': '',
>>>>> 'schemaVersion': 1,
>>>>> 'signatures': [{'header': {'alg': 'ES256',
>>>>> 'jwk': {'crv': 'P-256',
>>>>> 'kid': '...',
>>>>> 'kty': 'EC',
>>>>> 'x': '...',
>>>>> 'y': '...'}},
>>>>> 'protected': '...',
>>>>> 'signature': '...'}],
>>>>> 'tag': ''}
>>>>>
>>>>> *Manifest v2s2 for the same image:*
>>>>> {'config': {'digest':
>>>>> 'sha256:c17d63d23edcbe46b14fbe531e2e9b4411483947669087e90bdba3372eb2684d',
>>>>> 'mediaType':
>>>>> 'application/vnd.docker.container.image.v1+json',
>>>>> 'size': 1217},
>>>>> 'layers': [{'digest':
>>>>> 'sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f',
>>>>> 'mediaType':
>>>>> 'application/vnd.docker.image.rootfs.diff.tar.gzip',
>>>>> 'size': 208811520},
>>>>> {'digest':
>>>>> 'sha256:4d85b6fa2344380743db8ccf4f0404603f56c8289db0824c13bfc309b2009be8',
>>>>> 'mediaType':
>>>>> 'application/vnd.docker.image.rootfs.diff.tar.gzip',
>>>>> 'size': 589285888}],
>>>>> 'mediaType': 'application/vnd.docker.distribution.manifest.v2+json',
>>>>> 'schemaVersion': 2}
>>>>>
>>>>> сб, 20 окт. 2018 г. в 11:56, Konstantin M. Khankin <
>>>>> khankin.konstantin at gmail.com>:
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> I tried the tool Buildah (
>>>>>> https://www.projectatomic.io/blog/2017/06/introducing-buildah/) to
>>>>>> build containers without having docker binary. Containers are built (not
>>>>>> sure if they work), but I can't upload them to the pulp repo:
>>>>>>
>>>>>> 1) I copy container from "container-storage:" to "dir:", then create
>>>>>> tar archive per
>>>>>> https://docs.pulpproject.org/plugins/pulp_docker/user-guide/recipes.html#upload-v2-schema-2-and-schema-1-images-to-pulp
>>>>>> 2) pulp-admin -vvv docker repo uploads upload --repo-id
>>>>>> docker_centos7 -f centos7.tar
>>>>>>
>>>>>> [2] results in:
>>>>>> "traceback": "Traceback (most recent call last):\n File
>>>>>> \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 367, in
>>>>>> trace_task\n R = retval = fun(*args, **kwargs)\n File
>>>>>> \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529,
>>>>>> in __call__\n return super(Task, self).__call__(*args, **kwargs)\n File
>>>>>> \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107,
>>>>>> in __call__\n return super(PulpTask, self).__call__(*args, **kwargs)\n
>>>>>> File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 622, in
>>>>>> __protected_call__\n return self.run(*args, **kwargs)\n File
>>>>>> \"/usr/lib/python2.7/site-packages/pulp/server/managers/content/upload.py\",
>>>>>> line 223, in import_uploaded_unit\n unit_type=unit_type_id,
>>>>>> summary=result['summary'], details=result['details']\nPulpCodedException:
>>>>>> The importer docker_importer indicated a failed response when uploading
>>>>>> docker_manifest unit to repository docker_centos7.\n",
>>>>>>
>>>>>> "error": {
>>>>>> "code": "PLP0047",
>>>>>> "data": {
>>>>>> "unit_type": "docker_manifest",
>>>>>> "importer_id": "docker_importer",
>>>>>> "repo_id": "docker_centos7",
>>>>>> "details": {},
>>>>>> "summary": [
>>>>>> "Cannot resolve field \"digest\""
>>>>>> ]
>>>>>> },
>>>>>> "description": "The importer docker_importer indicated a failed
>>>>>> response when uploading docker_manifest unit to repository
>>>>>> docker_centos7.",
>>>>>>
>>>>>> I tried both v2s2 and v2s1, both don't work. I tried 2 different
>>>>>> Dockerfiles and both produce the same error.
>>>>>>
>>>>>> Is it an issue in Buildah or in Pulp? Or am I doing something wrong?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> --
>>>>>> Konstantin Khankin
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ханкин Константин
>>>>> _______________________________________________
>>>>> Pulp-list mailing list
>>>>> Pulp-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>>>
>>>>
>>>
>>> --
>>> Konstantin Khankin
>>> _______________________________________________
>>> Pulp-list mailing list
>>> Pulp-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>>
>
> --
> Konstantin Khankin
>
--
Ханкин Константин
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190917/6fcfc9ed/attachment.htm>
More information about the Pulp-list
mailing list