<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <style type="text/css" style="display:none"></style> </head> <body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;"> <p>I have tried '~', 'null' and '' (without the quotes), and they all get rendered as null. This leads to an issue with running the migrations when running the Ansible script.<br> </p> <p><br> </p> <p>I opened an issue: <a href="https://pulp.plan.io/issues/5687">https://pulp.plan.io/issues/5687</a><br> </p> <p><br> </p> <p>Maybe this should have gone in the pulp-ansible project?<br> </p> <p><br> </p> <p>Cheers,<br> </p> <p><br> </p> <div id="Signature"> <div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0"> <font style=""><font style="">Joey Dumont</font></font> <div style=""><br> </div> <div style="">Technical Advisor, Knowledge, Information, and Technology Services</div> <div style="">National Research Council Canada / Governement of Canada</div> <div style=""><a tabindex="0" href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="NoLP">joey.dumont@nrc-cnrc.gc.ca</a> / Tel: 613-990-8152 / Cell: 438-340-7436</div> <div style=""><br> </div> <div style="">Conseiller technique, Services du savoir, de l'information et de la technologie</div> <div style="">Conseil national de recherches Canada / Gouvernement du Canada</div> <div style=""><a tabindex="0" href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="NoLP">joey.dumont@nrc-cnrc.gc.ca</a> / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436</div> </div> </div> <div style="color: rgb(33, 33, 33);"> <hr tabindex="-1" style="display:inline-block; width:98%"> <div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Mike DePaulo <mikedep333@redhat.com><br> <b>Sent:</b> 09 November 2019 13:47<br> <b>To:</b> David Davis<br> <b>Cc:</b> Dumont, Joey; pulp-list@redhat.com<br> <b>Subject:</b> Re: [Pulp-list] Issues with using S3 storage when running pulp on Amazon EC2 (pulp3)</font> <div> </div> </div> <div> <div dir="ltr">I think that with Ansible syntax, you need to specify: <div> AWS_DEFAULT_ACL:<br> </div> <div><br> </div> <div>I am fairly certain the lack of any value at the end would get interpreted as Python None.</div> <div><br> </div> <div>- Mike</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Sat, Nov 9, 2019 at 6:57 AM David Davis <<a href="mailto:daviddavis@redhat.com">daviddavis@redhat.com</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex"> <div dir="ltr">You should be able to set aws_default_acl to null (ie "aws_default_acl: ~" or "aws_default_acl: null"). Sounds like that's not working though. What error did you get? Would you mind filing an issue? <div><br> </div> <div><a href="https://pulp.plan.io/projects/pulp/issues/new" target="_blank">https://pulp.plan.io/projects/pulp/issues/new</a></div> <div><br> </div> <div>Thanks.<br clear="all"> <div> <div dir="ltr"> <div dir="ltr"> <div> <div dir="ltr"> <div> <div dir="ltr"> <div><br> </div> <div>David<br> </div> </div> </div> </div> </div> </div> </div> </div> <br> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Fri, Nov 8, 2019 at 10:20 PM Dumont, Joey <<a href="mailto:Joey.Dumont@nrc-cnrc.gc.ca" target="_blank">Joey.Dumont@nrc-cnrc.gc.ca</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex"> <div dir="ltr" style="font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif"> <p>I encounted a new issue that escaped my notice by some weird coincidence.<br> </p> <p><br> </p> <p>It turns that you use the pulp_settings section of the playbook, aws_default_acl: None gets converted to AWS_DEFAULT_ACL = "None", instead of AWS_DEFAULT_ACL = None, which results in the cryptic <br> </p> <p><br> </p> <div>"An error occurred (InvalidArgument) when calling the PutObject operation: None"</div> <div><br> Is there a way to ensure that Ansible converts aws_default_acl: None to AWS_DEFAULT_ACL = None?<br> </div> <div><br> </div> <div>I tried setting it to null but I got an error.<br> </div> <div><br> </div> <div>Cheers,<br> <br> </div> <p><br> </p> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979Signature"> <div name="divtagdefaultwrapper"><font><font>Joey Dumont</font></font> <div><br> </div> <div>Technical Advisor, Knowledge, Information, and Technology Services</div> <div>National Research Council Canada / Governement of Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tel: 613-990-8152 / Cell: 438-340-7436</div> <div><br> </div> <div>Conseiller technique, Services du savoir, de l'information et de la technologie</div> <div>Conseil national de recherches Canada / Gouvernement du Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436</div> </div> </div> <div dir="ltr" style="font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif"> <hr style="display:inline-block; width:98%"> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979divRplyFwdMsg" dir="ltr"> <font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> <a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a> <<a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a>> on behalf of Dumont, Joey <<a href="mailto:Joey.Dumont@nrc-cnrc.gc.ca" target="_blank">Joey.Dumont@nrc-cnrc.gc.ca</a>><br> <b>Sent:</b> 06 November 2019 11:36<br> <b>To:</b> Mike DePaulo<br> <b>Cc:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] Issues with using S3 storage when running pulp on Amazon EC2 (pulp3)</font> <div> </div> </div> <div> <p>PR, as suggested: <a href="https://github.com/pulp/pulpcore/pull/366" target="_blank">https://github.com/pulp/pulpcore/pull/366</a>. <br> </p> <p><br> </p> <p>Cheers,<br> </p> <p><br> </p> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979Signature"> <div name="divtagdefaultwrapper"><font><font>Joey Dumont</font></font> <div><br> </div> <div>Technical Advisor, Knowledge, Information, and Technology Services</div> <div>National Research Council Canada / Governement of Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tel: 613-990-8152 / Cell: 438-340-7436</div> <div><br> </div> <div>Conseiller technique, Services du savoir, de l'information et de la technologie</div> <div>Conseil national de recherches Canada / Gouvernement du Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436</div> </div> </div> <div style="color:rgb(33,33,33)"> <hr style="display:inline-block; width:98%"> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979divRplyFwdMsg" dir="ltr"> <font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Mike DePaulo <<a href="mailto:mikedep333@redhat.com" target="_blank">mikedep333@redhat.com</a>><br> <b>Sent:</b> 06 November 2019 10:52<br> <b>To:</b> Dumont, Joey<br> <b>Cc:</b> David Davis; <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] Issues with using S3 storage when running pulp on Amazon EC2 (pulp3)</font> <div> </div> </div> <div> <div dir="ltr"> <div>Hi Joey,</div> <div><br> </div> It sounds like aws_default_acl should be documented here then: <div><a href="https://docs.pulpproject.org/en/3.0/nightly/installation/storage.html?highlight=aws" target="_blank">https://docs.pulpproject.org/en/3.0/nightly/installation/storage.html?highlight=aws</a><br> </div> <div><br> </div> <div>Care to submit a documentation PR?</div> <div> <div><a href="https://github.com/pulp/pulpcore/blob/master/docs/installation/storage.rst" target="_blank">https://github.com/pulp/pulpcore/blob/master/docs/installation/storage.rst</a><br> </div> <div><br> </div> <div>Thanks,</div> <div>-Mike</div> <div></div> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Wed, Nov 6, 2019 at 9:07 AM Dumont, Joey <<a href="mailto:Joey.Dumont@nrc-cnrc.gc.ca" target="_blank">Joey.Dumont@nrc-cnrc.gc.ca</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex"> <div dir="ltr" style="font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif"> Turns out the issue was on my end. I had to add aws_default_acl: None to the pulp_settings section of the playbook. The public-read ACL was incompatible with the BlockPublicAccess settings that I had set on my S3 bucket. <div> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663Signature"> <div name="divtagdefaultwrapper"><font><font><br> </font></font></div> <div name="divtagdefaultwrapper"><font><font>I'm now encountering a different issue, but I'll start another thread for that one.</font></font></div> <div name="divtagdefaultwrapper"><font><font><br> </font></font></div> <div name="divtagdefaultwrapper"><font><font>Thanks for the pointers, they were very helpful!</font></font></div> <div name="divtagdefaultwrapper"><font><font><br> </font></font></div> <div name="divtagdefaultwrapper"><font><font>Joey Dumont</font></font> <div><br> </div> <div>Technical Advisor, Knowledge, Information, and Technology Services</div> <div>National Research Council Canada / Governement of Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tel: 613-990-8152 / Cell: 438-340-7436</div> <div><br> </div> <div>Conseiller technique, Services du savoir, de l'information et de la technologie</div> <div>Conseil national de recherches Canada / Gouvernement du Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436</div> </div> </div> <div style="color:rgb(33,33,33)"> <hr style="display:inline-block; width:98%"> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663divRplyFwdMsg" dir="ltr"> <font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>><br> <b>Sent:</b> 01 November 2019 15:51<br> <b>To:</b> Dumont, Joey<br> <b>Cc:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] Issues with using S3 storage when running pulp on Amazon EC2 (pulp3)</font> <div> </div> </div> <div> <div dir="ltr">Unfortunately I don't know of a good way to debug the problem other than to dig into the code. If you want to debug from the Pulp code, you could stick a debugger in the artifact saver stage: <div><br> </div> <div><a href="https://github.com/pulp/pulpcore/blob/2203fee1407738a4ddd8e644fcbc741aab0bca63/pulpcore/plugin/stages/artifact_stages.py#L179-L200" target="_blank">https://github.com/pulp/pulpcore/blob/2203fee1407738a4ddd8e644fcbc741aab0bca63/pulpcore/plugin/stages/artifact_stages.py#L179-L200</a></div> <div><br> </div> <div>What I would probably do though is stick a debug statement here in django-storages to see what params it's passing to boto3:</div> <div><br> </div> <div><a href="https://github.com/jschneier/django-storages/blob/0ab2b1e3efd2bcaf0f24540a718993acc7742d9b/storages/backends/s3boto3.py#L511" target="_blank">https://github.com/jschneier/django-storages/blob/0ab2b1e3efd2bcaf0f24540a718993acc7742d9b/storages/backends/s3boto3.py#L511</a><br> </div> <div><br> </div> <div>You can see the location of django-storages with `pip show django-storages`.</div> <div><br> </div> <div>Sorry I don't have a better answer for you. Perhaps this is something we can improve in the future. Also, I'd be curious as to what the issue is as it sounds like everything should work in theory.<br clear="all"> <div> <div dir="ltr"> <div dir="ltr"> <div> <div dir="ltr"> <div> <div dir="ltr"> <div><br> </div> <div>David<br> </div> </div> </div> </div> </div> </div> </div> </div> <br> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Fri, Nov 1, 2019 at 2:26 PM Dumont, Joey <<a href="mailto:Joey.Dumont@nrc-cnrc.gc.ca" target="_blank">Joey.Dumont@nrc-cnrc.gc.ca</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex"> <div dir="ltr" style="font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif"> <p>I've installed the latest pulp3 using the Ansible installer using the following playbook:<br> </p> <p><br> </p> <div></div> <div> <div>---</div> <div>- hosts: mirrors</div> <div> vars:</div> <div> prereq_pip_packages:</div> <div> - django-storages</div> <div> - boto3</div> <div> pulp_use_system_wide_pkgs: True</div> <div> pulp_default_admin_password: !vault |</div> <div> $ANSIBLE_VAULT;1.1;AES256</div> <div> ...<br> </div> <div> pulp_settings:</div> <div> secret_key: !vault |</div> <div> $ANSIBLE_VAULT;1.1;AES256</div> <div> ...<br> </div> <div> default_file_storage: 'storages.backends.s3boto3.S3Boto3Storage'</div> <div> aws_storage_bucket_name: 'xxx-pulp-storage'<br> </div> <div> aws_s3_region_name: 'ca-central-1'</div> <div> aws_s3_addressing_style: "path"</div> <div> media_root: '/pulp3/'</div> <div> pulp_install_plugins:</div> <div> pulp-file: {}</div> <div> pulp-rpm:</div> <div> prereq_role: "pulp.pulp_rpm_prerequisites"</div> <div> # pulp-docker: {}</div> <div> roles:</div> <div> - pulp-database</div> <div> - pulp-workers</div> <div> - pulp-resource-manager</div> <div> - pulp-webserver</div> <div> - pulp-content</div> <div> environment:</div> <div> DJANGO_SETTINGS_MODULE: pulpcore.app.settings</div> <div><br> </div> <div>I also set up an RPM repo that uses S3 for storage. However, when I try to sync, I get an AccessDenied error. I know the instance profile is correct, as I can upload objects from that instance using both the AWS CLI and Boto3 without specifying credentials. <br> </div> <br> How can I debug this further? Is there a way for me know what parameters are passed to the put_object boto3 call by the sync task?<br> </div> <div><br> </div> <div>Cheers,<br> </div> <p><br> </p> <p><br> </p> <div id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663gmail-m_-2646072161816306239Signature"> <div name="divtagdefaultwrapper"><font><font>Joey Dumont</font></font> <div><br> </div> <div>Technical Advisor, Knowledge, Information, and Technology Services</div> <div>National Research Council Canada / Governement of Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663gmail-m_-2646072161816306239NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tel: 613-990-8152 / Cell: 438-340-7436</div> <div><br> </div> <div>Conseiller technique, Services du savoir, de l'information et de la technologie</div> <div>Conseil national de recherches Canada / Gouvernement du Canada</div> <div><a href="mailto:joey.dumont@nrc-cnrc.gc.ca" id="gmail-m_6731890020120963154gmail-m_3096176027992824979gmail-m_5589638655153726663gmail-m_-2646072161816306239NoLP" target="_blank">joey.dumont@nrc-cnrc.gc.ca</a> / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436</div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> <br clear="all"> <div><br> </div> -- <br> <div dir="ltr"> <div dir="ltr"> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-weight:bold; margin:0px; padding:0px; font-size:14px; text-transform:capitalize"> <span>Mike</span> <span>DePaulo</span><span style="text-transform:uppercase; color:rgb(170,170,170); margin:0px"></span></p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:12px; margin:0px 0px 4px; text-transform:capitalize"> He / Him / His</p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:12px; margin:0px; text-transform:capitalize"> <span>Service Reliability Engineer, Pulp</span></p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; margin:0px 0px 4px; font-size:12px"> <a href="https://www.redhat.com/" target="_blank" style="color:rgb(0,136,206); margin:0px">Red Hat</a></p> <div style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:medium; margin-bottom:4px"> </div> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; margin:0px; font-size:12px"> <span>IM: <span>mikedep333</span></span></p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:12px; margin:0px"> <span>GPG: 51745404</span></p> <div style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:medium; margin-top:12px"> <table border="0"> <tbody> <tr> <td width="100px"><a href="https://www.redhat.com/" target="_blank"><img width="90" height="auto" src="https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9ddafd5b2f1836854d7416a/Logo-RedHat-Email.png"></a></td> </tr> </tbody> </table> </div> </div> </div> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> </blockquote> </div> <br clear="all"> <div><br> </div> -- <br> <div dir="ltr" class="gmail_signature"> <div dir="ltr"> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-weight:bold; margin:0px; padding:0px; font-size:14px; text-transform:capitalize"> <span>Mike</span> <span>DePaulo</span><span style="text-transform:uppercase; color:rgb(170,170,170); margin:0px"></span></p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:12px; margin:0px 0px 4px; text-transform:capitalize"> He / Him / His</p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:12px; margin:0px; text-transform:capitalize"> <span>Service Reliability Engineer, Pulp</span></p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; margin:0px 0px 4px; font-size:12px"> <a href="https://www.redhat.com/" target="_blank" style="color:rgb(0,136,206); margin:0px">Red Hat</a></p> <div style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:medium; margin-bottom:4px"> </div> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; margin:0px; font-size:12px"> <span>IM: <span>mikedep333</span></span></p> <p style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:12px; margin:0px"> <span>GPG: 51745404</span></p> <div style="color:rgb(0,0,0); font-family:RedHatText,sans-serif; font-size:medium; margin-top:12px"> <table border="0"> <tbody> <tr> <td width="100px"><a href="https://www.redhat.com/" target="_blank"><img width="90" height="auto" src="https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9ddafd5b2f1836854d7416a/Logo-RedHat-Email.png"></a></td> </tr> </tbody> </table> </div> </div> </div> </div> </div> </body> </html>