<div dir="ltr"><div>That patch broke signed cloudfront URLs, as the S3 content-disposition query string has to be included in the URL which gets signed. Sigh. So, there's a PR upstream at <a href="https://github.com/jschneier/django-storages/pull/1004">https://github.com/jschneier/django-storages/pull/1004</a> which resolves that. PR 1003 also allows the signing key to work easily when passed in via env variable. There's a forked django-storages with both incorporated at <a href="https://github.com/Kong/django-storages/tree/release/kong-prod">https://github.com/Kong/django-storages/tree/release/kong-prod</a>, should someone else need to use that before upstream integrates the changes. It's fairly easy to just `pip3 install "django-storages[boto3] @ git+<a href="https://github.com/Kong/django-storages@release/kong-prod">https://github.com/Kong/django-storages@release/kong-prod</a>"` instead of the usual documented step. Hope that helps someone. :)</div><div><br></div><div>--Danny</div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Apr 16, 2021 at 11:35 AM Danny Sauer <<a href="mailto:danny.sauer@konghq.com" target="_blank">danny.sauer@konghq.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">FWIW, I worked around this by just patching pulp in our adjusted docker build.<br><a href="https://github.com/Kong/docker-pulp/blob/main/pulp-core/patches/content_parameter_filename_fix.patch" target="_blank">https://github.com/Kong/docker-pulp/blob/main/pulp-core/patches/content_parameter_filename_fix.patch</a><br><div><br></div><div>Upstream patch hasn't been submitted yet because I'm still scrambling to get this implemented before our current hosted provider goes away. Which is also why it took a week to share the workaround I had in place last week (and why my documentation PRs still don't have issues associated). :D</div><div><br></div><div>I know the project is preferring to go the Kube operator / Ansible route, but speaking of Docker and Kubernetes and a CDN, we do have a helm chart for this whole thing that I'm hoping we can open source soon as well. Someday...</div><div><br></div><div>--Danny</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 7, 2021 at 10:39 AM David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Interesting. Keep us posted.<br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><br></div><div>David</div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 6, 2021 at 9:37 PM Danny Sauer <<a href="mailto:danny.sauer@konghq.com" target="_blank">danny.sauer@konghq.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Thanks for following up. Yes, the query string *should* be there. I found this bug last week when I was looking in to it, though (basically, telling Django-storages to use cloudfront breaks the query string appending code). I'm back from away-from-keyboard vacation tomorrow, and should be able to get a some patches sent upstream. :)<div dir="auto"><div dir="auto"><br></div><div dir="auto"><a href="https://github.com/jschneier/django-storages/issues/997" target="_blank">https://github.com/jschneier/django-storages/issues/997</a><br></div><div dir="auto"><br></div><div dir="auto">--Danny</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 6, 2021, 2:07 PM David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Danny,<div><br></div><div>I don't know much about AWS logging but Pulp does set the filename in the response-content-disposition[0]. Could that be used to determine the filename for each request?</div><div><br></div><div>If not, I'm looking at the boto3 docs for get_object[1] to see if there's another parameter we could set to help you track the filename in requests but I'm seeing anything useful. My knowledge of s3 is a bit limited so if you have a suggestion how we can construct a request to S3 that would help you to track the filenames of requests to s3, I could probably look at how we could support it in Pulp 3.</div><div><br></div><div>[0] <a href="https://github.com/pulp/pulpcore/blob/f38f955425b185749b3c8d4d878a7e166cfc05b9/pulpcore/content/handler.py#L613-L614" rel="noreferrer" target="_blank">https://github.com/pulp/pulpcore/blob/f38f955425b185749b3c8d4d878a7e166cfc05b9/pulpcore/content/handler.py#L613-L614</a></div><div>[1] <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.get_object" rel="noreferrer" target="_blank">https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.get_object</a><br clear="all"><div><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><br></div><div>David</div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 30, 2021 at 10:43 AM Danny Sauer <<a href="mailto:danny.sauer@konghq.com" rel="noreferrer" target="_blank">danny.sauer@konghq.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I've got Pulp set up to serve all the content from S3 behind CloudFront. This works really well, except for a minor issue: the content URLs are all the UUIDs for artifacts, not, for example, the pretty name of the RPM being downloaded. That's an issue in my situation because we'd really like to generate download analytics using off-the-shelf tools which consume the AWS CDN standard log format.<div><br></div><div>My initial thought was that it might be easy to have the redirects include a query string in the generated URL which notes the original filename or relative path requested. But I don't have sufficiently developed Django skills to know the easiest way to do that (or if it's even reasonable to think that's easy). Using the content server's logs is another option, but I have some other content on the same S3 bucket which may not necessarily be reached solely through Pulp's content server, so that means two log locations, etc. If it was easy to make Django / Gunicorn log to an S3 bucket in a manner similar to Cloudfront, that might also be ok. Post-processing logs with a series of API calls to work out what artifact maps to what repository content would ideally be a last resort.</div><div><br></div><div>Anyone have some great insights which might help me out here? :) If it helps, I'm building my own Docker images which ultimately run in EKS. So patches / extra modules are an option, but I'd prefer to stay as close to vanilla upstream as possible with environment variable-based config adjustments.</div><div><br></div><div>Thanks.</div><div>--Danny</div></div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" rel="noreferrer" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://listman.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer noreferrer" target="_blank">https://listman.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div> </blockquote></div> </blockquote></div> </blockquote></div> </blockquote></div> </div>