<div dir="ltr">I tested and confirmed that the files are not being deleted. Mind opening a bug for that?<div><br></div><div><div>For the failed chunk uploads, httpie is timing out. You can set a higher timeout (like you did) or use smaller chunks.</div><div><br></div><div><div><div dir="ltr" class="m_-8561388604270746870gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>I looked into why artifact creation is failing for files < 2GB. The reason is that it's taking too long to calculate the checksums. There are 6 checksum types and each one takes about 4-8 seconds from the command line in my test environment. Calculating the digests in Python seems to add about 1-2 seconds. The default timeout in gunicorn is 30 seconds:</div><div><br></div><div>Jul 05 14:21:56 pulp3 gunicorn[13691]: [2019-07-05 14:21:56 +0000] [13691] [CRITICAL] WORKER TIMEOUT (pid:29843)<br>Jul 05 14:21:57 pulp3 gunicorn[13691]: [2019-07-05 14:21:57 +0000] [30031] [INFO] Booting worker with pid: 30031<br></div><div><br></div><div>You can raise this timeout or also you can pass in the checksums when creating the artifact[0]. I think the best solution though might be to make artifact creation a background task.</div><div><br></div><div>[0] http POST :24817/pulp/api/v3/artifacts/ upload=$UPLOAD sha256=abc...</div><div><br></div><div>David<br></div></div></div></div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 5, 2019 at 9:08 AM Juan Cabrera <<a href="mailto:juan.cabrera@unamur.be" target="_blank">juan.cabrera@unamur.be</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hi David</p>
<p>This morning I made a test sequence and open a ticket</p>
<p><a class="gmail-m_-8561388604270746870gmail-m_-4651838809158492575moz-txt-link-freetext" href="https://pulp.plan.io/issues/5087" target="_blank">https://pulp.plan.io/issues/5087</a><br>
</p>
<p>Yes, I see that the update file is deleted when the artifact is
created. <br>
</p>
<p>When I get the artifact creation error, the upload HREF is not
deleted, which could be normal as there was a error.</p>
<p>But when I clean all the server uploads using API <br>
</p>
<p><tt>for u in $(http $PORT/pulp/api/v3/uploads/ | jq -r
'.results[] | ._href'); do</tt><tt><br>
</tt><tt> echo $u</tt><tt><br>
</tt><tt> http DELETE $PORT$u</tt><tt><br>
</tt><tt>done</tt><br>
</p>
<p>The files are not deleted.</p>
<p>Juan<br>
</p>
<div class="gmail-m_-8561388604270746870gmail-m_-4651838809158492575moz-cite-prefix">On 5/07/19 12:44, David Davis wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Juan,
<div><br>
</div>
<div>There is in fact a 2GB limit currently on artifact size. I
consider this a bug and I filed this issue:</div>
<div><br>
</div>
<div><a href="https://pulp.plan.io/issues/4998" target="_blank">https://pulp.plan.io/issues/4998</a></div>
<div><br>
</div>
<div>The file in /var/lib/pulp/upload should be deleted once
it's imported as an artifact. I'm guessing it's maybe not
happening since the server is throwing an error.<br clear="all">
<div>
<div dir="ltr" class="gmail-m_-8561388604270746870gmail-m_-4651838809158492575gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div><br>
</div>
<div>David<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jul 4, 2019 at 12:31
PM Juan Cabrera <<a href="mailto:juan.cabrera@unamur.be" target="_blank">juan.cabrera@unamur.be</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
In my previous mail I forget to say that I updated the Pulp
version to<br>
3.0.0rc3<br>
<br>
pulp_source_dir: "git+<a href="https://github.com/pulp/pulpcore.git@3.0.0rc3" rel="noreferrer" target="_blank">https://github.com/pulp/pulpcore.git@3.0.0rc3</a>"<br>
pulp_plugin_source_dir:<br>
"git+<a href="https://github.com/pulp/pulpcore-plugin.git@0.1.0rc3" rel="noreferrer" target="_blank">https://github.com/pulp/pulpcore-plugin.git@0.1.0rc3</a>"<br>
pulp_install_plugins:<br>
pulp-rpm:<br>
app_label: "rpm"<br>
source_dir: "git+<a href="https://github.com/pulp/pulp_rpm.git@3.0.0b4" rel="noreferrer" target="_blank">https://github.com/pulp/pulp_rpm.git@3.0.0b4</a>"<br>
<br>
Juan<br>
<br>
_______________________________________________<br>
Pulp-list mailing list<br>
<a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote>
</div>
</blockquote>
<div class="gmail-m_-8561388604270746870gmail-m_-4651838809158492575moz-signature">-- <br>
<div style="font-family:Verdana,Helvetica,Arial,sans-serif;font-size:8pt;font-weight:normal;color:rgb(97,99,101)"> <img src="cid:16bc260db83f24ddb3f1" style="display: block; margin: 1em 0em;">
<p style="margin:1em"> <span style="font-weight:bold">Juan
CABRERA</span><br>
<span style="color:rgb(105,190,40)">Correspondant informatique</span><br>
<span>Département de Mathématiques</span> </p>
<p style="margin:1em">
<span style="font-weight:bold">T.</span> 081724919<br>
<a style="color:rgb(105,190,40)" href="mailto:juan.cabrera@unamur.be" target="_blank">juan.cabrera@unamur.be</a><br>
<a style="color:rgb(105,190,40)" href="http://staff.unamur.be/jbcabrer" target="_blank">http://staff.unamur.be/jbcabrer</a>
</p>
<p style="margin:1em"> <span style="font-weight:bold;font-size:8pt">Université de Namur ASBL</span><br>
<span>Rue de Bruxelles 61 - 5000 Namur</span><br>
<span>Belgique</span><br>
</p>
<p style="margin:1em;font-size:6pt;color:rgb(105,190,40)"> <span>Let’s
respect the environment together.</span><br>
<span style="font-weight:bold">Only print this message if
necessary!</span><br>
</p>
</div>
</div>
</div>
</blockquote></div>