[Pulp-list] /var/cache/pulp filling up?

Michael Hrivnak mhrivnak at redhat.com
Thu Mar 17 16:53:08 UTC 2016

This appears to be a regression in 2.7.

In 2.6 (and earlier), at the completion of each rpm download, the file
would be moved from its temporary location to the permanent location. That
means for any given sync, you would never have more files in that temporary
location than you did concurrent downloads. Using the default setting of 5
concurrent downloads, that's a maximum of 5 rpms. No problem so far.

This change turned the "move" into a "copy":


Now all of the rpm files pile up, and they don't get removed until the end
of the task. On quick glance, it appears that 2.8 suffers the same problem,
although the code has been refactored.

If you don't mind filing a bug report, it should be an easy fix: likely
just adding a delete call after the copy.



On Thu, Mar 17, 2016 at 11:39 AM, Randy Barlow <rbarlow at redhat.com> wrote:

> On Wednesday, March 16, 2016 12:23:48 AM EDT Matthew Madey
> wrote:
> > I think it's failing to remove the files when they are done. Currently
> > there are 2G of RPM's in the cache directory, most of them are
> 0bytes. I'm
> > syncing with the RHEL7 repository, and I'm seeing every package in
> the
> > repository in the cache directory, however, once it had filled /var,
> the
> > remaining RPM files were 0 bytes. Is there a way to limit to how
> much
> > content is able to be staged in the cache directory before being
> published
> > to /var/lib/pulp?
> Hello again Matthew!
> I believe that the sync task downloads all the RPMs into /var/cache/
> pulp and then moves them into /var/lib/pulp as a later step. Several of
> us developers want to redesign our plugin API to support stream
> processing, but in the meantime you will need enough storage in /var/
> cache/pulp to support all the bytes for an entire sync. I don't know off
> the top of my head how large distro repos are, but I'd say they're
> usually on the scale of 10s of GBs.
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20160317/8d9bfed5/attachment.htm>

More information about the Pulp-list mailing list