[Pulp-list] /var/cache/pulp filling up?

Michael Hrivnak mhrivnak at redhat.com
Tue Mar 22 15:13:06 UTC 2016


I filed a bug report and submitted a fix, which will appear in 2.8.1.

https://pulp.plan.io/issues/1789

Michael

On Thu, Mar 17, 2016 at 12:53 PM, Michael Hrivnak <mhrivnak at redhat.com>
wrote:

> This appears to be a regression in 2.7.
>
> In 2.6 (and earlier), at the completion of each rpm download, the file
> would be moved from its temporary location to the permanent location. That
> means for any given sync, you would never have more files in that temporary
> location than you did concurrent downloads. Using the default setting of 5
> concurrent downloads, that's a maximum of 5 rpms. No problem so far.
>
> This change turned the "move" into a "copy":
>
>
> https://github.com/pulp/pulp_rpm/commit/de1895ede2f24731dcf6f52321beaee6b71c15a4
>
> Now all of the rpm files pile up, and they don't get removed until the end
> of the task. On quick glance, it appears that 2.8 suffers the same problem,
> although the code has been refactored.
>
> If you don't mind filing a bug report, it should be an easy fix: likely
> just adding a delete call after the copy.
>
> https://pulp.plan.io/projects/pulp_rpm/issues/new
>
> Thanks,
> Michael
>
> On Thu, Mar 17, 2016 at 11:39 AM, Randy Barlow <rbarlow at redhat.com> wrote:
>
>> On Wednesday, March 16, 2016 12:23:48 AM EDT Matthew Madey
>> wrote:
>> > I think it's failing to remove the files when they are done. Currently
>> > there are 2G of RPM's in the cache directory, most of them are
>> 0bytes. I'm
>> > syncing with the RHEL7 repository, and I'm seeing every package in
>> the
>> > repository in the cache directory, however, once it had filled /var,
>> the
>> > remaining RPM files were 0 bytes. Is there a way to limit to how
>> much
>> > content is able to be staged in the cache directory before being
>> published
>> > to /var/lib/pulp?
>>
>> Hello again Matthew!
>>
>> I believe that the sync task downloads all the RPMs into /var/cache/
>> pulp and then moves them into /var/lib/pulp as a later step. Several of
>> us developers want to redesign our plugin API to support stream
>> processing, but in the meantime you will need enough storage in /var/
>> cache/pulp to support all the bytes for an entire sync. I don't know off
>> the top of my head how large distro repos are, but I'd say they're
>> usually on the scale of 10s of GBs.
>>
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20160322/e43aeae5/attachment.htm>


More information about the Pulp-list mailing list