[Pulp-list] pulp v1 vs pulp v2 rpm repo sync times
Steven Roberts
strobert at strobe.net
Mon Mar 25 20:22:10 UTC 2013
On Mon, Feb 25, 2013 at 03:03:40PM -0500, Jay Dobies wrote:
>
> >To give an example of a larger repo with latency, the RHEL 6Server
> >x86_64 repo:
> >
> >Pulp V1 (i believe 4 threads): 108 minutes
> >Pulp V2 with 4 threads: 192 minutes
> >Pulp V2 with 1 thread: 300 minutes
> >
> >-Justin
>
> Few points...
>
> Looks like the analysis for 4 threads v. 1 was incorrect. We'll
> address this in some capacity for 2.1.
>
> The simple fact is that it's just plain going to be slower than v1
> for a bit. In v1, Pulp was acting largely as a web interface to a
> yum repo on disk. In v2, the paradigm has significantly changed.
> Grinder, however, did not change. So what we're dealing with is
> grinder functioning one way, v2 advocating a different model, and a
> bunch of glue in between them.
>
> We've been investigating this since really the start of the year.
> Part one was us measuring different download approaches in Python.
> We settled on one and implemented a chunk that will handle
> concurrent downloads smoothly. Part two is taking place this sprint
> as we start to use this new approach to handle a yum repository
> instead of yum itself, which admittedly has always been a bit of a
> square peg, circle hole situation. I'm not saying the decision was
> wrong, but at the end of the day the new approach should fit
> cleaner.
>
> Since you guys are set up to run these tests easily, can I ask you
> to take it past 4 threads and see what you find? There has to be a
> place where the increases are less significant than you saw from 1
> to 4, but I am curious just how far we should take it.
FYI,
0.26-beta with the pull requests from bug 873313 is looking a LOT
better.
https://bugzilla.redhat.com/show_bug.cgi?id=873313
Just did a fresh rhels6 (current so 10K rpms) and it finished at 39
minutes.
Steve
More information about the Pulp-list
mailing list