[Pulp-list] pulp v1 vs pulp v2 rpm repo sync times

Steven Roberts strobert at strobe.net
Mon Mar 25 20:22:10 UTC 2013


On Mon, Feb 25, 2013 at 03:03:40PM -0500, Jay Dobies wrote:
> 
> >To give an example of a larger repo with latency, the RHEL 6Server
> >x86_64 repo:
> >
> >Pulp V1 (i believe 4 threads):  108 minutes
> >Pulp V2 with 4 threads: 192 minutes
> >Pulp V2 with 1 thread:  300 minutes
> >
> >-Justin
> 
> Few points...
> 
> Looks like the analysis for 4 threads v. 1 was incorrect. We'll
> address this in some capacity for 2.1.
> 
> The simple fact is that it's just plain going to be slower than v1
> for a bit. In v1, Pulp was acting largely as a web interface to a
> yum repo on disk. In v2, the paradigm has significantly changed.
> Grinder, however, did not change. So what we're dealing with is
> grinder functioning one way, v2 advocating a different model, and a
> bunch of glue in between them.
> 
> We've been investigating this since really the start of the year.
> Part one was us measuring different download approaches in Python.
> We settled on one and implemented a chunk that will handle
> concurrent downloads smoothly. Part two is taking place this sprint
> as we start to use this new approach to handle a yum repository
> instead of yum itself, which admittedly has always been a bit of a
> square peg, circle hole situation. I'm not saying the decision was
> wrong, but at the end of the day the new approach should fit
> cleaner.
> 
> Since you guys are set up to run these tests easily, can I ask you
> to take it past 4 threads and see what you find? There has to be a
> place where the increases are less significant than you saw from 1
> to 4, but I am curious just how far we should take it.

FYI, 

0.26-beta with the pull requests from bug 873313 is looking a LOT
better.
https://bugzilla.redhat.com/show_bug.cgi?id=873313

Just did a fresh rhels6 (current so 10K rpms) and it finished at 39
minutes.

Steve




More information about the Pulp-list mailing list