[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] pulp v1 vs pulp v2 rpm repo sync times

On 02/25/2013 12:03 PM, Jay Dobies wrote:
To give an example of a larger repo with latency, the RHEL 6Server
x86_64 repo:

Pulp V1 (i believe 4 threads):  108 minutes
Pulp V2 with 4 threads: 192 minutes
Pulp V2 with 1 thread:  300 minutes


Few points...

Looks like the analysis for 4 threads v. 1 was incorrect. We'll address
this in some capacity for 2.1.

The simple fact is that it's just plain going to be slower than v1 for a
bit. In v1, Pulp was acting largely as a web interface to a yum repo on
disk. In v2, the paradigm has significantly changed. Grinder, however,
did not change. So what we're dealing with is grinder functioning one
way, v2 advocating a different model, and a bunch of glue in between them.

We've been investigating this since really the start of the year. Part
one was us measuring different download approaches in Python. We settled
on one and implemented a chunk that will handle concurrent downloads
smoothly. Part two is taking place this sprint as we start to use this
new approach to handle a yum repository instead of yum itself, which
admittedly has always been a bit of a square peg, circle hole situation.
I'm not saying the decision was wrong, but at the end of the day the new
approach should fit cleaner.

Since you guys are set up to run these tests easily, can I ask you to
take it past 4 threads and see what you find? There has to be a place
where the increases are less significant than you saw from 1 to 4, but I
am curious just how far we should take it.


= low latency pulp v2 syncs =

1 thread   : 3m10s
4 threads  : 2m20s
8 threads  : 2m18s
16 threads : 2m16s
32 threads : 2m19s

= high latency pulp v2 syncs =

1 thread   : 25m
4 threads  : 7m
8 threads  : 5m
16 threads : 5m
32 threads : 5m

seems like 8 is around the right sweet spot.


Mike McCune
mmccune AT redhat.com
Red Hat Engineering       | Portland, OR
Systems Management        | 650-254-4248

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]