[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] pulp v1 vs pulp v2 rpm repo sync times



To give an example of a larger repo with latency, the RHEL 6Server
x86_64 repo:

Pulp V1 (i believe 4 threads):  108 minutes
Pulp V2 with 4 threads: 192 minutes
Pulp V2 with 1 thread:  300 minutes

-Justin

Few points...

Looks like the analysis for 4 threads v. 1 was incorrect. We'll address this in some capacity for 2.1.

The simple fact is that it's just plain going to be slower than v1 for a bit. In v1, Pulp was acting largely as a web interface to a yum repo on disk. In v2, the paradigm has significantly changed. Grinder, however, did not change. So what we're dealing with is grinder functioning one way, v2 advocating a different model, and a bunch of glue in between them.

We've been investigating this since really the start of the year. Part one was us measuring different download approaches in Python. We settled on one and implemented a chunk that will handle concurrent downloads smoothly. Part two is taking place this sprint as we start to use this new approach to handle a yum repository instead of yum itself, which admittedly has always been a bit of a square peg, circle hole situation. I'm not saying the decision was wrong, but at the end of the day the new approach should fit cleaner.

Since you guys are set up to run these tests easily, can I ask you to take it past 4 threads and see what you find? There has to be a place where the increases are less significant than you saw from 1 to 4, but I am curious just how far we should take it.


--
Jay Dobies
Freenode: jdob @ #pulp
http://pulpproject.org


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]