[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] pulp v1 vs pulp v2 rpm repo sync times



Touch base with Randy. He tweaked a bunch of those numbers before v2 released and should be able to point you to the best places to start playing with.

On 02/22/2013 08:12 PM, Mike McCune wrote:
I looked at the list and didn't see any threads around a noticeable
difference in syncing content with pulp v1 vs v2 and wanted to start a
general thread on the performance differences.  I searched in BZ for
existing bugs around this subject but couldn't see anything.

== Test Conditions ==

hardware: 2 RHEL6.4 VMs with 1 VCPU, 2G RAM, 10G disk

Just some quick tests on 2 VM's I have here at my house.  Both are
identical, running RHEL6.4.  RPM versions:

# rpm -q pulp
pulp-1.1.15-1.el6cf.noarch

# rpm -q pulp-server
pulp-server-2.0.8-1.noarch

both are 'vanilla' installs, no Katello or anything non standard.  The
repo is public here:  http://mmccune.fedorapeople.org/repos/medium/ but
I brought it down local to remove any network from the test to a host on
my home network, 172.31.1.103

the repo is ~150M with 2k packages.

== Pulp V1 ==

[root g-02221312 ~]# pulp-admin repo create --id=test
--feed=http://172.31.1.103/medium/
Successfully created repository [ test ]

[root g-02221312 ~]# time pulp-admin repo sync --id=test -F
Sync for repository test started
Sync: Finished
2002/2002 new items downloaded
0/2002 existing items processed

Item Details:
RPMs: 2002/2002


TIME:  1m18.686s

== Pulp V2 ==

# pulp-admin -u admin -p admin  rpm repo create --repo-id=test
--feed=http://172.31.1.103/medium/
..

# time pulp-admin -u admin -p admin  rpm repo sync run --repo-id=test
+----------------------------------------------------------------------+
                     Synchronizing Repository [test]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
[..snip...]
... completed

TIME:  3m12.394s

So I'm hoping there is some simple tuning that can be done that can
improve this around 3X time increase in speeds between syncing the same
repo in V1 -> V2.  On much larger repos we have seen the sync times get
even worse between the 2 versions.

My first thought would be that settings in server.conf should get tweaked:

[tasks]
concurrency_threshold: 9
dispatch_interval: 0.5
archived_call_lifetime: 48
consumer_content_weight: 0
create_weight: 0
publish_weight: 1
sync_weight: 2

or should I specify the --num-threads when creating the repo?

Mike


--
Jay Dobies
Freenode: jdob @ #pulp
http://pulpproject.org


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]