[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Pulp-list] pulp v1 vs pulp v2 rpm repo sync times



I looked at the list and didn't see any threads around a noticeable difference in syncing content with pulp v1 vs v2 and wanted to start a general thread on the performance differences. I searched in BZ for existing bugs around this subject but couldn't see anything.

== Test Conditions ==

hardware: 2 RHEL6.4 VMs with 1 VCPU, 2G RAM, 10G disk

Just some quick tests on 2 VM's I have here at my house. Both are identical, running RHEL6.4. RPM versions:

# rpm -q pulp
pulp-1.1.15-1.el6cf.noarch

# rpm -q pulp-server
pulp-server-2.0.8-1.noarch

both are 'vanilla' installs, no Katello or anything non standard. The repo is public here: http://mmccune.fedorapeople.org/repos/medium/ but I brought it down local to remove any network from the test to a host on my home network, 172.31.1.103

the repo is ~150M with 2k packages.

== Pulp V1 ==

[root g-02221312 ~]# pulp-admin repo create --id=test --feed=http://172.31.1.103/medium/
Successfully created repository [ test ]

[root g-02221312 ~]# time pulp-admin repo sync --id=test -F
Sync for repository test started
Sync: Finished
2002/2002 new items downloaded
0/2002 existing items processed

Item Details:
RPMs: 2002/2002


TIME:  1m18.686s

== Pulp V2 ==

# pulp-admin -u admin -p admin rpm repo create --repo-id=test --feed=http://172.31.1.103/medium/
..

# time pulp-admin -u admin -p admin  rpm repo sync run --repo-id=test
+----------------------------------------------------------------------+
                    Synchronizing Repository [test]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
[..snip...]
... completed

TIME:  3m12.394s

So I'm hoping there is some simple tuning that can be done that can improve this around 3X time increase in speeds between syncing the same repo in V1 -> V2. On much larger repos we have seen the sync times get even worse between the 2 versions.

My first thought would be that settings in server.conf should get tweaked:

[tasks]
concurrency_threshold: 9
dispatch_interval: 0.5
archived_call_lifetime: 48
consumer_content_weight: 0
create_weight: 0
publish_weight: 1
sync_weight: 2

or should I specify the --num-threads when creating the repo?

Mike
--
Mike McCune
mmccune AT redhat.com
Red Hat Engineering       | Portland, OR
Systems Management        | 650-254-4248


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]