[Pulp-list] Stress Testing Script to Create Hundreds of Repos

[ben.stanley at exemail.com.au]

As I have created a script to create my repos, and it seems to cause pulp
a lot of trouble, I thought I should share it here as a stress testing
example.

Please find attached CreateRHELPulpRepos.sh .

I suggest you DON'T use it on an existing system, as it will likely stomp
all over your repos.

I use it like this:
--------------------------------------------------------------------
# source ./CreateRHELPulpRepos.sh
# ConfigureRedHatWorkstationRepos
--------------------------------------------------------------------

Note that this script will create 278 repos.

While I run this script, I monitor:
1) The output of ConfigureRedHatWorkstationRepos
2) /var/log/pulp.log
3) nload (note that I have an incoming bandwidth limit set)

Sometimes I observe that a particular task gets stuck, and won't progress,
despite no other tasks making progress either. Lack of network load is the
easiest indicator to track. In this case, I have had to re-start pulp to
get around this.

Note that the ConfigureRedHatWorkstationRepos does not actually sync the
repos. It does create a sync schedule, but if you don't want to wait for
the weekend, try this:

--------------------------------------------------------------------
# source ./CreateRHELPulpRepos.sh
# SyncAllRepos
--------------------------------------------------------------------
This will sync all of the repos in pulp, one by one (provided nothing gets
stuck...).

If everything stuffs up and you want to start again, you might need this
thermonuclear sequence:
--------------------------------------------------------------------
# source ./CreateRHELPulpRepos.sh
# while true; do sleep 10; date; CancelAllRunningTasks; done  # <---
CTRL-C when no tasks have been deleted for a while.
# DeleteAllRepos
--------------------------------------------------------------------
The packages remain on disk, it is just that the repos and the metadata
are removed from mongodb. The repos can be re-created using
ConfigureRedHatWorkstationRepos. Only the metadata will be downloaded
again (however, this does take time).

I hope that this script proves to be useful for stress-testing pulp. If
this isn't enough, I can send you a few more functions to create several
hundred more repos, although I'm sure you can see how to add a few more
repos yourselves. Hint: s/workstation/server/ .

Ben Stanley.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CreateRHELPulpRepos.sh
Type: application/x-shellscript
Size: 8914 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20150504/2d9ca826/attachment.bin>