Sugg: Improve for mock (caching)

[Enrico Scholz]

skvidal at phy.duke.edu (seth vidal) writes:

>> The speedup gained by caching is not (primarily) related to the
>> network connection but to the rpm installation and disk-IO. On the
>> fedora.us server we used caching also and the initial buildroot was
>> created in <1min (afair).
>> 
>> Local tests at that time showed, that the initial buildroot can be
>> created in 20 seconds when a tmpfs will be used instead of a real disk
>> with ext3.
>
> So the numbers I'm curious about that you may have:
> - remote repos - but lan-like connection times
> - local-disk repos - chroot creation - all on ext3
> - tmpfs targets for the chroot creation and repos
>
> do you have anything like that?

ok, a little bit late, but I implemented caching now so I can provide
some real numbers:

Traditional 'mock init' on a tmpfs takes

| Init buildroot...
|
| real    1m37.445s
| user    1m15.329s
| sys     0m8.009s

tar'ing this filesystem, and extracting it takes

| Extracting cached buildroot...
| 
| real    0m1.266s
| user    0m0.188s
| sys     0m0.944s


Both operations are resulting into the same root-fs; the executed code
was:

| mock=( env http_proxy= mock --resultdir="$results" -r "$MOCK_ROOT" )
| mock_yum=( env http_proxy= /usr/sbin/mock-helper yum --installroot "$BUILD_DIR" )
|
| function initBuildroot() {
|     "${mock[@]}" clean >/dev/null
| 
|     if test -e "$CACHE_NAME"; then
|         echo "Extracting cached buildroot..."
|         time /usr/sbin/mock-helper uncache "$MOCK_ROOT"
| 
|         echo "Updating metadata..."
|         "${mock_yum[@]}" check-update || :
|     else
|         lock
|         echo "Init buildroot..."
|         time "${mock[@]}" init >/dev/null
|         unlock
| 
|         echo "Cleaning up root..."
|         "${mock_yum[@]}" clean packages headers
|         echo "Creating cache..."
|         /usr/sbin/mock-helper cache "$MOCK_ROOT"
|      fi
| }

The 'cache' and 'uncache' operations of 'mock-helper' were added by me
and are simple 'tar' wrappers. The resulting tarball is uncompressed and
280MB sized, both the rootfs and the tarball are located in the tmpfs.

Environment:
  mock-machine:      Intel Celeron 2.80GHz, 1500MB RAM
  repository-server: PII 266MHz, 196MB RAM, http-access to repo (Apache2)
  100Mb/s LAN; 'wget' download rate between repo and mock machine appr. 7-8MB/s



Enrico