nothing from buildsys?

[Mike A. Harris]

Reuben Farrelly wrote:
> 
> 
> On 2/02/2006 11:39 p.m., Peter Robinson wrote:
> 
>>> Is buildsys still working on today's bake, or is there some other reason
>>> why no new rawhide today?
>>
>>
>> If there's a lot of rebuilds it sometimes doesn't appear until around
>> midday GMT time so it's probably still toiling away on the builds. I
>> saw on a blog somewhere that one of the redhat guys is playing with a
>> unified accross all platforms srpm repo so it could be due to that.
> 
> 
> Every time a nasty bug shows up in rawhide (like the one with yum and 
> python-sqlite from yesterday) I wonder if it would be possible or in 
> fact practical to have two builds a day or more to pick up rawhide changes.
> Is there a reason why only 1 a day is done - is it just historical?

Rawhide compose begins at 2:00am EST or so I'm told.  My understanding,
is that it is done at 2:00am because that is a time of day which the
buildsystems and infrastructure is usually the least busy, and has the
least number of engineers using it (if any), so it is not disruptive to
the regular flow of daily engineering that is happening.

If rawhide were to be composed every 12 hours instead, it would hit at
2:00am, and 2:00pm then, and most likely would lower everyone's
productivity horrendously, due to the increased load on the buildsystem
infrastructure.

> I'd also be curious to know what sort of machine is doing the actual 
> building, one would expect that given sometimes there are builds of gcc 
> and glibc on the go in one nightly build, it must be a seriously fast 
> piece of machinery to be able to produce so many binaries in a 
> reasonable amount of time.  Anyone who has tried compiling glibc from 
> scratch would testify how long that alone can take ;-)

The buildsystem infrastructure consists of numerous computers.  The
primary hub, is our infamous "porkchop", which is an SMP x86 box that
is the entranceway to the buildsystem and various other infrastructure.
Jobs submitted to the buildsystem (beehive) on porkchop, get enqueued
to be farmed out to all 7 architectures simultaneously, and the actual
builds occur on each architecture as a machine in the build pool is
free and available to accept a job.  All of the buildmachines are SMP
hardware with 2-8 CPUs (possibly 16 CPUs on occasion, not sure), with
whackloads of RAM.

While the whole setup is sometimes idle, more often than not, it is
very busy building whackloads of packages for Fedora, RHEL updates,
and running various automated tests and other stuff.  Tree composes,
and various other RELENG stuff also pounds the machines into the
ground, as do automated mass-rebuilds and other fun stuff.

Nonetheless, even with this much total computing power available,
due to the high load on the systems, it isn't always what you might
consider "fast" overall.  For example, I can build Xorg on my
personal workstation here at home in about 15 minutes completely,
however the same rpm takes anywhere from 1.5 hours to 3 or more
hours going through the buildsystem even when it's idle (usually
due to s390/s390x/ia64 taking forever to compile).

Anyhow, I figured I'd share a bit of mumbo-jumbo about the buildsystem,
as people often ask, and don't always get much info. ;)  If you've
got a 1.5Ghz CPU or faster, and install "ccache" from Fedora Extras,
your machine will build pretty much any package faster than our
buildsystem.  ;o)


-- 
Mike A. Harris  *  Open Source Advocate  *  http://mharris.ca
                       Proud Canadian.