For your consideration: Secondary Architectures in Fedora

[David Woodhouse]

On Wed, 2007-05-30 at 05:23 -0400, Jakub Jelinek wrote:
> > > Otherwise, why bother making a distinction at all?
> > 
> > A question which had occurred to me also. We seem to be trying to
> > 'solve' a problem which hasn't actually been demonstrated to exist yet.
> > 
> > For building (and scheduling) actual releases, there may be some point
> > in making the distinction. For the routine package builds, it seems
> > unnecessary.
> 
> There is a problem, while the ppc{,64} koji buildboxes aren't the slowest
> thing in the world, in my experience they are 25% to 100%
> slower than the i?86/x86_64 boxes (I'm always waiting on them for gcc/glibc
> etc. builds) 

Yeah, I'm often confused by that. My quad G5 is a _lot_ faster than
whatever's in our build system.

> and with addition of sparc{,64} that will be still orders of
> magnitude slower (not sure what hardware is planned for koji sparc* builds,
> but even 8way UltraII is horribly slow).  

It would perhaps be useful if Spot's document actually started with a
statement of the problem it was trying to solve. The one you put forward
seems to be the only real problem that exists, and the existing proposal
seems rather a convoluted and suboptimal way to address that.

> So for secondary arches we really need async builds instead of sync builds.

Most of what I build seems to be the kernel, and that's _always_ just
been 'fire and forget', because it takes so long. Perhaps I'm just used
to it, but it just doesn't seem to be that much of a problem. The speed
of the build system only ever really bothers me when I need to build
more than one package in succession, and the second package is to build
against the first. Is that really the common case?

Even if that case _is_ common enough for us to want to optimise for it,
the proposal is _very_ suboptimal for it anyway. Mostly when I've
encountered that situation, it's been building bluez-libs followed by
bluez-utils. I have to wait until the bluez-libs build is complete,
before I start the bluez-utils build.

But if the bluez-libs build were to appear complete when it isn't
actually finished, and then I submit the bluez-utils build which needs
it -- what happens on the architectures which haven't yet finished
building bluez-libs? Maybe if we're lucky they'll do the builds one
after the other and work by luck -- but most of the time either they'll
fail, or silently build against the older version of the libs. That's
_much_ worse than the existing situation, where I just have to wait a
little longer for the original build to complete.

It would be nice if I could submit them both together, and let the build
system handle building one after the other. And that shouldn't be
impossible to achieve...

$ make -C bluez-libs/devel build
 job id = 54321
$ make -C bluez-utils/devel DEPENDS_ON_JOB=54321 build
 job id = 54322

We can do asynchronous builds that way -- you just don't "commit" the
package and allow it into the real repo until it's finished on all
architectures, but it can still be available for subsequent builds in
koji on the same architecture before then. Obviously those subsequent
dependent builds can't be "committed" until both the original and the
dependent packages have finished on all architectures. And obviously if
a build fails, you have to 'roll back' and fail any of the dependent
builds which were using its output.

That way, you get the builds out of the faster architectures (and can
download them from koji and test them) as quickly as is physically
possible -- but we still don't let any build through to rawhide
until/unless it actually built everywhere.

If this is the problem you're trying to address, you _certainly_ don't
need to allow builds to fail silently on some architectures and yet
still proceed into the repo without maintainer attention.

-- 
dwmw2