Extras Rebuild?

[Michael Schwendt]

On Sat, 21 May 2005 18:42:40 -0700, Toshio Kuratomi wrote:

> [...]  But we have a bunch of broken packages in the devel repository that
> need dealing with some way before FC4 is released.  And we have a bunch of
> packages in unknown state because they haven't been rebuilt since FC3.

I believe it's the same set of packages: whatever failed to rebuild on
April 7th exists in the repository as a copied FC3 binary. Is this a wrong
assumption?

> The major problem in my mind being that previous rebuilds didn't result in
> bugzilla bugs being filed.

Right. But notice that we imported Seth's build failure report into the
Wiki for tracking purposes, and later on I filed a bug report about every
package on that list. The FE4Target tracker depends on all those bugs
plus other issues, such as crashes.

> The packages that are currently out of sync,
> imported to the devel tree but not rebuilt, etc aren't all being tracked so
> it takes effort for a non-maintainer (or maintainer of many (too many?)
> packages to figure out what stll needs fixing.

That's why we need a roadmap. It's the maintainer's responsibility to
(re)build packages in preparation for FE4.

Basically, for each package in the devel repo, which is out-of-sync with
devel CVS, somebody would need to make sure that it is really the right
version that shall be built and released. It's the packager who would tag
that version and request a build. I mean, we don't want to build something
without the "okay" from the package owner and possibly create a situation
where a released binary must be pulled from the repo because it was a
premature version upgrade that wasn't meant to be built.

> If it's community members who are merging fixes, then something that would
> help me, as a community member, to figure out which packages that I care
> about are in need of work, would be a complete record of what packages have
> failed rebuild.  This info isn't in bugzilla.

It is.

Or let me ask a question: what other packages should be added to
bug 157183?

> It isn't in the build logs.
> It *is* in the repository but isn't easy to extract:
>   Is the package available on all architectures and has been rebuilt at a
>   sufficiently recent date that all its buildrequirements had been updated
>   to a final enough version.

That assumes that [minor] version upgrades introduced ABI/API
incompatibility without the package owner paying attention to it. Can you
provide any examples?

> > Bill Nottingham some days ago posted a list of Extras packages to
> > maintainers-list, which need a rebuild, because of version mismatches on
> > the three archs we build for. We should assume that package maintainers
> > have noticed that list.
> > 
> But it wasn't.  Or maintainers don't all have enough time to work on them.

"Bump release, make tag, make build" is all that would be needed.

> We need to have these filed in bugzilla and become a dependency of the
> FE4Tracker bug so the community can take a look at these problems.

Are these due to "problems"? I still think most of the version mismatches
are due to missing powerpc build support earlier on.

For the noarch cases, packages have been copied over. A series of
other packages has been rebuilt for all archs meanwhile.

> My problem with the way FE4 stands is that we haven't recorded the status of
> our packages.  So where do we currently stand?  Which packages from Bill's
> list are still a problem?

A re-run of the script, which created that list, should tell.

> Which packages pulled down from the yum 
> repository will work and which will not?

Again, we do have package maintainers, don't we?  Also, there are fine
community people reporting broken packages in bugzilla. We should not
disappoint users, who use our packages actually.

> Which packages have been
> compiled against a sufficiently new version of FC4 and which have been built
> with older versions that A) will evoke bugs when run on FC4 or B) won't
> rebuild against FC4.

Same here as above. Unattended mass rebuilds should not be expected to fix
broken packages magically. They are unlikely to create good builds for
packages, which simply miscompile in some way. It really needs a package
developer to take care of a package (and e.g. verify whether version
checks done by a configure script still give good results with FCn+1).

> > In addition to that, the FE4Target tracker contains a list of broken
> > packages, which failed to rebuild for FC4 or which fail due to serious
> > defects. These are issues a mass rebuild is unlikely to fix. As some of us
> > have attached patches to some of the bugs, how long do we wait for a
> > reaction from the package owner?
> > 
> Email the owner.  Try to get one other person to review the change.
> Eventually merge yourself.  We need to evolve some policy here.  To be
> useful for FC4, though, we need to come up with something sane quickly
> as we need to merge those fixes and test them.

The thing here is, bugzilla tickets are _assigned to_ the package owner
and _mailed to_ the package owner. So, that is a way already to address
the package owner. Assume that package owners do see your bug report and
your attachments (if any) and could add a comment like "go ahead with
the fix" or set status to ASSIGNED to give you some feedback.

> Side note: Although I've been browsing the FETracker bugs to find things to
> fix, it strikes me as the wrong way to address FE (as we have it setup.)  If
> individual maintainers are responsible for bringing the packages to
> readiness, then the packages will make it into the FE4 repository when
> they're ready rather than the FE4 repository will be released when the FE4
> packages are ready.  This is in contrast to the FC4 Trackers which, in
> theory, describe all the changes that must occur before FC4 can ship.

This tracker is called FE4Target with good reason. It's not called
FE4Blocker, as we don't have a release date, because Fedora Extras has
been a rolling release so far.

It _could_ influence when to make available the current devel repository
in an "extras/4" channel.  That would not be fair, though, since some
contributors make sure their packages are ready, and other packagers
perhaps depend on upstream projects catching up with API changes.

> For Extras in its present incarnation, it makes more sense to have
> per-package trackers.  If a package has remaining issues, it doesn't
> make it into the FEx repository.

That is the reason why I didn't like it when the public repository was
filled with copies of FC3 binaries without a guide for the packagers when
to rebuild their packages. We should have built Fedora Extras Development
from scratch, since the only thing we depend on is Fedora Core (well,
except for future programming language packages which need bootstrapping).

> Among other things, this means build failures should go in bugzilla (perhaps
> the buildsystem can do this just as it emails out failures.  The problem is
> a bit harder than email as we probably want to update older build failures
> rather than create new ones and close out old ones when the build succeeds.
> Perhaps an approach such as having a buildsystem bugzilla account be the
> submitter or a BUILDFAIL keyword.)  The bugzilla tickets can be used to
> track the current status of a package.

Here once more: if Fedora Extras wants to scale, every package should have
at least one person taking care of it. Build requests are done by human
beings. Build status reports are mailed back to human beings. It is
assumed that the person, who requests a build, does something useful with
a build failure log and e.g. decides whether it should be tracked in
bugzilla, because it might take some time until it is fixed.

> > I consider mass rebuilds a bottom-up "brute force" approach to try fixing
> > something where no top-down approach is available. More helpful in my
> > opinion would be, at least, to try coming up with a roadmap for Fedora
> > Extras and then find out where we encounter any difficulties.
> > 
> I agree.
> 
> Do we have a top-down approach tha will allow us to get FE4 out
> when FC4 hits the streets or do we have nothing better than the brute force
> approach at this point?
> 
> I agree with your assessment (pretty much your entire message) of where we
> want to be.  My question is can we get there in time for FC4 or do we need a
> stop gap for FC4 and something that is designed to work well for FC5-devel?

You mentioned "overworked packagers". If you think you know what you are
doing and a packager seems overworked, I doubt he minds if you applied a
patch yourself to fix a package.

For bigger changes, such as version upgrades, an explicit "okay" from the
package owner should be asked for.  If such a question is not answered
after several weeks, with FC4 being very close, that could be an
indication that the packager is occupied with other stuff and that the
package might benefit from another person taking care of it.

We should not complicate matters too much. If the most simple
communication fails, we have bigger problems than just temporarily failed
builds of some packages.

-- 
Fedora Core release Rawhide (Rawhide) - Linux 2.6.11-1.1323_FC4
loadavg: 1.10 1.43 1.75