Noarch subpackages: Upcoming Feature Freeze

[Toshio Kuratomi]

Mike Bonnet wrote:
> Toshio Kuratomi wrote:
>> Florian Festi wrote:
>>> Please add the packages you changed or plan to change to
>>> https://fedoraproject.org/wiki/Features/NoarchSubpackages/PackagesChanged
>>> Put the later in parenthesis. That way it will be easier to justify a
>>> release note entry and to argue that the Feature is (at least a bit)
>>> finished.
>>>
>> I explored what would be changed if we enabled a lenient-hash check on
>> these files I discovered these things:
>>
>> 1) Asking reviewers to check this is pretty hard.  I'll add the steps at
>> the end.
> 
> It's actually very easy to script given a few calls to the xmlrpc interface.
> 
So.... where's the recipe?  Give us a script that reviewers can run to
check this themselves.

>> 2) If reviewers are expected to do this, we need to get our user
>> interface changes to rpmdiff merged upstream.  rpmlint's rpmdiff can
>> only ignore timestamp.  Reviewers are going to want to have something
>> like lenient-hash as well.
>> 3) devhelp documentation should be marked as %doc
>> 4) It might be reasonable for --lenient-hash to not check
>> f.startswith('/usr/share')... I'm on the fence about this.
>> 5) Most of these would have checked fine even if we used a full hash
>> check instead of lenient
>> 6) I discoverd a package with architecture specific differences.
>> Luckily, it's just a problem in documentation.
>>
>> Check results with:
>> ./rpmdiff.mine -iS -iT --lenient-hash
>>
>> Script attached.
> 
> I noticed your lenient-hash changes special-case .pyc and .pyo files.

Yes.  From past discussions it seemed that the biggest concern was that
known good things would fail so I added the heuristic to keep that from
happening.

> It's this kind of special-casing that worries me.  What about other
> types of bytecode, Java, OCaml, haskell?  What about firmwares?  What
> about game datafiles?

Here's my thinking.  It's better to err on the cautious side and not let
through things that are broken while catching some things that are legal
in the net.  (I already presented .startswith('/usr/share') as another
heuristic, for instance) we can proceed to whitelist them.

noarch subpackages are an additional, optional feature after all, not
something that will cause the world to end if we can't do it.  There's a
lot of potential noarch subpackages that will pass the current filter...
all doc files.  Headers that actually are noarch.  All pure python.
Anything that doesn't change from build to build (where a lot of game
data falls).

Note: java byte code would fall under the '/usr/share' heuristic if we
chose to use it.  ocaml installs its byte code into %{_libdir} so it's
currently not a candidate for noarch.  If we fix that, we might as well
put the byte code under %{_datadir} so the heuristic would fit there as
well.  Most of our haskell is for use with ghc which I believe compiles
to native code.. I don't know if we have any byte-code except in the
hugs package itself....

>  We can't be having build system outages every
> time we realize there's another special-case we need to handle in the
> rpmdiff script.

Good point.  Why will updating the rpmdiff script cause a buildsystem
outage?  Are you executing the script or importing it into the koji process?

>  I think we need to trust packagers to review their
> packages and only convert to noarch where appropriate.
> 
But this is not always an easy thing to see.  So instead of throwing
this out to the packagers where everyone needs to be aware of it and
know when and how to check for it and have the poor reviewers take more
time to check for errors and to the users who might not be able to
diagnose it at all we should automate this so it only bothers people who
have failures.

> And if someone wants to setup a more rigorous package checker outside of
> the build system and file bugs against packages with problems, noarch or
> otherwise, more power to them.
> 
Sure.  A package checker that queues up every build and runs a wider
variety of automated checks on them and if the packages fail, refuses to
let them be pushed or files a bug would be great.  But that's currently
vapourware and we have a problem currently and a way to address the
problem until a better solution comes along.

That solution might take some packages out of the mix that would
otherwise be noarchable but to what extent?  Of the packages that are
currently built and testable with rpmdiff, only one of them had a false
negative and that was caused by a packaging bug.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20090226/4bde8f78/attachment.sig>