RFC: X library package changes, dependancy changes, freedesktop.org xlibs, etc.

[Mike A. Harris]

On Mon, 2 Feb 2004, Toshio wrote:

>> 1) Allow multiple X11 implemenations to be available in rpm 
>>    packaging, and be substituted with each other easily.
>> 
>If you are going to package freedesktop-X11 as separate packages for
>each library, then the XFree86 libraries also should be split into
>separate binary packages so people can actually play with alternative
>implementations of a subset of the X libraries.  (May have been implied,
>but it wasn't actually mentioned as being part of the plan.)

No, I have no intention of repackaging XFree86 libraries into
individual rpm packages.  There is no useful reason to do so, and 
it would just increase the complexity of the XFree86 package 
rather signficantly.  I don't want people having one lib from one 
implementation and another from a different implementation for 
example.  That serves no useful purpose to my goals at least.

However, it is entirely possible that someone else out there 
might want to whip up their own rpms with things split out like 
that.  I wont stop them, however if they do so, they're on their 
own, and can likely expect massive problems.  ;o)


>> 2) Allow X11 software to compile, link, build and produce 
>>    packages irrespective of which implemenation is present.  This 
>>    is important, because X11 is a standard, and so any compliant 
>>    implementation should be substitutable, at least for the bits 
>>    that are actually official standards.
> 
>Are we going to be troubleshooting this or upstream?  Or is it "upstream
>cares about this and when we find it doesn't work as expected we'll
>point it out to them"?

That depends on the nature of the hypothetical problem that you
haven't stated, and where it would lay in the mix of things.  
Obviously if someone finds an implementation bug in some library,
in any implementation, both the authors of the upstream library,
as well as Red Hat will be concerned.  In comment #2 above
however, I'm more concerned about API and linkage time problems
than runtime ABI problems.  ABI issues are an important thing
also, but unrelated to coming up with universal and generic
distribution packaging for various X11 implementations.  ABI
problems would be best handled via upstream bug reports, but
would be useful in Red Hat bugzilla also (for implementations we
ship that is).  Again that's a problem unrelated, and not covered
by this proposal however.


>> 3) Allow X11 software binary rpm packages which were compiled 
>>    with any X11 implementation, to install cleanly on a system 
>>    which has any X11 implementation installed.  Of course, in 
>>    this particular case, only "official standard" parts of the 
>>    given X11 implementation should be interchangeable 100%, 
>>    however implementation specific features may require 
>>    additional dependancies.
> 
>So we'll have to break apart X11 providing packages into standard's
>providing pieces and extras providing pieces wherever possible.

Not sure what you mean by that, but perhaps my #3 above wasn't 
clear either.  Let me expand upon it.  What I mean above is that 
"libX11" is a standard API, which should be compatible at the 
source and binary level between implementations.  libXinerama 
however is not standard at all.  It is an XFree86 specific custom 
extension.  X.org has an "official" Xinerama extension proposed 
which is not yet "standard" so to speak, and not included in any 
implementation out there yet.  The two APIs are not compatible 
with each other, however X.org's API is in libExt aparently 
instead of a separate lib, so they should co-reside if needbe I 
am told.

Another example is "XFIXES".  That extension is experimental and 
only present in kdrive currently.  It is very likely to be 
included in future X implementations in some form or another, 
however if it is included in one implementation and some 
application gets built to require it, there needs to be a way to 
specify that dependancy.  As these issues get discovered, I'll 
handle them one at a time, however it's at least partially a 
different problem to solve at least for future stuff like 
XFIXES.  There are other extensions present in XFree86 which are 
present in some other implementations, such as RENDER and RANDR. 
If an app requires those to compile, then you need an 
implementation that provides those APIs installed to develop 
with, and a way of specifying those dependancies.  My proposed 
method should handle this, as an implementation that doesn't 
provide these "not-yet-standard" APIs just wont Provide: them 
anywhere.  That makes sense, as apps that use these non-standard 
implementations - depend on them.  Any implementation that 
doesn't supply them, is well....   useless, and thus not in the 
scope of the problem being solved.  ;o)

In other words, GNOME, KDE, and large numbers of other software
components out there, _require_ things like Xft, RENDER, RandR
now (although some or all of that is likely compile time
configurable, but you'd have to ask GNOME developers to be sure).  
So gnome needs a way of building that says "I need RandR-devel at
compile time, or there is no Gnome resolution switcher applet to 
change res on the fly", etc.

Build GNOME against rpmified X.org's X11R5 and you wont get 
RENDER or RANDR.  ;o)


>Regarding the new Requires: Is it really necessary to Require:
>XFree86-libs right now?

Yes.

$ rpm -q --whatrequires XFree86-libs
kdenetwork-2.2.2-0.71.0
XFree86-xfs-4.1.0-50
XFree86-4.1.0-50
XFree86-devel-4.1.0-50
XFree86-twm-4.1.0-50

$ rpm -q --whatrequires XFree86-libs
XFree86-4.3.0-15.1
XFree86-devel-4.3.0-15.1
XFree86-twm-4.3.0-15.1
XFree86-xfs-4.3.0-15.1

The question then becomes "are those above dependancies 
necessary?"  The answer depends on the package of course.

I've no idea why kdenetwork requires it, but that OS release is 
rather old, so it might not be valid anymore.  XFree86-devel 
requires it because the .so libs in -devel require the .so.* 
files to be present in order to function correctly.  The 
XFree86-xfs and XFree86-twm packages would drag in XFree86-libs 
on their own via autodependancies, however *any* -libs package 
would satisfy it.  I explicitly want the ver-rel of twm, xfs 
installed to explicitly require the exact ver-rel of XFree86-libs 
to be installed, due to some problem from ages ago which I don't 
remember.  Of course, those dependancies would have to change in 
the future to work with the new proposal.  ;o)

However, the general answer aside from certain specific 
exceptions is "no, autodeps handle almost everything".  For the 
cases which they don't handle, I believe Conflicts: lines might 
suffice.


>Shouldn't rpm's automatic dependencies drag in libX11.so which
>is found in XFree86-libs.  Couldn't we get rid of dependency on
>XFree86-libs and not replace it?

Yes and no.  Right now I have intentional dependancies present 
explicitly, such as:

Requires: XFree86-xfs = %{version}-%{release}, XFree86-libs = %{version}-%{release}

That is intended to FORCE people who upgrade XFree86-xfs to also 
upgrade XFree86-libs.  I don't remember the original reason it 
was added, but I don't add such dependancies usually unless some 
ugly problem occured in which I dropped a shotgun in to fix it.  
Some people want to save 2 minutes download time for example so 
might only upgrade -xfs if they have a problem, but the fix might 
be in XFree86-libs current package.  This way, when a new XFree86 
release is released, and someone upgrades one component, they are 
forced to upgrade them all, so I don't get bug reports coming in 
due to long since fixed problems.

That sledgehammer "upgrade your system please" approach might not 
work with the new proposal, and so I will probably have to 
revisit these individual special cases again.  Other packages 
probably have similar sledgehammers in place here and there.  I'm 
sure there's another solution though.

>Separate problem: we have to identify packages which can simply depend
>on finding any /usr/lib/{X11/}libX11.so in the filesystem and those
>which need a specific implementation of libX11.  So I think we need the
>capability to Provide/Require: XFree86-libX11 and freedesktop-libX11. 
>This will be something of a headache as we really need finer grained
>tracking than rpm provides.

Virtual provides can handle that problem well enough I believe.

>Maybe we'll need extra web infrastructure that tells us XFree86-libX11
>provides (standards, X11_nonstandard_gizmo) while freedesktop-libX11
>provides (standards, X11_nonstandard_whizzo) and developers can check if
>those things are ever reconciled/build separate packages for the
>different xlibs, etc (ugh).

I'd prefer to not get that complicated if possible, but if it 
turns out to be something universally required, it's not outside 
the realm of possibility.  I think if some app requires a 
specific implementation's lib and no other implementation 
provides it, a quick hack would be to Require: 
freedesktop-libXfixes for example.

In other words, there are some apps that WILL still require 
specific implementation present most likely.  The only way to 
avoid that, is to completely disallow anything as an X extension 
that isn't approved by X.org directly.  However reality is that 
developers working on new standards don't always have 10 years to 
wait for X.org to gather interest in a proposed new API and then 
to standardize it.  Look at Xinerama for example. It's been 
implemented in XFree86 forever more or less now.  X.org is just 
now planning on coming out with a totally incompatible Xinerama 
standard with a different API.  Would everyone who has used 
Xinerama for the last 3-4 years have preferred to wait for X.org 
to release Xinerama?  ;o)

So there will definitely be APIs that are unique to an 
implementation out there, and a properly coded app should check 
both at compile time, *and* at runtime, and if the extension 
isn't available at runtime, the app should fallback to not using 
it.  Most apps do this already of course.


>Hopefully most API/ABI enhancements will be done in true add-ons rather
>than entangled with standard pieces like this....

They generally are on the client side.  Client side stuff is in
things like libXinerama, libXrender, libXrandr, etc. Whereas
standardized stuff tends to go into libXext.  The difference is
more on the server side, where the X server may or may not
support a given extension, however it is up to all applications
to test at runtime for nonstandard extensions, as they are
non-standard by definition.  They should provide fallbacks in 
that case.

Implementation defined stuff will appear in implementation 
defined libraries.  There are some "issues" with this, such as 
the XFree86 "MISC" extension, which provides special things 
specific to XFree86's server, however apps should be properly 
querying for that stuff anyway.  Apps that *require* it, should 
by definition:  Require: XFree86-libs or whatever as they're 
implementation specific.

You do bring up some interesting complexities of course, and 
they'll need to be explored also once the immediate issues are 
dealt with.  I forsee an "ok, let's try this and see what all 
breaks" day coming up in the future.  Should be fun.  ;)

Thanks for feedback!

TTYL



-- 
Mike A. Harris     ftp://people.redhat.com/mharris
OS Systems Engineer - XFree86 maintainer - Red Hat