Re: RPM 4.0 and 4.0.2 analysis, resolution, and patch.

[Jeff Johnson <jbj JBJ ORG>]

On Mon, Mar 05, 2001 at 03:32:09AM -0500, Vladimir Vukicevic wrote:
> 
> So, despite all the discussion that has gone on, and all the blame
> that has been thrown left and right, no one has proposed a fix for the
> incompatability between rpm 4.0 and 4.0.2. I was bored this Sunday
> evening and decided to try to do just that, so I built rpm 4.0 and
> 4.0.2 on my system. Note that I could be very wrong; this is my first
> foray into the rpm codebase.
> 

Your analysis appears correct, although doesn't address other breakage
with ximian, on Red Hat 6.2 using db1 for example.

> First a minor build issue; the rpm 4 build process assumes that you'll
> be building with db 3.0 or db 3.1 and not db 3.2, so in lib/db3.c and
> lib/db2.c there are a number of
> 	#if DB_VERSION_MAJOR == 3 && DB_VERSION_MINOR == 1
> that need to read
> 	#if DB_VERSION_MAJOR == 3 && DB_VERSION_MINOR >= 1
> 

There's no support for db-3.2.9 in rpm-4.0 anything, there's API changes
that need figgering, already in bugzilla at #29721.

> 
> This is yet another example where rpm arbitrarily breaking
> compatability between microversions, except in this case it's database
> backwards-compatability.  I understand that the rpm in question is
> 4.0.2 which is "unreleased" -- unfortunately, it has had a wide enough
> deployment with Nautilus PR3 and the Red Hat betas to cause trouble.
> In the red carpet case, this situation could have been avoided if the
> API didn't change between micro versions and if the shared library
> version number were updated to reflect when it did.  Until that
> happens, it is impossible to deploy any software that links in a
> shared manner against rpm. (Unless you happen to control both rpm and
> the software in question.)
> 

The change isn't exactly "arbitrary breakage", it's called "header regions".

> Note that this problem only affects 4.0 clients. It also only appears
> with certain packages -- some packages get 'accidentally' written with
> the right sort order into the database. No data is lost, and any 4.0.2
> applications will continue to work fine. But it's a one-way move,
> which IMHO is unacceptable.
> 

Yup, header regions will be a one-way move, as your early deployment
of rpm-4.0.2 through a Ximian pipe is triggering a bug in rpm-4.0, and
the only fix will be to release another version of rpm to "fix".

AFAIAC, the "fix" is rpm-4.0.2.

> The claim that data was lost is also false; all the data is still
> there, but rpm 4.0 cannot see it.  If the rpm 4.0 patch is applied,
> doing a rpm --rebuilddb causes the data to get rewritten to the
> database in the correctly-sorted format.
> 

Yup.

> The right way to handle this is to have a database version number
> (which may or may not exist -- maybe it exists only for major
> versions, I don't know) that needs to be bumped so that when something
> like this happens, the older rpm can complain and say "the database
> format changed." But considering that the shared library is still
> librpm.so.0, I don't foresee this happening any time soon.
> 

Maybe. A new scheme requires releasing another version of rpm,
basically the same as releasing rpm-4.0.2.

FWIW, there's another way to handle the problem, with an incompatibility
tracking dependency, the provides

    rpmlib(HeaderLoadSortsTags) = 4.0.1-1
	header tags are always sorted after being loaded.

is already in rpm-4.0.2.  It is possible to add the corresponding
Requires: during package installation with rpm-4.0.2. The end result
is exactly the same, a one-way move to rpm-4.0.2, this time enforced
by the dependency (but overridable at your peril using --nodeps).

FWIW, silently adding the requirement is entirely at odds with what is being
attempted with header regions, i.e. preserving the original header as a
contiguous region so that signatures can be verified after a package is
installed, as the added tag will force 2 copies of the Requirename,
Requireversion, and Requireflags values in each and every header in
the database.

> So, the patches needed here are a patch for 4.0 to deal with the
> unsorted entries in the database (below), and a patch for 4.0.2 to not
> unsort the header before unloading it, which I don't have the
> inclination to write.
> 
> For 4.0, the patch is simple -- we simply take away the sorted
> assumption.
> 
> --- orig/rpm-4.0/lib/header.c	Sun Jul  9 14:56:22 2000
> +++ rpm-4.0/lib/header.c	Mon Mar  5 02:35:17 2001
> @@ -378,7 +378,10 @@
>      h->usageCount = 1;
>  
>      /* This assumes you only headerLoad() something you headerUnload()-ed */
> -    h->sorted = 1;
> +    /* --- the sort assumption is not valid if the db has been touched by
> +     * rpm 4.0.2
> +     */
> +    h->sorted = 0;
>  
>      pe = (struct entryInfo *) p;
>      dataStart = (char *) (pe + h->indexUsed);
> 

The patch fixes the original problem, but there's another one line patch to
deal with the missing RPMTAG_FILESTATES that is segfaulting as well.

FWIW, both patches are already in rpm-4.0.2.

> 
> For 4.0.2, the situation is harder, since header.c seems to have
> received a rather large overhaul between 4.0.  The doHeaderUnload
> needs to write the data out into the database in a sorted format.  If
> this is not possible, then this is a binary database incompatability,
> and at least a new rev of 4.0 should be released that includes the
> aforementioned patch.
> 
> I hope that this has clarified what has turned into a somewhat ugly
> situation.  RPM is an essential component on many systems; as such, it
> needs to be heavily tested, especially for backward compatability,
> before it is released (officially or unofficially) so problems such as
> this do not happen in the future.
> 

FWIW, this change/incompatibility was announced here back in December, and
code has been available for testing since then as well. Too bad the problem
wasn't detected sooner.

I see little reason for Yet Another rpm release, as this problem is already
fixed in rpm-4.0.2

Thanks for the analysis.

73 de Jeff

-- 
Jeff Johnson	ARS N3NPQ
jbj@jbj.org	(jbj@redhat.com)
Chapel Hill, NC