[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Reading little endian RPM db files on big endian machine

[Yet more info, and a work-around]

I found rpmdbNextIterator in rpmdb/rpmdb.c does a dbiGet(...key...)
and then memcpy(&mi->mi_offset, key->data ...).  The mi_offset is the
value that has unaccounted for endian issues, and since it is used in
calculating the allocation size for PBM_REALLOC, we get huge
allocation requests in PBM_REALLOC.

I did an endian swap on mi_offset whenever I found it was a huge
value, and also did an "rpm --rebuilddb" (which used to fail with the
same huge allocation problem) and now things seem to work sanely.

I noted that the code was unchanged in 4.3 (I was using/testing 4.2),
so maybe you RPM gurus have a cleaner/better fix in mind than what I
did, but at least you know where the problem lies.


On 4/27/05, Paul Gortmaker <paul gortmaker gmail com> wrote:
> More info on this:
> It seems that PBM_REALLOC in rpmdb/rpmdb.c is the culprit; asking for
> huge memory amounts.  On a normal i386 box, you see the region starts
> out as 176k and after many iterations as it goes through the package
> listing, it eventually grows to about 3MB, while growing about 50%
> each time.  On the PPC,  it starts out at a whopping 8MB(!!!) and more
> than doubles each iteration, so after maybe  four iterations it is up
> to 250MB and then it fails.
> Does this trigger an "Oh, I know what it is!" for anybody.... ?
> Paul.
> On 4/26/05, Paul Gortmaker <paul gortmaker gmail com> wrote:
> > I was wondering if anyone is aware of any issues regarding
> > reading/using /var/lib/rpm/*  db files on a big endian machine (e.g.
> > PPC) that were generated on an ix86 host?   A simple "rpm -qa"  run on
> > the PPC lists off about 25 packages and then fails with:
> >
> > memory alloc (260046852 bytes) returned NULL.
> >
> > which in hex is $F800004. (This happens when it
> >
> > I noted that "file" listed the db files as being little endian, and
> > based on what I'd read, I did a:
> >
> > /usr/lib/rpm/rpmdb_dump $i.le | /usr/lib/rpm/rpmdb_load $i
> >
> > after which "file" reports them as being in native endian format;
> > however the result from "rpm -qa" is identical.
> >
> > Searching the archives, I found:
> >
> > https://www.redhat.com/archives/rpm-list/2003-May/msg00227.html
> >
> > where it says:
> > ----------------------------------------------------
> > I believe (but have never testsed, Red Hat didn't have any big-endian platforms
> > in it's current product line for a couple years, go figger) that rpm databases
> > can be accessed from other arches (again, not through NFS, but possible
> > through sunrpc).
> >
> > There are 3 layers of compatibility, 2 are known to be there already:
> >         a) Berkeley DB is endian neutral.
> >         b) Package headers are always big-endian
> >         c) join keys for inverted list indices.
> >
> > The join keys in c) are 1 or 2 ints that used to be native-endian. I believe
> > I have the byte swaps in place for the join keys, can/will certainly
> > get you a fix if you show me a problem that I can reproduce.
> > ----------------------------------------------------
> >
> > I was wondering if this is something that is known to be
> > broken/unsupported, or if it should simply work.    I've noted that
> > other rpm functions seem broke against these LE db files too (e.g.
> > "rpm -qf /bin/bash" segfaults).
> >
> > Thanks,
> > Paul.
> >

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]