Re: --retry <num> and --sleeptime <num>

[Peter A Jonsson <Peter A Jonsson epl ericsson se>]

> There's a better way IMHO. The Berkeley DB permits a CDB model
> (Concurrent DataBase) that has finer grained locking than the exclusive
> lock traditionally used by rpm. I know that CDB works fine for the rpmdb, but
> other mechanisms are gonna be needed to synchronize rpm install operations that
> currently implicitly assume that an exclusive lock on the database provides
> a mutex. Anyways a CDB model will permit concurrent access from, say, an
> rpm query run from a %post scriptlet, and should remove the need to
> sleep while waiting for an exclusive lock.

I have been reading some parts of the docs on sleepycat.com for db. This 
concurrent db sounds interesting to me. Changing to the cdb-model seems to 
involve a few things:

* Creating/Opening the db with the DB_INIT_CDB and DB_INIT_MPOOL. 
* Making sure no cursors are read/write when already holding a cursor with 
read/write access.
* Make sure the error return codes are checked for.
* Make sure no cursors are open when doing put/del.

Am I missing something?

Some creative use of grep makes me believe that lib/db3.c lib/depends.c and 
lib/rpmdb.c could need modification. Any other file? Is this something wanted 
in rpm?

However, this does not solve the "problem" with two rpm's trying to write to 
the same db as far as I can understand. It will still die unconditionally. 
Several processes can read from the db at the same time but only one can 
write. See below for more discussion.

> There's nothing other than rpm that calls rpmInstall() that I'm aware of.
> Meanwhile, that interface is changing anyways in rpm-4.0.3, as rpm install
> and erase modes need to be merged into a common routine. The new args will look
> much more like the rpmQuery()/rpmVerify() interface when I'm done.

-- Rearrangement --

> There are other problems with several machines accessing an rpmdb
> simultaneously, not the least of which is that Berkeley DB does not
> support locking across NFS. I believe the fcntl scheme might "work"
> across NFS, haven't looked seriously at all. There's also the Berkeley DB RPC
> that could be used for several machines accessing an rpmdb. I believe that
> most of the pieces are there, but, again, I haven't looked seriously at
> using, only at configuring.

Without looking at the code, can there be an option put in for retrying to 
lock the db (if the first lock fails) after a delay in that API? If the 
parameters have sane values (maybe through a macro) it should have very little 
impact on the other code. An alternative could be to wait until the lock is 
gotten, I don't know if that is what one want though. Something to the 
commandline so it doesn't have a default behaviour that is dangerous. I am 
open for suggestions.

The locking over NFS is flakey (to say the least) in several operatingsystems. 
Another possible usage for the feature of retrying the db-opening is that 
several build-/install-scripts might be running at the same time, either 
building from different pools or building several rpms due to the fact that 
the load on the machine is low (ie it is nighttime). Ofcourse this mainly 
applies to SMP-machines. Veritas has a cluster filesystem, etc..

I believe that there could be uses for several rpm's trying to write to the 
same db at the same time. The user might be one of those who happens to have 
some odd hardware and wants to do something odd.

> Bug reports and enhancement requests in bugzilla
> 	http://bugzilla.redhat.com
> please, as I sometimes abuse tha delete key when reading mail.

Ok, will do.

> You might also look at patching against rpm-4.0.3 from the rpm-4_0
> branch in CVS

My firewall blocks CVS. Are there any snapshots availible via ftp/http? Will 
the ones in pub/rpm/test-4.0.3 do or are the changes in CVS too big? Will 
patches against 4.0.3 be much more helpful to you than 4.0.2?

/ Peter