[Spacewalk-list] Postgresql slowness

Jan Pazdziora jpazdziora at redhat.com
Thu Jul 21 18:16:49 UTC 2011


On Thu, Jul 21, 2011 at 05:36:19PM +0100, John Hodrien wrote:
> On Thu, 21 Jul 2011, Jan Pazdziora wrote:
> 
> >After each package "push" is done, the code tries to make sure the
> >rhnChannelNewestPackage "cache" is in sync, so it recalculates the
> >whole channel.
> >
> >We could do it after the whole channel is synced -- but what if the
> >sync fails before it gets chance to call the refresh?
> 
> I'm not clear why this slows down significantly with every package pushed (to
> the extent it's taking over 6 seconds per package), but then cheer up if I
> restart the sync.

I'm not sure about PostgreSQL -- I assume the database server is
caching some statistics about table content and since this table
content changes while the session is still active, maybe the initial
statistics become stale?

> /Perhaps/ this could be done periodically by taskomatic rather than per
> package push?  I'm not sure, but given it's adding several seconds per package
> pushed on my system, I just thought I'd question whether it needs to happen in
> the way it is at the moment.  I concede my database isn't necessarily
> sprightly, but this is adding hours and hours to a full push at the minute.
> 
> Given that a failed sync would hopefully be a rare event (it'd have to fail to
> the extent repo-sync fairly ungracefully quit I'd assume), would once per sync
> anda a nightly call be unreasonable?

If I'm not mistaken, spacewalk-sync (unlike spacewalk-repo-sync)
is inserting the data in sets of 100 packages. So maybe that could
be the way to go -- bundle multiple packages into transaction and only
run the rhn_channel.refresh_newest_package after each transaction,
not after each package inserted.

Another possibility would be to only regenerate the values for
any given (channel_id, name_id, package_arch_id) -- because obviously,
evaluating newest "bash" version when you just pushed "zsh" does not
make much sense.

It should also make the thing much gentler on the redo logs in Oracle
and on MVCC in PostgreSQL.

I'll be happy to review a patch.

-- 
Jan Pazdziora
Principal Software Engineer, Satellite Engineering, Red Hat




More information about the Spacewalk-list mailing list