[Fedora-infrastructure-list] Mirror stats

[seth vidal]

On Sat, 2006-10-14 at 23:54 -0500, Mike McGrath wrote:
> I've been working on some scripts for mining information out of our
> mirror logs.  In addition to the awstats scipts I'm keeping a total
> archive of all requests in a special format.  This will allow us to
> mine the information as we need to.  The current plan is to keep the
> archives as is until it becomes unmanageable.  At present about one
> months worth of logs gets reduced to about 50 M, about 5M after
> compression.
> 
> In case anyone is wondering the logs are being kept on
> fpserv:/srv/mirrorStats/stats.txt
> 
> The format is:
> 
> [08/Oct/2006:05:05:03 a05543977dc854b859da8e365de818b8 rawhide i386
> 
> its quick, and easy to run tools against.  The second column is a md5
> encrypted version of the ip address.  Its not perfect but at least its
> not plain text.  I'm sure if someone really wanted to convert these
> back to IP's they could create a rainbow table or some such silliness
> but this should pre-empt anyones fears about us stockpiling ip's or
> any personal information for that matter.
> 
> Eventually I'll write static frontend pages for this similar to our
> awstats pages.  For those keeping count there's been 17800 unique IP's
> that have contacted the mirrors server looking for rawhide.

maybe we could put the stats into rrd's

-sv