stop web spiders

[Mike Klinke]

On Wednesday 25 January 2006 12:30, Anze Vidmar wrote:
> On Wed, 2006-01-25 at 18:24 +0000, Sean O Sullivan wrote:
> > Look into robots.txt
>
> Thanks!
>
> I've terminated them successfully ;-)
>

Bear in mind that only those robots that choose to play nicely will 
honor your "robots.txt" files.  Some robots simply ignore the file 
and collect whatever links they can follow.

Regards, Mike Klinke