stop web spiders

[Anze Vidmar]

hi!

I know this topic sounds like a spam, but it's not ;-)

I'm talking about web spiders that are collecting information from web
sites.

Is it possible to stop spiders to access specific folder on my web
server? e.g. http://myweb.com is public, but
http://myweb.com/confidential has some confidential files in it and I
don't want them to show while searching google for example. The password
protection (.htaccess) is *not* an option. I have "-Indexes" option set
on this folder, but still not sure if spiders can read them or not?

Thanks.
-- 
Anze Vidmar <anzevi at gmail.com>