T TechBoyJK Lifer Oct 17, 2002 16,701 60 91 Sep 4, 2003 #1 My server is being ransacked by spider bots. How can I utilize something like IP tables to prevent this?
My server is being ransacked by spider bots. How can I utilize something like IP tables to prevent this?
I Isaiah Senior member May 31, 2000 453 0 0 Sep 4, 2003 #4 Make a file called "robots.txt" with the following text: User-agent: * Disallow: * Click to expand... Then upload the robots.txt to the root of your public www folder That will stop all(ones that follow the rules) spiders from indexing your site Hope that helps, -Isaiah
Make a file called "robots.txt" with the following text: User-agent: * Disallow: * Click to expand... Then upload the robots.txt to the root of your public www folder That will stop all(ones that follow the rules) spiders from indexing your site Hope that helps, -Isaiah