Stopping GET requests in Apache

[vash]

For the last few weeks, I've seen a significant amount of plain ol 'GET /" requests to my Apache server.

I'm at a loss as to what is causing the significant jump to the site. My site is quite small and if you point a browser to the page, you will download a few images from index.html. Instead, these get requests don't get the main page, all they are looking for is "/".

Anyone know what is floating around that is doing this? What can I do to prevent someone from doing this to my server (Debian, woody, stable)

vash

 

[xcript]

Err, "GET /" is a request for the index page.

Edit: Yeah, so maybe I'm not getting what the problem is. Sorry if this is the case.

 

[vash]

They really aren't getting the / page at all. If you point to our website, you will definitely download a number of images. Also, we would see the refer from either yahoo or google if they came from a search engine.

Anyone else?

vash

 

[xcript]

Originally posted by: vash
If you point to our website, you will definitely download a number of images.

Not necessarily, if it's a robot or e-mail harvester what use would it have for the images?

Probably none, so they wouldn't be requested.

I get seemingly random hits like this in my access_log all the time. Doesn't do any harm.

 

[CTho9305]

If you go to http://google.com/, google sees a "GET /". If you go to http://google.com/index.html, google sees a "GET /index.html". As others speculated, it is probably robots hitting you. Log user-agents and referers - you can then see if it bots, and where any other traffic is arriving from.

 

[vash]

Originally posted by: CTho9305
If you go to http://google.com/, google sees a "GET /". If you go to http://google.com/index.html, google sees a "GET /index.html". As others speculated, it is probably robots hitting you. Log user-agents and referers - you can then see if it bots, and where any other traffic is arriving from.

Thanks for the response and yes, I am logging user-agents and referers (default I guess in Debians install). Here are a few of the lines in the log file:

ool-18b9a80f.dyn.optonline.net - - [04/Sep/2003:18:28:29 -0700] "GET / HTTP/1.1" 200 3510 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)"
user-0c8hj8s.cable.mindspring.com - - [04/Sep/2003:18:32:40 -0700] "GET / HTTP/1.1" 200 3510 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)"
adsl-155-252-151.asm.bellsouth.net - - [04/Sep/2003:18:35:19 -0700] "GET / HTTP/1.1" 200 3510 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)"
cvg-65-27-232-245.cinci.rr.com - - [04/Sep/2003:18:35:35 -0700] "GET / HTTP/1.1" 200 3510 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)"

I'm getting thousands of these per day. If someone comes from yahoo or google, I will see the refer in the log, but with so many like this, I'm not sure what is hitting the site.

vash

 

[CTho9305]

Ok, that just looks like random people hitting the front page for no apparent reason .

 

[sciencewhiz]

If his front page has several images, he should have get requests for each image. Also, if it was random people, the user agent wouldn't always be the same user-agent.

vash, is it always the same user-agent? What date exactly did it start?

 

[vash]

Originally posted by: sciencewhiz
If his front page has several images, he should have get requests for each image. Also, if it was random people, the user agent wouldn't always be the same user-agent.

vash, is it always the same user-agent? What date exactly did it start?

This started a few weeks ago when the net started seeing the SoBig virus. The user agent is consistantly MSIE 5.5/Windows 98 machines, there are very hits that are different in that reguards. Again, if someone were hitting my main page, they would get a series of images, not nothing, as the requests come.

vash

 

[sciencewhiz]

you could use packet inspecting firewall to block that specific http header based on the user-agent. However, you run the risk of blocking a small amount of legitimate traffic. You could also use mod_rewrite to give a smaller index page perhaps that only links to your main page. This will save you bandwidth, but will probably cost you CPU time.