And you base this on what exactly?

Not sure about your sites, but my traffic increases during the holiday season... Besides, I started this almost 3 weeks before.

Ummm, port 80 is of course open, but the firewall does stateful packet inspection. The firewall's Intrusion Prevention system does block such attacks. Tens of thousands of them a day, in fact. It uses a realtime list of over 1,700 attack & exploit profiles to spot the illicit content in web traffic. I stay on top of patches and security updates, but a multi-layered defense offers added protection. Eric

Sure, you can enter such rules into YOUR firewall to screen out such things, but the vast majority of users either do not have this feature, are ignorant on how to implement it, or do not have proper privi's to institute it.

Curious, does your solution block these IP's from further attacks?

The security policy system in Astaro is very robust. Rules can be written to act on findings of the IDS. This software costs less than $500 for a ten user license, users being unique domain names served. It needs to run on a dedicated server, but I imagine that someone out there offers a firewall script, with similar features, that will run as a service on an existing server. Eric

BTW, my comments were not intended to minimalize what you did with the script described in the OP. Like I said, I believe in a multi-layered approach to security. Your script sounds like a good idea. EP

kmike runs a small site with only 9 million posts and 4,000 concurent users online. :)
I think his experiences are to be trusted among other big boarders...
Try to listen more and communicate, instead of being defensive. A script like yours will not do much good to an attack coming from a decent russian hacker. Pray that you will not piss anyone from the East side. :)

Well, apparently our forum audience is different from yours - we see a steady traffic drop during the Christmas week (up to 30% on 25th) and then a sharp increase on 26th when the members get back to the Net after the Christmas festivities.

But my point still stands, I don't think the vulnerability sniffing requests add much load to your forum. Their pattern is usually "hit and run" - no point hammering the forum with them when the first request (or the first few) fails. And besides, the target script will drop a parameter error at the very first stages of execution when the URI parameters are checked.

Of course, your efforts were not in vain, dropping the sniffing requests won't hurt the performance, but I think the effect is negligible compared let's say to the load the rampaging Yahoo spider can cause. Have you checked the search spider activity, maybe it's lower near the end of the year?

When you watch the logs in real time ( I use this ) along side with the server load, you can watch it happen right in front of you. The same IP will send out a list of attacks, each one firing up VB (i.e. loading files and connecting to DB). These same IP's will also hit other scripts on the server looking for vulnerabilities.

If you use a good stats program, you can isolate these IP's and show how many times they hit the server. At that point, you can check it against after you start banning the IP's. The amount of hits take a nosedive.

I pipe the output of the script to a log file so I can see how it goes over a period of time. I execute the job every 10 minutes and at first, every execution would have X amount of attempts. After running the script for a couple days, you begin to notice that several executions result in 0 attempts, and that grows in of itself.

To date, the script has banned 2,425 IP's. It started by banning about 35 - 45 IP's an hour, it has now dropped to about 4 - 8 an hour.

So what are the hit numbers you were seeing?

I checked the logs for the last two days, and there were less than 1000 hits with "=http://" URI parameter per day. It's certainly something I wouldn't worry about at this point, since it constitutes probably about a hundrenth of percent of our total daily hits. Also about 20% of these hits had 404 or other not-OK HTTP status.

Maybe for a small forum with less hardware resources 1000 stray hits per day could be a problem, but then again, I'm pretty sure the number of bogus hits depends linearly on the forum position in the search engines, and this in turn depends indirectly on the forum activity. So smallish forums should see less sniffer bot activity.
If I find time before the year end, I'll try to prove that theory by looking through the small forum logs I have access to.

PLEASE email me this perl script. Our forum just got shutdown because of these hack attempts and they won't turn us back on until we have a script in place. Big thanks! kstiever at hot mail

It very well could be that my script would not work on your server. It parses my log file, which is an apache log file. You may not even be using apache, may be using a different version, may not have access to the log, may not have access to crontab, may not have access to shell (ssh, telnet) etc..

If you are using a shared server, then there is a 99.9% chance this method will NOT work for you. You need access to tools usually only root has, such as iptables.

If you do have root to your server, you should contact someone who is fluent in Perl to write you up a script, as it should only take about an hour or two.