Bots are devouring my monthly execution limit/bandwidth. Love some help

[obglobal.net]

Hello, vb.org! A Merry Xmas to you all.

So my site is still getting pillaged by bots after trying everything. These bots are a-holes!

These 2 in particular are having their way with my site, and I cannot manage to block them.

Unknown robot (identified by empty user agent string)
Unknown robot (identified by 'bot*')

Could anyone please help me regarding .htaccess? I'd appreciate the input because I suck at this stuff.

Thanks a lot.

[final kaoss]

Hello one idea that I have for you is to use cloudflare. From there you can easily block bots, also, cloudflare is capable of serving many static resources from you to help you save your servers resources. However I believe that it will only serve the static resources once you get a paid plan ($5 or more), the user agent/bad behaviour thing is available on the free plan.

I'll also leave you with these 2 links to check out to do further research.
http://www.fleiner.com/bots/

https://www.google.com/search?q=slow...sm=93&ie=UTF-8

[Max Taxable]

I used Cloud Flare for two years. It is not worth the downtime it causes and if your host goes down, you still go down. it doesn't do anything, really except put your site on a proxy.

Obglobal - please define "pillaged."

[obglobal.net]

Originally Posted by Max Taxable

I used Cloud Flare for two years. It is not worth the downtime it causes and if your host goes down, you still go down. it doesn't do anything, really except put your site on a proxy.

Obglobal - please define "pillaged."

Pillaged as in getting invaded by. Bots are invading my site.

So Cloud Flare is no go, hey. I can't believe it's so tricky to get rid of bots!

Thanks for helping.

--------------- Added [DATE]1387670459[/DATE] at [TIME]1387670459[/TIME] ---------------

Originally Posted by final kaoss

Hello one idea that I have for you is to use cloudflare. From there you can easily block bots, also, cloudflare is capable of serving many static resources from you to help you save your servers resources. However I believe that it will only serve the static resources once you get a paid plan ($5 or more), the user agent/bad behaviour thing is available on the free plan.

I'll also leave you with these 2 links to check out to do further research.
http://www.fleiner.com/bots/

https://www.google.com/search?q=slow...sm=93&ie=UTF-8

Thanks a lot man!!! I'll be sure to check that out.

[Max Taxable]

Originally Posted by obglobal.net

Pillaged as in getting invaded by. Bots are invading my site.

How many? 10s, 100s, 1000s? What are they actually doing?

I can't believe it's so tricky to get rid of bots!

It's not.

[obglobal.net]

Originally Posted by Max Taxable

How many? 10s, 100s, 1000s? What are they actually doing?

Tens of thousands over a month's time. They're doing basically everything they can to sweep the site, including 200+ registrations per day.

I've added mods to stop them from registering, but it still racks up on my monthly executions limit.

--------------- Added [DATE]1387670759[/DATE] at [TIME]1387670759[/TIME] ---------------

Sorry, I meant tens of thousands of executions. Not tens of thousands of bots.

[Max Taxable]

Originally Posted by obglobal.net

Tens of thousands over a month's time. They're doing basically everything they can to sweep the site, including 200+ registrations per day.

I've added mods to stop them from registering, but it still racks up on my monthly executions limit.

To help further I would need access to see this for myself and check some things out in the backend.

However testing your site shows you have bigger problems than bots - bandwidth usage just for pageload is off the scale and that is not bots causing that:

http://www.webpagetest.org/result/13..._8R/1/details/

LATER EDIT: C'mon Man, you don't have a bot problem at all. Never did have. Here's stats from your forum home page:

Currently Active Users

There are currently 20 users online. 0 members and 20 guests

Most users ever online was 61, 05-23-2013 at 05:06 PM.

That's nothing.

[obglobal.net]

Originally Posted by Max Taxable

To help further I would need access to see this for myself and check some things out in the backend.

However testing your site shows you have bigger problems than bots - bandwidth usage just for pageload is off the scale and that is not bots causing that:

http://www.webpagetest.org/result/13..._8R/1/details/

Yeah, the load time is pretty enormous. I'm happy to give you access, but would you mind holding off for a while? My second son's just been born, I'm travelling overseas for Xmas, and I have a pretty nasty hangover. Haha.

I also have the admincp ip denial mod active, but don't have access to my computer to turn it off at the moment.

I really appreciate your help dude.

[Max Taxable]

Originally Posted by obglobal.net

Yeah, the load time is pretty enormous. I'm happy to give you access, but would you mind holding off for a while? My second son's just been born, I'm travelling overseas for Xmas, and I have a pretty nasty hangover. Haha.

I also have the admincp ip denial mod active, but don't have access to my computer to turn it off at the moment.

I really appreciate your help dude.

You don't have a bot problem. Please disable the hook system in adminCP so i can test just your native vBulletin installation, to see how the host server behaves that way.

Your sidebar alone is doing about 100 executions every pageload and and about 100 calls to the db, THIS is what is using up your resources. You have way too much junk in it like Post thanks feed, tag cloud, activity stream. It's making the page about twice as long as your tables go.

[obglobal.net]

Originally Posted by Max Taxable

You don't have a bot problem. Please disable the hook system in adminCP so i can test just your native vBulletin installation, to see how the host server behaves that way.

Your sidebar alone is doing about 100 executions every pageload and and about 100 calls to the db, THIS is what is using up your resources. You have way too much junk in it like Post thanks feed, tag cloud, activity stream. It's making the page about twice as long as your tables go.

Really? Thanks for having a look at it, Max Tax.