vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vB4 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=251)
-   -   Bots are devouring my monthly execution limit/bandwidth. Love some help (https://vborg.vbsupport.ru/showthread.php?t=305095)

obglobal.net 12-21-2013 10:32 PM

Hello, vb.org! A Merry Xmas to you all.

So my site is still getting pillaged by bots after trying everything. These bots are a-holes!

These 2 in particular are having their way with my site, and I cannot manage to block them.

Unknown robot (identified by empty user agent string)
Unknown robot (identified by 'bot*')

Could anyone please help me regarding .htaccess? I'd appreciate the input because I suck at this stuff.

Thanks a lot.

final kaoss 12-21-2013 10:39 PM

Hello one idea that I have for you is to use cloudflare. From there you can easily block bots, also, cloudflare is capable of serving many static resources from you to help you save your servers resources. However I believe that it will only serve the static resources once you get a paid plan ($5 or more), the user agent/bad behaviour thing is available on the free plan.

I'll also leave you with these 2 links to check out to do further research.
http://www.fleiner.com/bots/

https://www.google.com/search?q=slow...sm=93&ie=UTF-8

Max Taxable 12-21-2013 10:57 PM

I used Cloud Flare for two years. It is not worth the downtime it causes and if your host goes down, you still go down. it doesn't do anything, really except put your site on a proxy.

Obglobal - please define "pillaged."

obglobal.net 12-21-2013 10:59 PM

Quote:

Originally Posted by Max Taxable (Post 2469908)
I used Cloud Flare for two years. It is not worth the downtime it causes and if your host goes down, you still go down. it doesn't do anything, really except put your site on a proxy.

Obglobal - please define "pillaged."

Pillaged as in getting invaded by. Bots are invading my site.

So Cloud Flare is no go, hey. I can't believe it's so tricky to get rid of bots!

Thanks for helping.

--------------- Added [DATE]1387670459[/DATE] at [TIME]1387670459[/TIME] ---------------

Quote:

Originally Posted by final kaoss (Post 2469899)
Hello one idea that I have for you is to use cloudflare. From there you can easily block bots, also, cloudflare is capable of serving many static resources from you to help you save your servers resources. However I believe that it will only serve the static resources once you get a paid plan ($5 or more), the user agent/bad behaviour thing is available on the free plan.

I'll also leave you with these 2 links to check out to do further research.
http://www.fleiner.com/bots/

https://www.google.com/search?q=slow...sm=93&ie=UTF-8

Thanks a lot man!!! I'll be sure to check that out.

Max Taxable 12-21-2013 11:01 PM

Quote:

Originally Posted by obglobal.net (Post 2469909)
Pillaged as in getting invaded by. Bots are invading my site.

How many? 10s, 100s, 1000s? What are they actually doing?
Quote:

I can't believe it's so tricky to get rid of bots!
It's not.

obglobal.net 12-21-2013 11:05 PM

Quote:

Originally Posted by Max Taxable (Post 2469913)
How many? 10s, 100s, 1000s? What are they actually doing?

Tens of thousands over a month's time. They're doing basically everything they can to sweep the site, including 200+ registrations per day.

I've added mods to stop them from registering, but it still racks up on my monthly executions limit.

--------------- Added [DATE]1387670759[/DATE] at [TIME]1387670759[/TIME] ---------------

Sorry, I meant tens of thousands of executions. Not tens of thousands of bots.

Max Taxable 12-21-2013 11:12 PM

Quote:

Originally Posted by obglobal.net (Post 2469915)
Tens of thousands over a month's time. They're doing basically everything they can to sweep the site, including 200+ registrations per day.

I've added mods to stop them from registering, but it still racks up on my monthly executions limit.

To help further I would need access to see this for myself and check some things out in the backend.

However testing your site shows you have bigger problems than bots - bandwidth usage just for pageload is off the scale and that is not bots causing that:

http://www.webpagetest.org/result/13..._8R/1/details/

LATER EDIT: C'mon Man, you don't have a bot problem at all. Never did have. Here's stats from your forum home page:
Currently Active Users

There are currently 20 users online. 0 members and 20 guests

Most users ever online was 61, 05-23-2013 at 05:06 PM.
That's nothing.

obglobal.net 12-21-2013 11:21 PM

Quote:

Originally Posted by Max Taxable (Post 2469918)
To help further I would need access to see this for myself and check some things out in the backend.

However testing your site shows you have bigger problems than bots - bandwidth usage just for pageload is off the scale and that is not bots causing that:

http://www.webpagetest.org/result/13..._8R/1/details/

Yeah, the load time is pretty enormous. I'm happy to give you access, but would you mind holding off for a while? My second son's just been born, I'm travelling overseas for Xmas, and I have a pretty nasty hangover. Haha.

I also have the admincp ip denial mod active, but don't have access to my computer to turn it off at the moment.

I really appreciate your help dude.

Max Taxable 12-21-2013 11:22 PM

Quote:

Originally Posted by obglobal.net (Post 2469921)
Yeah, the load time is pretty enormous. I'm happy to give you access, but would you mind holding off for a while? My second son's just been born, I'm travelling overseas for Xmas, and I have a pretty nasty hangover. Haha.

I also have the admincp ip denial mod active, but don't have access to my computer to turn it off at the moment.

I really appreciate your help dude.

You don't have a bot problem. Please disable the hook system in adminCP so i can test just your native vBulletin installation, to see how the host server behaves that way.

Your sidebar alone is doing about 100 executions every pageload and and about 100 calls to the db, THIS is what is using up your resources. You have way too much junk in it like Post thanks feed, tag cloud, activity stream. It's making the page about twice as long as your tables go.

obglobal.net 12-21-2013 11:43 PM

Quote:

Originally Posted by Max Taxable (Post 2469922)
You don't have a bot problem. Please disable the hook system in adminCP so i can test just your native vBulletin installation, to see how the host server behaves that way.

Your sidebar alone is doing about 100 executions every pageload and and about 100 calls to the db, THIS is what is using up your resources. You have way too much junk in it like Post thanks feed, tag cloud, activity stream. It's making the page about twice as long as your tables go.

Really? Thanks for having a look at it, Max Tax.


All times are GMT. The time now is 02:11 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01429 seconds
  • Memory Usage 1,751KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (9)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (3)pagenav_pagelink
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete