vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Miscellaneous Hacks - Ban Spiders by User Agent (https://vborg.vbsupport.ru/showthread.php?t=268208)

DemOnstar 09-14-2013 01:07 PM

Quote:

Originally Posted by ForceHSS (Post 2445517)
You can change that if u want

Thank you... I will leave as is for the present...

Simon Lloyd 09-14-2013 08:43 PM

You have the option to redirect to a site (i.e the one already installed) or directly back to the ip of the banned useragent, its all about choice really :)

DemOnstar 09-17-2013 04:50 AM

How do I know if this is working?

Haven't seen any evidence so far...What do I look for?

Simon Lloyd 09-17-2013 05:13 AM

it will take over 30 minutes to start to see differences in the WOL as the spiders get the message and a bit longer until they stop trying altogether.

The easiest way to see it working is to turn on writing to the log file, or if you dare have threads made in a forum of your choice, i advise against it as you can get thousands of posts quickly!!!!! it's only there for test purposes.

Max Taxable 09-17-2013 01:58 PM

Quote:

Originally Posted by DemOnstar (Post 2446119)
How do I know if this is working?

Haven't seen any evidence so far...What do I look for?

Smoke. Smoke starts coming out of your hard drive. :D

Simon Lloyd 09-17-2013 03:37 PM

Quote:

Originally Posted by Max Taxable (Post 2446219)
Smoke. Smoke starts coming out of your hard drive. :D

Thats only if you're a power user :D

K4GAP 11-06-2013 06:24 AM

Should I make any changes to my robot.txt file?
Right now it is blank.

Simon Lloyd 11-06-2013 07:25 AM

Hi Gary this thread has nothing to do with robots.txt files, the mod bans anything whose useragent contains any string you enter in to it.

And as a standard you should have something in your robots file as you've been shown here https://vborg.vbsupport.ru/showthread.php?t=304164, there are many threads here that contain details of robots.txt.

K4GAP 11-06-2013 08:47 AM

Oh, I guess I need to learn more about user agents and robots.

Thanks'

Simon Lloyd 11-06-2013 09:13 AM

Hi Gary, all that you need to know about useragents...etc is in the thread description. Not all bots follow the robots.txt so, with this mod you can block those bots completely and many others. What you need to do is identify your target audience, so if you are not catering for China then you'd want to block Chinese traffic, to sort the bots out you can block the likes of Baidu Sogou....etc.

I'll try and help you with whatever you need along the way so that you get to keep your bandwidth for more important users :)


All times are GMT. The time now is 01:05 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01779 seconds
  • Memory Usage 1,735KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (3)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete