vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Miscellaneous Hacks - Ban Spiders by User Agent (https://vborg.vbsupport.ru/showthread.php?t=268208)

Simon Lloyd 11-09-2011 04:59 AM

Quote:

Originally Posted by bigtree (Post 2265918)
Thats a serious list. I would like to get rid of all BS spiders on my site but I worry about ranking? What effect does a list like this have?

How do we tell if a guest is a bad spider?

You can't other than doing lots of net searching, check out honeypot project and sites like that.

BadgerDog 11-09-2011 09:43 AM

1 Attachment(s)
Quote:

Originally Posted by Simon Lloyd (Post 2265920)
If you want to pm me access i'll take a look but it wont be until around 5pm gmt

Thanks ... :)

Here's a pic of my current "Who is On-line" display showing "Baidu" spiders with the mod active...

Regards,
Doug

Simon Lloyd 11-09-2011 04:39 PM

Doug i see, can you pm me a link to your site..etc, there's nothing much i can do without being able to investigate your site, one thing you might want to check is that you don't have any leading or trailing spaces around the Baidu in your list, so it shouldn't look like this
" Baidu" or "Baidu " (without the quotes of course :))

EasyEazy 11-12-2011 09:16 AM

Just what im looking for. Installed and will see how it goes.

Thank you.

fly 11-14-2011 05:50 PM

Thanks. I recently switched from Apache to Nginx. I used to have an .htaccess that worked great, but Nginx doesn't use em.

This saved me from having to figure out how to do it with Nginx. :D

KissOfDeath 11-16-2011 03:57 PM

Is it just spiders this bans or users aswell?

got a few spambots making spam posts for example

AtaTuvisio "173.193.227.141" resolves to 173.193.227.141-static.reverse.softlayer.com, so i've added "static.reverse.softlayer.com" to the list but it does not seem to be doing anything, am i understanding this wrong?

also have some the resolve to "localhost" what do i do about those?

Simon Lloyd 11-16-2011 04:19 PM

Localhost? thats your server then???
If you have banned that useragent then after your WOL timeout (and after they have tried moving to view another thread..etc) will be gone!

Anyway aside from that the resolved UA that you are showing isn't their UA its their IP resolution, go to WOL at the bottom there's a dropdown "Display User Agent" change that to yes and then find them :)

KissOfDeath 11-16-2011 05:29 PM

Quote:

Originally Posted by Simon Lloyd (Post 2268571)
Localhost? thats your server then???
If you have banned that useragent then after your WOL timeout (and after they have tried moving to view another thread..etc) will be gone!

Anyway aside from that the resolved UA that you are showing isn't their UA its their IP resolution, go to WOL at the bottom there's a dropdown "Display User Agent" change that to yes and then find them :)

Ah i understand now, thanks a lot of the spam posters are mozzila so i guess i can't really ban that, trying to find a ballance that does not wrongfully ban people, tried the plugins that use stopforum spam but the block someone just for using an isp like AOL or Ttiscali and you lose legitimate traffic.....

the local host ones are weird it's not my server the ip address is completely different to my server "113.165.164.99" for example

Simon Lloyd 11-16-2011 08:13 PM

No matter :), just ban their UA and you'll be good!

Alan_SP 11-25-2011 04:00 AM

Quote:

Originally Posted by Simon Lloyd (Post 2264649)
To NOT create threads (which of course was my recommendation) then simply uncheck the radio button for "Create Thread" :)

I installed vB4 version (I followed at first vB3 version). I have vB4.1.2PL4.

I read that someone had problems with "phantom" threads created after installing mod. I too just had this experience.

Of course in settings I selected that threads aren't created at all. Also, I had empty place for forum and user ID.

But threads where created, using user ID 1 (first user I created when installing forum). Forum ID is missing, I see empty name instead forum's name.

All of it I see with Valters Advanced Forum Statiscs: https://vborg.vbsupport.ru/showthread.php?t=235841

It looks like:

Activity from Bot No. 7 (Baidu) in your list 0 0 primus ▲ 25-11, 05:31

Note that the last thing is no forum name, i.e. empty space. :(

I can't see them in What's New. Or anywhere else in forum. When I try to access them I get error message. I don't know where they are really.

This is link to last one: http://slobodni.net/showthread.php?t...8&goto=newpost

Threads are created, but I don't know where. Also I don't know where can I remove them and how? 6 Threads are created at the moment.

Trying to prevent this, I put forum ID in settings as well as user ID, so I could at least delete threads upon creation.

So, mod creates threads somehow even if option is disabled. How and where, not sure, but I see them. :(

Interesting thing is, it created only this six threads and at the moment there aren't any new threads. I think there should be more spiders activity from banned bots. Very strange.


EDIT: Just noticed this:

You had this in conditions:
PHP Code:

$vbulletin->options ['bs_report_createthread' 

I just changed this to:
PHP Code:

$vbulletin->options['bs_report_createthread' 

Note there was one space. Not sure if this is bug that caused thread creation or not, but I just removed that one space.


All times are GMT. The time now is 06:29 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01655 seconds
  • Memory Usage 1,751KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_php_printable
  • (4)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete