vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.8 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=235)
-   -   Miscellaneous Hacks - Ban Spiders by User Agent (https://vborg.vbsupport.ru/showthread.php?t=264932)

Simon Lloyd 07-26-2013 05:45 AM

I doubt there's a pc on the planet that still uses MSIE 1,2 or 3 :)

Remember this mod was built to ban spiders or bots which usually display their bot name in their UA, using the mod outside of that scope is bound to take you in to some grey areas, make sure you use the tools shown in the original post of this thread to check new UA's before you ban them.

Dan49 07-26-2013 11:24 AM

Is there a way to block MSIE 1 without affecting MSIE 10?

Simon Lloyd 07-26-2013 02:19 PM

All you can do is add a space after it, but again all it will do is look for MSIE 1_ where _ is the space (it doesn't physically look like _ that was for illustration purposes), but if it doesn't appear in the UA with a space after it, it wont ban it.

Dan49 07-26-2013 02:56 PM

Thanks.

Another general question. I understand that malicious bots can fake the UA. Why don't they all fake it to display as Google? This way anybody would be hesitant to block them? BTW I do use your IP ban mod also.

Max Taxable 07-26-2013 03:55 PM

Quote:

Originally Posted by Dan49 (Post 2435415)
Max, you may be interested in this. I had a legit user blocked out of my forum. I checked and rechecked his UA and could not figure out why he was blocked
Code:

Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
Of course turning off the mod unblocked him, so I knew it had to be something.

On a hunch I looked at the MSIE entries I copied from your old post:

MSIE 1
MSIE 2
MSIE 3
MSIE 4
MSIE 5
MSIE 6

I realized that MSIE 1 is the culprit being the first part of MSIE 10.0;

Have you since removed MSIE 1 from your list?

Is there another way to make MSIE 1 work without affecting MSIE 10? As well as the other MSIE, like MSIE 2 for when IE gets to version 20 :)?

Yes this was covered earlier in the thread, and have since removed MSIE 1 from my list. This was months ago. Like Simon said, the number of surviving computers on the planet still using MSIE1 is so tiny, it's not worth blocking.

Max Taxable 07-26-2013 03:57 PM

Quote:

Originally Posted by Dan49 (Post 2435482)
Thanks.

Another general question. I understand that malicious bots can fake the UA. Why don't they all fake it to display as Google? This way anybody would be hesitant to block them? BTW I do use your IP ban mod also.

Some of them do so, but it's very easy to spot those fakes since Google only uses one or two IP addresses. I am pretty sure there's a Mod for blocking spoofed google UA's somewhere.

Dan49 07-26-2013 07:42 PM

I found this information about verifying Google bot. And used this tool http://ipadmin.junkemailfilter.com/rdns.php. Is this the best way? Or is there a list of the IP addresses google uses?

I'd appreciate a link to the mod you mentioned, I searched and can't find it.

valdet 07-26-2013 08:46 PM

Hi Simon,

Hope you got your forum problems sorted :)

I noticed that the mod will not create new threads on private forums, where only admins can post/read.
I am using my admin account as thread creator, so it has permissions to create threads. The threads are created only on public forums where other members and guests can see them, which of course isn't nice.

The output.txt file writing is also not logging anything.

But from my server logs, I see this mod has been doing incredibly well in blocking some nasty spiders (mainly Magpie and Spinn3r).

For those who don't check their server logs, I would advise to check them and see if Magpie is sucking up to 40% of bandwidth as it was until recently in one of my sites.

Max Taxable 07-26-2013 08:48 PM

Quote:

Originally Posted by Dan49 (Post 2435539)
I found this information about verifying Google bot. And used this tool http://ipadmin.junkemailfilter.com/rdns.php. Is this the best way? Or is there a list of the IP addresses google uses?

I'd appreciate a link to the mod you mentioned, I searched and can't find it.

I only vaguely remember seeing such, and it was sometime back. And I might even be wrong.

Simon Lloyd 07-26-2013 09:15 PM

Quote:

Originally Posted by valdet (Post 2435544)
Hi Simon,

Hope you got your forum problems sorted :)

I noticed that the mod will not create new threads on private forums, where only admins can post/read.
I am using my admin account as thread creator, so it has permissions to create threads. The threads are created only on public forums where other members and guests can see them, which of course isn't nice.

The output.txt file writing is also not logging anything.

But from my server logs, I see this mod has been doing incredibly well in blocking some nasty spiders (mainly Magpie and Spinn3r).

For those who don't check their server logs, I would advise to check them and see if Magpie is sucking up to 40% of bandwidth as it was until recently in one of my sites.

Glad it's working for you, on my forum I simply added a forum where only staf had permissions to view and made the threads there, I also had no issue with logging.

I did get my forum issues sorted, turns out it was a half assed script kiddie attempt to kill it but got it sorted (one of my forums allowed html in the post environment....really bad idea!). When I get chance (which will be after Thursday as my wife is having an op) I will retest the mod from scratch and post specifics here.


All times are GMT. The time now is 12:25 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01280 seconds
  • Memory Usage 1,747KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_code_printable
  • (4)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete