The Arcive of Official vBulletin Modifications Site.

Ban Spiders by User Agent · **Released**: 08-08-2011

What this mod does
With this mod you can enter User Agents to watch or ban, you can also recieve emails or have an Output.txt created and updated with time and date of visits. It doesn't just have to be spiders, you can watch, log or ban any useragent!

How to install
Simply import the product ban_spider, the mod is active by default but none of the other options are turned on.

What is a UserAgent?
http://en.wikipedia.org/wiki/User_agent

Understanding a UserAgent string
http://user-agent-string.info/parse

Genuine User Getting Blocked?
https://vborg.vbsupport.ru/showpost....&postcount=105

Tools to help
http://whatsmyuseragent.com/SwitchingUserAgents.asp
http://www.botsvsbrowsers.com/SimulateUserAgent.asp

FAQ
https://vborg.vbsupport.ru/showpost....&postcount=137

How does it work?
https://vborg.vbsupport.ru/showpost....&postcount=381

What's a bot?
http://en.wikipedia.org/wiki/Spambot

How do i ban a bot?
https://vborg.vbsupport.ru/showpost....&postcount=318
https://vborg.vbsupport.ru/showpost....7&postcount=51

Where's output.txt located?
https://vborg.vbsupport.ru/showpost....&postcount=216

Bad bot lists
https://vborg.vbsupport.ru/showpost....&postcount=259
https://vborg.vbsupport.ru/showpost....&postcount=224
https://vborg.vbsupport.ru/showpost....&postcount=281

Tested on vb3.7.x, vB3.8.x , vB4.x.x but should work on any version.

__________________________________________________ __________________
Special thanks to:
Lior
KH99
BoP5
for helping me sort out a few issues

...and beta testers
ForceHSS (Special thanks to Force for latest testing)
ozzy47
GreyHost

If you use this please mark as INSTALLED

History
9th June 2011 Orginal xml added
12th June 2011 Added both email notification and text file logging
22nd June 2011 Version 2.0.0, Added create thread on activity

Added match facility you can now use something like Yandex and it will match MOZILLA/5.0 (COMPATIBLE; YANDEXBOT/3.0; +HTTP://YANDEX.COM/BOTS)
Added clickable link to visited thread

22nd September 2011 added user redirect url selection
08th October Beta testing started for thread creation.
20th October Beta testing started for emailing.
21st October Beta testing complete Ver 3.0.0 uploaded
29th October minor fix added to cope with empty userid on thread creation
30th October Beta testing automatic redirection to spiders/bots IP
31st October New xml uploaded with automatic redirect to IP
25th November Minor fix for blank forumid fixed
26th November 2011 Fixed version check & create thread Off by default
17th December 2014 Version 3.1.0 uploaded, Hook changed extra logging and statistics added by Ozzy47 (Chris)
18th December 2014 Version 3.1.1 uploaded, prevented spiders being counted when mod turned off.
17th December 2014 Version 3.1.2 uploaded, due to rogue code from another mod
The Bad Bots list is now included in the product
Please prune out all those that you wish to be able to see your site (i suggest you definately prune out "DA" and "Custo" :

Support will now only be given to those who have this mod marked as INSTALLED

ozzy47 · #**682** 12-12-2014, 10:56 AM

Yes that is a option, but it helps in no way to help figure out what the issue is with them making it through.

But I may have that solved, but I need to do further testing.

CAG CheechDogg · #**683** 12-12-2014, 11:11 AM

The thing is Baidu has tons of ips from where it comes from .... some aren't actually Baidu spiders or bots but simply bad bots disguising themselves as Baidu...check this list out for all the known ips and some user agents for Baidu:

USER AGENT: "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
IP: 119.63.196.10
IP: 119.63.196.102
IP: 119.63.196.103
IP: 119.63.196.104
IP: 119.63.196.105
IP: 119.63.196.106
IP: 119.63.196.107
IP: 119.63.196.108
IP: 119.63.196.109
IP: 119.63.196.11
IP: 119.63.196.110
IP: 119.63.196.111
IP: 119.63.196.112
IP: 119.63.196.113
IP: 119.63.196.114
IP: 119.63.196.115
IP: 119.63.196.116
IP: 119.63.196.117
IP: 119.63.196.119
IP: 119.63.196.12
IP: 119.63.196.120
IP: 119.63.196.13
IP: 119.63.196.15
IP: 119.63.196.16
IP: 119.63.196.17
IP: 119.63.196.18
IP: 119.63.196.19
IP: 119.63.196.20
IP: 119.63.196.21
IP: 119.63.196.22
IP: 119.63.196.23
IP: 119.63.196.24
IP: 119.63.196.25
IP: 119.63.196.26
IP: 119.63.196.27
IP: 119.63.196.39
IP: 119.63.196.40
IP: 119.63.196.41
IP: 119.63.196.42
IP: 119.63.196.43
IP: 119.63.196.45
IP: 119.63.196.47
IP: 119.63.196.48
IP: 119.63.196.50
IP: 119.63.196.51
IP: 119.63.196.52
IP: 119.63.196.53
IP: 119.63.196.55
IP: 119.63.196.56
IP: 119.63.196.57
IP: 119.63.196.73
IP: 119.63.196.74
IP: 119.63.196.75
IP: 119.63.196.76
IP: 119.63.196.77
IP: 119.63.196.78
IP: 119.63.196.79
IP: 119.63.196.80
IP: 119.63.196.81
IP: 119.63.196.82
IP: 119.63.196.83
IP: 119.63.196.84
IP: 119.63.196.85
IP: 119.63.196.86
IP: 119.63.196.88
IP: 119.63.196.89
IP: 123.125.71.100
IP: 123.125.71.101
IP: 123.125.71.102
IP: 123.125.71.103
IP: 123.125.71.104
IP: 123.125.71.105
IP: 123.125.71.106
IP: 123.125.71.107
IP: 123.125.71.108
IP: 123.125.71.109
IP: 123.125.71.110
IP: 123.125.71.111
IP: 123.125.71.112
IP: 123.125.71.113
IP: 123.125.71.114
IP: 123.125.71.115
IP: 123.125.71.116
IP: 123.125.71.117
IP: 123.125.71.70
IP: 123.125.71.71
IP: 123.125.71.81
IP: 123.125.71.83
IP: 123.125.71.94
IP: 123.125.71.95
IP: 123.125.71.96
IP: 123.125.71.97
IP: 123.125.71.98
IP: 123.125.71.99
IP: 180.76.5.100
IP: 180.76.5.103
IP: 180.76.5.111
IP: 180.76.5.137
IP: 180.76.5.141
IP: 180.76.5.142
IP: 180.76.5.145
IP: 180.76.5.146
IP: 180.76.5.147
IP: 180.76.5.148
IP: 180.76.5.150
IP: 180.76.5.151
IP: 180.76.5.155
IP: 180.76.5.159
IP: 180.76.5.161
IP: 180.76.5.162
IP: 180.76.5.166
IP: 180.76.5.167
IP: 180.76.5.170
IP: 180.76.5.171
IP: 180.76.5.172
IP: 180.76.5.176
IP: 180.76.5.177
IP: 180.76.5.178
IP: 180.76.5.179
IP: 180.76.5.180
IP: 180.76.5.181
IP: 180.76.5.182
IP: 180.76.5.183
IP: 180.76.5.185
IP: 180.76.5.187
IP: 180.76.5.189
IP: 180.76.5.190
IP: 180.76.5.192
IP: 180.76.5.193
IP: 180.76.5.194
IP: 180.76.5.195
IP: 180.76.5.196
IP: 180.76.5.197
IP: 180.76.5.49
IP: 180.76.5.51
IP: 180.76.5.54
IP: 180.76.5.55
IP: 180.76.5.57
IP: 180.76.5.58
IP: 180.76.5.59
IP: 180.76.5.62
IP: 180.76.5.63
IP: 180.76.5.64
IP: 180.76.5.65
IP: 180.76.5.67
IP: 180.76.5.87
IP: 180.76.5.88
IP: 180.76.5.90
IP: 180.76.5.92
IP: 180.76.5.93
IP: 180.76.5.94
IP: 180.76.5.95
IP: 180.76.5.96
IP: 180.76.5.97
IP: 180.76.5.98
IP: 180.76.5.99
IP: 180.76.6.21
IP: 180.76.6.212
IP: 180.76.6.213
IP: 180.76.6.222
IP: 180.76.6.223
IP: 180.76.6.224
IP: 180.76.6.231
IP: 180.76.6.28
IP: 180.76.6.29
IP: 180.76.6.36
IP: 180.76.6.37
IP: 199.36.73.116
IP: 220.181.108.100
IP: 220.181.108.107
IP: 220.181.108.110
IP: 220.181.108.81
IP: 220.181.108.84
IP: 220.181.108.93
IP: 220.181.108.95
IP: 220.181.108.97
IP: 222.186.24.59

USER AGENT: "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
IP: 67.159.56.162
IP: 72.11.144.119
IP: 119.63.196.10
IP: 119.63.196.103
IP: 119.63.196.105
IP: 119.63.196.106
IP: 119.63.196.11
IP: 119.63.196.113
IP: 119.63.196.116
IP: 119.63.196.117
IP: 119.63.196.118
IP: 119.63.196.120
IP: 119.63.196.124
IP: 119.63.196.125
IP: 119.63.196.13
IP: 119.63.196.16
IP: 119.63.196.19
IP: 119.63.196.20
IP: 119.63.196.23
IP: 119.63.196.25
IP: 119.63.196.26
IP: 119.63.196.27
IP: 119.63.196.28
IP: 119.63.196.30
IP: 119.63.196.31
IP: 119.63.196.32
IP: 119.63.196.38
IP: 119.63.196.40
IP: 119.63.196.42
IP: 119.63.196.43
IP: 119.63.196.44
IP: 119.63.196.45
IP: 119.63.196.47
IP: 119.63.196.53
IP: 119.63.196.54
IP: 119.63.196.57
IP: 119.63.196.58
IP: 119.63.196.60
IP: 119.63.196.61
IP: 119.63.196.76
IP: 119.63.196.78
IP: 119.63.196.80
IP: 119.63.196.82
IP: 119.63.196.88
IP: 119.63.196.89
IP: 119.63.196.9
IP: 119.63.196.93
IP: 119.63.196.94
IP: 119.63.196.96
IP: 123.125.67.164
IP: 123.125.71.100
IP: 123.125.71.101
IP: 123.125.71.102
IP: 123.125.71.103
IP: 123.125.71.104
IP: 123.125.71.105
IP: 123.125.71.106
IP: 123.125.71.107
IP: 123.125.71.108
IP: 123.125.71.109
IP: 123.125.71.110
IP: 123.125.71.111
IP: 123.125.71.112
IP: 123.125.71.113
IP: 123.125.71.114
IP: 123.125.71.115
IP: 123.125.71.116
IP: 123.125.71.117
IP: 123.125.71.12
IP: 123.125.71.13
IP: 123.125.71.14
IP: 123.125.71.15
IP: 123.125.71.16
IP: 123.125.71.17
IP: 123.125.71.18
IP: 123.125.71.19
IP: 123.125.71.20
IP: 123.125.71.21
IP: 123.125.71.22
IP: 123.125.71.23
IP: 123.125.71.24
IP: 123.125.71.25
IP: 123.125.71.26
IP: 123.125.71.27
IP: 123.125.71.28
IP: 123.125.71.29
IP: 123.125.71.30
IP: 123.125.71.31
IP: 123.125.71.32
IP: 123.125.71.33
IP: 123.125.71.34
IP: 123.125.71.35
IP: 123.125.71.36
IP: 123.125.71.38
IP: 123.125.71.39
IP: 123.125.71.40
IP: 123.125.71.41
IP: 123.125.71.42
IP: 123.125.71.43
IP: 123.125.71.44
IP: 123.125.71.45
IP: 123.125.71.46
IP: 123.125.71.47
IP: 123.125.71.48
IP: 123.125.71.49
IP: 123.125.71.50
IP: 123.125.71.51
IP: 123.125.71.52
IP: 123.125.71.53
IP: 123.125.71.54
IP: 123.125.71.55
IP: 123.125.71.56
IP: 123.125.71.57
IP: 123.125.71.58
IP: 123.125.71.59
IP: 123.125.71.60
IP: 123.125.71.69
IP: 123.125.71.70
IP: 123.125.71.71
IP: 123.125.71.72
IP: 123.125.71.73
IP: 123.125.71.74
IP: 123.125.71.75
IP: 123.125.71.76
IP: 123.125.71.77
IP: 123.125.71.78
IP: 123.125.71.79
IP: 123.125.71.80
IP: 123.125.71.81
IP: 123.125.71.82
IP: 123.125.71.83
IP: 123.125.71.84
IP: 123.125.71.85
IP: 123.125.71.86
IP: 123.125.71.87
IP: 123.125.71.88
IP: 123.125.71.89
IP: 123.125.71.90
IP: 123.125.71.91
IP: 123.125.71.92
IP: 123.125.71.94
IP: 123.125.71.95
IP: 123.125.71.96
IP: 123.125.71.97
IP: 123.125.71.98
IP: 123.125.71.99
IP: 125.39.78.168
IP: 125.39.78.171
IP: 125.39.78.173
IP: 125.39.78.174
IP: 125.39.78.177
IP: 125.39.78.179
IP: 125.39.78.181
IP: 125.39.78.183
IP: 125.39.78.185
IP: 125.39.78.187
IP: 125.39.78.188
IP: 125.39.78.189
IP: 125.90.93.141
IP: 173.236.136.101
IP: 180.76.5.100
IP: 180.76.5.101
IP: 180.76.5.103
IP: 180.76.5.107
IP: 180.76.5.110
IP: 180.76.5.111
IP: 180.76.5.113
IP: 180.76.5.136
IP: 180.76.5.137
IP: 180.76.5.138
IP: 180.76.5.139
IP: 180.76.5.140
IP: 180.76.5.141
IP: 180.76.5.142
IP: 180.76.5.143
IP: 180.76.5.144
IP: 180.76.5.145
IP: 180.76.5.146
IP: 180.76.5.147
IP: 180.76.5.148
IP: 180.76.5.149
IP: 180.76.5.150
IP: 180.76.5.151
IP: 180.76.5.153
IP: 180.76.5.154
IP: 180.76.5.155
IP: 180.76.5.156
IP: 180.76.5.157
IP: 180.76.5.158
IP: 180.76.5.159
IP: 180.76.5.160
IP: 180.76.5.161
IP: 180.76.5.162
IP: 180.76.5.163
IP: 180.76.5.164
IP: 180.76.5.165
IP: 180.76.5.166
IP: 180.76.5.167
IP: 180.76.5.168
IP: 180.76.5.169
IP: 180.76.5.170
IP: 180.76.5.171
IP: 180.76.5.172
IP: 180.76.5.173
IP: 180.76.5.175
IP: 180.76.5.176
IP: 180.76.5.177
IP: 180.76.5.178
IP: 180.76.5.179
IP: 180.76.5.180
IP: 180.76.5.181
IP: 180.76.5.182
IP: 180.76.5.183
IP: 180.76.5.184
IP: 180.76.5.185
IP: 180.76.5.186
IP: 180.76.5.187
IP: 180.76.5.188
IP: 180.76.5.189
IP: 180.76.5.190
IP: 180.76.5.191
IP: 180.76.5.192
IP: 180.76.5.193
IP: 180.76.5.194
IP: 180.76.5.195
IP: 180.76.5.196
IP: 180.76.5.197
IP: 180.76.5.48
IP: 180.76.5.49
IP: 180.76.5.50
IP: 180.76.5.51
IP: 180.76.5.52
IP: 180.76.5.53
IP: 180.76.5.54
IP: 180.76.5.55
IP: 180.76.5.56
IP: 180.76.5.57
IP: 180.76.5.58
IP: 180.76.5.59
IP: 180.76.5.60
IP: 180.76.5.61
IP: 180.76.5.62
IP: 180.76.5.63
IP: 180.76.5.64
IP: 180.76.5.65
IP: 180.76.5.66
IP: 180.76.5.67
IP: 180.76.5.87
IP: 180.76.5.88
IP: 180.76.5.89
IP: 180.76.5.90
IP: 180.76.5.91
IP: 180.76.5.92
IP: 180.76.5.93
IP: 180.76.5.94
IP: 180.76.5.95
IP: 180.76.5.96
IP: 180.76.5.97
IP: 180.76.5.98
IP: 180.76.5.99
IP: 180.76.6.20
IP: 180.76.6.21
IP: 180.76.6.211
IP: 180.76.6.212
IP: 180.76.6.213
IP: 180.76.6.222
IP: 180.76.6.223
IP: 180.76.6.224
IP: 180.76.6.225
IP: 180.76.6.227
IP: 180.76.6.230
IP: 180.76.6.231
IP: 180.76.6.232
IP: 180.76.6.233
IP: 180.76.6.26
IP: 180.76.6.28
IP: 180.76.6.29
IP: 180.76.6.35
IP: 180.76.6.36
IP: 180.76.6.37
IP: 204.45.133.74
IP: 220.181.108.165
IP: 220.181.108.166
IP: 220.181.108.167
IP: 220.181.108.168
IP: 220.181.108.169
IP: 220.181.108.170
IP: 220.181.108.171
IP: 220.181.108.172
IP: 220.181.108.173
IP: 220.181.108.174
IP: 220.181.108.175
IP: 220.181.108.176
IP: 220.181.108.177
IP: 220.181.108.178
IP: 220.181.108.179
IP: 220.181.108.180
IP: 220.181.108.181
IP: 220.181.108.182
IP: 220.181.108.183
IP: 220.181.108.184
IP: 220.181.108.185
IP: 220.181.108.186
IP: 220.181.108.187
IP: 220.181.108.79

USER AGENT: "Mozilla/5.0+(compatible;+Baiduspider/2.0;++http://www.baidu.com/search/spider.html)"
IP: 222.76.212.176

CAG CheechDogg · #**684** 12-12-2014, 11:12 AM

Just block those ips and get it over with ...

Gadget_Guy · #**685** 12-12-2014, 02:26 PM

agreed that I can just deal with it via my firewall.

However, I am providing these reports to help the team with improving the mod.

I get so much from this site that I want to try and give back as much as I can.... so if it can help others who don't have firewalls etc then I feel good about trying to help.

I will continue to try things this way to give my experiences... once they get the mod doing what they want it to do as Ozzy eludes to, I will most likely just go the firewall route.

D.

ozzy47 · #**686** 12-12-2014, 10:51 PM

Well back to the drawing board.

ozzy47 · #**687** 12-13-2014, 09:41 AM

Ok here they are showing up in WOL, and they should not be.
I went back to the way the mod is now, and I disabled any other mod running on style_fetch so it is not a hook conflict.

Now I will go back to a recoded version, and try some different hook locations.

Simon Lloyd · #**688** 12-14-2014, 08:08 AM

If anyone has a test vb4 install that is open to bot access i'd appreciate having admin access to try a few things, I ask this because I can only install an early version of vb4 that comes with my 3.8.7 licence. and I suspect its the later versions that are having this anomaly.

ozzy47 · #**689** 12-14-2014, 12:56 PM

Simon, I can send you a PM on my site, which would contain the XML's for vB3 and vB4.

The new way has so far seemed to successfully blocked them, as well they do not show up in Paul's mod anymore.

I also have added the ability to change the name of the log file, to a different name, in case admins want to change it, as well as a counter of how many bots are blocked using the mod, that shows up in the site stats on the forum home. Now this is of course starting from the time you install the updated version, as there is no way to get a correct count from before. There is also a setting to select which usergroups can see the stats.

Max Taxable · #**690** 12-14-2014, 04:48 PM

Quote:

Originally Posted by Simon Lloyd

If anyone has a test vb4 install that is open to bot access i'd appreciate having admin access to try a few things, I ask this because I can only install an early version of vb4 that comes with my 3.8.7 licence. and I suspect its the later versions that are having this anomaly.

It's going to be v4s that are heavily modded, having the issues. My 4.2.2 is lightly modded, and this hack works perfectly and always has.

Gadget_Guy · #**691** 12-14-2014, 04:53 PM

I have 4.2.2 and I think I am heavily modded(ish)

Your call.

D.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.13733 seconds Memory Usage 2,388KB Queries Executed 29 (?)
More Information
Template Usage: (1)SHOWTHREAD (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)ad_showthread_beforeqr (1)bbcode_quote (1)footer (1)forumjump (1)forumrules (1)gobutton (1)header (1)headinclude (1)modsystem_post (1)navbar (4)navbar_link (120)option (1)pagenav (1)pagenav_curpage (4)pagenav_pagelink (3)pagenav_pagelinkrel (11)post_thanks_box (20)post_thanks_box_bit (11)post_thanks_button (1)post_thanks_javascript (1)post_thanks_navbar_search (5)post_thanks_postbit (11)post_thanks_postbit_info (10)postbit (3)postbit_attachment (11)postbit_onlinestatus (11)postbit_wrapper (1)spacer_close (1)spacer_open (1)tagbit_wrapper Phrase Groups Available: global inlinemod postbit posting reputationlevel showthread	Included Files: ./showthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/functions_bigthree.php ./includes/class_postbit.php ./includes/class_bbcode.php ./includes/functions_reputation.php ./includes/functions_post_thanks.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_postinfo_query fetch_postinfo fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete showthread_start showthread_getinfo forumjump showthread_post_start showthread_query_postids showthread_query bbcode_fetch_tags bbcode_create showthread_postbit_create postbit_factory postbit_display_start post_thanks_function_post_thanks_off_start post_thanks_function_post_thanks_off_end post_thanks_function_fetch_thanks_start fetch_musername post_thanks_function_fetch_thanks_end post_thanks_function_thanked_already_start post_thanks_function_thanked_already_end post_thanks_function_fetch_thanks_bit_start post_thanks_function_show_thanks_date_start post_thanks_function_show_thanks_date_end post_thanks_function_fetch_thanks_bit_end post_thanks_function_fetch_post_thanks_template_start post_thanks_function_fetch_post_thanks_template_end postbit_imicons bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete postbit_attachment postbit_display_complete post_thanks_function_can_thank_this_post_start pagenav_page pagenav_complete tag_fetchbit_complete forumrules navbits navbits_complete showthread_complete
Messages:

16 благодарности(ей) от:
ASCONAN, BlueCheri, Disco_Dave, ForceHSS, Gn_Snake, GreyGhost, htscpl, Michael.A, midnz, obglobal.net, puertoblack2003, RichieBoy67, rootsxrocks, tanzeelniazi, too_cool_3, victorvu

Благодарность от:
CAG CheechDogg

Благодарность от:
CAG CheechDogg

The Arcive of Official vBulletin Modifications Site.

It is not a VB3 engine, just a parsed copy!