vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Miscellaneous Hacks - Ban Spiders by User Agent (https://vborg.vbsupport.ru/showthread.php?t=268208)

ozzy47 12-12-2014 10:56 AM

Yes that is a option, but it helps in no way to help figure out what the issue is with them making it through.

But I may have that solved, but I need to do further testing. :)

CAG CheechDogg 12-12-2014 11:11 AM

The thing is Baidu has tons of ips from where it comes from .... some aren't actually Baidu spiders or bots but simply bad bots disguising themselves as Baidu...check this list out for all the known ips and some user agents for Baidu:

USER AGENT: "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
IP: 119.63.196.10
IP: 119.63.196.102
IP: 119.63.196.103
IP: 119.63.196.104
IP: 119.63.196.105
IP: 119.63.196.106
IP: 119.63.196.107
IP: 119.63.196.108
IP: 119.63.196.109
IP: 119.63.196.11
IP: 119.63.196.110
IP: 119.63.196.111
IP: 119.63.196.112
IP: 119.63.196.113
IP: 119.63.196.114
IP: 119.63.196.115
IP: 119.63.196.116
IP: 119.63.196.117
IP: 119.63.196.119
IP: 119.63.196.12
IP: 119.63.196.120
IP: 119.63.196.13
IP: 119.63.196.15
IP: 119.63.196.16
IP: 119.63.196.17
IP: 119.63.196.18
IP: 119.63.196.19
IP: 119.63.196.20
IP: 119.63.196.21
IP: 119.63.196.22
IP: 119.63.196.23
IP: 119.63.196.24
IP: 119.63.196.25
IP: 119.63.196.26
IP: 119.63.196.27
IP: 119.63.196.39
IP: 119.63.196.40
IP: 119.63.196.41
IP: 119.63.196.42
IP: 119.63.196.43
IP: 119.63.196.45
IP: 119.63.196.47
IP: 119.63.196.48
IP: 119.63.196.50
IP: 119.63.196.51
IP: 119.63.196.52
IP: 119.63.196.53
IP: 119.63.196.55
IP: 119.63.196.56
IP: 119.63.196.57
IP: 119.63.196.73
IP: 119.63.196.74
IP: 119.63.196.75
IP: 119.63.196.76
IP: 119.63.196.77
IP: 119.63.196.78
IP: 119.63.196.79
IP: 119.63.196.80
IP: 119.63.196.81
IP: 119.63.196.82
IP: 119.63.196.83
IP: 119.63.196.84
IP: 119.63.196.85
IP: 119.63.196.86
IP: 119.63.196.88
IP: 119.63.196.89
IP: 123.125.71.100
IP: 123.125.71.101
IP: 123.125.71.102
IP: 123.125.71.103
IP: 123.125.71.104
IP: 123.125.71.105
IP: 123.125.71.106
IP: 123.125.71.107
IP: 123.125.71.108
IP: 123.125.71.109
IP: 123.125.71.110
IP: 123.125.71.111
IP: 123.125.71.112
IP: 123.125.71.113
IP: 123.125.71.114
IP: 123.125.71.115
IP: 123.125.71.116
IP: 123.125.71.117
IP: 123.125.71.70
IP: 123.125.71.71
IP: 123.125.71.81
IP: 123.125.71.83
IP: 123.125.71.94
IP: 123.125.71.95
IP: 123.125.71.96
IP: 123.125.71.97
IP: 123.125.71.98
IP: 123.125.71.99
IP: 180.76.5.100
IP: 180.76.5.103
IP: 180.76.5.111
IP: 180.76.5.137
IP: 180.76.5.141
IP: 180.76.5.142
IP: 180.76.5.145
IP: 180.76.5.146
IP: 180.76.5.147
IP: 180.76.5.148
IP: 180.76.5.150
IP: 180.76.5.151
IP: 180.76.5.155
IP: 180.76.5.159
IP: 180.76.5.161
IP: 180.76.5.162
IP: 180.76.5.166
IP: 180.76.5.167
IP: 180.76.5.170
IP: 180.76.5.171
IP: 180.76.5.172
IP: 180.76.5.176
IP: 180.76.5.177
IP: 180.76.5.178
IP: 180.76.5.179
IP: 180.76.5.180
IP: 180.76.5.181
IP: 180.76.5.182
IP: 180.76.5.183
IP: 180.76.5.185
IP: 180.76.5.187
IP: 180.76.5.189
IP: 180.76.5.190
IP: 180.76.5.192
IP: 180.76.5.193
IP: 180.76.5.194
IP: 180.76.5.195
IP: 180.76.5.196
IP: 180.76.5.197
IP: 180.76.5.49
IP: 180.76.5.51
IP: 180.76.5.54
IP: 180.76.5.55
IP: 180.76.5.57
IP: 180.76.5.58
IP: 180.76.5.59
IP: 180.76.5.62
IP: 180.76.5.63
IP: 180.76.5.64
IP: 180.76.5.65
IP: 180.76.5.67
IP: 180.76.5.87
IP: 180.76.5.88
IP: 180.76.5.90
IP: 180.76.5.92
IP: 180.76.5.93
IP: 180.76.5.94
IP: 180.76.5.95
IP: 180.76.5.96
IP: 180.76.5.97
IP: 180.76.5.98
IP: 180.76.5.99
IP: 180.76.6.21
IP: 180.76.6.212
IP: 180.76.6.213
IP: 180.76.6.222
IP: 180.76.6.223
IP: 180.76.6.224
IP: 180.76.6.231
IP: 180.76.6.28
IP: 180.76.6.29
IP: 180.76.6.36
IP: 180.76.6.37
IP: 199.36.73.116
IP: 220.181.108.100
IP: 220.181.108.107
IP: 220.181.108.110
IP: 220.181.108.81
IP: 220.181.108.84
IP: 220.181.108.93
IP: 220.181.108.95
IP: 220.181.108.97
IP: 222.186.24.59

USER AGENT: "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
IP: 67.159.56.162
IP: 72.11.144.119
IP: 119.63.196.10
IP: 119.63.196.103
IP: 119.63.196.105
IP: 119.63.196.106
IP: 119.63.196.11
IP: 119.63.196.113
IP: 119.63.196.116
IP: 119.63.196.117
IP: 119.63.196.118
IP: 119.63.196.120
IP: 119.63.196.124
IP: 119.63.196.125
IP: 119.63.196.13
IP: 119.63.196.16
IP: 119.63.196.19
IP: 119.63.196.20
IP: 119.63.196.23
IP: 119.63.196.25
IP: 119.63.196.26
IP: 119.63.196.27
IP: 119.63.196.28
IP: 119.63.196.30
IP: 119.63.196.31
IP: 119.63.196.32
IP: 119.63.196.38
IP: 119.63.196.40
IP: 119.63.196.42
IP: 119.63.196.43
IP: 119.63.196.44
IP: 119.63.196.45
IP: 119.63.196.47
IP: 119.63.196.53
IP: 119.63.196.54
IP: 119.63.196.57
IP: 119.63.196.58
IP: 119.63.196.60
IP: 119.63.196.61
IP: 119.63.196.76
IP: 119.63.196.78
IP: 119.63.196.80
IP: 119.63.196.82
IP: 119.63.196.88
IP: 119.63.196.89
IP: 119.63.196.9
IP: 119.63.196.93
IP: 119.63.196.94
IP: 119.63.196.96
IP: 123.125.67.164
IP: 123.125.71.100
IP: 123.125.71.101
IP: 123.125.71.102
IP: 123.125.71.103
IP: 123.125.71.104
IP: 123.125.71.105
IP: 123.125.71.106
IP: 123.125.71.107
IP: 123.125.71.108
IP: 123.125.71.109
IP: 123.125.71.110
IP: 123.125.71.111
IP: 123.125.71.112
IP: 123.125.71.113
IP: 123.125.71.114
IP: 123.125.71.115
IP: 123.125.71.116
IP: 123.125.71.117
IP: 123.125.71.12
IP: 123.125.71.13
IP: 123.125.71.14
IP: 123.125.71.15
IP: 123.125.71.16
IP: 123.125.71.17
IP: 123.125.71.18
IP: 123.125.71.19
IP: 123.125.71.20
IP: 123.125.71.21
IP: 123.125.71.22
IP: 123.125.71.23
IP: 123.125.71.24
IP: 123.125.71.25
IP: 123.125.71.26
IP: 123.125.71.27
IP: 123.125.71.28
IP: 123.125.71.29
IP: 123.125.71.30
IP: 123.125.71.31
IP: 123.125.71.32
IP: 123.125.71.33
IP: 123.125.71.34
IP: 123.125.71.35
IP: 123.125.71.36
IP: 123.125.71.38
IP: 123.125.71.39
IP: 123.125.71.40
IP: 123.125.71.41
IP: 123.125.71.42
IP: 123.125.71.43
IP: 123.125.71.44
IP: 123.125.71.45
IP: 123.125.71.46
IP: 123.125.71.47
IP: 123.125.71.48
IP: 123.125.71.49
IP: 123.125.71.50
IP: 123.125.71.51
IP: 123.125.71.52
IP: 123.125.71.53
IP: 123.125.71.54
IP: 123.125.71.55
IP: 123.125.71.56
IP: 123.125.71.57
IP: 123.125.71.58
IP: 123.125.71.59
IP: 123.125.71.60
IP: 123.125.71.69
IP: 123.125.71.70
IP: 123.125.71.71
IP: 123.125.71.72
IP: 123.125.71.73
IP: 123.125.71.74
IP: 123.125.71.75
IP: 123.125.71.76
IP: 123.125.71.77
IP: 123.125.71.78
IP: 123.125.71.79
IP: 123.125.71.80
IP: 123.125.71.81
IP: 123.125.71.82
IP: 123.125.71.83
IP: 123.125.71.84
IP: 123.125.71.85
IP: 123.125.71.86
IP: 123.125.71.87
IP: 123.125.71.88
IP: 123.125.71.89
IP: 123.125.71.90
IP: 123.125.71.91
IP: 123.125.71.92
IP: 123.125.71.94
IP: 123.125.71.95
IP: 123.125.71.96
IP: 123.125.71.97
IP: 123.125.71.98
IP: 123.125.71.99
IP: 125.39.78.168
IP: 125.39.78.171
IP: 125.39.78.173
IP: 125.39.78.174
IP: 125.39.78.177
IP: 125.39.78.179
IP: 125.39.78.181
IP: 125.39.78.183
IP: 125.39.78.185
IP: 125.39.78.187
IP: 125.39.78.188
IP: 125.39.78.189
IP: 125.90.93.141
IP: 173.236.136.101
IP: 180.76.5.100
IP: 180.76.5.101
IP: 180.76.5.103
IP: 180.76.5.107
IP: 180.76.5.110
IP: 180.76.5.111
IP: 180.76.5.113
IP: 180.76.5.136
IP: 180.76.5.137
IP: 180.76.5.138
IP: 180.76.5.139
IP: 180.76.5.140
IP: 180.76.5.141
IP: 180.76.5.142
IP: 180.76.5.143
IP: 180.76.5.144
IP: 180.76.5.145
IP: 180.76.5.146
IP: 180.76.5.147
IP: 180.76.5.148
IP: 180.76.5.149
IP: 180.76.5.150
IP: 180.76.5.151
IP: 180.76.5.153
IP: 180.76.5.154
IP: 180.76.5.155
IP: 180.76.5.156
IP: 180.76.5.157
IP: 180.76.5.158
IP: 180.76.5.159
IP: 180.76.5.160
IP: 180.76.5.161
IP: 180.76.5.162
IP: 180.76.5.163
IP: 180.76.5.164
IP: 180.76.5.165
IP: 180.76.5.166
IP: 180.76.5.167
IP: 180.76.5.168
IP: 180.76.5.169
IP: 180.76.5.170
IP: 180.76.5.171
IP: 180.76.5.172
IP: 180.76.5.173
IP: 180.76.5.175
IP: 180.76.5.176
IP: 180.76.5.177
IP: 180.76.5.178
IP: 180.76.5.179
IP: 180.76.5.180
IP: 180.76.5.181
IP: 180.76.5.182
IP: 180.76.5.183
IP: 180.76.5.184
IP: 180.76.5.185
IP: 180.76.5.186
IP: 180.76.5.187
IP: 180.76.5.188
IP: 180.76.5.189
IP: 180.76.5.190
IP: 180.76.5.191
IP: 180.76.5.192
IP: 180.76.5.193
IP: 180.76.5.194
IP: 180.76.5.195
IP: 180.76.5.196
IP: 180.76.5.197
IP: 180.76.5.48
IP: 180.76.5.49
IP: 180.76.5.50
IP: 180.76.5.51
IP: 180.76.5.52
IP: 180.76.5.53
IP: 180.76.5.54
IP: 180.76.5.55
IP: 180.76.5.56
IP: 180.76.5.57
IP: 180.76.5.58
IP: 180.76.5.59
IP: 180.76.5.60
IP: 180.76.5.61
IP: 180.76.5.62
IP: 180.76.5.63
IP: 180.76.5.64
IP: 180.76.5.65
IP: 180.76.5.66
IP: 180.76.5.67
IP: 180.76.5.87
IP: 180.76.5.88
IP: 180.76.5.89
IP: 180.76.5.90
IP: 180.76.5.91
IP: 180.76.5.92
IP: 180.76.5.93
IP: 180.76.5.94
IP: 180.76.5.95
IP: 180.76.5.96
IP: 180.76.5.97
IP: 180.76.5.98
IP: 180.76.5.99
IP: 180.76.6.20
IP: 180.76.6.21
IP: 180.76.6.211
IP: 180.76.6.212
IP: 180.76.6.213
IP: 180.76.6.222
IP: 180.76.6.223
IP: 180.76.6.224
IP: 180.76.6.225
IP: 180.76.6.227
IP: 180.76.6.230
IP: 180.76.6.231
IP: 180.76.6.232
IP: 180.76.6.233
IP: 180.76.6.26
IP: 180.76.6.28
IP: 180.76.6.29
IP: 180.76.6.35
IP: 180.76.6.36
IP: 180.76.6.37
IP: 204.45.133.74
IP: 220.181.108.165
IP: 220.181.108.166
IP: 220.181.108.167
IP: 220.181.108.168
IP: 220.181.108.169
IP: 220.181.108.170
IP: 220.181.108.171
IP: 220.181.108.172
IP: 220.181.108.173
IP: 220.181.108.174
IP: 220.181.108.175
IP: 220.181.108.176
IP: 220.181.108.177
IP: 220.181.108.178
IP: 220.181.108.179
IP: 220.181.108.180
IP: 220.181.108.181
IP: 220.181.108.182
IP: 220.181.108.183
IP: 220.181.108.184
IP: 220.181.108.185
IP: 220.181.108.186
IP: 220.181.108.187
IP: 220.181.108.79

USER AGENT: "Mozilla/5.0+(compatible;+Baiduspider/2.0;++http://www.baidu.com/search/spider.html)"
IP: 222.76.212.176

CAG CheechDogg 12-12-2014 11:12 AM

Just block those ips and get it over with ...

Gadget_Guy 12-12-2014 02:26 PM

agreed that I can just deal with it via my firewall.

However, I am providing these reports to help the team with improving the mod.

I get so much from this site that I want to try and give back as much as I can.... so if it can help others who don't have firewalls etc then I feel good about trying to help.

I will continue to try things this way to give my experiences... once they get the mod doing what they want it to do as Ozzy eludes to, I will most likely just go the firewall route.

D.

ozzy47 12-12-2014 10:51 PM

Well back to the drawing board. :(

ozzy47 12-13-2014 09:41 AM

1 Attachment(s)
Ok here they are showing up in WOL, and they should not be.
I went back to the way the mod is now, and I disabled any other mod running on style_fetch so it is not a hook conflict.

Now I will go back to a recoded version, and try some different hook locations.

Simon Lloyd 12-14-2014 08:08 AM

If anyone has a test vb4 install that is open to bot access i'd appreciate having admin access to try a few things, I ask this because I can only install an early version of vb4 that comes with my 3.8.7 licence. and I suspect its the later versions that are having this anomaly.

ozzy47 12-14-2014 12:56 PM

Simon, I can send you a PM on my site, which would contain the XML's for vB3 and vB4.

The new way has so far seemed to successfully blocked them, as well they do not show up in Paul's mod anymore.

I also have added the ability to change the name of the log file, to a different name, in case admins want to change it, as well as a counter of how many bots are blocked using the mod, that shows up in the site stats on the forum home. Now this is of course starting from the time you install the updated version, as there is no way to get a correct count from before. There is also a setting to select which usergroups can see the stats. :)

Max Taxable 12-14-2014 04:48 PM

Quote:

Originally Posted by Simon Lloyd (Post 2527150)
If anyone has a test vb4 install that is open to bot access i'd appreciate having admin access to try a few things, I ask this because I can only install an early version of vb4 that comes with my 3.8.7 licence. and I suspect its the later versions that are having this anomaly.

It's going to be v4s that are heavily modded, having the issues. My 4.2.2 is lightly modded, and this hack works perfectly and always has.

Gadget_Guy 12-14-2014 04:53 PM

I have 4.2.2 and I think I am heavily modded(ish)

Your call.

D.


All times are GMT. The time now is 05:17 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.04290 seconds
  • Memory Usage 1,792KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete