![]() |
Yes that is a option, but it helps in no way to help figure out what the issue is with them making it through.
But I may have that solved, but I need to do further testing. :) |
The thing is Baidu has tons of ips from where it comes from .... some aren't actually Baidu spiders or bots but simply bad bots disguising themselves as Baidu...check this list out for all the known ips and some user agents for Baidu:
USER AGENT: "Baiduspider+(+http://www.baidu.com/search/spider.htm)" IP: 119.63.196.10 IP: 119.63.196.102 IP: 119.63.196.103 IP: 119.63.196.104 IP: 119.63.196.105 IP: 119.63.196.106 IP: 119.63.196.107 IP: 119.63.196.108 IP: 119.63.196.109 IP: 119.63.196.11 IP: 119.63.196.110 IP: 119.63.196.111 IP: 119.63.196.112 IP: 119.63.196.113 IP: 119.63.196.114 IP: 119.63.196.115 IP: 119.63.196.116 IP: 119.63.196.117 IP: 119.63.196.119 IP: 119.63.196.12 IP: 119.63.196.120 IP: 119.63.196.13 IP: 119.63.196.15 IP: 119.63.196.16 IP: 119.63.196.17 IP: 119.63.196.18 IP: 119.63.196.19 IP: 119.63.196.20 IP: 119.63.196.21 IP: 119.63.196.22 IP: 119.63.196.23 IP: 119.63.196.24 IP: 119.63.196.25 IP: 119.63.196.26 IP: 119.63.196.27 IP: 119.63.196.39 IP: 119.63.196.40 IP: 119.63.196.41 IP: 119.63.196.42 IP: 119.63.196.43 IP: 119.63.196.45 IP: 119.63.196.47 IP: 119.63.196.48 IP: 119.63.196.50 IP: 119.63.196.51 IP: 119.63.196.52 IP: 119.63.196.53 IP: 119.63.196.55 IP: 119.63.196.56 IP: 119.63.196.57 IP: 119.63.196.73 IP: 119.63.196.74 IP: 119.63.196.75 IP: 119.63.196.76 IP: 119.63.196.77 IP: 119.63.196.78 IP: 119.63.196.79 IP: 119.63.196.80 IP: 119.63.196.81 IP: 119.63.196.82 IP: 119.63.196.83 IP: 119.63.196.84 IP: 119.63.196.85 IP: 119.63.196.86 IP: 119.63.196.88 IP: 119.63.196.89 IP: 123.125.71.100 IP: 123.125.71.101 IP: 123.125.71.102 IP: 123.125.71.103 IP: 123.125.71.104 IP: 123.125.71.105 IP: 123.125.71.106 IP: 123.125.71.107 IP: 123.125.71.108 IP: 123.125.71.109 IP: 123.125.71.110 IP: 123.125.71.111 IP: 123.125.71.112 IP: 123.125.71.113 IP: 123.125.71.114 IP: 123.125.71.115 IP: 123.125.71.116 IP: 123.125.71.117 IP: 123.125.71.70 IP: 123.125.71.71 IP: 123.125.71.81 IP: 123.125.71.83 IP: 123.125.71.94 IP: 123.125.71.95 IP: 123.125.71.96 IP: 123.125.71.97 IP: 123.125.71.98 IP: 123.125.71.99 IP: 180.76.5.100 IP: 180.76.5.103 IP: 180.76.5.111 IP: 180.76.5.137 IP: 180.76.5.141 IP: 180.76.5.142 IP: 180.76.5.145 IP: 180.76.5.146 IP: 180.76.5.147 IP: 180.76.5.148 IP: 180.76.5.150 IP: 180.76.5.151 IP: 180.76.5.155 IP: 180.76.5.159 IP: 180.76.5.161 IP: 180.76.5.162 IP: 180.76.5.166 IP: 180.76.5.167 IP: 180.76.5.170 IP: 180.76.5.171 IP: 180.76.5.172 IP: 180.76.5.176 IP: 180.76.5.177 IP: 180.76.5.178 IP: 180.76.5.179 IP: 180.76.5.180 IP: 180.76.5.181 IP: 180.76.5.182 IP: 180.76.5.183 IP: 180.76.5.185 IP: 180.76.5.187 IP: 180.76.5.189 IP: 180.76.5.190 IP: 180.76.5.192 IP: 180.76.5.193 IP: 180.76.5.194 IP: 180.76.5.195 IP: 180.76.5.196 IP: 180.76.5.197 IP: 180.76.5.49 IP: 180.76.5.51 IP: 180.76.5.54 IP: 180.76.5.55 IP: 180.76.5.57 IP: 180.76.5.58 IP: 180.76.5.59 IP: 180.76.5.62 IP: 180.76.5.63 IP: 180.76.5.64 IP: 180.76.5.65 IP: 180.76.5.67 IP: 180.76.5.87 IP: 180.76.5.88 IP: 180.76.5.90 IP: 180.76.5.92 IP: 180.76.5.93 IP: 180.76.5.94 IP: 180.76.5.95 IP: 180.76.5.96 IP: 180.76.5.97 IP: 180.76.5.98 IP: 180.76.5.99 IP: 180.76.6.21 IP: 180.76.6.212 IP: 180.76.6.213 IP: 180.76.6.222 IP: 180.76.6.223 IP: 180.76.6.224 IP: 180.76.6.231 IP: 180.76.6.28 IP: 180.76.6.29 IP: 180.76.6.36 IP: 180.76.6.37 IP: 199.36.73.116 IP: 220.181.108.100 IP: 220.181.108.107 IP: 220.181.108.110 IP: 220.181.108.81 IP: 220.181.108.84 IP: 220.181.108.93 IP: 220.181.108.95 IP: 220.181.108.97 IP: 222.186.24.59 USER AGENT: "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" IP: 67.159.56.162 IP: 72.11.144.119 IP: 119.63.196.10 IP: 119.63.196.103 IP: 119.63.196.105 IP: 119.63.196.106 IP: 119.63.196.11 IP: 119.63.196.113 IP: 119.63.196.116 IP: 119.63.196.117 IP: 119.63.196.118 IP: 119.63.196.120 IP: 119.63.196.124 IP: 119.63.196.125 IP: 119.63.196.13 IP: 119.63.196.16 IP: 119.63.196.19 IP: 119.63.196.20 IP: 119.63.196.23 IP: 119.63.196.25 IP: 119.63.196.26 IP: 119.63.196.27 IP: 119.63.196.28 IP: 119.63.196.30 IP: 119.63.196.31 IP: 119.63.196.32 IP: 119.63.196.38 IP: 119.63.196.40 IP: 119.63.196.42 IP: 119.63.196.43 IP: 119.63.196.44 IP: 119.63.196.45 IP: 119.63.196.47 IP: 119.63.196.53 IP: 119.63.196.54 IP: 119.63.196.57 IP: 119.63.196.58 IP: 119.63.196.60 IP: 119.63.196.61 IP: 119.63.196.76 IP: 119.63.196.78 IP: 119.63.196.80 IP: 119.63.196.82 IP: 119.63.196.88 IP: 119.63.196.89 IP: 119.63.196.9 IP: 119.63.196.93 IP: 119.63.196.94 IP: 119.63.196.96 IP: 123.125.67.164 IP: 123.125.71.100 IP: 123.125.71.101 IP: 123.125.71.102 IP: 123.125.71.103 IP: 123.125.71.104 IP: 123.125.71.105 IP: 123.125.71.106 IP: 123.125.71.107 IP: 123.125.71.108 IP: 123.125.71.109 IP: 123.125.71.110 IP: 123.125.71.111 IP: 123.125.71.112 IP: 123.125.71.113 IP: 123.125.71.114 IP: 123.125.71.115 IP: 123.125.71.116 IP: 123.125.71.117 IP: 123.125.71.12 IP: 123.125.71.13 IP: 123.125.71.14 IP: 123.125.71.15 IP: 123.125.71.16 IP: 123.125.71.17 IP: 123.125.71.18 IP: 123.125.71.19 IP: 123.125.71.20 IP: 123.125.71.21 IP: 123.125.71.22 IP: 123.125.71.23 IP: 123.125.71.24 IP: 123.125.71.25 IP: 123.125.71.26 IP: 123.125.71.27 IP: 123.125.71.28 IP: 123.125.71.29 IP: 123.125.71.30 IP: 123.125.71.31 IP: 123.125.71.32 IP: 123.125.71.33 IP: 123.125.71.34 IP: 123.125.71.35 IP: 123.125.71.36 IP: 123.125.71.38 IP: 123.125.71.39 IP: 123.125.71.40 IP: 123.125.71.41 IP: 123.125.71.42 IP: 123.125.71.43 IP: 123.125.71.44 IP: 123.125.71.45 IP: 123.125.71.46 IP: 123.125.71.47 IP: 123.125.71.48 IP: 123.125.71.49 IP: 123.125.71.50 IP: 123.125.71.51 IP: 123.125.71.52 IP: 123.125.71.53 IP: 123.125.71.54 IP: 123.125.71.55 IP: 123.125.71.56 IP: 123.125.71.57 IP: 123.125.71.58 IP: 123.125.71.59 IP: 123.125.71.60 IP: 123.125.71.69 IP: 123.125.71.70 IP: 123.125.71.71 IP: 123.125.71.72 IP: 123.125.71.73 IP: 123.125.71.74 IP: 123.125.71.75 IP: 123.125.71.76 IP: 123.125.71.77 IP: 123.125.71.78 IP: 123.125.71.79 IP: 123.125.71.80 IP: 123.125.71.81 IP: 123.125.71.82 IP: 123.125.71.83 IP: 123.125.71.84 IP: 123.125.71.85 IP: 123.125.71.86 IP: 123.125.71.87 IP: 123.125.71.88 IP: 123.125.71.89 IP: 123.125.71.90 IP: 123.125.71.91 IP: 123.125.71.92 IP: 123.125.71.94 IP: 123.125.71.95 IP: 123.125.71.96 IP: 123.125.71.97 IP: 123.125.71.98 IP: 123.125.71.99 IP: 125.39.78.168 IP: 125.39.78.171 IP: 125.39.78.173 IP: 125.39.78.174 IP: 125.39.78.177 IP: 125.39.78.179 IP: 125.39.78.181 IP: 125.39.78.183 IP: 125.39.78.185 IP: 125.39.78.187 IP: 125.39.78.188 IP: 125.39.78.189 IP: 125.90.93.141 IP: 173.236.136.101 IP: 180.76.5.100 IP: 180.76.5.101 IP: 180.76.5.103 IP: 180.76.5.107 IP: 180.76.5.110 IP: 180.76.5.111 IP: 180.76.5.113 IP: 180.76.5.136 IP: 180.76.5.137 IP: 180.76.5.138 IP: 180.76.5.139 IP: 180.76.5.140 IP: 180.76.5.141 IP: 180.76.5.142 IP: 180.76.5.143 IP: 180.76.5.144 IP: 180.76.5.145 IP: 180.76.5.146 IP: 180.76.5.147 IP: 180.76.5.148 IP: 180.76.5.149 IP: 180.76.5.150 IP: 180.76.5.151 IP: 180.76.5.153 IP: 180.76.5.154 IP: 180.76.5.155 IP: 180.76.5.156 IP: 180.76.5.157 IP: 180.76.5.158 IP: 180.76.5.159 IP: 180.76.5.160 IP: 180.76.5.161 IP: 180.76.5.162 IP: 180.76.5.163 IP: 180.76.5.164 IP: 180.76.5.165 IP: 180.76.5.166 IP: 180.76.5.167 IP: 180.76.5.168 IP: 180.76.5.169 IP: 180.76.5.170 IP: 180.76.5.171 IP: 180.76.5.172 IP: 180.76.5.173 IP: 180.76.5.175 IP: 180.76.5.176 IP: 180.76.5.177 IP: 180.76.5.178 IP: 180.76.5.179 IP: 180.76.5.180 IP: 180.76.5.181 IP: 180.76.5.182 IP: 180.76.5.183 IP: 180.76.5.184 IP: 180.76.5.185 IP: 180.76.5.186 IP: 180.76.5.187 IP: 180.76.5.188 IP: 180.76.5.189 IP: 180.76.5.190 IP: 180.76.5.191 IP: 180.76.5.192 IP: 180.76.5.193 IP: 180.76.5.194 IP: 180.76.5.195 IP: 180.76.5.196 IP: 180.76.5.197 IP: 180.76.5.48 IP: 180.76.5.49 IP: 180.76.5.50 IP: 180.76.5.51 IP: 180.76.5.52 IP: 180.76.5.53 IP: 180.76.5.54 IP: 180.76.5.55 IP: 180.76.5.56 IP: 180.76.5.57 IP: 180.76.5.58 IP: 180.76.5.59 IP: 180.76.5.60 IP: 180.76.5.61 IP: 180.76.5.62 IP: 180.76.5.63 IP: 180.76.5.64 IP: 180.76.5.65 IP: 180.76.5.66 IP: 180.76.5.67 IP: 180.76.5.87 IP: 180.76.5.88 IP: 180.76.5.89 IP: 180.76.5.90 IP: 180.76.5.91 IP: 180.76.5.92 IP: 180.76.5.93 IP: 180.76.5.94 IP: 180.76.5.95 IP: 180.76.5.96 IP: 180.76.5.97 IP: 180.76.5.98 IP: 180.76.5.99 IP: 180.76.6.20 IP: 180.76.6.21 IP: 180.76.6.211 IP: 180.76.6.212 IP: 180.76.6.213 IP: 180.76.6.222 IP: 180.76.6.223 IP: 180.76.6.224 IP: 180.76.6.225 IP: 180.76.6.227 IP: 180.76.6.230 IP: 180.76.6.231 IP: 180.76.6.232 IP: 180.76.6.233 IP: 180.76.6.26 IP: 180.76.6.28 IP: 180.76.6.29 IP: 180.76.6.35 IP: 180.76.6.36 IP: 180.76.6.37 IP: 204.45.133.74 IP: 220.181.108.165 IP: 220.181.108.166 IP: 220.181.108.167 IP: 220.181.108.168 IP: 220.181.108.169 IP: 220.181.108.170 IP: 220.181.108.171 IP: 220.181.108.172 IP: 220.181.108.173 IP: 220.181.108.174 IP: 220.181.108.175 IP: 220.181.108.176 IP: 220.181.108.177 IP: 220.181.108.178 IP: 220.181.108.179 IP: 220.181.108.180 IP: 220.181.108.181 IP: 220.181.108.182 IP: 220.181.108.183 IP: 220.181.108.184 IP: 220.181.108.185 IP: 220.181.108.186 IP: 220.181.108.187 IP: 220.181.108.79 USER AGENT: "Mozilla/5.0+(compatible;+Baiduspider/2.0;++http://www.baidu.com/search/spider.html)" IP: 222.76.212.176 |
Just block those ips and get it over with ...
|
agreed that I can just deal with it via my firewall.
However, I am providing these reports to help the team with improving the mod. I get so much from this site that I want to try and give back as much as I can.... so if it can help others who don't have firewalls etc then I feel good about trying to help. I will continue to try things this way to give my experiences... once they get the mod doing what they want it to do as Ozzy eludes to, I will most likely just go the firewall route. D. |
Well back to the drawing board. :(
|
1 Attachment(s)
Ok here they are showing up in WOL, and they should not be.
I went back to the way the mod is now, and I disabled any other mod running on style_fetch so it is not a hook conflict. Now I will go back to a recoded version, and try some different hook locations. |
If anyone has a test vb4 install that is open to bot access i'd appreciate having admin access to try a few things, I ask this because I can only install an early version of vb4 that comes with my 3.8.7 licence. and I suspect its the later versions that are having this anomaly.
|
Simon, I can send you a PM on my site, which would contain the XML's for vB3 and vB4.
The new way has so far seemed to successfully blocked them, as well they do not show up in Paul's mod anymore. I also have added the ability to change the name of the log file, to a different name, in case admins want to change it, as well as a counter of how many bots are blocked using the mod, that shows up in the site stats on the forum home. Now this is of course starting from the time you install the updated version, as there is no way to get a correct count from before. There is also a setting to select which usergroups can see the stats. :) |
Quote:
|
I have 4.2.2 and I think I am heavily modded(ish)
Your call. D. |
Quote:
"heavily" modded is kind of subjective though too.:D I believe either there is a hook conflict or, too much time waiting in the "queue" so to speak, since the hook this mod uses is like 15th or something in order of fire. |
If you need any details about my site or want me to test things... I am happy to do it.
Shoot me a PM and I will respond with my e-mail address.... happy to work with you all to improve the world. ;) D. |
Quote:
|
Just to make sure I clarify - there's nothing inherently wrong with this mod. I merely believe it is being interfered with, on some v4 installations.
|
Quote:
And yes, there is nothing wrong with the mod, the original code works just fine. I think it is a mod conflicting, or quite possibly a vBoption, not sure which but it's something. I just have it running in a much better hook location for vB4 now, as well as the added goodies. :) |
I have been trying to keep an eye kn this thread to see if anyone can block baidu 100%. Has this been achieved yet? I see ozzy47 talking about a new way to block them but see no instructions. Am I missing something?
|
Yes I have them blocked completely, now it's up to Simon to look at my changes and see if he wants to update the mod. :)
|
Will take a look tomorrow guys :)
|
Quote:
|
Right, but the changes I made should stop it on all sites now. :)
|
Looking forward to Max's thumbs up!
Kudos to all the people collaborating on this thread! I really is good to see people working together as a community for a common goal! I for one could not have a successful site if it wasn't for people like yourselves! D. |
Yeah in the past 72 hrs, I have successfully blocked Baidu over 5400 times. :)
|
Can you share with us how you done this?
|
Simon has the changes, when he has time he will go over it, and if he wants, release a update. :)
|
Ozzy47's xml has been uploaded and an update sent out, thanks Chris nice work! :)
|
Not a problem sir, glad to help make this great mod just a bit better. :)
|
Installed and already working.
I will post a screen shot of WOL before and after in a little while... I want to have it in place for a little bit so it can start to really do it's thing. Home page stats in 2 min is reporting 43 blocked D. |
Working Brilliantly!
In less than 10min I went from 30+ Searchbots according to WOL to 14 and they are all the good ones (Google, Bing, Yahoo) My server loads went from high 3 and 4 to 1.5 (ish) Output file is showing Baidu being blocked among others. (most seem to be coming from Yandex) You should all be commended on great work! D. |
Excellent, glad to hear it has worked so well so far. :)
|
Version 3.1.1 uploaded, slight bug fix, stopped spiders/bots being counted when mod is off!
|
Nice to see this venerable, useful and very reliable plugin updated. Nice going guys.
|
Max if you downloaded the new version before I posted this message please download it again and overwrite the previous one, the first update had a bug :)
|
Quote:
|
That's fine, as I made a mistake and that post should have been for the VB3 version :)
|
Just very happy to see this little issue solved for folks. Drove Ozzy and I crazy for months, before I bought a existing v4 board and didn't have the same problem. That opened the door for investigation, as it were. Gave us the angle of attack.
|
I think it's earlier versions of VB4 start up sequence is pretty much the same as VB3 later versions order was different, Ozzy tried the bootstrap hook and that did it! I wasn't aware of the issue until this thread resurrected itself with these posts.
|
Quote:
Originally we thought cron jobs were interfering, on Ozzy's board. Hence why only relatively few Baidu's were getting through. It wasn't until much later on we began to suspect hooks, and firing order, and etc. Originally after working on it for a couple of weeks, we just blew it off for a time. Wrote it off as just a fluky deal. Until others started reporting the issue, then when I bought that v4 and didn't have the problem it really ramped up interest in solving this again. This goes back almost a year or maybe a little more, this mystery. For us at least. Kudos to you Simon for having such a open mind about this - few devs would have listened especially on such a well established and widely used Mod as this. |
Quote:
|
Ozz and I never truly let go of a problem. It haunts us day and night until it is solved!
https://vborg.vbsupport.ru/external/2015/02/1.gif |
Hate to be the one to have to say this.
My site just started throwing these errors: Database error in vBulletin 4.2.2: Invalid SQL: UPDATE `bad_bots_blocked` SET ban_useragent = ban_useragent + 1; MySQL Error : Table 'subaru_test.bad_bots_blocked' doesn't exist Error Number : 1146 Request Date : Wednesday, December 17th 2014 @ 08:11:41 PM Error Date : Wednesday, December 17th 2014 @ 08:11:41 PM Script : http://www.toronto-subaru-club.com/f.../index.php?how Referrer : IP Address : 5.255.253.29 Username : Classname : vB_Database_MySQLi MySQL Version : D. |
All times are GMT. The time now is 09:31 AM. |
Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
![]() |
|
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|