99% of the chinese bots will bring no traffic so won't hurt your adsense revenue, on my other sites i ban ALL chinese bots as they index far too agressively, these are the ones i ban at my other sites:
Yandex
Yeti
Baidu
soso
sogou
ichiro
speedy
spinn3r
mlbot
psbot
SBIder
Ezooms
snap shots
metauri
YoudaoBot
youdao

Hope that helps you, but of course its a personal thing ;)

Well, there's nothing really to look at except your settings ... (see pic)...

Are they correct?

Regards,
Doug

That looks ok, next you need to check your session timeout settings and see what it's set at as nothing goes missing from the WOL until that has expired, if the timeout has passed and you've been watching WOL and they remain after that time then click WOL to view all those online, from the dropdown select yes for useragent and copy the UA then try it here http://www.botsvsbrowsers.com/SimulateUserAgent.asp and see what results you get, the UA will look something like this:
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)

In fact you can try that at the link i gave you, make sure you set it to look at your site :)

Thank you. Helps.
After checking my session expiration setting, and just watched the lil' critters disapear!

Will watch for the fix upcoming, and if al works after testing, will most certainly vote motm!

It's set for default 20 minutes, but PaulM's guest mod is showing dozens of accesses (logins) from those bots that have occurred in the last 24 hours, so am I misunderstanding what this mod is supposed to do?

Shouldn't there be NO logins by Baidu and Yandex spiders for at least 23 hours ago, since this mod has been running with your corrected settings for days?

Thanks .. :)

Regards,
Doug

What you forget is that they have to attempt access to your site to get banned (redirected 301) so thats why Pauls mod is showing those to you, also bots don't access homepage then select a forum then select a thread, they just go straight for a thread (or post), so as soon as that happens Pauls mod will log them, but if you look at WOL are they there now?

I doubt it :), Pauls mod is doing the job it's set out to, mine should be doing the job too, did you test that UA i gave above at the link i gave? If so what were the results?

I'm close to a fix for this but it will probably mean an additional php file to be uploaded as it seems that it can't work comfortably with the bots being redirected the moment they call the forum to load as it's leaving nothing for the notification to notify, all the others work comfortably together i.e Output.txt logging, email and create thread, it's just when you ban the bot you either ban it late which means it always will be seen in WOL or ban it early so it's very rarely seen there, it's the early bit thats causing the issue!

Using this site and useragent tag to test, I get varying results.

1 - if I use just my home page (cms) it doesnt seem to be working. Not sure if this is even an issue really as my baidu bot count is nil now with this mod, maybe just doesnt work with cms.

2 - when I add the necessary /forums/ to my url on the test page, it seems to be working, but it redirects to google.com.hk (is that normal?)

Right, it wont work with cms as thats outside of the /forum folder, and yes they are getting redirected to a chinese google :):)

Ahh, ok that explains it then.

1 - Are there plans to make this work with the vB suite (ie-cms/forum/blog/groups,etc)?

2 - Can you when you are able, make it so the admin can set where they want the redirect to? (I would rather redirect to baidu themselves, I dont want to play mean with google as they can get real pissy if they were to not like it and track back the redirects. Dont want to be on googles bad side ya know)

3 - (and last I promise) Are the 'redirects' true permenant 301's by definition?

301 is set in the redirect, i don't think i will be able to set the redirect back to the bots own source but certainly i can make the redirect selectable (and will next day or so) and i don't know whether i'll venture in to getting it to work with cms at the moment (as i have a lot on) but anything in /forum and lower i.e /forum/blog.....etc will benefit from the mod (or should!)

Thanks. Looking forward to this. (is there a suggested redirect url that is effective and safe?)

No problem.

Thanks for everything on this.

Added user redirect entry box, now you can select where you send them bad bots ;)

Suggestion to redirect them to:
http://www.klikhierniet.net/
It's dutch ( meaning, don't click here )
I think you'll find it's very annoying and I think you could find simular ones in english

Thanks for the option of where to redirect.

I set it for www.baidu.com,... checked it on the checking link you posted,.. and it shows a successful redirect back to them.

I know all bots entered will go to baidu.com, but I am not concerned with that.
(I thought funny the link you suggested ;) )

Thanks for reporting back that it works ok, i just dashed it off and was about to mark this as a beta when i realised i hadn't tested it so THANKS for being a guinea pig!

That link is annoying though :)

Is there a setting yet for the redirect link?

Yes, and I tested and it works

any word when the Forum ID and Thread Username options will work

Im still working on that, i'm trying my best to keep it all in one xml product, at the moment i'm experimenting with a seperate php file called from the product but thats not my goal.

You can use the thread creation...etc but without actually banning the spider at the moment.

There will be an update notice sent to you when i replace the xml or files here just as you did for the "use your own redirection url" change :)

installed and thanks :D

thanks ur the best

I will be releasing a beta back end of next week if anyone wants to try it pm me, remember i said BETA!, so it may have some issues that i'll need feedback on, i'm still trying to keep this in one product without additional files :)

Beta for Thread creation and Banning working together to be released - target date 7th October 2011

I just had a member get rejected using his android phone

This mod can ONLY block those useragents that you have entered in the list, firstly get your user to go here http://whatsmyuseragent.com/ (via his phone) and find out what his useragent is then you go here http://www.botsvsbrowsers.com/SimulateUserAgent.asp and paste his UA string in and test it to see if you get denied or not.

Something in his useragent string is in your list so it's not the mods fault as it's banning what you ask it to :)

Thanks

If you get stuck let me know :)

Hi im trying to reinstall but now i get XML Error: Not well-formed (invalid token) at Line 1
can anyone help

Uninstal what you have and then install the latest one here, if your still having problems let me know and i'll check the xml out, it's possible that its a minimum version or thread fault, whichever it's an easy fix :)

Awesome Mod....I was soooo sick of seing "Baidu".....got my eye on your "Insert Objects/ads anywhere", also.....

Glad you like it :) if you haven't and you like it that much you can visit the MOTM link above ;)

Hi all, the planned release of the Beta i mentioned won't happen today, our daughter struggled terribly to bring our grandson in to the world so all my thoughts and efforts have been toward helping her.

For the next week or so my support here may be sporadic because of that, so if by chance you don't get an answer from me right away please be patient and i will answer. :)

Ok, i said i wouldn't but it's been theraputic, i have the beta ready for release, if any of you want to try it and report back then please PM me.

Without testing this in a few dufferent environment i can't say that it will work ok for you, however it works on my test board :)

Installed on forum version 4.1.7 and works great! We used to have 150+ spider bots crawling the site (mostly Baidu) and after we're down to 5-10: Google, Bing, Yahoo, etc.

Good stuff :)

Glad you like it :) do you want to try the beta? the beta is where i'm attempting to get both the banning AND the create thread working together, right now they will only work one at a time so either you can ban OR you can create thread (on detection) OR create log OR send email notification.

No pressure but im looking for testers ;)

I will test pm me with the beta

Great thanks, pm'd you details :)

So far the beta seems to be working great, I have Baidu in there now and it is posting threads, and showing in the output.txt, it was not doing either before.

Thats great, thanks for testing, It is also being banned?

Oh man, your mod deserve gold. Baidu piece of shit was spamming my forum so bad . I had 129 baidu spiders was spamming my forum every day, and now all of them are gone thanks to you <3

Is it ok if i keep creat new thread for each UA detection turned off?


BTW i voted :)