Yes, the current product only allows you to either ban the spider OR have a notification like output to log, email or create a thread, i have a beta which allows you to do both at the same time, a couple of people are trialing it right now.

You only need to have the mod active, ban spiders in list set to yes and some spiders in the list to ban :)

Thanks for voting MOTM, you will recieve an update when this beta is proven ;)

As far as I can tell it is.

This mod is awesome. Thank you so much. I was so sick of the Baidu spider on my site. I installed this mod and watch as they left one by one, and haven't been back. :D

Glad you're happy with it - watch out for the new release after this beta has been tested a while longer :)

Is anyone getting this in your error log?

[11-Oct-2011 08:33:56] die 3
[11-Oct-2011 08:35:19] die 3
[11-Oct-2011 08:35:26] die 3
[11-Oct-2011 08:35:29] die 3
[11-Oct-2011 08:35:30] die 3
[11-Oct-2011 08:35:40] die 3
[11-Oct-2011 08:35:40] die 3
[11-Oct-2011 08:35:49] die 3
[11-Oct-2011 08:35:49] die 3
[11-Oct-2011 08:35:51] die 3
[11-Oct-2011 08:36:00] die 3
[11-Oct-2011 08:36:00] die 3
[11-Oct-2011 08:36:08] die 3
[11-Oct-2011 08:36:08] die 3
[11-Oct-2011 08:38:35] die 3
[11-Oct-2011 08:38:58] die 3
[11-Oct-2011 11:32:59] die 4 true
[11-Oct-2011 11:32:59] die 4 true
[11-Oct-2011 13:24:17] die 3
[11-Oct-2011 13:27:56] die 3

are you using the beta or the one from the first post

It would be natural i suspect to get those, the mod performs all the redirects...etc at style_fetch, so in other words even though spiders...etc go straight for a url they never get to see it as it never completes for them hence the connection for them dying, however this mod does NOT give a Die(); command, it gives a 301 redirect.

@ForceHSS there's only you and one other using the beta and another has just requested to be a tester so that will be 3 of you :)

Did not know how many thought that might of been from the beta the reason I asked

Hows the testing going ForceHSS?

Not the beta

Are you isning my Ban Ip mod? that mod actually gives the Die(); command which breaks the connection. This mod allows first connection but redirects immediately so their request never completes.

Nope, only this mod of yours

Well you can rest assured it's only banning or stopping complete connection for those in your list :)

going good no bugs in it so far that I can see
one thing would be nice to see as it seems to miss Thread Prefixes even if I make it forced to use them on a section it wont add them

If a spider is banned, how do I get them to crawl my site again, I tried your full ban list, and now my website monitor services are no longer checking my site.

I removed all spiders from admin except Baidu.

Hi Simon, I just sent PM to test beta.

I have the released version installed on our vBCMS 4.1.7 but it doesn't seem to be banning Baidu. Our forums are located in the root with the CMS (so no /forums/), not sure if it's to do with this.

I have Track Guest Visits installed and it still shows 40-50 Baidu every day.

I've double checked my settings... only have "Ban Spiders In List" selected, no logging etc.

My List is:
Yandex
Yeti
Baidu
soso
sogou
ichiro
speedy
spinn3r
mlbot
psbot
SBIder
Ezooms
snap shots
metauri
YoudaoBot
youdao

Anyway, will try beta and see if that fixes it.

8-)

PS. I hope your daughter and grandson are doing well.

It wont add prefixes as they are added when the forum loads, your actual url stays the same, a prefix is never added to them - have you ever seen a url like this http:www.mysite.com/showthread?t=[solved]12345 ??? :)

You added your site monitoring service as a bad bot? bad move!, remember we're sending them a 301 which is a permanent redirect, if you don't see them back in a week check with them, you may ask for your url to crawled again.

Right, firstly, thanks they're now doing great :), your "Track Guest Visits" mod will ALWAYS show the spiders but your native vBulletin WOL will not, the reason why the TGV mod picks them up is because they are actually accessing your site (so that mods doing it's job and recording them) but my mod prevents them from having their request completed i.e direct request for a url is a forum access but they are redirected permanently before the thread loads (so my mod is ALSO doing its job :))

Hope that clears things up for you all.

@GreyGhost i'll PM you details of the beta ;)

Yeah my site monitoring site was in your bad bot list, and I did not see it.

OK I got them back, 1 was missing, the other one was showing as guest, I upgraded and forgot to re up the spiders xml from WolfsHead

Excellent! :)

Yes, suspected this was the case. Just after posting I tested it @ http://www.botsvsbrowsers.com/SimulateUserAgent.asp and Baidu were indeed being redirected (straight back to Baidu :D).

Great stuff!

8-)

In your ban list you have InternetSeer.com

That would not be the same as this, Internet Seer Spider
because that is a web site monitoring service.

You may see in your WOL Internet Seer Spider but you need to know it's full UA (here's one: Mozilla/5.0 (compatible; Atwatch-InternetSeer monitoring; MSIE 6.0; ) Gecko) so we can decipher which part if any is being blocked, Maybe this will help you get them back http://www.internetseer.com/help/faq.xtp

Just a word though, internetseer has always been just an email scraper, they trade off with "monitoring" your site or of course you can pay them to do it, either way they are archiving email addresses found on your site - still want to use 'em?

As a final word, i did mention when posting the list of bad bots (both here in the thread and the txt file) to prune out those that you want to be able to see your site, you might have like to also prune out ia_archiver, you should definitely prune out DA and custo (as should most people).

Beta testing is going well :), those that are using the beta please pm me any results you have or bugs. If all is well i'll release it as stable next week ;)

Will post here and not pm. So far no bugs in the bot posting still have to test the other settings but I am sure others have but will test them as well

Posting here is great, other folk get to know how it's going and if anything what to look out for :)

You're welcome to quote my PMs here Simon, if you feel they'll be at all informative.
Although I will be removing the images after a few days so you may want to leave them out.

8-)

the email part of it dont seem to work

I still see Baidu hitting my site every so often, and the last one that was banned was 10/11/2011 according to forum post and output.txt

Well baidu is persistant :), remember that you are getting notification BEFORE a thread or forum loads properly for them.

@ForceHSS & GreyGhost i'll work on the emailing a little later :)

I received this error after activation of "Write to log" option

The highlighted word in the photo below is missing "i"

https://vborg.vbsupport.ru/external/2011/10/28.jpg

btw, does the url of redirect work? or I've to put something else?

thnx

this seems like you do not have write permissions to your server!

The typo i will fix (if i remember in future releases, it has nothing to do with the operation of the mod), the url redirect works fine, as in the instructions you cannot add http:// in that box but can use www. or just the domain - a word of warning - redirecting to Google is a very bad idea!

This what I asking about. What do you suggest ?

I made some suggestions a page or two back!, didn't the product you downloaded already have a url in the box?

No, it haven't.

https://vborg.vbsupport.ru/showpost....7&postcount=94

use this one
www.klikhierniet.net

Thanks ForceHSS it must just be in the beta, i will release fix for emailing in the next day or so when i've finished work.

Youdon't have this marked as installed so i have no idea of which version you downloaded!

Just installed, works as described.
My problem: I had "Create New Thread" activated, didn't see the checkbox -.-"
No forum ID was given, so now my Top10 Stats shows two threads called "Activity from Bot No. 0 (Baidu) in your...", with no user and no forum. If i try to delete them it tells me, that i dont have access :S

Any tipps on how to delete those two posts?
Tanks!