It shouldn't create threads and it may have been that typo (thanks for spotting it) in the updated xml i have also added a fix so that if the forumid is blank the routine is exited :)

I had the same problem. However, being stupid I turned ON option to create threads, but have not specified the forum. Obviously threads should not be created if forum was not specified and I believe author is going to fix it.

Anyway, the best way to clear the threads (I got 20 of them) is to use your AdminCP -> Threads And Post -> Prune and purge them. I basically removed them using "Other Option'" and put the string: "Activity From Bot" in the TITLE field and it took care of it. I checked DB and they were removed. After this operation Advanced Statistics Plugin no longer reports them. :)

Hope it helps.

EB

Does anyone have a pruned list of just the really BAD spiders? Honestly, my only problem has been with Baidu (80 spiders at one point) ....and I'm not really knowledgeable enough to know which of that LONG list are: "really bad", "kindof bad", "annoying", etc.

Thanks!

It really depends on your target audience, if you were catering for China then you'd allow Baidu, if you were catering for Russia then you'd allow Yandex (Russian Yahoo), most of the bots in that short list (trust me it is short as there are many more that add no value to your site or stats) are known scrapers, bad bots or foriegn spiders that you just don't need.

Check other sites and see which bots they are allowing, see if they are in your country, research the ones they show to see if they're good or bad for your site.

Just as a FYI, the "Check Version" option in the admincp->Plugins&Products->Installed Products listing doesn't work.

The error message is:

The hard link in the listing does correctly point to this thread; however, the version check doesn't work properly.

Thanks i'll sort that out :)

Thanks.

I found other method. Go to ACP->Maintenance->Update Counters->Remove Orphan Threads (at the end of list)

This sorted it out quickly and I too didn't see threads anymore.

Yes, I know. It's very strange that it created this six threads.

Probably you should also put 0 instead 1 in options for thread creating by default? Maybe that could help?

Good thinking. I missed that one.

EB

Fixed :)

Fixed :)

Fixed :)

is it work on vb 4.1.7?

Yes, and also on 4.1.8

if you check out the version above it works on 4.x.x that means any version from 4.0.0 and onwards :)

Works like a charm!!! Thank you so much :)

Glad you like it, check your resource usage after a month and you'll see a dramatic drop in your bandwidth usage and cpu time :)

Simon, you might want to add:

Twiceler
xpymep2.exe

to the spiders list in the mod.

Hi Boofo, thanks for that however there are quite a few maintained lists around Lee that posts here has posted his list at vbulletin.com and at vbseo.com his pet hate is bad bots n spiders so he has a hugeeeee list, the ones i added to the mod are just to get people started and to see how to enter their bots :)

A link to those lists would be a good addition, also.

Try these:
http://www.forumpostersunion.com/showthread.php?t=1644
http://www.vbseo.com/f34/how-create-...78/index4.html

But to be honest it's nothing that a little googling or binging wont reslove :)

I was referring to those that aren't familiar with bots and how they act. The more knowledge available when releasing any mods here is always a GOOD thing. Sorry if I interrupted anything with that request.

Well i suppose the secondd of those two links has more explanation but i didn't get that from your post, i thought you wanted links to lists posting, but for my money that can be a bad thing, people would then blindly enter all the bots to the banning list when really, dependant on their content, they will want some of those scraping or visiting their site, it's a personal preference really.

I built this because the bots were killing my bandwidth and making the forum slow so i ban the more agressive ones like Baidu, why on earth would they need 215 bots indexing my site?, so ban them in favour for the less aggressive :)

Anyway as an information post this link explains what they are http://en.wikipedia.org/wiki/Spambot

Yeah, 215 Chinese bots leeching resources, to allegedly index your site for people who most likely can never see it. Most US sites are blocked in China. Baidu makes no sense at all, it certainly doesn't help anybody. By far the worst behaving bot out there, it totally ignores robots.txt.

Lol, thanks Force ;)

Since upgrading to vB4.1.10, I've noticed (occasionally) some spiders in the list still showing up online.
Anyone else having this issue, and any ideas what's going on?

TiA

You shouldn't unless they are using a different UA, the spiders xml that you can download from Mosh's site (vb.com thread) may have them in the list with the same name but their UA may be different to that in the list :)

A couple of examples of ones in my list that (recently) show as online;
AhrefsBot
Exabot
gigabaz
Heritrix
Majestics MJ12bot

I though this hack picked up on all things similar (name based)?... ie "Majestics" should also take out "Majestics MJ12bot", regardless of the software they're using to access the site.

How do we include same name spiders in our list that are using different UA's?

You are right entering Exabot should prevent that bot viewing your site, however entering "Majestics MJ12bot" will not stop a bot with either "Majestics" or "MJ12bot" it will only stop bots with that entire phrase as part of their UA, so if you want to ban that bot enter the seperated values.

Can you give me a link to your site?

Simon, I thought about that and included the individual entries when the issue first arose.

nscale.net ... however, the WGO block is only visible to members... PM me if you want temporary access.

what does this option mean? " Create new thread for each UA detection (be aware that this can cause hundreds of threads at first until spiders get the mesage!)
You can use this even if you aren't using the ban option above" also what does this option do. will it creat a thread in my forum?

ALSO WHY IN A DISCRIPTION FROMA MEMBER DO THEY HAVE THIS OPTION SET TO YES??

Firstly that does exactly as it says, It WILL create threads!, hundreds and hundreds of them :) and one of the members here who say they have it set to "yes" chose to do so!

Downloaded, suggested this by a friend.

Think must be useful, let us see how it works.

Thanx.

G!

Well done Simon.

We were having an infestation of Baidu Spiders but your Mod took care of them. I installed it last night and this morning they (and others) were gone.

Glad it's helping you :)

hmmm,

why do i have all this in my ""VSa - Advanced Forum Statistics"

this:

* activity from bot no.7 (baidu) in your list*

also notice from 2 other bots like: yandex and webster.

thanks for any help...

In forum statistics you will see the bot activity as they do access your site but are then redirected before they can open the thread...etc that they were attempting, if you are seeing the actual phrase "activity from bot no.7 (baidu) in your list" then it's because you have the notifications turned on in the mod :)

so just turn off the option at: "Create New Thread" ?
thanks

Yep, turn it off, it's just a way of monitoring it working or if you are only testing one bot at a time, otherwise you'll get thousands of the damn things ;)

Awesome mod, thanks!

Saved our community from Baidu, hundreds of bots were online persistenly to the point of crashing our server.

Please mark it as installed :)