Alright, I did what you said, still no change.

Also, is there an easy way to test this to see it works? It sometimes takes days for new stuff to appear in that page.

No, not really. Just gets lots of spiders to your pages. ;-)

As for the problems you're having, the only thing I can think of is that something is up with your spiders_vbulletin.xml file. You say you pulled down that latest file, but did you rename it to spiders_vbulletin.xml and place it in your includes/xml directory? Also, check the permissions on the file, it should be 644. If it's not chmod it to that.

Actually mikel, I am also having the exact, same problem. I've got many unknown spiders on my forums. Here have a look...

http://www.bonethugsforums.com/spiders.html

Do you have any idea what's wrong?

mike, i did what you said, still nothing, chmod is 644, and i renamed the file and its in correct location.

Yours is normal, the "unknown spiders" are ones that are identified as spiders but are not listed in the spiders_vbulletin.xml file.

we need a massive update of spiderlist.xml..

mikelbeck..the spiders.php page execute a HUGE SQL query..in our page its 416 query..:S ..do u have any idea to reduce sql query in next release???

someone needs to update the spiders.xml file.

Yes, I have a couple of ideas. But I'm not going to be able to get to them for a little while...

thnx man...hope u will impliment those idea to reduced sql query as earlier as possible..:)


TIA.

am I the only one having this unknown spider bot problem?

I dont have a SINGLE known spider in the list:

http://www.total-clan.com/forums/spiders.php

I have the LATEST xml file, I formatted it right, chmod is right, everything is right. What's wrong?

99% of the time mine are unknown too...I used the spiders from hambil's BOL hack, and it gives me a known one every once in a blue moon...but the googlebot, MSNbot both show up...so it might be a step in the direction your after.

I almost posted where did they go however glad this is there!

Why is adv_index showing up in the list I thought this was replaced when I renamed it to index.php, or am I just loosing my mind.

You could be losing your mind, but that's something you'll have to deal with on your own. ;-)

That page probably calls itself adv_index (in the code of the page), so that's what the spider watcher sees when the page is loaded. You can rename the file to whatever you want, but if you don't change the code it'll still have it's original name.

BTW - don't go changing the page's name if you don't know what you're doing... It's used in other places than just the file itself, if you just change it there you can screw other stuff up.

Hello..
the spider list in vbulletin.com now being updated...get it from there
http://www.vbulletin.com/forum/showp...5&postcount=12
Thanx to Stadler

:) also thnx to mikel..unless this watcher it wasnt possible to get all those spiders name :)

umm can u say..now most of the spider is now being catagorised...
after update the new spiderlist.....
unknown spider will changed to known spider?

can u have look on this lists catagori & update the watchers catagory stats..

plz give a serious thought abt sql query.
[high]* Zia offer mikel an ice cold chilled bear[/high]

hi mike,

here my german translation.

bye und best regards,
svenna

mine isn't working either..... no spiders display

the format that is used on vbulletin.com wont work with it... even the one zia post wont either... i posted a few pages back a working one... the format they have one vbulletin.com is wrong and its not even like that stock so i have no clue why they update the list there cause its still a wrong format

i guess i dont know enough about spiders to ask the right questions.. but i do know enough to read threads to determine what is good or bad for my forum. and i guess some spiders are good for search engine purposes... while others milk your bandwidth....i installed this watcher, but i am now looking for a bad spider SNIPER.....
any ideas?

DementedMindz..
its really a strange matter...it should not be wrong..its published by Stadler on vb.com.

And the latest one is working for me. the new spider is listed/update/entried there..they are showing over on my spiders.php page..

do u mind to drop few lines on vb.com ,in the thread regarding the error u found ?

I think :
let me xplain :
a site got a spider Google-feedfetcher - 100 times.

Google-Feedfetcher : wastnt listed on the old spiderlist.xml , so it was appeared as un-known spider's block.

after replace the new spiderlist.xml : if again google-feedfecther spider again crawl the site
visit no#101(100 times on old list.xml) , it will be appeard in rss-spider block.

The old data base will not be modified...
its my point of view.

Edit: plz see this hacks description mikelbeck refer the same thread of vb.com. So i dont think mikel use any other format.


Thnx.

take a look at stock vbulletin and take a look at stadlers its wrong... i have fixed it on mine but i dont use this no more as of the high number of queries on that page... alot of members here were saying it wasnt working for them either and then once i posted mine with how the format was with the stock vbulletin file it worked perfect for them... i havent looked at the file now but it was way off last time...

here is the top of stadlers


here is how it should be just like it is on the stock vbulletin file

Ok, people, I've had some time to get back to this. Sorry it's taken so long....

I've got Beta 10 running here: http://www.heavymetalscene.com/spiders.php

Notice how much faster it is. I've pulled out the "rolling up" code and placed it in a file that's run once an hour via a cron job. So the data isn't 100% up to date, but it uses much (much) less queries than the previous version.

What else do you want to see in this release?

how many queries are on that page? also will it auto update once a new spider is added?

8! :classic:

No matter how much data is in the database, it's still 8 queries.

http://www.heavymetalscene.com/spiders.php (not a whole lot of spider data): Page generated in 0.19605 seconds with 8 queries
http://www.forumowners.com/spiders.php (quite a bit of spider data): Page generated in 0.51553 seconds with 8 queries
http://www.listedwrong.com/forums/spiders.php (very little spider data): Page generated in 0.14583 seconds with 8 queries

When a new spider is added to the xml file you mean? I'll see if I can make it so it'll evaluate the spider xml file and update the "unknown" spiders if there's new data.

yes that is what i ment sorry... yeah that would be great and glad to see you brought them down...

its nice mikelbeck to knowing abt the new version..

please give thought about not lost old spider visitng data after install new version..
:) Less query is really delicious things

[high]* Zia is waiting for the new release[/high]

Can you please explain what you changed in my file?

btw: you shouldn't relay on the root-tag. Freddie implemented the xml-file before I've finished the DTD. However: online.php of vBulletin works with both root-tags, since it simply ignores it.

Regards,
Christian Stadler

excellent

why do you save all data without parsing in field PAGE?
Example:
So, as you see, in this field has many repetitions like showthread, index, member, etc.

I think, in DB must be just one mention about each page.

PS: you can do that in this statistics was include information when spiders visited forums archived pages? Thanks.

Can someone throw me the code to add this to my navbar please I know Im learning thank for the great hack installed 3.5.4 no issues at all Just need quick code to add to nav bar

thxs
Mark

I got it I added to tool bar fav I dont need this inside my actually forums works great thanks

Markus

Oh that is great but have you updated the zip pack yet?

Not yet, give me a couple of days.

Alrighty then, I'll be waiting here for the update.

Hi guys,

like the mod, I have installed by I am unsure if it is working correctly.

mysite http://www.bowls-aust.com/spiders.php

The problem is no type display for spiders except unknown and the google spider seems to be in the correct address range to be identified by the spiders_vbulletin.xml file data.

I uploaded the latest spider file and rename to spiders_vbulletin.xml

Am I just to impatient or is something wrong.

I don't know, if this is already suggested, but I suggest, that you add an option to only list spiders with at least X Hits in the list of unknown spiders and perhaps add a link to 'List all unknown spiders' then. especially these garbled user agents, like 'cblbshjhbjdmpvrbfdrvv5arh5wi' for example seem to me like some crappy configured firewall or proxy, which tries to hide the user-agent ...

I straight one mySQL errors indicated got. Where does the problem lie?


Any can help?

what you do when this error going? Query like this may be possible SQL-injection did posible hackers, but I'm not sure about it, so don't worry early

mikelbeck
can you do display IP of spiders via popupable div? Because JScript popup don't usable :(

This plugin don't want add in list spider WebAlta. In table spider_watcher I see always random fileds like this:
So, as you see - script do not count rollup. Why?

No update yet?