Thank you mikelbeck

Really nice :D

Install

How do I update the spider list?

I had a look on there but I dont know how to

On this post its from 2001
http://www.vbulletin.com/forum/showpost.php?p=565415

Thank you for any help :)

helo mikelbeck..

is there any update gonna release for vb3.6.0 ?

its an important module.

thanks

this product compatible with vB 3.6 Beta 2

Quick question:

I see the numbers going up for example on the googlebot entry, but the date is not allways updated.

Is this because you only update the date when it's a GET and not a HEAD request? or could this be a bug?

Love the hack btw.

Great plugin, installed without a hitch and works perfectly for me on 3.5.4

Thanks! :)

On my forum, it's not recognizing what bots are what.. how can I remedy this?

They are currently all listed as "Unknown Spider"

edit:

Ok, I downloaded that spiderlist.xml from vbulletin.com and replaced my spiders_vbulletin.xml with it.. is there a way to "update" the current entries so I'm able to see which bots are which easier, or will that never update?

Thanks! easy install, works great.

Had some new spider break the script today, looks like some of the code in the user_agent made the sql bomb, I have removed the script until this can be resolved, this isn't affecting my forums, just when you load the spiders page.

I realized it's been a ridicously long time since I've updated this mod, but I did it today...

This version dramatically cuts down on the number of SQL queries that occur when somebody views the spiders page. Instead of rolling up the spider data at that point, a cron job rolls up the data once an hour.

Please note that the installation and upgrade instructions have changed, and a new file is included.

ILTK - the error you've just posted should be corrected in this latest version.

I don't have 3.6 installed on any of my sites yet, so I can't confirm it works with it. I don't see why it wouldn't, but I can't say that it does for sure.

This is cool *installed*

Does this new version take some time before it shows any spiders? I'm guessing it does. I'm going to see if I can kick some spiders over to check my forums and gen some stats.

Thanks

Sweet, thanks for the update, my hostings cpanel is dog slow so I used it alot instead of going into awstats to see spidering trends.

Yes, the rollup runs once an hour, so it'll take that time until you see some stats. I looked at your site, I see Google and MSN on the spiders page. ;-)

mikelbeck.. Any plan to release it for vB 3.6 ?

Once I can upgrade my sites to 3.6 and test it, yes.

Does it not work with 3.6?

Sure enough, thank you. I love it. I'm always wondering if I'm getting index'd. :up: and by who...

Okay I will know in an hour if it is working fine with Vbulletin 3.6 Gold. The installation went fine, The instructions was a bit confusing for the cron job, hopefully I have set that up correctly for the hourly run of it. (Maybe that could be re-worded to be more noob friendly for people like myself ^^ lol)

Hope this helps you or anyone else that needs help in the cron per his instructions, here is a visual for you inside a cPanel CRON Job area see the attachment below ;)

I have just upgraded it and all the spider counts are still there :D

Working great! now 8 queries less and very pleased with this

Thank you :)

Right, it said when you upgrade from Beta 7 to Beta 8, you just upgraded to Beta 10 so this line doesn't apply.

Good to hear!

Something I been wondering, why is it that the hits seem to update allways, but the last visit date doesn't allways update?

I'll look into that, I think it has something to do with how the data is selected during the rollup. The latest timestamped record may not be the last record selected, so the timestamp is wrong. I think, I'm not positive...

Not sure if anyone has said this yet, but this seems to be working fine on vB 3.6.0 Gold :)

Good to hear.

Since I upgraded my last visits arent updating at all.

It seems as though the task dissapeared, I readded it but am not sure what to put for the Varname section. My last visit time still isnt updating, when its a new spider then it shows up but for google the last time it shows was before I upgraded to 3.6

Add a link to spiders.php on your navbar or footer.???
If this is correct:http://outerzone.us/forum/includes/spiders.php or http://www.outerzone.us/spiders.php I'm getting a 404 error
Yes I'm a noob....Any help would be greatly appreciated.

Amesome Thank You for helping me out

Ok one last request for this great hack,I can add a link to This other Great Hack ,but I would really like to add this to my 1st or 2nd navbar but I'm not exactly sure how to. I do know how to get to navbar, I see items that are in the navbar,but I dont know how to edit an existing item with proper wording to get The Spider Page to show up on 1st or 2nd navbar.......

Go to -> vBulletin Options -> Menu Control Version 1.3 Final Settings

Then add your link.

Name of menu Item: spiders
Menu link one: outerzone.us/forums/spiders.php

That's it. customize anything else you want to use.

Actually, I just looked and it looks like you figured it out.

I'm wondering..

Database error in vBulletin 3.5.4:

Invalid SQL:
DELETE FROM spider_watcher WHERE bot = '';show processlist;';

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ';show processlist;'' at line 1
Error Number : 1064

What exactly is this?
I don't see 'show processlist' anywhere in the php script, making me wonder it might be someone trying to exploit the script?

havnt look into it more but..

edit:
I see some entries in the mysqldatabase in the table 'bot'.

51219 ';show processlist; Unknown Spider 2006-08-16 22:33:48 1113767976 index 0
51220 ';show processlist; Unknown Spider 2006-08-16 22:33:52 1113767976 index 0
51223 ';show processlist; Unknown Spider 2006-08-16 22:34:08 1113767976 index 0
51399 ';show processlist; Unknown Spider 2006-08-16 22:54:18 1113767976 index 0
51402 ';show processlist; Unknown Spider 2006-08-16 22:54:22 1113767976 index 0
51403 ';show processlist; Unknown Spider 2006-08-16 22:54:27 1113767976 index 0
51404 ';show processlist; Unknown Spider 2006-08-16 22:54:31 1113767976 index 0
51405 ';show processlist; Unknown Spider 2006-08-16 22:54:35 1113767976 index 0
51406 ';show processlist; Unknown Spider 2006-08-16 22:54:38 1113767976 index 0
51407 ';show processlist; Unknown Spider 2006-08-16 22:54:43 1113767976 index 0
51408 ';show processlist; Unknown Spider 2006-08-16 22:54:47 1113767976 index 0
51409 ';show processlist; Unknown Spider 2006-08-16 22:54:51 1113767976 index 0

and some more nasty entry's

I have the latest one installed, and ive not had any problems with it and had it for about a week. Then I had to restart my sever today and then got this database error over and over and over hundreds of times.


Any ideas why?

Some records from MySql Dump:
My board visit crawler WebAlta (info for this crawler exist in my spiders_vbulletin.xml), in db records for this crawler exist, but this hack don't count visits of this crawler (field rolup already = 0). With other spiders I don't have this problem, only with WebAlta. Why ?

PS: I'm using version of this hack Beta 10, but when I use version beta 9 - this error still happened.

Need help!

It's because that spider is identifying itself as "Windows NT", so the rollup process thinks it's a user and not a spider. I guess it should check to see if there's anything in the "type" field before deleting it...

I'll fix that in the next version.

Good to see it's working fine on 3.6. I'm on it! :D

thx, waiting for the next version :)

So when is it gonna be released? Anytime soon? :)