vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.5 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=113)
-   -   Spider Watcher (https://vborg.vbsupport.ru/showthread.php?t=104582)

The Notorious 04-22-2006 07:01 PM

Good hack, installed.

Trana 04-27-2006 03:24 PM

What does everyone recommend I do with this nasty beast:

Unknown Spider
Mozilla/5.0 (000000000; 0; 000 000 00 0 000000; 00000; 0000000000) 00000000000000 000000000000000

82.48.249.204
82.56.186.64

Is there an ID for it? If it won't identify itself should I just start blocking the IPs? Can I restrict them in my robots.txt?

Thanks!

DementedMindz 04-27-2006 04:00 PM

personally im about to remove this hack... it has high queries and it sucks they dont update...

Trana 04-27-2006 04:35 PM

Quote:

Originally Posted by DementedMindz
personally im about to remove this hack... it has high queries and it sucks they dont update...

Its pretty stupid that it won't update old records in the DB, and since it uses so many queries it is obviously coded poorly.
But won't the high number of queries on affect the server when you load the spider page? I only allow admins to load it, so people shouldn't be hitting it all the time.

DementedMindz 04-27-2006 04:55 PM

yeah they affect the server load... im removing it since i dont think it will be updated anytime soon... if it is then i may reinstall but as of right now it sucks editing from sql or having 500 bots unknown

Trana 04-27-2006 05:15 PM

But it only affects the server when you actually load the page right? Its not an ongoing thing with cron or anything is it?

mikelbeck 04-27-2006 10:04 PM

It's not "stupid" that it doesn't update, the installation instructions say to update your spiders xml file BEFORE you install this. If you do that, then you won't have anything to update, as it'd be using the latest data.

The only time there should be a lot of queries is when it hasn't "rolled up" the data recently. After that first hit the number of queries should drop.

Quote:

and since it uses so many queries it is obviously coded poorly
That may be the stupidiest thing I've ever heard.

Look, people, I wrote this thing for myself. I posted it here thinking that other people may benefit from it. If you want to use it, use it. If you don't, then don't. But don't go bashing it or my code. If you think you can do better, then write your own.

I will update it when I can, right now my top priority is finding a new job so I can support my family. Updating the the freebie hacks that I write when the mood strikes me aren't even in the top 10 of my list of priorities right now.

{edit}
One other thing... The first version that I put out deciphered the spider data when the user viewed the spiders page. If the spiders xml file had been updated after the data was collected it would be interpreted properly. But that version had way too many queries and everybody complained about it. Now the spider data is decoded when the spider visits and it's written to the database. The spiders page just collates all of that data and doesn't do any deciphering.

xStylezx 04-27-2006 10:40 PM

This hack is great bro.The only real issue i see with it is that sometimes the date and time doesnt get updated on a fresh spider visit to the forum.Sometimes the date reflects the true last visit and sometimes it doesnt.That is about the only thing i wish could be fixed up.Other than that,this hack is excellent and dont worry about harsh opinions of it,i cant wait to see the next update

Trana 04-28-2006 02:36 PM

Mike,

I appreciate this hack, it gives me good visibility to what the spiders are actually doing. I'm not a very good coder so I can't improve on your work.

My previous message was just stating that 58 queries is an extraordinarily large amount for such a simple page. It was my impression that the data would be regenerated when the spider page was loaded only, now I realize that its doing a lot of work each time a spider hits the site.

Thanks again.

Logikos 04-29-2006 05:48 AM

great hack, but there lays a very crtitcal problem for me at least. When viewing the page.

Page generated in 0.59078 seconds with 50 queries [Server Loads: 0.41 0.22 : 0.10]

Thats quite alot of quires. The more spiders it fetches, the more queires this adds.


All times are GMT. The time now is 05:05 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01289 seconds
  • Memory Usage 1,740KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (2)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete