vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.5 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=113)
-   -   Spider Watcher (https://vborg.vbsupport.ru/showthread.php?t=104582)

mikelbeck 01-08-2006 08:15 PM

Quote:

Originally Posted by bashy
Hi m8

MySQL Variables
system_time_zone GMT

My time zone is GMT
Forum Timezone is GMT also

I dont suppose this helps you any now does it lol

Bashy

Not even a little bit!

I'm clueless as to why those dates are wrong, I'll have to look into it and will come up with a fix.

bashy 01-08-2006 08:18 PM

Didnt think it would help you with them all being GMT ;)
Will wait for a fix, no rush as its still recording them regardless :)

Regards

Bashy


Quote:

Originally Posted by mikelbeck
Not even a little bit!

I'm clueless as to why those dates are wrong, I'll have to look into it and will come up with a fix.


mikelbeck 01-08-2006 08:22 PM

Can run an SQL query? I'm interested to see the results of this:

SELECT now() , curdate() , curtime();

Can you run that and post the results? Also please post what the current time in your area is.

Zia 01-09-2006 04:01 AM

hello mikelbeck

need help....can u plz check the link

http://www.golpo.net/forum/spiders.php

There are some unknown-spiders & they faced ErrorDocs..we need to know in which location spider faced the error docs...

and we need to know more about spider list..mean updated spider list..
can u tell me where to submit the spiders details plz??

nitro 01-09-2006 04:05 AM

You would need to add unknown spiders to the xml file in an editor afaik

to stop spiders roaming unauthorised areas you would if I have it right want to place a robots.txt file in your webroot with details on what pages you dont want them to view. It can take sometime for the robots.txt file to get cached by the spiders.

Zia 01-09-2006 05:22 AM

Quote:

Originally Posted by nitro
You would need to add unknown spiders to the xml file in an editor afaik

to stop spiders roaming unauthorised areas you would if I have it right want to place a robots.txt file in your webroot with details on what pages you dont want them to view. It can take sometime for the robots.txt file to get cached by the spiders.

Can u help me a bit? how could i stop robot to visit inside a thread.
just dont want spider face any error docs?

how could I do it??

bashy 01-09-2006 06:13 PM

hi mikelbeck

Just a little more on this time issue, i thought i'd mention that although the 20 Hours Ago for example is incorrect the 08:08 PM actually is correct, strange 'eh lol

Regards

Bashy

mikelbeck 01-09-2006 06:15 PM

Quote:

Originally Posted by bashy
hi mikelbeck

Just a little more on this time issue, i thought i'd mention that although the 20 Hours Ago for example is incorrect the 08:08 PM actually is correct, strange 'eh lol

Regards

Bashy

Did you do this yet? https://vborg.vbsupport.ru/showpost....8&postcount=43

DementedMindz 01-11-2006 05:22 AM

mikelbeck i like your site i was wondering is there somewhere where we can get the Drinks Menu that you have on your site ?

Devil Woman 01-11-2006 09:33 AM

I was wondering if there is anyway you can prevent members from being able to see the spider page at all and prevent members from seeing that admin and mods are in the spider page on the who is online page

Thanks

samu2 01-11-2006 11:04 AM

When does it start showing the spiders? I have got 6 spiders on now but it is showing that I have none.

http://www.mumsonthenet.co.uk/forum/spiders.php

Thanks for the hack

mikelbeck 01-11-2006 11:46 AM

DementedMindz: That's a database that I bought and a custom interface that I wrote... For now, it's not available.

Devil Woman: Not right now.

samu2: It should start showing spiders right away.

samu2 01-11-2006 01:33 PM

Darn,it hasn't shown one yet :(

mikelbeck 01-11-2006 01:40 PM

Quote:

Originally Posted by samu2
Darn,it hasn't shown one yet :(

That's odd, it should pick them up right away. Did you get any errors when you installed it?

Could you try un-installing and re-installing?

Devil Woman 01-11-2006 01:55 PM

Will this ever be added to this hack maybe or will you ever consider adding this in at all?

Thanks

samu2 01-11-2006 01:57 PM

No erros.Just un installed and re installed.No change.

mikelbeck 01-11-2006 02:16 PM

Devil Woman: I was thinking about adding permissions, so you can set who can and who can't see the page. I guess I'll do it.

samu2: Can you verify that you have a plugin called "Spider Watcher" that's hooked into "global_start"?

samu2 01-11-2006 02:19 PM

Yeah,I do :)

Devil Woman 01-11-2006 02:32 PM

Thanks for that, is there anyway you can password protect the page for now at all to prevent members getting in, or is that not possible either?

Thanks

DementedMindz 01-11-2006 03:11 PM

hmm thats strange cause as soon as i submitted my site i had spiders crawl it and they showed up right away

DementedMindz 01-11-2006 03:14 PM

another question is there a way we can add more spiders? so they dont show up as unknown?

samu2 01-11-2006 03:14 PM

I think my site just hates me lately lol

The spiders have been stuck on the home page all day.I wonder if that has anything to do with it? mind you ad sense has been active.

DementedMindz 01-11-2006 03:14 PM

lol which spiders are they?

samu2 01-11-2006 03:18 PM

Google and slurp have been on and seems to have gotten stuck lol

I uploaded buttons and saw them three days after everyone else could.Had to put up a redirect last night,works ok for everyone else,bar me.

DementedMindz 01-11-2006 03:23 PM

thats really strange im not sure at all i would ask someone who knows alot more then me i never see mine on site long at all

Zia 01-11-2006 03:34 PM

Hello....we are getting spider...most of them are unown..
confused


http://www.golpo.net/forum/spiders.php



*Unknown Spider
Dillo/0.8.5 01-09-2006
10:49 PM 221.135.208.159

*Unknown Spider
Googlebot-Image/1.0 01-07-2006
01:09 AM 66.249.65.202

*Unknown Spider
Microsoft Data Access Internet Publishing Provider Protocol Discovery 01-07-2006
01:48 PM 203.189.231.13

*Unknown Spider
Mozilla/2.0 (compatible; MS FrontPage 5.0) 01-08-2006
02:35 PM 203.189.231.13

*Unknown Spider
Mozilla/3.01 (compatible 1-08-2006
10:50 PM 12.20.58.68, 69.88.30.146, 202.52.197.55, 202.52.197.56, 210.50.143.21

*Unknown Spider
Mozilla/4.0 (compatible; Google Desktop) 01-08-2006
05:11 PM 70.236.181.179

*Unknown Spider
Mozilla/4.0 (compatible; MSIE 6.0) 01-09-2006
10:58 PM 202.63.231.150

*Unknown Spider
MSFrontPage/5.0 01-07-2006
01:48 PM 203.189.231.13

*Unknown Spider
Nokia6600/1.0 (5.27.0) SymbianOS/7.0s Series60/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.0 01-08-2006
11:25 AM 202.56.4.109


Seemd enugh of unknown spiders..confused any of them bandwidth theaf or not?

Any one can say which bots they are ?


Thanx

XtremeOffroad 01-11-2006 04:59 PM

Installed fine on 3.5.3 but no spiders are showing.
Been watching them on who's online though.
Any Idea why this isnt seeing them?

mikelbeck 01-11-2006 10:15 PM

Quote:

Originally Posted by Devil Woman
Thanks for that, is there anyway you can password protect the page for now at all to prevent members getting in, or is that not possible either?

Thanks

You can do that with an .htaccess file. GIS for "password protect page htaccess".

For everybody with the "unknown" spiders. If the spider's user agent isn't in the vbulletin_spider.xml file, then it's going to be reported as "unknown". You can take a whack at adding the spiders to that file yourself, or just keep an eye on the thread related to that file and wait for it to be updated.

Devil Woman 01-11-2006 10:19 PM

Lol I know many things but I wouldn't know where to start with that, are you familiar with that sort of thing at all and if I pm'd would you be able to help at all?

would be greatly appreciated

Thanks

Lizard King 01-11-2006 10:22 PM

I had to remove this because this script cannot track my spiders activity :) The log is too big and i always receive Maximum execution time of 30 seconds exceeded error with this.

DementedMindz 01-11-2006 11:22 PM

You would have to add more names of the bots in the spiders_vbulletin.xml file.
PHP Code:
Code:

<spider ident="METATAG INFO"> 
<name>SpiderName of Your Desire</name> 
</spider>


mikelbeck 01-11-2006 11:34 PM

Lizard King: On what page do you get that message? And what do you mean, "the log is too big"? There is no log, it's a table in the database.

Devil Woman: Have a look at this page: http://www.4webhelp.net/tutorials/misc/htaccess.php

Lizard King 01-12-2006 05:30 AM

Quote:

Originally Posted by mikelbeck
Lizard King: On what page do you get that message? And what do you mean, "the log is too big"? There is no log, it's a table in the database.

Devil Woman: Have a look at this page: http://www.4webhelp.net/tutorials/misc/htaccess.php

When i tried to view spiders.php the first day it worked but after first day it never worked. ıt might be because i have a lot of spider activity lately. Checked the attached log because of this i always get timeout errors.

NexDog 01-12-2006 05:45 AM

This is great - installs. I want admins only to be able to see this so what conditional would I need for forumhome...if condition="$bbuserinfo['user.....?

armagan 01-12-2006 08:55 AM

Quote:

Originally Posted by Lizard King
When i tried to view spiders.php the first day it worked but after first day it never worked. ıt might be because i have a lot of spider activity lately. Checked the attached log because of this i always get timeout errors.

!!!!

mikelbeck 01-12-2006 12:26 PM

Quote:

Originally Posted by NexDog
This is great - installs. I want admins only to be able to see this so what conditional would I need for forumhome...if condition="$bbuserinfo['user.....?

Yeah, something like this:

Code:

<if condition="$permissions['adminpermissions']">
<a href="$vboptions[homeurl]/spiders.php">Spider Activity</a>
</if>


Zia 01-12-2006 01:03 PM

Quote:

Originally Posted by samu2
Google and slurp have been on and seems to have gotten stuck lol

I uploaded buttons and saw them three days after everyone else could.Had to put up a redirect last night,works ok for everyone else,bar me.

ahha...same here...Tahoo !Slurp just sticky my place...even 3/4 at a time..
Google very frequently...

just wondering too many spider can slow down forum ??

NexDog 01-12-2006 01:16 PM

Quote:

Originally Posted by mikelbeck
Yeah, something like this:

Code:

<if condition="$permissions['adminpermissions']">
<a href="$vboptions[homeurl]/spiders.php">Spider Activity</a>
</if>


I did it with a usergroup ID:

Code:

<if condition="is_member_of($bbuserinfo, 6)">
6 being the ID of the admin usergroup. Can you think of any drawbacks to doing it that way?

The hack is so cool though. Since I installed just a few hours ago it's logged over 900 hits from google and 400 from slurp. :)

Club3G 01-12-2006 04:19 PM

Installed, but getting the following error in 3.5.3:

Code:

Database error in vBulletin 3.5.3:

Invalid SQL:
START TRANSACTION;

MySQL Error  : You have an error in your SQL syntax near 'START TRANSACTION' at line 1
Error Number : 1064
Date        : Thursday, January 12th 2006 @ 01:18:47 PM
Script      : http://www.sevenstring.org/forum/spiders.php
Referrer    :
IP Address  : xxx
Username    : Chris
Classname    : vb_database

:(

mikelbeck 01-12-2006 04:21 PM

NexDog: I'm able to view your spiders page as a guest, so I'm going to assume that your code isn't working properly.

Club3G: What database & version are you using?


All times are GMT. The time now is 01:04 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.03086 seconds
  • Memory Usage 1,829KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (5)bbcode_code_printable
  • (11)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (3)pagenav_pagelink
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete