vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vB3 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=111)
-   -   Should this concern me? (bots) (https://vborg.vbsupport.ru/showthread.php?t=165379)

dawn100 12-17-2007 02:45 PM
Should this concern me? (bots)
 
Ok, I am *very* new to this, so I need some guidance on bots. I did create the robots.txt file, as suggested in another thread. This morning I checked my stats, and saw this:

Robots/Spiders visitors (Top 25) - Full list - Last visit
3 different robots* Hits Bandwidth Last visit
Googlebot 41 239.85 KB 16 Dec 2007 - 15:09
Unknown robot (identified by 'bot/' or 'bot-') 2+1 42.31 KB 17 Dec 2007 - 04:27
Unknown robot (identified by hit on 'robots.txt') 0+1 1.64 KB 16 Dec 2007 - 15:13

I am fine with the googlebot. What do the other 2 lines tell me? Do I need to do something to further restrict access?

Any insight appreciated!

pcoskat 12-17-2007 02:49 PM
You might want to also check vbulletin.com's board for posts on bots. There are some great threads on it. (especially check out the ones on the Yahoo bots)


All times are GMT. The time now is 02:11 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.00997 seconds
  • Memory Usage 1,700KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (2)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 
Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 
Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete