The Arcive of Official vBulletin Modifications Site.

dawn100 · #1 12-17-2007, 02:45 PM

Ok, I am *very* new to this, so I need some guidance on bots. I did create the robots.txt file, as suggested in another thread. This morning I checked my stats, and saw this:

Robots/Spiders visitors (Top 25) - Full list - Last visit
3 different robots* Hits Bandwidth Last visit
Googlebot 41 239.85 KB 16 Dec 2007 - 15:09
Unknown robot (identified by 'bot/' or 'bot-') 2+1 42.31 KB 17 Dec 2007 - 04:27
Unknown robot (identified by hit on 'robots.txt') 0+1 1.64 KB 16 Dec 2007 - 15:13

I am fine with the googlebot. What do the other 2 lines tell me? Do I need to do something to further restrict access?

Any insight appreciated!

pcoskat · #2 12-17-2007, 02:49 PM

You might want to also check vbulletin.com's board for posts on bots. There are some great threads on it. (especially check out the ones on the Yahoo bots)

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.03297 seconds Memory Usage 2,160KB Queries Executed 13 (?)
More Information
Template Usage: (1)SHOWTHREAD (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)ad_showthread_beforeqr (1)ad_showthread_firstpost (1)ad_showthread_firstpost_sig (1)ad_showthread_firstpost_start (1)footer (1)forumjump (1)forumrules (1)gobutton (1)header (1)headinclude (1)navbar (3)navbar_link (120)option (2)post_thanks_box (2)post_thanks_button (1)post_thanks_javascript (1)post_thanks_navbar_search (2)post_thanks_postbit_info (2)postbit (2)postbit_onlinestatus (2)postbit_wrapper (1)spacer_close (1)spacer_open (1)tagbit_wrapper Phrase Groups Available: global inlinemod postbit posting reputationlevel showthread	Included Files: ./showthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/functions_bigthree.php ./includes/class_postbit.php ./includes/class_bbcode.php ./includes/functions_reputation.php ./includes/functions_post_thanks.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete showthread_start showthread_getinfo forumjump showthread_post_start showthread_query_postids showthread_query bbcode_fetch_tags bbcode_create showthread_postbit_create postbit_factory postbit_display_start post_thanks_function_post_thanks_off_start post_thanks_function_post_thanks_off_end post_thanks_function_fetch_thanks_start post_thanks_function_fetch_thanks_end post_thanks_function_thanked_already_start post_thanks_function_thanked_already_end fetch_musername postbit_imicons bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete postbit_display_complete post_thanks_function_can_thank_this_post_start tag_fetchbit_complete forumrules navbits navbits_complete showthread_complete
Messages:

The Arcive of Official vBulletin Modifications Site.

It is not a VB3 engine, just a parsed copy!