The Arcive of Official vBulletin Modifications Site.

GraphiX2004 · #1 05-12-2009, 04:25 PM

Some admins reported to me today that our community is now seeing google spider.
when we look where the spider is viewing its normally search or a certain forum.

today i noticed the spider was on the site so i checked where it was viewing.
it was actually viewing a members profile page??? wtf

why would google spider bot be singling out a member's personal profile page?
i understand kind of why we have this google spider bot but cant we just turn it off?
or is this built into vbulletin so it's mandatory for search engines to sqirm all over our forum?

Any help how to get rid of the google spider or why its now singling out members
would really be helpful thanks guys. and girls.

BlueNinjaGo · #2 05-12-2009, 05:35 PM

It's indexing your site. So if someone searches "GraphiX2004" for example, it'll pull it up.

As far as I know, a spider bot can only view what a guest can view. So by stopping guests from viewing profiles it should stop it. But it's not really causing any harm.

Cryo · #3 05-12-2009, 06:47 PM

The point of a spider is to crawl through your pages and index them. If you want to prevent it from indexing your member pages I'd suggest reading this site. You can add a specific exception for member.php.

Brandon Sheley · #4 05-12-2009, 07:02 PM

add a line in your robots.txt file to exclude the profile pages
The bot is only doing what it was designed for

GraphiX2004 · #5 05-13-2009, 01:09 AM

ok guys thanks for the headups

Biker_GA · #6 05-13-2009, 01:28 AM

Also, if there was a stop sign associated with the entry, it means the spider was not able to index the page. No access.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.03739 seconds Memory Usage 2,198KB Queries Executed 11 (?)
More Information
Template Usage: (1)SHOWTHREAD (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)ad_showthread_beforeqr (1)ad_showthread_firstpost (1)ad_showthread_firstpost_sig (1)ad_showthread_firstpost_start (1)footer (1)forumjump (1)forumrules (1)gobutton (1)header (1)headinclude (1)navbar (3)navbar_link (120)option (6)post_thanks_box (6)post_thanks_button (1)post_thanks_javascript (1)post_thanks_navbar_search (6)post_thanks_postbit_info (6)postbit (6)postbit_onlinestatus (6)postbit_wrapper (1)spacer_close (1)spacer_open (1)tagbit_wrapper Phrase Groups Available: global inlinemod postbit posting reputationlevel showthread	Included Files: ./showthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/functions_bigthree.php ./includes/class_postbit.php ./includes/class_bbcode.php ./includes/functions_reputation.php ./includes/functions_post_thanks.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete showthread_start showthread_getinfo forumjump showthread_post_start showthread_query_postids showthread_query bbcode_fetch_tags bbcode_create showthread_postbit_create postbit_factory postbit_display_start post_thanks_function_post_thanks_off_start post_thanks_function_post_thanks_off_end post_thanks_function_fetch_thanks_start post_thanks_function_fetch_thanks_end post_thanks_function_thanked_already_start post_thanks_function_thanked_already_end fetch_musername postbit_imicons bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete postbit_display_complete post_thanks_function_can_thank_this_post_start tag_fetchbit_complete forumrules navbits navbits_complete showthread_complete
Messages:

The Arcive of Official vBulletin Modifications Site.

It is not a VB3 engine, just a parsed copy!