Go Back   vb.org Archive > vBulletin 3 Discussion > vB3 General Discussions
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools Display Modes
  #1  
Old 05-12-2009, 04:25 PM
GraphiX2004 GraphiX2004 is offline
 
Join Date: Mar 2009
Posts: 28
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default Google Spider? why is it doing this?

Some admins reported to me today that our community is now seeing google spider.
when we look where the spider is viewing its normally search or a certain forum.

today i noticed the spider was on the site so i checked where it was viewing.
it was actually viewing a members profile page??? wtf

why would google spider bot be singling out a member's personal profile page?
i understand kind of why we have this google spider bot but cant we just turn it off?
or is this built into vbulletin so it's mandatory for search engines to sqirm all over our forum?

Any help how to get rid of the google spider or why its now singling out members
would really be helpful thanks guys. and girls.
Reply With Quote
  #2  
Old 05-12-2009, 05:35 PM
BlueNinjaGo's Avatar
BlueNinjaGo BlueNinjaGo is offline
 
Join Date: Mar 2009
Posts: 668
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

It's indexing your site. So if someone searches "GraphiX2004" for example, it'll pull it up.

As far as I know, a spider bot can only view what a guest can view. So by stopping guests from viewing profiles it should stop it. But it's not really causing any harm.
Reply With Quote
  #3  
Old 05-12-2009, 06:47 PM
Cryo Cryo is offline
 
Join Date: Dec 2003
Location: Buffalo, NY
Posts: 197
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

The point of a spider is to crawl through your pages and index them. If you want to prevent it from indexing your member pages I'd suggest reading this site. You can add a specific exception for member.php.
Reply With Quote
  #4  
Old 05-12-2009, 07:02 PM
Brandon Sheley's Avatar
Brandon Sheley Brandon Sheley is offline
 
Join Date: Mar 2005
Location: Google Kansas
Posts: 4,678
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

add a line in your robots.txt file to exclude the profile pages
The bot is only doing what it was designed for
Reply With Quote
  #5  
Old 05-13-2009, 01:09 AM
GraphiX2004 GraphiX2004 is offline
 
Join Date: Mar 2009
Posts: 28
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

ok guys thanks for the headups
Reply With Quote
  #6  
Old 05-13-2009, 01:28 AM
Biker_GA Biker_GA is offline
 
Join Date: Oct 2004
Location: Where my hat is
Posts: 829
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Also, if there was a stop sign associated with the entry, it means the spider was not able to index the page. No access.
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 12:22 AM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.03448 seconds
  • Memory Usage 2,198KB
  • Queries Executed 11 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (6)post_thanks_box
  • (6)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (6)post_thanks_postbit_info
  • (6)postbit
  • (6)postbit_onlinestatus
  • (6)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete