vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vB3 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=111)
-   -   Google Spider? why is it doing this? (https://vborg.vbsupport.ru/showthread.php?t=213493)

GraphiX2004 05-12-2009 04:25 PM

Google Spider? why is it doing this?
 
Some admins reported to me today that our community is now seeing google spider.
when we look where the spider is viewing its normally search or a certain forum.

today i noticed the spider was on the site so i checked where it was viewing.
it was actually viewing a members profile page??? wtf

why would google spider bot be singling out a member's personal profile page?
i understand kind of why we have this google spider bot but cant we just turn it off?
or is this built into vbulletin so it's mandatory for search engines to sqirm all over our forum?

Any help how to get rid of the google spider or why its now singling out members
would really be helpful thanks guys. and girls.

BlueNinjaGo 05-12-2009 05:35 PM

It's indexing your site. So if someone searches "GraphiX2004" for example, it'll pull it up.

As far as I know, a spider bot can only view what a guest can view. So by stopping guests from viewing profiles it should stop it. But it's not really causing any harm.

Cryo 05-12-2009 06:47 PM

The point of a spider is to crawl through your pages and index them. If you want to prevent it from indexing your member pages I'd suggest reading this site. You can add a specific exception for member.php.

Brandon Sheley 05-12-2009 07:02 PM

add a line in your robots.txt file to exclude the profile pages
The bot is only doing what it was designed for ;)

GraphiX2004 05-13-2009 01:09 AM

ok guys thanks for the headups

Biker_GA 05-13-2009 01:28 AM

Also, if there was a stop sign associated with the entry, it means the spider was not able to index the page. No access.


All times are GMT. The time now is 11:45 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.00946 seconds
  • Memory Usage 1,717KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (6)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete