Go Back   vb.org Archive > vBulletin 3 Discussion > vB3 General Discussions
  #1  
Old 06-22-2009, 04:23 PM
Bosetto Bosetto is offline
 
Join Date: Nov 2008
Posts: 5
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default Google indexing by means of sitemap

Dear all,
this is my forum:

http://lnx.scaricone.it/forums

I have big issues with Google because:
1) the spider is always on calendar pages and members profile
2) the indexed pages are few (about 500) in comparison with the high number of discussions (more than 400.000)
3) about 1 week ago I have added in my forum the "vbseo sitemap generator" : now in my google account I can see that the "url sent" are more than 450.000 but the "url indexed" are about 400 and each day this number seems to be updated starting again from zero. But never it is more than 450.

In my opinion the problem is in the point no.1

Could you please help me???

Thanks in advance!!!
Reply With Quote
  #2  
Old 06-23-2009, 06:05 PM
Bosetto Bosetto is offline
 
Join Date: Nov 2008
Posts: 5
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Please, could anyone help me?
pleaseeeeee
Reply With Quote
  #3  
Old 06-23-2009, 06:15 PM
HMBeaty's Avatar
HMBeaty HMBeaty is offline
 
Join Date: Sep 2005
Posts: 4,141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

One way to stop them from crawling pages you don't want them to is by creating a new text file and saving it as robots.txt. Then add the following text inside the robots.txt file:
Quote:
User-agent: *
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/member.php
Disallow: /forums/memberlist.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php
And upload this to your root directory. So basically it would come up as www.yoursite.com/robots.txt
Reply With Quote
  #4  
Old 06-25-2009, 05:34 PM
Bosetto Bosetto is offline
 
Join Date: Nov 2008
Posts: 5
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Dear Redline,
I uploaded the robots file and it was succesfully read by google (I checked on my webmaster tool account) but the situation is not changed!!! Google is still on calendar and profiles pages!!!!
What have I to do??
Please can anyone help me??????? Please!!!!!!!!
Reply With Quote
  #5  
Old 06-26-2009, 07:19 PM
Bosetto Bosetto is offline
 
Join Date: Nov 2008
Posts: 5
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Please,
is there an expert who can help me????

--------------- Added [DATE]1246132907[/DATE] at [TIME]1246132907[/TIME] ---------------

Quote:
Originally Posted by Bosetto View Post
Please,
is there an expert who can help me????
I cannot believe that nobody can solve this issue!!!
please!!!
Reply With Quote
  #6  
Old 06-27-2009, 07:41 PM
Bosetto Bosetto is offline
 
Join Date: Nov 2008
Posts: 5
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

I cannot believe that nobody can solve this issue!!!
please!!!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 11:24 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.12925 seconds
  • Memory Usage 2,204KB
  • Queries Executed 13 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (2)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (6)post_thanks_box
  • (6)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (6)post_thanks_postbit_info
  • (6)postbit
  • (6)postbit_onlinestatus
  • (6)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete