vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vB3 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=111)
-   -   High CPU usage by my site>>help me?? (https://vborg.vbsupport.ru/showthread.php?t=289516)

CAG CheechDogg 01-20-2013 03:09 PM

Bing, Facebook and Baidu bots do cause those issues, for me it was facebook's bots not Bings or Baidu's.

Dr_Mo7ammad 01-22-2013 07:36 AM

they sent me these instructions to do

You can also prevent robots from crawling parts of your site, while allowing them to crawl other sections. The following example would request search engines and robots not to crawl the cgi-bin folder, the tmp folder, and the junk folder and everything in those folders on your website.
If you want to stop bots from visiting you site and stop search engines from ranking you, use this code:

#Code to not allow any search engines!
User-agent: *
Disallow: /

2nd choice which i used and iam asking is this enough
# Blocks robots from specific folders / directories
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/

Simon Lloyd 01-22-2013 04:17 PM

That only works if the bots obey the robots.txt and many dont!

Dr_Mo7ammad 01-30-2013 07:00 PM

thanks
but how they donot obey???

also
they suspend again
and said to me to enable caching

how to enable them???

--------------- Added [DATE]1359576211[/DATE] at [TIME]1359576211[/TIME] ---------------

does this topic help

https://vborg.vbsupport.ru/showthread.php?t=139151

iam sorry iam annoying you
but help me
really iam depressed from recurrent suspension

final kaoss 01-30-2013 07:07 PM

Try the article in my signature, as for bingbot I think you can regulate it's crawling via the bing webmaster tools.

Dr_Mo7ammad 01-30-2013 07:15 PM

my version is 3.8

final kaoss 01-30-2013 07:16 PM

Some of the tips also work for 3.8. The ones that don't clearly state that they don't.

final kaoss 01-30-2013 07:37 PM

1 Attachment(s)
Once you sign in to BWT & verify your site ownership, navigate to

Dashboard > Configure My Site > Crawl Control

Less bars = Lower Crawl Speed

Dr_Mo7ammad 01-30-2013 08:21 PM

1 Attachment(s)
for sorry
i cannot find crawl control
attached what i can see

final kaoss 01-30-2013 08:27 PM

Well if you like you can tell it to block urls & force it to stop visiting completely.


All times are GMT. The time now is 09:47 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01141 seconds
  • Memory Usage 1,729KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete