Go Back   vb.org Archive > Community Central > Community Lounge
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools Display Modes
  #1  
Old 07-26-2007, 11:57 PM
theFAILURE's Avatar
theFAILURE theFAILURE is offline
 
Join Date: Jan 2005
Posts: 141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default Will the Slurp ever calm down?

Does anyone know if Yahoo is going to calm down with the attack of the Yahoo Slurp Spider?

Tonight I had quite a few of them:



On boards with a shared hosting packages, it must really suck.

If you have limited bandwidth, it also must kill you.

Tonight they almost beat my "most users ever online" stat!
Reply With Quote
  #2  
Old 07-27-2007, 12:12 AM
d8tabyte's Avatar
d8tabyte d8tabyte is offline
 
Join Date: Nov 2005
Location: Michigan
Posts: 239
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

you can throttle the slurp in your robots file
Reply With Quote
  #3  
Old 07-27-2007, 12:13 AM
nexialys
Guest
 
Posts: n/a
Default

it's not a big amoung of bandwidth actually, because each slurp is a page hit... yahoo is not using the same technique as google... google is more to fear actually because if you have more than one google connection, you have some bots hitting your board at the same time...
Reply With Quote
  #4  
Old 07-27-2007, 01:03 AM
theFAILURE's Avatar
theFAILURE theFAILURE is offline
 
Join Date: Jan 2005
Posts: 141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by d8tabyte View Post
you can throttle the slurp in your robots file
I read on several occasions that they are ignoring the robots.txt file.

True?
Reply With Quote
  #5  
Old 07-27-2007, 08:01 AM
projectego's Avatar
projectego projectego is offline
 
Join Date: Feb 2006
Location: UK
Posts: 724
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Wow. That's a whole lot of spiders you have online, tF. I'm stll waiting for the day that many decide to hit my boards.
Reply With Quote
  #6  
Old 07-27-2007, 04:57 PM
theFAILURE's Avatar
theFAILURE theFAILURE is offline
 
Join Date: Jan 2005
Posts: 141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

I have realized that there have been a WHOLE LOT MORE spiders since I turned on the "forum archive".

This leads me to believe if you are having a problem with the spiders and have the forum archive turned on, you can turn it off in your admincp to release some pressure if there is any.
Reply With Quote
  #7  
Old 07-27-2007, 05:49 PM
nexialys
Guest
 
Posts: n/a
Default

forum archive is actually the best way for search engine to access your site... this is the quickest and simpliest way, but also better for your server, and the archive is 10 times less server-intense...
Reply With Quote
  #8  
Old 07-27-2007, 05:59 PM
theFAILURE's Avatar
theFAILURE theFAILURE is offline
 
Join Date: Jan 2005
Posts: 141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

With that in mind, is there a way to keep them off regular threads, and keep them crawling through the Archives only?
Reply With Quote
  #9  
Old 07-27-2007, 06:07 PM
nexialys
Guest
 
Posts: n/a
Default

i suppose there is a hack that control that... did not search.
Reply With Quote
  #10  
Old 07-27-2007, 06:11 PM
theFAILURE's Avatar
theFAILURE theFAILURE is offline
 
Join Date: Jan 2005
Posts: 141
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Essentially if its the same EXACT result, I can not see why anyone with Archiving turned on would not opt for it
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 09:57 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.06808 seconds
  • Memory Usage 2,245KB
  • Queries Executed 13 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (1)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (1)pagenav_pagelink
  • (10)post_thanks_box
  • (10)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (10)post_thanks_postbit_info
  • (10)postbit
  • (7)postbit_onlinestatus
  • (10)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete