The Arcive of Official vBulletin Modifications Site.It is not a VB3 engine, just a parsed copy! |
|
#1
|
|||
|
|||
crawl-0c.cuil.com
Is anyone else having problems with this search engine crawler. I am assuming thats what it is, I did a google search and came up with cuil.com and that is what it says.
This thing is constantly on my site, and I keep getting mysql database errors from it. That IP addy listed below is registered to crawl-0c.cuil.com. Database error in vBulletin : mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Lost connection to MySQL server at 'reading initial communication packet', system error: 111 /home/marin49/public_html/includes/class_core.php on line 312 MySQL Error : Error Number : Request Date : Monday, January 18th 2010 @ 11:28:16 AM Error Date : Monday, January 18th 2010 @ 11:28:16 AM Script : http://www.boatinghowto.com/external...S2&forumids=38 Referrer : IP Address : 216.129.119.10 Username : Classname : vB_Database MySQL Version : |
#2
|
|||
|
|||
Create a robots.txt file and place in it. Or if you have a robots.txt simply append it to the end.
User-Agent: twiceler Crawl-delay: 30 You can also completely block the crawler, since their search sucks any way, but that's up to you. |
#3
|
|||
|
|||
thanks bsmedia, I may just go ahead and block them. But just curious what does the robots.txt file actually do. I am new to this so excuse me if it is a dumb question...lol
tim |
#4
|
|||
|
|||
<a href="http://robotstxt.org" target="_blank">http://robotstxt.org</a>
Robots.txt is a file that reputable search engine spiders follow as a set of instructions of directories to include in their search results, to delay their requests to pages, etc. I forgot to mention it needs to be placed in your top level directory, so you'll access it from domain.com/robots.txt or subdomain.domain.com/robots.txt |
#5
|
|||
|
|||
Thank you, I will do a google search on that and see if what I need to put in there and learn all the ins and outs.
|
|
|
X vBulletin 3.8.12 by vBS Debug Information | |
---|---|
|
|
More Information | |
Template Usage:
Phrase Groups Available:
|
Included Files:
Hooks Called:
|