vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)
-   -   vbArchive - Search Engine Indexer for vBulletin (https://vborg.vbsupport.ru/showthread.php?t=47667)

TLucent 02-05-2003 07:25 PM

"googlebot.com (64.68.82.xxx) - Spider/Robot
04 Feb -- 18:33:47 -- -- Code 301 Moved Permanently = /forum"

I get this nearly everyday, and have never really been indexed. Is this normal?

Thx

saint_seiya 02-05-2003 07:28 PM

This is weird, i pay for lycos insite select ( http://insite.lycos.com ) and it still has not been indexed :( Any ideas why? I added a link to the archive today, from my forum page in case that was it.

As you see it should be 48 hour spider refreshes and i installed this a while ago :) Am I doing something wron Teck? I did your archive this weekend, i will wait this week and then email lycos for support ;)

BTW, my site is www.vgcity.com , archive: http://www.vgcity.com/forum/archive :chinese:

PS.- Teck when you finish your other project can you tell me, i think it was vBHL . Thanks :) :smoke:

jjj0923 02-05-2003 07:32 PM

google - don't think that's normal:

I get this:

Quote:

2/5/2003 4:22:53 PM
Search String: googlebot
Replace String:
Path: D:\logs
File Mask: *.*
Search Subdirectories
crawler10.<googlebot>.com - - [23/Dec/2002:06:22:22 -0500] "GET / HTTP/1.0" 302 0 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
crawler10.googlebot.com - - [23/Dec/2002:06:22:22 -0500] "GET / HTTP/1.0" 302 0 "-" "<Googlebot>/2.1 (+http://www.googlebot.com/bot.html)"
crawler10.googlebot.com - - [23/Dec/2002:06:22:22 -0500] "GET / HTTP/1.0" 302 0 "-" "Googlebot/2.1 (+http://www.<googlebot>.com/bot.html)"
crawler10.<googlebot>.com - - [23/Dec/2002:06:22:24 -0500] "GET /robots.txt HTTP/1.0" 404 279 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
crawler10.googlebot.com - - [23/Dec/2002:06:22:24 -0500] "GET /robots.txt HTTP/1.0" 404 279 "-" "<Googlebot>/2.1 (+http://www.googlebot.com/bot.html)"
crawler10.googlebot.com - - [23/Dec/2002:06:22:24 -0500] "GET /robots.txt HTTP/1.0" 404 279 "-" "Googlebot/2.1 (+http://www.<googlebot>.com/bot.html)"
crawler10.<googlebot>.com - - [23/Dec/2002:06:22:28 -0500] "GET /upload/index.php HTTP/1.0" 200 122266 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"



TECK 02-05-2003 07:49 PM

302 is not an error, check w3c related sites for the error number.
The 404 you get it because you don't have a robots.txt file where resides the main files, not the forum ones.

wooolF[RM] 02-06-2003 09:03 AM

]Just got an idea...

Imagine Forum home page :
Users Currently Online: 200 [ 100 users + 100 guests ] <-- just an EXAMPLE

idea is to trace IPs of all users and if they match any of the IPs owned by any of search crawlers like googlebot, altavista etc, show this :

Users Currently Online: 200 [ 100 users + 80 guests + Google + Altavista ] <-- just an EXAMPLE


Maybe looks ugly... maybe add extra queries... instead of dnsing/tracing all IPs u can just look after its ident (like Mozilla for IE).


PS: maybe it's not clever to add it on the forum home, but I would REALLY like to see this feature implemented on Who's Online page :)

I know u can do it, TECK ;)

Overgrow 02-06-2003 05:57 PM

From the Google Webmasters FAQ:

What is cloaking?

The term "cloaking" is used to describe a website that returns altered webpages to search engines crawling the site. In other words, the webserver is programmed to return different content to Google than it returns to regular users, usually in an attempt to distort search engine rankings. This can mislead users about what they'll find when they click on a search result. To preserve the accuracy and quality of our search results, Google may permanently ban from our index any sites or site authors that engage in cloaking to distort their search rankings.

http://www.google.com/webmasters/faq.html


I'm assuming what you mean to do is give the spider a different page than a user gets if they click through from the search results.

That is Link Cloaking and that is grounds for banning, no matter how similar the pages are. Do at your own risk.

Overgrow 02-06-2003 05:59 PM

>>Yes we've been listed for months by Google, but none of the last 3 searches has found archives

Hahahah let's blame Teck for Google's spidering. People using his hack are in Google. If you can't get your archive in there, that is your fault. Even people using my old vBSpiderFriend are doing very well in Google... DevShed was serving me answers with it just last week from the top Google 1-5 result spots.

Floris 02-06-2003 07:21 PM

Tonight I received a google attack :)

49 guests online, 5 members :)

saint_seiya 02-06-2003 07:23 PM

How did you all get that part where it says from where the guest came. I am going to read the readme again :p

Floris 02-06-2003 07:23 PM

damn
IT DOESN"T STOP

4 Members and 53 Guests

The hosts are a vbulletin option > resolve hosts on whois online : yes.


All times are GMT. The time now is 05:49 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01407 seconds
  • Memory Usage 1,745KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete