Teck - why the need to rename to archive.html? I have 20 Inktomi spiders in there now so don't want to interrupt anything.

Might do on vbHome. Mainly doing this for added content and links and not so worried that everything gets listed in Google...

I'm telling you what I did, only.
That doesn't mean you have to do the same like me...

Hey man, it's your hack so I just want to benefit from your skills. ;)

Woke up this morning to find all bots in the Archive - 30 googlebots doing their thing so me is a happy chappy. :D

Would just like to know the theory behind naming the archive with a .html extension.....

alright, I'm getting a parse error after I add the add_to_functions.php to admin/functions.php

Any ideas?

nevermind, a small little tiny typo

it seems bots arent interested in visiting my site ;)
still waiting for bot rush but only sometimes googlebot10-11-12 visits my board then leave ;) they cant find archive ... they try to "retrieve password" , "register" and similiar things and they go out after visiting main page of archive finally ...

do i have to put every php file as it listed by TECK to robots.txt ... or "removing all those and allowing bots to visit all those" is better ?

thanks

What's your board's URL? I would be happy to take a look for you. And yea, put all php pages in the robots.txt except index.php.

http://www.trojanforge.net
vbhome installed as home.php ,
via htaccess top of domain is home.php
board index is at : http://www.trojanforge.net/index.php
archive is at : http://www.trojanforge.net/archive/

and for robots.txt ; as they will get no permission notices they can browse other pages from that pages i think via top and bottom links ... ermm ;)

Hmmm, don't know why they aren't finding the archive. I would implement the mod that links the on/off gif to the corresponding forum in the archive.

I've installed this 3 times for clients with absolute ease ...... until today ;)

In the server when i go to rename the archive.txt and then when i rename it to 'archive' it disappears from the file list. And upon a refresh it reappears as the old filename. So i thought i'd try uploading it as just 'archive' and not archive.txt and then in the ftp log thingy it says:

I don't know what to do or how to fix it :(

Regards

- miSt

allright tnx , i added that also ...
i am thinkin of not editin robots.txt ... i think that wont cause anything only no permission pages ;) maybe bots pisses of and goes away ;)

anyway tnx for tips ... i will wait for next check/rush of bots ... i hope they will ;)

thanks for this great hack ...
greetz

Got a lot of new guests from the archive, but not many are registering? Read this:

https://vborg.vbsupport.ru/showthrea...threadid=49721

I attempted this install. when I go to my archive it points my forums to .../forumdisplay/f-1.html, etc. and I get the "forum not specified" error.

My question... what process creates these .html files? Should my install have created them? or are these created dynamically?

Great Hack!

Just installed it!

I was banging my head for about 2 hours trying to get it to work, Then I noticed that I missed the period infront of "htaccess", then bingo, everything is working great!

Thanks
:)

in step 3 of the installation, when you are to rename the files... are we supposed to rename them without any file extension?? Ie. forumdisplay rather than forumdisplay.txt or do they need the .php extension?

I am having problems getting this install to work.. my forums arent linked to anything... ie f-1.html is not found

-James

How long does it usually take before the bots start invading?

I think I need to install this hack with some modifications ... Thanks teck.

No one is crawling on my page :(

How long does it usually take?

You need to submit your archive to google et all.

Bump!!!

:)

There is no set time. It took google about a month to crawl mine and another month for them to actually show up on google. Others have had theirs crawled days after they installed this hack. Just be patient.

>>You need to submit your archive to google et all.

NO! Don't ever submit anything to Google. It will rank you higher if it finds you on it's own. Make sure you get other sites to link to yours and let the spider come naturally. To have your archive is listed, make sure it is linked FROM YOUR HOME PAGE. Even a small link is fine.

Thanks, Teck.. Dance, Google, Dance! I'm getting listed..

that's good to know. thanks for the tip!

I have the GoogleDeepCrawler on our forum right now. I'm wondering if I should add forumdisplay.php and showthread.php to the robots.txt as I don't know what Google will think if it spiders the forum and archive, finding the same content. Could drop one or the other. What has everyone else done with successful listings in Google?

Just to let you know that Google's Freshie and DeepCrawler come in on different IPs:

So you're saying DeepCrawlers identify with "crawl#" and FreshCrawlers idenfity with "crawler#"? Thanks, good info.

>>I'm wondering if I should add forumdisplay.php and showthread.php to the robots.txt as I don't know what Google will think if it spiders the forum and archive, finding the same content.

I don't think so.. while they contain some of the same content, they are in different formats.. which should equal different content to the spider.

I see you had success and have your archive set up like mine. Did google's bots hit your forum and archive? Do you have showthread.php listed in your robots.txt?

WOW, PR7! What was your page rank before you messed with the archive? Due to our server move and new site, our dropped to 5 from 6 - not happy. :(

Looks like you're getting slammed:

Currently Active Users (977):450 members, 489 guests, and 38 web robots

I take it you have your own server. :D

/me blushes at mention of his 7" PR

Google has just started digesting the archive. I started getting hits in from it a day or two ago so I'm not sure it has affected the PageRank yet. I'm not even sure the archive has been listed fully in Google, we may be in for the short dance right now, to be re-added later.

>>Did google's bots hit your forum and archive? Do you have showthread.php listed in your robots.txt?

Google has always gotten some of the posts since they are linked from the front page, then it finds the forums. It never got into the deepcrawl of the archive though. Funny, as I think I was the first person to release a vB archive hack that lots of people still use successfully... but I never linked mine from my front page so it was never picked up. Only when I saw Teck's was I inspired to get it working and linked.

But oh yea, no, I don't have anything special listed in robots.txt

>>38 web robots, I take it you have your own server.

hehe those robots have been here for weeks. Only in the past few days have I had some really high guest numbers. Usually members / guests is even. Now I have a hundred or more guests on than members thanks to the archive. Hopefully they are getting into the rest of the site as well.

wow Overgrow, this must produce a huuuuge portion of traffic :bunny:

How do you get your link on othere websites?

Well I suppose you could hack into the other websites and put your link up.. or you can email the administrators and ask if they do link exchanges?

Had this bot on my forum for days:

argon.oxeo.com

Have no idea what it is. Does anyone know and should I deny it access?

Sorry, just caught this.

Google's Freshbot comes in on IP range 64.68.82 and the deep crawler that goes out in Google dance time comes in on 216.239.46.

OK there is a point where the archive can work too well. I was getting hundreds of off-topic hits from Google searchers thanks to the spidering of my off-topic chat areas. The last thing I want is to expand the off-topic areas and get more general chatters in, so I added disallows to my robots.txt to the chatting archives and I made the actual archive threads offlimits in showthread.txt.. One question:

Will Google DE-list my pages once they are disallowed there?


I'm getting great results for things like:

Christina Agulera Naked
and
Internet Security Procedures

My hat is off to you TECH!

5600 Pages (threads) indexed by Yahoo and Google.

You da man!

Question...I have to delete threads once in a while that are bad content. When someone lands on a deleted thread from the search hack, they get the page with no images, etc..

Is there a re-direct we could use that sends them somewhere else to fix that?

Thanks again!

I don't know if this add-on was already provided in the thread or not, my apologies if it did:

This code keeps updating the original thread's view count even if your archive thread is visited:

Edit "showthread" (hack file! not showthread.php!), find:
Before that add:

Enjoy..

Hmmm... Google results for vBulletin.org Archive:
Results 1 - 10 of about 10,600. Search took 0.04 seconds

Google results for TeckWizards.com Archive:
Results 1 - 10 of about 727. Search took 0.06 seconds
(I don't allow the robots to visit any .php forum file and all archive files are still indexed. I have my archive set outside the forum folder.)

In your .htaccess file, add this line:
[high]ErrorDocument 404 http://www.yoursite.com/[/high]