vbSpiderFriend - Search Engine Friendliness

[Learner29]

> You need to put an .htaccess file like this in the /forums/archive folder:
> ErrorDocument 404 /forums/archive/index.php
> Change forums to the correct folder name for you board directory. This is
all that's required along with the index.php and the class.mysql.php
files.

thank you so much for trying to help.
this .htaccess file in the "archive" subfolder of the "vb" folder is
something I triple checked.
The file is there, and it is working, and this is why when clicking on any
of those broken links generated on the /vb/archive/index.php page, the page
itself is reloaded, for it is itself the 404 page.

[patvdv]

Originally posted by mvigod
Has anyone ever witnessed Google going more than one level deep? Since they get the 404 respose code with this hack they don't index those pages or go any deeper. They are not in the habit of indexing "Not Found 404 error" pages or following the links in them. They will follow the first set of links from the index.php but each of those really doesn't exist so the 404 error is the end of the line even though the pages have content.

I think this has to be rewritten with mod rewrite to rewrite the requests for all pages back to index.php so a 200 OK response code is given and it will be archived.

With all these posts didn't anybody realize this?

mvigod, I think you are right about this. Alot of webrippers will also trip over this. I know WebWhacker does handle the 404. Does anybody can give some input as to make this work with mod rewrite?

[Learner29]

Originally posted by patvdv, quoting above
...
I think this has to be rewritten with mod rewrite to rewrite the requests for all pages back to index.php so a 200 OK response code is given and it will be archived.

Patvdv and everyone else,

see what overgrow mentioned in the second post of this thread, about updating this..... look at the last of the fixes he mentioned.


Originally posted by Overgrow
...

FIXES:

v1.1b release

-Added header("Status: 200 OK"); for the really picky engines

is not that the 200OK response you are talking about ???


__________________________

I am trying to help here...... waiting for some courageous person to step forward and help me and others having problems to make this work.

I described my problem above, please read and try to help.

[Destee]

Oh My Gosh Overgrow !!!!! I am using this !!!!!

I have known about this thread for a very long time but never tried it because I thought I'd not be able to do it !!! Just thought that ... no real reason why ... but anyway ... :) ... I've just installed it ... PAINLESS ... BEAUTIFUL ... THANK YOU !!!!

http://destee.com/forums/archive

Whew ... Wow ... I missed out for a long time ... for no reason!

Thanks Again.

Destee

[Learner29]

well, if you have been that successful, would you kindly help your fellows a
bit ???

Please read me above, I have described my problem in detail.

[Destee]

Hi Learner29 ... it would be my pleasure to help you ... unfortunately, I don't have any tips to provide. I followed Overgrow's instructions step by step and it worked for me. Well ... as I think about it ... I did chmod 777 .htaccess ... which was not in his instructions. Perhaps this will help? Let us know.

Destee

[Learner29]

Detsee, that is so kind of you. I really appreciate your help.

well,

I hope overgrow would have a minute to help here.

mode of `.htaccess' changed to 0777 (rwxrwxrwx)

and

mode of `archive' changed to 0777 (rwxrwxrwx)

and

mode of `class.mysql.php' changed to 0777 (rwxrwxrwx)
mode of `index.php' changed to 0777 (rwxrwxrwx)

and still......... I have only index.php and no one single folder there...

I am getting pessimistic about this hack... frankly, I do not have the impression many people do really care to help.

[Destee]

Hi Learner ... sorry it's not working for you. If it is any consolation, vB3 will have something similiar to this hack included in it. So you won't be without this ability for long.

Don't be too discouraged, I happen to know that this original hack predates vb.org (which means he's probably given a lot of support since its creation), and I don't know if you've visited Overgrow's site before ... but it is humongous ... which I'm sure keeps him pretty busy ... even if he has nothing else to do, which I doubt.

Stay encouraged ... this really is a great community with lots of folk willing to help.

Destee

[Learner29]

I thank you a million my friend for your very kind answer.
I just wish anybody can help me get this to work.

[grooveh]

I am wondering if anyone can help me with this, please? I have seemingly installed everything as per directions, but still have a few errors, and cant figure out why?

Link to Archive.php (http://select.curlingtour.com/forums/archive/index.php)

Is it me? Please let me know. I'm thoroughly stumped.

TIA

[grooveh]

bump. :rambo:

[Shepski]

I have installed this hack. its seems to run but all i get is the forums displayed. if i click on any link it does not display the threads underneath.

http://www.dvd-swaps.co.uk/dvd2001/dvdforums/archive/

index.php code as follows (its on an nt server)

$privateForums="|17|12|"; // Hidden forumids, enclosed by | pipes
$firstPost="09/12/2001"; // MM/DD/YYYY of your forum's first post
$spacer="  "; // The characters or spaces to use as one indent
$forumURL="/dvd2001/dvdforums"; // Base URL of your forum
$refresh=0; // Change this to 1 if you want the archive to automatically
// forward the user if they come in from a search engine.
// This option is initially turned off because Google considers
// this to be 'cloaking.'


Is it crucial that i have my first post date correct? how do i find it? i have upgraded recently from xmb and things are a bit cocked up.

[Shepski]

double post!!

[Shepski]

anyone help?

[grooveh]

Shepski: It seems that overgrow and anyone that knows about this hack arent around right now. I posted a question too, and still havent heard from anyone. ITs too bad. It is a good and useful hack. I'd really like to be able to use it.

[Shepski]

Originally posted by JohnL
THANK YOU...THANK YOU...THANK YOU :)

OK all you IIS guys. Thanks to Razzie...

Do a search for REQUEST_URI and replace it with QUERY_STRING and you are good to go!



Thanks again Razzie :)

Forget my orginal question...the above sorted my problem out :) thanks to all involved

[Learner29]

Originally posted by Learner29 who is feeling so terrible about nobody caring to help him a bit...
oh. I am sorry to bother but I really tried to find by myself but could not !!

I installed the hack and the index.php page of the /vb/archive directory is displaying nicely.

Yet all the links on this index.php page point toward non-existent folders!!!

example

http://www.mydomain.com/vb/archive/29
and
http://www.mydomain.com/vb/archive/13

but checking that archive folder by FTP, there are NO Folders called 13 or 29 ....

I am so frustrated as I DID install this hack in the past and it worked nicely then.....

Whoever would help, I would be more than grateful and thankful.

[FFMania]

Hmm...edit the path variables inside index.php :P
It works great for me....:D *clicks install*

[Learner29]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Hmm...edit the path variables inside index.php :P
It works great for me....:D *clicks install*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Thank you ; I will let you know if this works.

[Ninth Dimension]

Installed and working fine on 2.2.6 :)

[Learner29]

Ninth Dimension

Installed and working fine on 2.2.6 :)

would you kindly help then ?? if it works for you, it should as well work
for others, right ??

Please Ninth Dimension take a while to help

[grooveh]

Oh, I'm so glad I finally got this going on my board. It works great! Thanks overgrow for your very useful hack!

Cheers all! :laugh:

[SkuZZy]

lol. I uploaded the files and had this hack working in under 5 minutes. I don't know how it could have been any easier!!! Awesome hack... can't wait till google's next update so I can see all my threads spidered :)

[Learner29]

maaaan, I just can't get it why it is not working for me.

Would you kindly help a bit skuzzy ???

[SkuZZy]

Originally posted by Learner29
maaaan, I just can't get it why it is not working for me.

Would you kindly help a bit skuzzy ???

yeah i'll help you as much as I can... what do you need help with? You're running apache? What version? This hack is the easiest one i've installed in all of vb.org... so it's probably just something small that you're missing... or else it's not compatible with your server for some reason. if you'd like me to send you all 3 files, the way I have them customized (so you can see a working modified version of it), e-mail me at support@battleforums.com and i'll send them to you in a .zip file, so you can see how I have them. BTW, my archives can be seen at http://www.battleforums.com/archive

- SkuZZy

[Learner29]

thank you a million. I will write you now.

I looked at your archives..... amasing....

[Learner29]

i tried this hack once more...... it does not want to work.

I am waiting for your help friend Skuzzy ...

[SkuZZy]

e-mailed you back, as well as the other 4 people who e-mailed me :D Now I know why overgrow stopped offering support heheh.

[pool-werbung]

It works very good but it does not have an effect. Google machine had spiderd my site with this new archive. The archive site is in the google-index (archive/index.php) but all of the rest are not in the google-index.

Can yot think why?

[Prince]

Google does not feel that the pages in the archive are worthy of being indexed. That simple.

[Learner29]

Originally posted by Prince
Google does not feel that the pages in the archive are worthy of being indexed. That simple.

I hope you are wrong... and I am afraid you might be right ...

[Learner29]

Originally posted by Prince
Google does not feel that the pages in the archive are worthy of being indexed. That simple.


Hi,

I searched on google for www.mydomain.com/vb/archive/

and I got this answer.....

Sorry, no information is available for the URL www.mydomain.com/vb/archive/

If the URL is valid, try visiting that web page by clicking on the following link: www.mydomain.com/vb/archive/
Find web pages that contain the term "www.mydomain.com/vb/archive/"

I tried clicking on that link that says : Find web pages that contain the term "www.mydomain.com/vb/archive/"

and it gave me this :

Your search - "+www.mydomain.com/vb/archive/" - did not match any documents.

Suggestions:
Make sure all words are spelled correctly.
Try different keywords.
Try more general keywords.


that means that my archive url has not even been indexed or crawled by google .... right ???

that means my submission to google is still in the queue and has not been crawled yet, right ???

[Prince]

it means that your archive/index.html page is not in the Google index, it does not mean it has not been crawled though.

Google crawls sites and then determines which pages are worthy of being included in it's index.

this site can be of more help to you >>
http://www.ihelpyouservices.com/forums/

[Learner29]

Originally posted by Prince
it means that your archive/index.html page is not in the Google index, it does not mean it has not been crawled though.

Google crawls sites and then determines which pages are worthy of being included in it's index.

this site can be of more help to you >>
http://www.ihelpyouservices.com/forums/

thank you very much


and how do we know what guidelines Google uses to decide whether to include or not to include any pages in its index......

[SkuZZy]

Unfortunately, this script doesn't work at all. I've given up on it now. After watching google come to my site time after time and sit at the index.php file and never go any deeper, I finally realized why. This script uses 404 errors to redirect. Google loads the archive page and thinks it's seeing a custom 404 page, so it doesn't spider any deeper (for obvious reasons). So i've removed this hack completely.

I've now installed a new hack (originally written by xenon who is awesome!!!) and you guys can view it at http://www.battleforums.com/history (note: mine has been modified a bit... you can view an unmodified copy at http://www.noxmedia.net/support/archive )

I'd recommend those of you who want to get spidered, give up on this hack and install xenon's. Google hasn't spidered it yet, but i'm confident they will. The reason xenon's hack works much better is because it doesn't use .htaccess (mine uses htaccess, but the original one @ noxmedia DOESNT), error redirects or even mod_rewrite. Xenon hasn't released it officially though (I don't think) and I don't think he wants to and I know for sure he isn't offering tech support on it... so the scripts are pretty much untested and in beta, but for those of you who want it, you can PM xenon and ask nicely.

It's too bad it took me two months to figure out that overgrow's hack didn't work. For those reading this, hopefully it will save you the time wasted of installing this script and you can go straight to another hack. There are several others out there as well that do work. If you don't want to install an archive, then try FastForward's hack or Filburts recentely released hack (@ vbulletin.com).

SKuZZy

[Learner29]

Originally posted by SkuZZy
Unfortunately, this script doesn't work at all. I've given up on it now. After watching google come to my site time after time and sit at the index.php file and never go any deeper, I finally realized why. This script uses 404 errors to redirect. Google loads the archive page and thinks it's seeing a custom 404 page, so it doesn't spider any deeper (for obvious reasons). So i've removed this hack completely.

It's too bad it took me two months to figure out that overgrow's hack didn't work.

that is too bad too sad too disappointing.... :confused: :bored: :dead:

[Thomas P]

Originally posted by SkuZZy
It's too bad it took me two months to figure out that overgrow's hack didn't work. For those reading this, hopefully it will save you the time wasted of installing this script and you can go straight to another hack. There are several others out there as well that do work. If you don't want to install an archive, then try FastForward's hack or Filburts recentely released hack (@ vbulletin.com).

SKuZZy


Sorry that it didn't work out for you, but the hack does work, and it helped me to get my site indexed...

Just a sample: http://www.google.de/search?q=m%C3%B6glich+die+NTUSER.+DAT+von+einem&ie=UTF-8&oe=UTF-8&hl=de&meta= (random pick)

Xenon's hack does look good indeed, very sad, he doesn't want to release it.
cu,
-Tom

[Sergio68]

What can be happened to my installation please?

http://www.bodyweb.it/vbullettin/archive/

[Sergio68]

And what's happened to directory /archive/1 .../2 ...3/ ?

Doesn't exist ?!?

[Sergio68]

UP

[Sergio68]

Please :ermm:

[SloppyGoat]

This won't make hidden forums be spidered too, will it? NM, it doesn't sound like it will work, from what I read above. :dead:

[grooveh]

Sergio: Are you running this on an NT box? I had the exact same thing happen to me on my forums, until I switched hosting to a UNIX box. As soon as I did that, my vBSpiderFriend worked like a charm.

I know thats not specific help with your problem...but I think the problem lies moreover with your host, and not the script.

good luck

[Sergio68]

Yes, I'm on windows 2000 server sistems and I'm hosted by myself ;)

About the errors, I mean, there is something wrong with the variables?

And why I can't see directory /archive/1 .../2 ...3/ ?

[grooveh]

its all tied around the way the code works on your windoze system. I think specifically with how the win system deals with the .htaccess

I had the exact same problem as you...to the letter. It was fixed only when I switched to a unix based system.

[AWS]

Are you using Apache with w2k?
If you are I can help you get it to work. I have it working on my forum.

[Prince]

for anyone that cares, this hack does not work, meaning Google (the only search spider that means anything) will not spider your threads with this hack installed.

I have had it installed for 2 months now, and Google did not pick up any more of my vB than it had before.

[pekay]

Originally posted by Prince
for anyone that cares, this hack does not work, meaning Google (the only search spider that means anything) will not spider your threads with this hack installed.

I have had it installed for 2 months now, and Google did not pick up any more of my vB than it had before.


You sure? :(

Damn..

[Sergio68]

Originally posted by AWS
Are you using Apache with w2k?
If you are I can help you get it to work. I have it working on my forum.


No, IIS5 ...

[Sergio68]

<a href="http://www.bodyweb.it/vbullettin/archive/" target="_blank">http://www.bodyweb.it/vbullettin/archive/</a>

Now seems Ok, but where are the directory /2 /13 /7 /.....

[Sergio68]

And why :


View an online example:

http://www.overgrow.com/edge/archive

Then :

Parse error: parse error, expecting `T_OLD_FUNCTION' or `T_FUNCTION' or `T_VAR' or `'}'' in /home/overgrow/HTML/globals/class.mysql.php on line 1

Fatal error: Cannot instantiate non-existent class: mysql in /home/overgrow/HTML/edge/archive/index.php on line 78

:( :(

[Sergio68]

I got it....

IIS5 doesn't have the .htacces file , you need to go inside the Site "Properties" and then "Custom Errors" and finally Edit the 404 Error File...

But Still not work at all, look : http://www.bodyweb.it/vbullettin/archive/


Help, please... ;)

[Sergio68]

Now I got the redirection, but the script (index.php) doesn't detect the forwarding.


http://www.bodyweb.it/vbullettin/archive/9


is the same of :


http://www.bodyweb.it/vbullettin/archive








Why ?

[Prince]

Originally posted by pekay



You sure? :(

Damn..

Positive, and now that I think about it I installed this hack 3 months ago.

[Sergio68]

Finally goes ! :D

[Sergio68]

Originally posted by Prince


Positive, and now that I think about it I installed this hack 3 months ago.

Now submit your site and than wait ....

http://128.121.225.20/submit/index.html

[Sergio68]

Search REQUEST_URI and replace it with QUERY_STRING in index.php if you are running IIS5 .

[tpearl5]

Originally posted by Prince
for anyone that cares, this hack does not work, meaning Google (the only search spider that means anything) will not spider your threads with this hack installed.

I have had it installed for 2 months now, and Google did not pick up any more of my vB than it had before.

This is my old URL.. but google did index it

:surprised:
http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=+site%3Adegster.com+midi+overgrow

[Prince]

hmmm....glad to see that, wonder why it has not indexed mine?

Google visits my site often too.

[dwh]

You need a link from your main page to the archive...just submitting it to google isn't enough..google needs to see links to that page or it won't like it...

[SoloX]

Is there any way to feed the archive URL to something like website downloaders to create a full text archive of the VBB? This way, the search engines wont see the nasty 404s. I tried using some win based site downloaders and all stopped at the first page of the archive index (due to 404's).

[SloppyGoat]

I don't think I'll try this one. "http://www.overgrow.com/edge/archive" gives me this....

Parse error: parse error, expecting `T_OLD_FUNCTION' or `T_FUNCTION' or `T_VAR' or `'}'' in /home/overgrow/HTML/globals/class.mysql.php on line 1

Fatal error: Cannot instantiate non-existent class: mysql in /home/overgrow/HTML/edge/archive/index.php on line 78

[lalo landa]

there 's an option to make work the vbcodes?

REally cool hack Thnkxs!!

[floridaideal]

Hi all

Thanks for this great hack, I installed it about 3 months ago but so far its not been spidered.

I had the link added to my index page www.top-forums.com and google has now picked the link up and has listed it as http://www.top-forums.com/forum/archive/index.php which is correct.

Its all working ok as far as I can see so why isnt it spidering the whole archives?

Someone please help.

Thanks

Stu

[Prince]

Originally posted by floridaideal
Hi all

Thanks for this great hack, I installed it about 3 months ago but so far its not been spidered.

I had the link added to my index page www.top-forums.com and google has now picked the link up and has listed it as http://www.top-forums.com/forum/archive/index.php which is correct.

Its all working ok as far as I can see so why isnt it spidering the whole archives?

Someone please help.

Thanks

Stu

Google. will not spider mine either, I have had it installed for around 4 months.

Apparently my page rank on my forums is not high enough.

[floridaideal]

Can anyone please help Ironman and me ?

Thanks

[ryancooper]

I get a "The page cannot be found" error when I click on the forum links? Any idea why??

Thanks!!

[aquariumpro]

Hi:

Since switching hosts ( a much better host!) and upgrading to VBulletin 2.30, vbspiderfriend is still working and we have thousands of links in Google and other SE. Thanks Overgrow!

However, a new development has started, which does not seem to affect the actual function of the script, but is generating hundreds of Kb a day in our server error log that look like:

[Wed Jun 25 09:00:06 2003] [error] [client 211.28.92.157] File does not exist: /home/aquariumpros/websites/www.aquariumpros.com/html/forums/archive/19/2003/01/2/785


These errors are being generated every time a user posts in our forum, and with about 100 posts a day or more, the error log reaches over 20,000 Kb a week!


What's up? and do you think it's possible to fix this?

URL: http://aquariumpros.com/forums/archive/

The script is working fine. It picks up all the new posts and can be easily browsed. Other than those error log entries, it works perfectly! Problem is that that error log bloating is running me ouyt of disk space and it must be emptied every other day.

[KevinM]

I have installed this hack on my test forum, and it works great on 15 out of 20 of my forums.

Of the ones that don't work, all the forum id's are single digit e.g.
http://www.ukgardeningchat.co.uk/archive/5 where the forum id is 5.

However, if I manually type in 05 at the end of the url instead, it works great. eg.

http://www.ukgardeningchat.co.uk/archive/05

but it does not do this automatically. All the forums from 10 upwards are fine. Any suggestions on how to make it work or force the url to read 05?

[michaelab]

05-21-01 at 05:29 PM Overgrow said this in Post #83 (https://vborg.vbsupport.ru/showthread.php?postid=109736#post109736)
Streicher: Sorry :( I have no idea on your problem-- you are the only one who has reported this and I cannot duplicate it.

Well - I am getting exactly the same problem! To re-cap, the problem is that for forums with an ID < 10 clicking on the forum link just reloads the top level arcihive list, and doesn't go into the forum itself.

If you manually add a 0 to the forum ID in the URL (eg .../archive/06) then it loads find and what's more, all subsequent links off the forum also work fine.

I couldn't find the exact problem but for some reason the code that splits up the URL:

if (ereg("/archive/[0-9]",getenv('REQUEST_URI'))) {

$urlArray=explode("/",getenv('REQUEST_URI')); //split the URL path
$a=0;

while($urlArray[$a] != "archive") {
$a++;
}

$forumID=$urlArray[($a+1)];
$year=$urlArray[($a+2)];
$month=$urlArray[($a+3)];
$week=$urlArray[($a+4)];
$threadID=$urlArray[($a+5)];

if (eregi("[a-z]",$forumID) or $forumID < 1 or strstr($privateForums,"|".$forumID."|"))
$forumID=1;
}


...is parsing the forum ID as nothing (empty string) for forum IDs < 10. I couldn't be bothered to work out why that was so I fixed the problem using a workaround:

In the forumList() function I changed:


if(!empty($data)) {

while ( list ( $key,$forum ) = each ($data)) {
echo spacer(1)."<a href=\"$baseURL/$forum[forumid]\">$forum[title]</a><br>";
}
}


to...


if(!empty($data)) {

while ( list ( $key,$forum ) = each ($data)) {
$forumNo = $forum[forumid];
if ($forumNo < 10)
$forumNo = "0" . $forumNo;

echo spacer(1)."<a href=\"$baseURL/$forumNo\">$forum[title]</a><br>";
}
}


That way, the 0 is prepended to the forum ID in the output and it all works fine.

I know this thread is pretty old but hopefully it will help some of you out there!

BTW, what's the status on this Hack working with Google or not re. the 404 issue? I have to use this hack rather than any of the others because I can't seem to get the mod_rewrite or ForceType directives to work using Skuzzy or Teck's hacks.

Michael.

[michaelab]

07-10-03 at 02:45 PM KevinM said this in Post #319 (https://vborg.vbsupport.ru/showthread.php?postid=416698#post416698)
I have installed this hack on my test forum, and it works great on 15 out of 20 of my forums.

Of the ones that don't work, all the forum id's are single digit e.g.
http://www.ukgardeningchat.co.uk/archive/5 where the forum id is 5.

However, if I manually type in 05 at the end of the url instead, it works great. eg.

http://www.ukgardeningchat.co.uk/archive/05

but it does not do this automatically. All the forums from 10 upwards are fine. Any suggestions on how to make it work or force the url to read 05?

Kevin - see my post above. I think my fix will solve your problem also (looks like the same problem).

BTW, on the 404 issue and Google not liking it, I just checked my archive URLs using a little bit of Java I just knocked up and the response codes are all 200 OK so shouldn't be an issue.

Michael.

[Till]

I was wondering, what exactly is the use of those pages that list the months and week 1 through week 4 for each one? Or is that just your "design"?

[smestas]

Im getting the following when clicking on any of the archive links.

"No posts, please go back."

I have the newest version (v1.1b) which I just d-loaded. If anyone can lead me in the right direction I'd appreciate it.

Link to my archive pages:
http://www.alfabb.com/bb/forums/archive/index.php

[Till]

Im getting the following when clicking on any of the archive links.

"No posts, please go back."

I have the newest version (v1.1b) which I just d-loaded. If anyone can lead me in the right direction I'd appreciate it.

Link to my archive pages:
http://www.alfabb.com/bb/forums/archive/index.php

Could be multiple reasons.

1) no posts ;)
2) certain requirements not fulfilled

If I remember correctly, there's some sort of debug mode, maybe turn it on and see what it says.

In any case, no offense to the author, I would not recommend running this script. It never worked for me (although posts showed up, none were indexed due to a bug that's been fixed now though (see: status code)) and there's a ton of scripts on vB.org that do the same or maybe a better job. (I wrote one myself to get the job done.)

[smestas]

Till,

Thanks for the reply. My main problem is that I just moved to a new host and they do not support the mod_mime/ForceType directive. I previously was running (old server) a archive hack called "easy archive" and it worked great. I currently have around 4,000 pages getting indexed and I need to get somthing back up there for the bots to chew on.

I guess im going back to the drawing board. Any suggestions?

[Till]

Till,

Thanks for the reply. My main problem is that I just moved to a new host and they do not support the mod_mime/ForceType directive. I previously was running (old server) a archive hack called "easy archive" and it worked great. I currently have around 4,000 pages getting indexed and I need to get somthing back up there for the bots to chew on.

I guess im going back to the drawing board. Any suggestions?

I am not familiar with easy archive, but why not rename the file to whatever.html and use that? Instead of "whatever". Then search and replace the source code and append the extension. Would be the easiest work around in order to not use forcetype.

I would also add a rewrite rule, so everything from /whatever gets rewritten to /whatever.html. You won't lose search engine traffic this way.

Then again, I don't see why they would not enable it for you. Especially if they want your business. Just remind them that there are 3 million companies out there offering webhosting. :)

[MrNase]

Hey :)

Is someone using that hack for 2.3.2 ?? Does it work?
I don't want to mess my forums ;)

[Skyline_GT]

Can anyone help me ??

http://www.gamerzonline.net/forum/archive/

[michaelab]

BTW, on the 404 issue and Google not liking it, I just checked my archive URLs using a little bit of Java I just knocked up and the response codes are all 200 OK so shouldn't be an issue.

Confirmed! :banana: My vbSpiderFriend archive pages are showing up on Google. Took their time but they're definitely there now so there is no 404 issue. YES!

Michael.

[michaelab]

Hey :)

Is someone using that hack for 2.3.2 ?? Does it work?
I don't want to mess my forums ;)

Yes - works fine with 2.3.2

Michael.

[michaelab]

In any case, no offense to the author, I would not recommend running this script. It never worked for me (although posts showed up, none were indexed due to a bug that's been fixed now though (see: status code)) and there's a ton of scripts on vB.org that do the same or maybe a better job. (I wrote one myself to get the job done.)
That's a bit harsh. This hack works fine and, unlike some similar ones, requires virtually no messing with original vB scripts. It also doesn't require the mod_mime/ForceType directives which a surprising number of hosts (including mine) don't support.

Michael.

[anonscape]

Hi,
Great hack. Very nice feature of the dynamically geterated subject and meta tag lines. I have this installed and everything seems to be working fine. Google has been by twice, gone to the archive and has gone to the depth of /archive/4 /archive 23 etc. for all of the archives. On both visits it has not proceeded deeper to index the months weeks and posts. Is this because it takes some time for googlebot to decide to move deeper in a future spidering of the site or is this because googlebot is seeing a 404 error and then not indexing further?
TIA for any info.

[michaelab]

It isn't a 404 problem, that was solved a long time ago by the script forcing the status code to 200 (OK). It took a little while for Google to fully index my forum but it now has done and gone all the way to all the posts. Very, very effective hack.

Michael.

[anonscape]

Thanks Michael for such quick feedback to my question. I will be patient and let google do its thing. :)

I'm REALLY glad to know that Overgrow's script will work because it is my ONLY option. Like you, my server does not allow force type directives. That is because my hosting enviornment has phpsuexe installed. Apparently this is useful for improving the security of the server but it has a horrible consequence for php scripts. When phpsuexe is installed all .htaccess php calls to apache are void therefore force type etc. lines in the .htaccess are useless. This is because php is no longer running as an apache module and apache will not handle those directives any longer. Apparently most server enviornments and hosts are going to install phpsuexe and that will render all hacks like xenon's, scuzzy's indexers, which rely on force type mime mod, features void.
Quote from "PHPSUEXEC - What do I need to know?" - "All servers will be running phpsuexec within the near future."

Overgrow's hack will be the only SE Friendly option for the majority of vBs soon.

[michaelab]

Ah - I don't know anything about phpsuexec but if it doesn't handle .htaccess directives then I'm not sure how the script works at all because allthough it doesn't require "force type" it does require the:

ErrorDocument 404 /forum/archive/index.php

(alter according to forum directory) line. If however you can hit your archive link and follow the links through in your browser then it must be working.

The ways that Google works are weird and wonderful and I don't think anyone who doesn't work there really understands them. One thing you'll need to do of course in case you haven't already is have a link to your archive directory somewhere in your forum (I hacked the footer so it appears on every page next to the "Contact Us" link). Otherwise, Google will have nothing to start with.

Michael.

[anonscape]

phpsuexec does block force type commands but allows for the .htaccess file to issue the error document line.
I put a link to the archive at the footer as well.

I checked out your forum at zerogain.com ... very nice and also noticed that it is number 1 for a google search for "vbSpiderFriend by ~shabang~ of Overgrow"

Therefore the google spider is really happy with what it finds at your archives - congrats and thanks for your help.

[vbmechanic]

That's a bit harsh. This hack works fine and, unlike some similar ones, requires virtually no messing with original vB scripts. It also doesn't require the mod_mime/ForceType directives which a surprising number of hosts (including mine) don't support.

Michael.
It was also the first spider-friendly archive for vBulletin and probably the inspiration for all the following scripts and the archive now included with vB3. Overgrow also posted a good article somewhere around here about making your archive more inviting with H1 tags, other optimizations, and more links into your site.

[michaelab]

I checked out your forum at zerogain.com ... very nice and also noticed that it is number 1 for a google search for "vbSpiderFriend by ~shabang~ of Overgrow"

Therefore the google spider is really happy with what it finds at your archives - congrats and thanks for your help.
Whoa! :surprised: I didn't expect that. I guess most people are using other spider hacks these days.

btw, useful tip for finding all Google records for your site is to use the "allinurl:" command ie, to search for all Google hits of my site I'd search for:

"allinurl:www.zerogain.com"

Then, click on the "repeat the search with the omitted results included" link to expand out everything and hey presto, all your hits :)

Michael.

[Thomas P]

Hello,

we are using the original hack from overgrow since it was introduced in 2001 and it does a hell of a good job :)
Best hack ever for vB2 imho.

I have a question regarding compatibility with vB3.

Given we upgrade from vB2 to vB3, what happens to all the archive links out there?
Will they still work?
What's the best strategy here? Put an 404.php online which redirects to the "new" archive? Would this be considered as "cloaking" by google?

I tried overgrow's hack with vB3, which can run parallel to the new archiv for compatibility reasons - seems to work.

Can anyone confirm or has someone a better idea?

Thanks,
-Tom