vB Easy Archive FINAL - Search Engine Spiderable Hack! Get your posts listed @ google

[Rand M]

Thanks for a great hack SkuZZy... installed and running :)

http://www.toyotaimportsforum.co.uk/forum/archive/

One more for your list!

[FlyBoy73]

SkuZZy,
Thank you so much for a great hack and all your personal help installing it. I can't wait for the dance..
Please feel free to add me to the list also if you would like.
Thanks,
David

[SkuZZy]

Originally posted by Ember
You can change http://a51gaming.madsims.net/forum/archive

to

www.a51forums.com/archive

I am just uploading it now :) Thought I should notify you of the change!

Congrats on the domain name!!! I updated your URL. I also added 3 other new URL's to the list ;) If anyone has an archive not on the list and wants to be added, PM me. Never can have too many demos ;)

[Guidster]

Originally posted by SkuZZy


Congrats on the domain name!!! I updated your URL. I also added 3 other new URL's to the list ;) If anyone has an archive not on the list and wants to be added, PM me. Never can have too many demos ;)

You can add mine if you like:

www.allthingsmoto.com/archive

[SkuZZy]

Originally posted by Guidster


You can add mine if you like:

www.allthingsmoto.com/archive

Added sir :D

[smestas]

SkuZZy,

I was having problems with v2 and a blank page but I just installed the new script v2.3 and it works great.

Thanks!

[SkuZZy]

Google is dancing! For those of you who installed this hack in early to mid-janary, check to see if google spidered your pages. There is a simple way to check:

1. Go to http://www2.google.com (notice the www2).

2. Type in: site:www.yoursite.com yoursite. (So for instance, I would type in "site:www.battleforums.com battleforums")

3. Check to see how many pages are listed. I notice some sites have alot of pages listed. Thrillnetwork has over 8000 pages indexed for example (not all of them from the archive).

BE SURE TO POST YOU RESULTS! :D

[SkuZZy]

Some interesting statistcs, for those who doubt the power of this hack. Over 29,000 archive pages have been spidered in the first month alone! Search for Archive SkuZZy Xenon in google and see the results!

http://www3.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=Archive+SkuZZy+Xenon

[floridaideal]

Hi Skuzzy, I just added my link to the index on the site and also submitted it to Google.

Do you think I will get a chance of spidering? or is it too late?

By the way do you think my link is ok on my index page http://www.top-forums.com (its right at the bottom of the page)

Also the link to the http://www.top-forums.com/forum/archives/

Does that look ok to you guys? Thanks all this is a great hack, I could not get vbspider to spider on google so I hope this one works better for me.

Thanks

Stuart

[Guidster]

Originally posted by SkuZZy
The dance is usually at the end of the month, often between the 25th and 30th and the "major crawl" happens 5 days after the dance BEGINS (approximately). So if the dance started on January 28th, the major crawl would probaly begin around the 3rd.

I saw the first deep crawl last evening into my archive. It would appear that they have been dancing all week and have begun the deep spidering.

[Guidster]

Originally posted by SkuZZy
Google is dancing! For those of you who installed this hack in early to mid-janary, check to see if google spidered your pages. There is a simple way to check:

1. Go to http://www2.google.com (notice the www2).

2. Type in: site:www.yoursite.com yoursite. (So for instance, I would type in "site:www.battleforums.com battleforums")

3. Check to see how many pages are listed. I notice some sites have alot of pages listed. Thrillnetwork has over 8000 pages indexed for example (not all of them from the archive).

BE SURE TO POST YOU RESULTS! :D

My site was only a couple of weeks old when I added this hack and got in before the last dance. Still, this hack has allowed Google to index over 300 pages of archive content and is currently deep crawling it again this month. This is great and is really going to help get my fledgling board off to the races! Thanks, Skuzzy!

Hey, Skuzzy--What is the sequence for the data at www2 to be added to the www index? Cudos for figuring all of this stuff out! I have also heard of a tool that can display the PR of a page. Is this a worthwhile thing to have and if so, where would one get it?

[kuska]

Does anyone know the dates of google dances? I had one google spider visit my forum yesterday but it did not pick up the link to my archive : \
I have the link to my archive visible but in my footer. Inktomi Slurp picked up on the link and was all over my archive forums but not Google. Any ideas?

[Guidster]

Originally posted by kuska
Does anyone know the dates of google dances? I had one google spider visit my forum yesterday but it did not pick up the link to my archive : \
I have the link to my archive visible but in my footer. Inktomi Slurp picked up on the link and was all over my archive forums but not Google. Any ideas?

This is a good sign. I saw the exact same thing earlier this month. At first they just nibble at it, then they really get after it. Watch for lots of activity in the coming days.

[NTLDR]

Seems to have worked pretty well, I think it has indexed every viewable thread by guests up till the time it spidered which is pretty good indeed :D

Results 1 - 10 of about 979. Search took 0.23 seconds.

[subduck]

This is the coolest hack ever!!!!!!!!

[John]

"The Google Dance" has indeed started - the 7 servers are currently out of sync.

Check progress here: http://www.google-dance.com/

[subduck]

lol! It looks like you guys don't need to worry about people backward links to your site! you have hundreds now!

[NTLDR]

I've just taken a look at my server stats and I've had 85 visitors come in via Google in the last 6.5 hours, thats about 14 an hour and the weekend is pretty inactive for me, this hack seems to have done a great job :D

[SkuZZy]

wow, i'm glad to hear all the success that everyone has had. The www2 server is what google uses to test out their new index, before they have it go "live". The test lasts 5 days and then the www server is fully updated. That is when the dance is considered "finished". Then the "big spidering" begins and it can last anywhere from 5 days to 3 weeks.

[Smoothie]

Originally posted by SkuZZy
Google is dancing! For those of you who installed this hack in early to mid-janary, check to see if google spidered your pages. There is a simple way to check:

1. Go to http://www2.google.com (notice the www2).

2. Type in: site:www.yoursite.com yoursite. (So for instance, I would type in "site:www.battleforums.com battleforums")

3. Check to see how many pages are listed. I notice some sites have alot of pages listed. Thrillnetwork has over 8000 pages indexed for example (not all of them from the archive).

BE SURE TO POST YOU RESULTS! :D Results from Macfora.com;
Results 1 - 10 of about 6,490. Search took 0.22 seconds

[Smoothie]

Is it wise to leave links in your header/navbar in your archive page? The reason I ask is because I understand that google starts at the first link it sees and works down from there. If it crawls those links, will it stop? Does anyone know?

@SkuZZy-

Have we found a way to eleiminate that blank page we get when clicking the category link?

[SkuZZy]

Originally posted by Smoothie
Is it wise to leave links in your header/navbar in your archive page? The reason I ask is because I understand that google starts at the first link it sees and works down from there. If it crawls those links, will it stop? Does anyone know?

@SkuZZy-

Have we found a way to eleiminate that blank page we get when clicking the category link?

The blank page when clicking a category would probably be fixable, but I haven't bothered to fix it because it doesn't really matter whether or not it works. The bottom line is, this archive works. It's by all means not a "pretty archive" but it gets the job done and it gets pages spidered. Google doesn't start at the first link it sees and goes from there... google will spider all links. Regardless of what is at the top. Don't worry about the first link being a dead one, google really doesn't care.

SkuZZy

[SkuZZy]

Originally posted by Smoothie
Results from Macfora.com;
Results 1 - 10 of about 6,490. Search took 0.22 seconds

Holy mother. Congrats, that is a great amount of pages! You need to remember, google only allows so many pages to be indexed per cycle. So even though you have more thn 6000 threads, that's all google is gonna give you this time around ;) The next month, another 6000, ect. This is to prevent googleboe from being overloaded, as spidering 50000 page sites isn't exactly easy to do. Please let me know if your hits increase. Right now google is still dancing, but watch in the next 3 days and see after that if your hits increase, they are bound to!

[Smoothie]

I noticed that past few days, google has been very active on my site.

And about the dead links in the navbar, yes, that was a big concern. Someone told me that if google tries to spide these first, there could be a problem, but judging from my results, I see that is not the case.

And yes, we get some forum members who actually go the archive to read it, thats why I asked about the dead category links.

BTW, here's my seach link;
Macfora.com (http://www2.google.com/search?hl=en&ie=ISO-8859-1&q=site%3Awww.macfora.com+macfora&btnG=Google+Search)

[SkuZZy]

muhahaha, smoothie. You are going to get tons of hits once google is done dancing. That is some spider you got! Congrats.

And now on a sad note, i've released the final version of this hack. I suggest everyone download it, because it's the final version... and it's stable. Hopefully this hack shall live on forever. Sorry to everyone for not getting around to adding cool new stuff, like the category thing mentioned above. Just no time lately and i've got to stop doing some stuff and with vb3 coming up, there realy is no need to continue this archive. All the die hards who stay with vb 2.2.9, this hack will be for you ;)

SkuZZy

[Smoothie]

Is there any need to upgrade if the version we use works? What is new about this final version?

[SkuZZy]

Originally posted by Smoothie
Is there any need to upgrade if the version we use works? What is new about this final version?

It won't hurt to upgrade. Just upload all files EXCEPT config.php and nothing will change at all, atleast not that google will know of
. Nothing much changed, just cleaned up the code a bit and changed the copyright around. Everyone should upgrade though, just because... final versions rock :D

[Smoothie]

@SkuZZy-

many thanks for this hack!

[subduck]

When we upgrade to vbulletin3, will our forums history in google still redirect to the proper pages? or will they all be dead links?

[SkuZZy]

Originally posted by subduck
When we upgrade to vbulletin3, will our forums history in google still redirect to the proper pages? or will they all be dead links?

Good question. I'm assuming the old links will work. The URL hasn't changed in vb3... it still looks the same.

[floridaideal]

Thanks for the great hack, its looking good now, I submitted it yesterday and hopefully crossed that I might get listed soon.

A quick question...do sub forums still not get spidered? Mine are not coming up on the archive?

Also how do you guys see that Google has been on your site? I got Urchine stats with my Ensim server from Rackshack but can't find dates that its been in?

Thanks

Stuart

[bgtking]

What do you mean by change the extensions? I take them off and go to the archive page and get an error.

[SkuZZy]

Bah ok everyone. I guess that wasn't my final version, cause I just released another one. This one now supports sub-forums. The previous one did also, except it relied on what you had your forums depth set as in the admin enter. I know lots of people have several levels of sub-forums, but dont want to show them on forumhome... so this hack will go 5 levels deep now, regardless of what your depth is set as in the admin. So I guess the current release is "vB Easy Archive Final v2.0" ..... lmao.

[floridaideal]

Nice one Skuzzy

I just installed it though and sub forum still won't work? Any idea's?

Thanks

Stuart

[SkuZZy]

I just updated it, please try now... what is your main forum depth set to btw?

[floridaideal]

Hi Skuzzy, thanks for that will try it again now, how do I set my forum depth?

Thanks

Stu

[floridaideal]

Hi Scuzzy

Working great now, nice one. Whats this forum depth thing about? Do I need to set it somewhere?

Thanks

Stuart

[SkuZZy]

nope, you don't need to set forum depth. Glad it's working now :D :D :D :D :D

[Minots]

Thank you for adding the sub-forums feature. Now it fits all my needs. :)

*Great Job*

[tpearl5]

I just installed the new version however I still don't see any sub-forums. There should be some under new files: http://board.midibuddy.net/archive2/

Google did reindex my site though :)
http://www.google.com/search?num=100&hl=en&lr=&ie=UTF-8&oe=UTF-8&q=+site%3Amidibuddy.net+midibuddy

[cklaszlo]

Results 1 - 10 of about 19,900. Search took 0.28 seconds

:)

In the latest update the only way for me to get it working was to add .php after "topic" and "forum" other then that it works great.

[Xenon]

have uploaded a new version
it has minor optimations to get the url (wouldn't affect the script, but jic)

also the copyrightnotice i've taken out from the scripts and add em into the footer.php

also the official vb copyright is now there :)

[cklaszlo]

help. For some reason everything just became centered. How do I correct this?

http://www.thrillnetwork.com/boards/archive/

Thanks

[Xenon]

at first please update to the current version, you don't have the copyright of jelsoft in the old one you use...

and then check if you have an unclosed <center> tag in your header file

[cklaszlo]

My bad. thanks. found the mistake. i will upgrade ASAP.

[SkuZZy]

As mentioned above, xenon made some changes to the hack and once again updated the .zip file. Everyone should upgrade, as it includes some code modifications and also the ever important vbulletin copyright.

[FrankR310]

Thank you!!!

....THANK YOU!!! :)

[cklaszlo]

Google went live with ww2 !!!!

Our results range between 19,900 and 20,300.

**********
SkuZZy, Xenon or anyone else:

I was wondering if you or someone else can help us create a similar script for our Roller Coaster and Amusement Park database. we have over 1600 roller coasters and 500 Amusement parks listed in our Database and not one page is listed in Google. It would be great to get all those pages listed too.

Coasters: http://www.thrillnetwork.com/index.php?page=database&func=allcoasters

Amusement parks: http://www.thrillnetwork.com/index.php?page=database&func=allparks

*******************
Thanks for this great hack. I see loads of people now viewing the archive (look at online.php). I guess they are getting there from searches on google. Question, we've all put on pop-up ads on the archive pages. but if users from google are getting to those do you think the pop-ups are more of a deterrent?

[Xenon]

cklaszlo: your links are not working...
but i think it should also be possible to write a quite similar script for these links, too.
but i don't have the time for it these time...

your popups are really annoying in my opinion
the archive is there to give a short plain overview of your board and a lot of popups and so on just lead to leave the page..

[cklaszlo]

Fixed the links. Thanks.

The reason I asked about the pop-ups was because of SkuZZy's post:

Making Money with the archive

I've recentely been using a great pop-up windows script that works great. They pay on time and pay $5.00 per 1000 unique visitors you send. It's monthly unique, but since most of the traffic coming to your forum is from search engines, you will make big bucks of it. I am currently making around $10 a day.

I removed most of the pop-ups and now only use a pop under. I agree about pop-ups. We have gone 100% pop-up free in the rest of the site except for the archive. But now I'm finding the bulk of new traffic is comming via Google.

Thanks for at least looking at our coaster/and park DB. Maybe someone else can help us and tack a crack at it.

[SkuZZy]

I use popups on my archive and i'm making quite a bit of money from it. Have a lost some potential members because of them? Most definately. I never have liked popups, but I do get a bit tired of everyone telling me to remove them. My main forums have no banner ads, no pop up windows and no advertising what-so-ever. The archive has popups. So what, people join the forums and they are all gone. If someone sees the popups and closes the site because of them, well, they just made me $0.008. Sure, i'd like more members, but tons of members doesn't do any good when you don't have enough money to pay for hosting. If others can run sites without popup windows, then kudos to them... but my site is gaming related and a forum, and no advertising company will accept me. Furthermore, gaming advertising sells for virtually nothing, therefore, i'm left with popups or selling my site :/

Just explaining why I use popups. If others don't want to use them, I understand and if they offend you in anyway, then please, don't visit my archive :D

[cklaszlo]

I didn't mean to offend you. I was really curious as to trying to get a handle on ratios of people coming to a live forum via the archive. And if pop-ups on archive pages were distracted people and made them not want to sign up.

SkuZZy, PM or email me. I might be able to host your site. It's the least I can do for you since because of your hack our site is getting tons of traffic and we are listed to the max (lol) in Google.

[SkuZZy]

Originally posted by cklaszlo
I didn't mean to offend you. I was really curious as to trying to get a handle on ratios of people coming to a live forum via the archive. And if pop-ups on archive pages were distracted people and made them not want to sign up.

SkuZZy, PM or email me. I might be able to host your site. It's the least I can do for you since because of your hack our site is getting tons of traffic and we are listed to the max (lol) in Google.

I wasn't offended, that was mainly directed towards xenon though :D I just wish people would stop telling other people what to do. It's like that hack Teck released, the pop-up windows for unregistered members. People spammed the thread with stuff about it being unethical and how you force your members to sign up and blah blah blah. Why can't people just mind their own business. Not saying xenon said anything wrong... maybe i'm just a little quick to pull the trigger this morning :D

Thanks for the offer on hosting, but battleforums does take alot of resources and we just moved hosts 3 days ago. My hosting actually doesn't cost that much (I make it seem worse than it is). But battleforums does use alot of resources, getting about 12,000 unique visitors (thanks google), 100,000 page views & 700,000 hits per day it's using about 65GB of bandwidth a month using GZIP and it's taking up alot of load on my current server... so I would not expect anyone to host it for free, but thanks for the offer :D

SkuZZy

[SkuZZy]

By the way, you do have an insane amount of pages in google! Over 8000 listed. Good job :D - How much traffic does your site get per day?

[cklaszlo]

I have 4 servers colo'd at my friends ISP. So if it becomes an issue for you let me know.

ThrillNetwork.com transfers over 85 gigs. We do 150,000 unique visitors per month. With over 1.7 mil page views. :) And we don't make any money!!! lol. Our big problem is stickyness. We are working very hard on that now. Building better content and easier navigation. Getting rid of pop-ups has helped with this. Also going ad free on the Index page of the main site.

How do we find the actual amount of Pages listed in Google? Do you think you might be able to help with our other Coaster/Park Database archive?

[Xenon]

@Skuzzy: i really understand your decicions why to use popus.
My post was never meant to offend anyone of you, i just answered cklaszlo's question what i think about the popups.

It's always the admins decision wether to use popups or not.
In my personal opinion i don't like em and close most of the sites immediately when i saw popups, but there are others who don't :)

I just wanted to show cklaszlo the other side of the sword :)

[Smoothie]

@Skuzzy-

This hack really screws up the who's online. A possible fix? I added FireFly's add who's online from the admin cp, but it won't work with these archive .html files.

[Destee]

it seems everyone is getting indexed but me ... :(

am i not being patient enough? i had the hack installed on 01/10

site:destee.com archive (http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=site%3Adestee.com+archive)

i had Overgrow's or Fastforward's hack installed at:

http://www.destee.com/forums/archive

but replaced it with Teck's hack tonight

my Skuzzy hack is at http://www.destee.com/forums/archives

::::::sigh:::::: i've installed them all ... :cry:

:)

Destee

[Javazoid]

same. i want to join in on the party! my index is at wcgateway.com/forums/archive and i belive im the only one that has even seen it. i submitted the url "http://www.wcgateway.com/forums/archive" to google but i have not been index. help me and destee out!

[Xenon]

@Javazoid: from what i can see you don't have a link to the archive on your forumhome.

Spiders can just index reachable sites...
if it's a 1x1 pixel link it will be considered as urlcloaking and won't work for google for example.

i've changed my footer template from a 1x1 image to a small textlink and the ammount of searchengines rises a lot..

[WebMasterAJ]

Originally posted by Smoothie
@Skuzzy-

This hack really screws up the who's online. A possible fix? I added FireFly's add who's online from the admin cp, but it won't work with these archive .html files.

Same here... are there any possible fixes for this? I have 16 guests and its all reading:

Unknown Location: /boardvb2/archive/topic/6785.html?

Which is a good thing... but... it doesn't look too good :D

Any help?

[TECK]

Stefan, from my experience with search engines (and I did searched a lot), indexes will drop your site for the following reasons:
1. no unique meta tags for each page (most important)
2. popups or any other external links, that direct your crawler outside the site
3. deprecated html code

Now, others will argue with me and tell me: "heh my archive is listed..."
Let me know for how long. I guarantee you that by the end of the month your links are dropped.
The best way to test your links is to go to Google and search for:
site:yoursite.com archive

So, I strongly recommend you Stefan to update your code, related to those issues I mentioned above, orelse the archive pages will NOT be indexed.
vBulletin.org decided to install my archive because of "no. 1 reason", listed above.

I know that "some" people (same who rated 1 my archive) will consider this as "spam" or "advertising" my own code... believe what you want, I simply try to help.
Feel free to read more here to see that what I say is true:
http://www.searchengineforums.com/

[cklaszlo]

@TECK: Our Results 1 - 10 of about 8,530 from google.

So Teck, are you saying that in a months time these pages will be removed?

Isn't a "title" a meta tag and there for each of these pages will be unique in the meta tags?


@Xenon: I put 2 links from our main index page (http://www.thrillnetwork.com) and also on our board pages. The bots loved this.

[Javazoid]

ok , i have added a big archive at the bottom of my forum home "http://www.wcgateway.com/forums" at the bottom to the archives ? will this help

[cklaszlo]

I'd also add it to http://www.wcgateway.com on the index page. Thats what we did and Google and inktomi have been spidering us for a few months now.

[FlyBoy73]

Did I miss google? I got hit by Inktomi hard on the 27th of January, but have not seen google yet. Have they done their main spidering yet?

[TECK]

Originally posted by cklaszlo
@TECK: Our Results 1 - 10 of about 8,530 from google.
Isn't a "title" a meta tag and there for each of these pages will be unique in the meta tags?
The meta tags are what users enter in the search when they request information from search engines.
Also, the index(crawler) associates each page with your meta tags.
Example:<meta name="keywords" content="indexer the lounge" />
<meta name="description" content="TeckWizards.com - Introduce yourself and discuss anything you want here" />
Those are the meta tags for the forum listed below:
http://www.teckwizards.com/forumdisplay/f-8.html

[cklaszlo]

@teck: OIC. But it seams that the Meta tags are not that imp to google as one might think. I mean this hack did index over 8000 of our pages. What I do like about your is the integration with VB. I like that it splits up pages.

[TECK]

Remember, is not me who say this, but the tech guys from Google and other search engines.
Search at http://www.searchengineforums.com for more details to see the real facts.

The split pages are good for large forums with millions of posts.
An site like overgrow's or avsforum.com will crash the server while displaying over 50,000 threads in one page, for 10 simultaneous guests, if split pages are not present.

[saint_seiya]

I know this is offtopic but can anyone help me, thanks:

-----------------

I want my urls to be rewritten with mod rewrite like:



http://www.vgcity.com/index.php?temp_view=game_info.php&game=ATV%20Offroad%20Fury%202&id=384

to

http://www.vgcity.com/index/php/temp_view/game_info/php/game/ATV%20Offroad%20Fury%202/id/384.html

. with /
? with /
= with /
& with /

At the end add .html (after 384)

[Crinos]

http://www.searchenginewatch.com/webmasters/index.html
http://www.searchenginewatch.com/webmasters/meta.html

As far as I know, meta tags are not given as high a priority as it once been given ... it's still all about relevancy ... there is no unifying rule that covers all search engines :)

[TECK]

Originally posted by Crinos
http://www.searchenginewatch.com/webmasters/index.html
http://www.searchenginewatch.com/webmasters/meta.html

As far as I know, meta tags are not given as high a priority as it once been given ... it's still all about relevancy ... there is no unifying rule that covers all search engines :)
http://www.searchenginewatch.com/webmasters/features.html

[SkuZZy]

meta tags dont matter in google, they are used very rarely. They are mainly only used for incomplete spidering sessions by google. Im not saying meta tags arent important for other search engines, but they dont matter much in google.

[SkuZZy]

Originally posted by TECK
Stefan, from my experience with search engines (and I did searched a lot), indexes will drop your site for the following reasons:
1. no unique meta tags for each page (most important)
2. popups or any other external links, that direct your crawler outside the site
3. deprecated html code

Now, others will argue with me and tell me: "heh my archive is listed..."
Let me know for how long. I guarantee you that by the end of the month your links are dropped.
The best way to test your links is to go to Google and search for:
site:yoursite.com archive

So, I strongly recommend you Stefan to update your code, related to those issues I mentioned above, orelse the archive pages will NOT be indexed.
vBulletin.org decided to install my archive because of "no. 1 reason", listed above.

I know that "some" people (same who rated 1 my archive) will consider this as "spam" or "advertising" my own code... believe what you want, I simply try to help.
Feel free to read more here to see that what I say is true:
http://www.searchengineforums.com/

I'm not going to bother trying to change your mind anymore, but you are wrong TECK and I will prove that to you. Google doesn't kick because of lack of unique meta tags. That is the biggest joke i've ever heard. And #2 is even more rediculous. Guess what, I use 3 popups on my archive and an exit window and I get over 7,000 uniques a day from google, just from this archive. Please don't spread these kind of remarks around, you are misunderstanding how google works. If you don't believe me, then post what you just posted below at SearchEngineForums (which you seem to preach) and they will tell you that you are wrong also. Google doesn't even open javascript files, so how would they know it contains a popup window? As far as #3 goes, there is no proof that google drops old pages that haven't been updated, but i'm sure that they will at some time. That is irrelevant anyways.

You seem to doubt the stuff I say teck, and i'm not too sure why. I've been working on search engines for over 2 years and i've been hired by some of Canada's top companies to search engine optimize their sites for them. My archive isn't as pretty as yours, but it works and I prefer it over yours for a few reasons. First off, the archive is meant for one purpous... to get your posts spidered, not to have users surf on! My archive is dull and has a nice big link pointing to the original thread. Yes, your archive is nice... so nice that most people will probably end up surfing it and never actually visiting the real forum and signing up. This isn't what I want, I want users to get to my forum.

As far as multiple pages, why does it matter? Listen. No one is going to be visiting the actual "index" of threads, only google. That is fine though, because each month it will continue to spider your site, so all your new threads will get added. I know what I am talking about, my forums have 1200+ threads and I continue to get 3000 pages added to google monthly. I have 5500 pages with them now and 3 months ago I had only about 10. Now, people coming from google will go directly to the post, so they wont have to see "50,000 threads" as you so called it. However, you are right, some larger forums that have forums with 10,000 or more threads in a single forums should not use my hack, they should use yours, otherwise alot of their current threads wouldn't get spidered.

As far as what you are saying being true, teck... man... you need some serious lessons. I don't know what threads you were reading over at searchengineforums, but I imagine you chose to read only the ones that would confirm your own opinions. I also urge anyone here to go surf around searchengineforums and you will see that my hack isn't "the devil" that teck makes it out to be. Is teck's hack better than mine? If you want a flashy archive and want your visitors surfing your archive (rather than your forum) then I suggest you go with his. I also suggest you use his if you have an overly large forum with several tens of thousands of threads. But for all the smaller forums out there, I guarentee you will get better results with my hack than teck's. Everything in mine is optimized, where as teck's archive has many small things that will hurt him in the search engine. For instance, on posts in your archive teck... you list the title of the forum first, then the title of the thread. Don't you know that the title is the most important part in google? And google only counts the first 59 characters of the title. anything after that is wasted space. So by putting the title first, you are wasting alot of characters.

There are many other small thinggs like that in teck's archive, but he can figure them out for himself. The bottom line is though, that my forum is recieving 7,000 unique hits from google per day. If you want to see proof, then go to my stats tracker, http://extremetracking.com/open?login=bforums - The proof is in the pudding. So teck, where are your results? Is your archive bringing you over 7,000 uniques a day from google?

- SkuZZy

[Smoothie]

google seems to love this script on my site;
Results 1 - 10 of about 6,510. Search took 0.21 seconds.

[SkuZZy]

Originally posted by Smoothie
google seems to love this script on my site;
Results 1 - 10 of about 6,510. Search took 0.21 seconds.

That's great smoothie. Have you noticed a traffic increase?

[Smoothie]

Yes I have! Not to mention 350 new members in the month of January.

[floridaideal]

Hi, I seem to have a problem being spidered

I have not had a link on Google yet although looking at the cashed index page that Google has recently stored the text link I put at the bottom of my page its clearly visable and works fine.

http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=%3Atop-forums.com

Google has spidered the first archive link I made that was another hack (not that it spidered all the pages just the link).

Will I have to wait or is there a problem?

Anyone got any ideas please?

Thanks

Stuart

[SkuZZy]

Google has probably spidered your archive already. Unfortunately, your pages won't be added right away. If you put your archive up after the 18th of January (which you must have) then your pages won't be added to google until next month's update, which will be around mid-late Febuary.

SkuZZy

Originally posted by floridaideal
Hi, I seem to have a problem being spidered

I have not had a link on Google yet although looking at the cashed index page that Google has recently stored the text link I put at the bottom of my page its clearly visable and works fine.

http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=%3Atop-forums.com

Google has spidered the first archive link I made that was another hack (not that it spidered all the pages just the link).

Will I have to wait or is there a problem?

Anyone got any ideas please?

Thanks

Stuart

[TECK]

Originally posted by SkuZZy


I'm not going to bother trying to change your mind anymore, but you are wrong TECK....

- SkuZZy Is obvious you are in the competition of having people installing your hack, I was expecting to see a reaction like that from you.
You are the person who's wrong thinking that I care people will install your hack and not mines, heh... is funny.

Is up to them what script they like better, not to you or me.
To prove you how much I care about this, I'm going to add a link to your archive in my first post so people can have an alternative, before they install my script.

[cklaszlo]

we've seen an increase of traffic too. but not as many people converting to registered users. Only 5-10 more a day. Which isn't horrible simce our site is very specific.

We have now over 8300 pages in Google. :)

What are your feelings forcing people to sign up? To even read the boards?

[SkuZZy]

Originally posted by TECK
Is obvious you are in the competition of having people installing your hack, I was expecting to see a reaction like that from you.
You are the person who's wrong thinking that I care people will install your hack and not mines, heh... is funny.

Is up to them what script they like better, not to you or me.
To prove you how much I care about this, I'm going to add a link to your archive in my first post so people can have an alternative, before they install my script.

I don't care either. I've reffered countless people to your script who didn't like mine for whatever reason. But i'm not trying to start up a "which script is better" thing. Both scripts have their UNIQUE features. I just don't appreciate you saying things which are completely not true. I think you have quite a bit to learn about search engines before you can start critisizing my script in my own thread.

SkuZZY

[cklaszlo]

As far as multiple pages, why does it matter? Listen. No one is going to be visiting the actual "index" of threads, only google.

Actually for us loads of people are browsing the archive for some reason. Is it possible to ad a meta tag redirect on the thread and post pages for the script? To redirect automatically to the real boards?

[TECK]

Originally posted by SkuZZy


I don't care either. I've reffered countless people to your script who didn't like mine for whatever reason. But i'm not trying to start up a "which script is better" thing. Both scripts have their UNIQUE features. I just don't appreciate you saying things which are completely not true. I think you have quite a bit to learn about search engines before you can start critisizing my script in my own thread.

SkuZZY
I'm sorry if you got this impression. I only posted what I read from others. I don't consider myself an expert that's why I asked fastforward and many other people about the best techniques used for search engine spiders.

My points listed above, were based on the information obtained from those people.

[saint_seiya]

Originally posted by saint_seiya
I know this is offtopic but can anyone help me, thanks:

-----------------

I want my urls to be rewritten with mod rewrite like:



http://www.vgcity.com/index.php?temp_view=game_info.php&game=ATV%20Offroad%20Fury%202&id=384

to

http://www.vgcity.com/index/php/temp_view/game_info/php/game/ATV%20Offroad%20Fury%202/id/384.html

. with /
? with /
= with /
& with /

At the end add .html (after 384)



Anyone???
Maybe just a link to a good tutorial :(

Btw, i cant wait until i see the increase of traffic :)
I paid inktomi for 2 day spidering so so... :nervous:

[SkuZZy]

Originally posted by cklaszlo


Actually for us loads of people are browsing the archive for some reason. Is it possible to ad a meta tag redirect on the thread and post pages for the script? To redirect automatically to the real boards?

Adding a redirect is against google's rules. If you were to add a meta-tag refresh or permanant redirect, google wouldn't spider the post pages. Someone a while back posted a method which seemed to make sense to me, which would forward only people using normal browsers directly to the thread, but if it was GoogleBot, it would refer it to the text archive as normal. This would probably work, however, it is still against google's rules and if someone reported you (it happens alot) then you'd get a nice vacation from google.

[cklaszlo]

Ok. Thanks for the Info. I see inktomi has found us again. :)

@ SkuZZy I just sent you a PM. thanks.

[SkuZZy]

Originally posted by cklaszlo
Ok. Thanks for the Info. I see inktomi has found us again. :)

@ SkuZZy I just sent you a PM. thanks.

Glad to hear that. Inktomi doesn't visit me much..... they never did, dont know why ;) But google wuvs me :D :D

I responded to your PM

[NTLDR]

Originally posted by SkuZZy
Someone a while back posted a method which seemed to make sense to me, which would forward only people using normal browsers directly to the thread, but if it was GoogleBot, it would refer it to the text archive as normal. This would probably work, however, it is still against google's rules and if someone reported you (it happens alot) then you'd get a nice vacation from google.

I posted how to do this towards the start of the thread, but I don't use it after taking your advice. I've allready had 1136 vistiors come in from google in the past 3 days compared with rarely getting any serach engine hits previously and the % of users comming in via a search engine compared to other methods is on the increase.

[kuska]

Im the other way around. Inktomi comes alot but google? Never :( And im still waiting for it, i dont care about intkomi because there isint that many polish people (i have a polish board) comming from inkomis search engines. Google has tons of polish sites and polish people visiting it.... :(
Im just gonna WAIT and bite my nails like i did for the past 3 weeks.
:)
Still great hack tho :)

[kuska]

Usefull tool:
Link Popularity Check
Popularity script checks 7 major Search Engines for inbound links to your site.
Checks Google, AllTheWeb, AltaVista, AOL, MSN, HotBot, and Lycos.
Comparison to your competitors.
Option to send the report to your Email address.
Fast and accurate.

THIS IS NOT A SPAM POST!!!!! Just thought it might be usefull for this hack Im in the 0 to 250 references spot : \ ...vb.org has 2,501 to 10,000 references
Here is the link:

http://www.free-webmaster-tools.com/link_popularity.htm

[Crinos]

Hmmm, weird ... I got more results from Alltheweb than Google ... news to me ... thanks for the link :)

[cklaszlo]

That site only showed that I have like 74 pages in Google. But Google says I have over 8500. Isn't that a big difference. :)

[SkuZZy]

I don't know what the deal is with marketleap, they are too unreliable for me ;)

[cklaszlo]

@SkuZZy - did you get my email?

[cklaszlo]

FYI guys,

Google is crawling in a major way. I noticed that Googlebot has made the leap from the archive to the real boards. :) anyone else see this?

[codewebs]

google seams to be viewing the printable version of alot of my threads :-/

[kuska]

Still waiting for Google, i just get 2-3 visite per every 4-5 days but never the archive :-/
Intkomi is all over it tho :)

[codewebs]

google has been majorly spidering my site for 2 days now :)

[limey]

Getting google every night, but no archive yet.

[limey]

Anyone getting hit with this bot? http://www.turnitin.com/robot/crawlerinfo.html#turnitin

I'm going to turn em away.

[NTLDR]

Originally posted by limey
Anyone getting hit with this bot? http://www.turnitin.com/robot/crawlerinfo.html#turnitin

I'm going to turn em away.

^^ Good on you! I gave them an IP block too.

[Guidster]

Originally posted by limey
Anyone getting hit with this bot? http://www.turnitin.com/robot/crawlerinfo.html#turnitin

I'm going to turn em away.
I put up a robots.txt file just for them and they have not been back since.

[Guidster]

Originally posted by cklaszlo
FYI guys,

Google is crawling in a major way. I noticed that Googlebot has made the leap from the archive to the real boards. :) anyone else see this?

Yes, I was just going to comment on this. Googlebot appears to be gorging on the threads, posts AND the archive. I have 35 instances of the bot on my site right now!

[Crinos]

Hehehe, yeah, there's a feeding frenzy in my site too:

http://www.opentechsupport.net/images/stats/google-crawl-small.jpg (http://www.opentechsupport.net/images/stats/google-crawl-big.jpg)

[Xenon]

yes, goolge indexed my archive and threads same time...

[SkuZZy]

I've also been hit by google. Their big spider should be ending soon. On a side note, I also got a cheque from PopUpSponsor yesterday ... made $168 US in 16 days with them, not too bad! :D ...... I've switched to another place though.

[julius]

Thanks, very simple and quick to install!

I have some threads with more then 2000 posts, is it possible to divide the threads to more pages?

[Xenon]

hmm, i've a thread with about 4k posts and it works also....

but, i'm sure i can work on a way to divide the threads (when i get more time to my hands :))

[julius]

This would be great! :) I think that nobody will open a search engine result page of 2 megabites.

[Xenon]

not? :D

will see what i can do for you soon :)

[Javazoid]

Everyone is getting archived but me! what am i doing wrong, http://www.wcgateway.com/forums/archive and i submitted to google have links on my main page and forums to it. what am i doing wrong!?!?

also where do i go to submit to inktomi? does it cost money?

[codewebs]

you probly will eventually, i cant tell why you havnt :-/

[kuska]

Javazoid have patience. I waited 4 EVER !!! (3-4 weeks)
To deal with the pressure just imagine you never installed the hack and go about your normal buisness :) And one day ... WHAM !!! You will be swormed with robots :)

[Xenon]

Originally posted by Javazoid
also where do i go to submit to inktomi? does it cost money?

i never invited them, just one day there was an inktomi crawler..

[cklaszlo]

@ SkuZZy --- Did you get my email? :)

Googlebot is going nuts. Most of our traffic is now comming from Google searches and users finding us via the archive. One reason I don't want to place too many pop-ups on the archive. Now here is the question of the day, and this might not be the right place, but, how do we convert guests into registered users?

[FlyBoy73]

Google just showed up on my site last night and has aprox 42-65 spiders running since that time. :) THANK YOU SKUZZY!!!!!!!!

[FlyBoy73]

What is the maximum number of spiders you usually see with Google? Inktomi blasted up to around 125 spiders.. I have about 50 on my site right now.

[FlyBoy73]

Originally posted by TECK
Is obvious you are in the competition of having people installing your hack, I was expecting to see a reaction like that from you.
You are the person who's wrong thinking that I care people will install your hack and not mines, heh... is funny.



I've kept quiet about this for some time but after these comments I am going to speak. Throughout this thread you have interjected yourself, Tech. On no other hack discussion have I seen someone with a "competing" (similar) hack visit the others thread so much and make comments such as the ones that you have made on numerous occasions - arguing against SkuZZy's info in regards to Google.com inclusion, etc. Maybe your hack works and looks pretty.. Good.. Whatever.. But, keep it out of this thread.. That is just rude.
As for your differing opinion on Google. I did go over to searchengineforums and read, read, read and then post. It seems that SkuZZy is right on the money with what he has been saying.

[bromyaur]

installed and clicked install

thanks for the great hack, wish I found it earlier

http://www.graphicsteck.ca/vb/archive/

now to make it look pretty lol and get the proper links from my forum

[cklaszlo]

Google is hitting us again today with about 75 - 90 bots on average. Inktomi at about 30.

[Xenon]

@julius (and all others of course):
i've uploaded a new version, it divides a thread into 200 posts per page, hope u like it :)

[julius]

Originally posted by Xenon
@julius (and all others of course):
i've uploaded a new version, it divides a thread into 200 posts per page, hope u like it :) Xenon, Thanks very much indeed! :)

[Steve St.Lauren]

I've installed this hack on our boards and I'm getting the forum list but when I click on the forum name then it just bounces to my error page (which is the front page of the site). I've added mod_mime and mod_rewrite to my httpd.conf and restarted apache so I'm pretty sure they are installed. Any way to verify that with a command? Any help would be appreciated.

Also, I have 56,312 threads on our site and I'd like to keep the loads off of our primary server (we get ~1,000,000 hits a day). I have a second server that is mysql slaved to the master so the entire database is on that server as well. Any recommendations on how I could change these scripts so that the actual information (mysql queries) come from the second server (which is on an IP address - no domain name) but it looks like it came from the primary server domain name. TIA

[Xenon]

hmm, weird.
are you sure you have placed the .htaccess right?
it seems that it doesn't run the php code just assuming site does not exists..

[Javazoid]

so you don't have to submit to inktomi? what! where is inktomi anyways.. inktomi.com ? you always have to submit other wise how they know your site is tehre, is it connect to google or something?

[Steve St.Lauren]

Yes, .htaccess is there - just verified that.

[Steve St.Lauren]

I figured it out - in my httpd.conf file AllowOverride was set to none for my entire htdocs directory. All I had to do was turn on AllowOverride for that directory and it's working now.

I could still use help on how to have it pull the data from my slave server but look like it's coming from the main. TIA

[julius]

Originally posted by Xenon
@julius (and all others of course):
i've uploaded a new version, it divides a thread into 200 posts per page, hope u like it :) Xenon, it seems that google downloads twice the same page:

ex: for thread id 2000

.../topic/2000.html

and

.../topic/2000-1.html

[Xenon]

@Steve: good to hear :)
sorry but the second thing i couldn't answer i just work on one server systems and hav noe experience for you then...

@julius:
yes, but that shouldn't be a problem :)
it's like it here, too
you can call page 1 by adding the pagenumer=1 or just leave it alone on showthread...
if it really disturbs you, just put an -1 after the $threadid in forum. where the link is..

[julius]

Originally posted by Xenon
@julius:
yes, but that shouldn't be a problem :)
it's like it here, too
you can call page 1 by adding the pagenumer=1 or just leave it alone on showthread...
if it really disturbs you, just put an -1 after the $threadid in forum. where the link is.. [/B]I put -1 after the $threadid in forum and the $forumid in index.php.
Thanks

[fello9]

Originally posted by Xenon
@julius (and all others of course):
i've uploaded a new version, it divides a thread into 200 posts per page, hope u like it :)

I have this installed on my board. But the page numbers on top are placed BEFORE the <HTML> tag. They should be placed after the </head> and baseurl tags.

I don't think Google will follow the page links if they're outside of the <html> tags.

I would rather prefer to move those page numbers to the bottom of the page before the footer. Can you please tell me how to do it?

Also, how do I add the Meta Robots tag to all pages?

thanks!

[cklaszlo]

Googlebot has jumped again to the real forums. It looks as if the Archive jump starts google. Wierd. But its now crawling huge amounts of pages in the active boards. I hope it leaps to our main site soon.

[Xenon]

@fello9: hmm, didn't know that...
have changed the files a bit again now the page nav should be within the htmltags :)

[fello9]

Originally posted by Xenon
@fello9: hmm, didn't know that...
have changed the files a bit again now the page nav should be within the htmltags :)

Thanks!

Can you also plese tell me how to add the <meta name="robots" content="index,follow"> tag?

[codewebs]

Originally posted by fello9


Thanks!

Can you also plese tell me how to add the <meta name="robots" content="index,follow"> tag?

if you dont have that, and you dont have for them not to follow, then the bots will asume you want them to index,follow.

[fello9]

Originally posted by codewebs


if you dont have that, and you dont have for them not to follow, then the bots will asume you want them to index,follow.

I have it on all my other pages so I think it might help. So can anyone just tell me how to add it?

(I don't know php :disappointed: )

[julius]

Is it possible in FORUM to add in
<head><title>$title</title></head> the forum title?

[Xenon]

change $title into $foruminfo[title]

[julius]

Thanks!

[codewebs]

look at all the googles :) i think there the fresh bots or whatever you call them tho..

http://www.codewebs.com/fresh.gif

[Javazoid]

Sign :disappointed: what am i doing wrong. Mine have been up for a very long time, couple of months and nothing has happened..

my archives are located at wcgateway.com/forums/archive

and the exact url i submitted to google was

http://www.wcgateway.com/forums/archive

i dunno maybe it hates me?

[Guidster]

Originally posted by Javazoid
Sign :disappointed: what am i doing wrong. Mine have been up for a very long time, couple of months and nothing has happened..

my archives are located at wcgateway.com/forums/archive

and the exact url i submitted to google was

http://www.wcgateway.com/forums/archive

i dunno maybe it hates me?

Maybe the spider is annoyed by the popups that are on your archive! :) Good looking archive. Have you submitted it again?

[Guidster]

Originally posted by dFrisk


I've kept quiet about this for some time but after these comments I am going to speak. Throughout this thread you have interjected yourself, Tech. On no other hack discussion have I seen someone with a "competing" (similar) hack visit the others thread so much and make comments such as the ones that you have made on numerous occasions - arguing against SkuZZy's info in regards to Google.com inclusion, etc. Maybe your hack works and looks pretty.. Good.. Whatever.. But, keep it out of this thread.. That is just rude.
As for your differing opinion on Google. I did go over to searchengineforums and read, read, read and then post. It seems that SkuZZy is right on the money with what he has been saying.

I have been so impressed that I uninstalled vBHome Lite in lieu of PHPPortal

[julius]

I hate Google!
I submitted the link to the archive to google and put the link on my home page.
After 2 days 50 google bot were on my site during about 2 days.
After 2 more days I made a search and Google returns me more than 1,000 pages (before were 190), with more than 800 from the archive.
Now, after 1 week google returns me only 191 pages, with only 1 (the main page) from the archive.

Why?

[SkuZZy]

Originally posted by julius
I hate Google!
I submitted the link to the archive to google and put the link on my home page.
After 2 days 50 google bot were on my site during about 2 days.
After 2 more days I made a search and Google returns me more than 1,000 pages (before were 190), with more than 800 from the archive.
Now, after 1 week google returns me only 191 pages, with only 1 (the main page) from the archive.

Why?

What you saw was called everflux. Your pages were only in google for a short time, then taken out. This is common practice. Your pages will be re-added again at the end of the month, this time permanantely.

[SkuZZy]

Originally posted by Javazoid
Sign :disappointed: what am i doing wrong. Mine have been up for a very long time, couple of months and nothing has happened..

my archives are located at wcgateway.com/forums/archive

and the exact url i submitted to google was

http://www.wcgateway.com/forums/archive

i dunno maybe it hates me?

Nice setup. It will spider you soon, chances are, it already has and you don't even know it. Watch for a bunch of your pages to be added next update. Even if google did spider you, they wouldnt be added until the end of the month during the "dance".

[Steve St.Lauren]

In installing this hack I noticed that the categories link to a blank page rather than just showing as a category. If you search through index.php for this line: (around line 82)

echo "<li><a
href=\"forum/$forum[forumid].html\">$forum[title]</a><br><font
size=1>$forum[description]</font>";

and change it to:

if ($forum['cancontainthreads']==0) {
echo "<li>$forum[title]<br><font size=1>$forum[description]</font>";
}
else
{
echo "<li><a
href=\"forum/$forum[forumid].html\">$forum[title]</a><br><font
size=1>$forum[description]</font>";
}

Then the categories won't show up as a link to a blank page.

[julius]

Originally posted by SkuZZy


What you saw was called everflux. Your pages were only in google for a short time, then taken out. This is common practice. Your pages will be re-added again at the end of the month, this time permanantely. Thanks.

[Xenon]

@Steve: thx for noticing, i've updated th file with that :)

[cklaszlo]

@Steven -

Thanks for finding that. I was wondering about it.

[99SIVTEC]

By adding the following lines of code to index.php you can gzip the output of this hack making it use far less bandwidth when users view it. Best of all the search engines can still spyder gzipped pages.


<?php
global $nozip;
$nozip=1;
ob_start("ob_gzhandler");


Just cut and paste that into the very top of the index.php file i nthe archive directory

[FlyBoy73]

I may have missed this earlier but how do I check to see how many "pages" or threads Google has indexed of my community?
Thanks

[Javazoid]

same.. how do you even know you have been indexed. i may have been but im not sure!

[SkuZZy]

Originally posted by dFrisk
I may have missed this earlier but how do I check to see how many "pages" or threads Google has indexed of my community?
Thanks

go to google and type in:

site:www.yourdomain.com yourdomain

So for instance, I type in:

site:www.battleforums.com battleforums

This will show you all the pages indexed. Keep in mind, this number only updates once per month, usually between the 25th and 30th.

SkuZZy

[FlyBoy73]

If that is so, then it only shows 4 pages for battleforums.com on Google? I thought using site:www.yourdomain.com would only show who is linking to you? I don't see any type of page information on how many of my threads (or yours) are available via google.
Am I doing something incorrectly?

[SkuZZy]

Originally posted by dFrisk
If that is so, then it only shows 4 pages for battleforums.com on Google? I thought using site:www.yourdomain.com would only show who is linking to you? I don't see any type of page information on how many of my threads (or yours) are available via google.
Am I doing something incorrectly?

I don't know what you are talking about. To see how many pages you have on your site, type:

site:www.yourdomain.com yourdomain


(Example) (http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&safe=off&q=site%3Awww.battleforums.com+battleforums)


To see how many people are linking to you, type:

link:www.yourdomain.com


(Example) (http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=link%3Awww.battleforums.com)

Get it?

[FlyBoy73]

Ok! I was not entering the second name.. ie site:www.yourdomain.com yourdomain

Thanks for the hand holding.

[cklaszlo]

@SkuZZy

Are you still going to help us with are roller coaster and amusement park DB?

Thanks

Alex

[SkuZZy]

Google dance should be coming soon all! A usefull tool is www.google-dance.com , check it out ;)

[Javazoid]

ok. is www.wcgateway.com/forums all set?

the archives are at wcgateway.com/forums/archive

the url i submitted a while ago was

http://www.wcgateway.com/forums/archive

should i submit again or just wait

[Guidster]

Originally posted by Javazoid
same.. how do you even know you have been indexed. i may have been but im not sure!

Check your logs. Parse for googlebot or if you want to see more, a good clue that spiders are about is a request for robots.txt. Right after the request for this file, you will most likely see a spider there.

The Google Dance site also features a Google tracking Perl application that I have yet to install, but this is a possible resource for you as well.

[kuska]

With the last dance i did not get indexed. Mabye i installed it and subbmited it to late, but i had spiders for 5 days in a row in my forums. No indexing tho :(

[kuska]

Mabye im doing something wrong? Take a look,
Here is the addy to the archive hack ....
http://www.nakazdytemat.com/forum

[julius]

Originally posted by 99SIVTEC
By adding the following lines of code to index.php you can gzip the output of this hack making it use far less bandwidth when users view it. Best of all the search engines can still spyder gzipped pages.


<?php
global $nozip;
$nozip=1;
ob_start("ob_gzhandler");


Just cut and paste that into the very top of the index.php file i nthe archive directory My vb is zipped with zlib. Is it possible to gzip the archive in the same way?

[SkuZZy]

Originally posted by kuska
Mabye im doing something wrong? Take a look,
Here is the addy to the archive hack ....
http://www.nakazdytemat.com/forum

Nope it looks fine. You all really need to give google some patience. Just because google spiders your site, doesn't mean your pages will be added.

For example, if you put your archive up on the 23rd, got spidered on the 24th, then google danced on the 25th, there is no way in hell your site would get listed. It's just how google works. Usually it takes 2 dances to get listed. The first dance to spider your site, and the 2nd dance to get listed.

Bottom line, if you had your archive up before Febuary 5th and you were spidered by google before Febuary 5th, then you will get listed in google in the upcoming dance. Anything after Febuary 5th..... no guarentees.

[cklaszlo]

any word on when the dance will take place?

[Steve_S]

Thank you. Great work you guys :)

Question: freshbot hit my Archive yesterday and all is well. I'm searching for a browser based "stats/logger" which records freshie and the deep crawl.

I tried www.darrinward.com/googletrax/ and inserted the following in the footer but it's not working:

include 'http://www.site.com/cgi-bin/vb_counter/googletrax.cgi';

I know I can do this with logs but any help on some code and exactly where to put it would be great?


Edit: I also found the following script but have not installed it. Anyone got it running with this hack and perhaps where you stuck the include? :

http://www.googlestats.com/english/

My guess: The "dance" will start 3/3/03

[SkuZZy]

Just a shout to everyone..... the dance hasnt started yet. I dunno what the hold up is. Should happen really soon though. I can't imagine it not happening within' the next couple of days. I will post here when it starts!

[FlyBoy73]

Skuzzy,
Could that also have an effect on when our new results show up on google? We got ours installed before the Jan/Feb dance and still don't see any new results.
Thanks,
David

[FlyBoy73]

"Mr. Google" is saying it will start within 48 hours.
http://www.webmasterworld.com/forum3/9955.htm

[SkuZZy]

Google is dancing everyone! To check out if your site got indexed, go to:

http://www3.google.com

Then, type in:

site:www.yourdomain.com yourdomain

For instance, I type in:

site:www.battleforums.com battleforums



...... Dont just type in "site:www.yourdomain.com" ... you need to add the additional "yourdomain" after it. This will tell you how many pages got indexed. Me personally, I went from 5500 to over 17,000. I'm shocked ;)

Great update so far! Good luck all!

[FlyBoy73]

AWESOME SkuZZy! I am showing 76 pages myself right now and that is a lot more than I had before. I am sure it will go up each month like you have stated.
I do have a question though. I seem to have a lot of printthread vs archive links that were also not there before? Heck, I won't argue with it. I'm just thrilled to be getting some love from google, thanks to your wonderful hack.
Cheers,
David

[FlyBoy73]

SkuZZy,
When I type in "archive" in place of the domain name, after site:www.yourname.com archive , I get 98 pages more. Does this work as well in determining how much content google indexes?

[SkuZZy]

Originally posted by dFrisk
SkuZZy,
When I type in "archive" in place of the domain name, after site:www.yourname.com archive , I get 98 pages more. Does this work as well in determining how much content google indexes?

You are showing a massive increase in pages: You went from 555 last month, to over 23,000 this month. You will notice and insane increase in hits im willing to bet. Congrats!!!!

http://www2.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=site%3Awww.prisontalk.com+prisontalk

[FlyBoy73]

:) Again, I can't thank you enough. For the hack and for the help you gave me installig it.
David

[Overgrow]

Even with increased traffic, you have to now turn those guests into members-->

https://vborg.vbsupport.ru/showthread.php?s=&threadid=49721

(just an info-post about improving the look of your archive for search results)

[insanctus]

How long does it take them to add a site?

[cklaszlo]

:(

we went from 19,900 to 10,200. Do we know why this happened or is it till being indexed?

Our site is very active and pages change daily. Also Googlebot was spidering our fourms not just the archive and those don't seem to have been indexed.

[SkuZZy]

Im showing you went from 8680 to 10,200. Where did ya see 19,900 from?

As far as the archive getting spidered..... eh, over 5910 of your pages are from the archive:

http://www2.google.com/search?q=site:www.thrillnetwork.com+skuzzy+thrilln etwork&hl=en&lr=&ie=UTF-8&oe=UTF-8&safe=off&filter=0

[cklaszlo]

My bad. sorry. I did a basic search for

thrillnetwork

That stayed at 19,900

What about the issue where the live forums were being indexed? And that they aren't appearing in google?

[cnczone]

How do I fix this http://www.cnczone.com/archive/

[cnczone]

Help!!

[ImportPassion]

what i had to do was hard code the path to the header and footer instead of using the variable. worked great.

Derek

[NexDog]

I only put the archive up a week ago and Google's Freshbot did crawl it. However, I'm pretty sure Google is always a month behind as we see loads of pages indexed from our old site.

Google is still dancing though. I'm seeing some of our placements jump around from page 1 to page 3 and back again.

[FlyBoy73]

Has google "danced" on anyone's site yet this month? I have yet to see any of them on my sites yet. Sure they are busy elsewhere..

[Craigr]

Works great for me.

Thanks
Craig

[Guidster]

Has anyone else noticed that this month Google indexed the forums as well as the archive. I saw it spider the forums and sure enough, the forums got it in addition to the archive.

With the site just 3 months old, I wound up with over 1100 entries for my site. The conversion rate from the archive is not what I would like, but I am seeing them make their way to the real board from the archive about 90% of the time.

Great hack!

[SkuZZy]

Originally posted by Guidster
Has anyone else noticed that this month Google indexed the forums as well as the archive. I saw it spider the forums and sure enough, the forums got it in addition to the archive.

With the site just 3 months old, I wound up with over 1100 entries for my site. The conversion rate from the archive is not what I would like, but I am seeing them make their way to the real board from the archive about 90% of the time.

Great hack!


What conversation rate would you want? Your forums only have 900 threads... remember, this hack is for threads, not posts. So if your forum has 5000 threads and 100,000 posts, then don't expect to get 100,000 threads in google... expect to get 5000 ;)

[Chruser]

Yay, my site got relisted. :)
http://www.google.com/search?q=Zelaron&ie=UTF-8&oe=UTF-8&hl=sv&lr=

[SkuZZy]

Originally posted by Chruser
Yay, my site got relisted. :)
http://www.google.com/search?q=Zelaron&ie=UTF-8&oe=UTF-8&hl=sv&lr=

:)

[Mickie D]

this is fantastic :)

also i have clicked install :)

is there anyone who has made a better template want to share with me :), i dont know what file to edit to make it look nicer :)

P.s and also do all i need to do is add the url

www.mydomain.com/forums/archive

to google to get it do the whole site or does evey page need to be added ????

please let me now :)

[Mickie D]

also can anyone tell me why it takes so long for the pages to change ????

is it ment to take long and would that stop the spider from going through the whole lot ????

[Mickie D]

scrap that last bit i re-uploaded the index.php and it seems fine for now :)

but still interested in making this look better :) can someone tell me what file to edit please or maybe if i am lucky give me a copy lol

also i need to know the answers to the submitting ... do i need to submit every page or just the url i said earlier ???

[CityInet]

does this work on windows?

[SloppyGoat]

Yeah, has anyone actually done this with IIS? I'd love to know.

[nintendo]

Yes, it works allright!! The 'Unknown Location' members are the Googlebot eating the 'HTML version' of the site thanks to this script.
http://www.vgchat.com/html_version/
https://vborg.vbsupport.ru/

[Craigr]

Anyone got a solution for editing online.php so it shows up correctly rather than unknown location.

I've added a few locations using Chens Custom Online Locations hack, but there are thousands of posts in the archive.

Thanks
Craig

[Craigr]

Today at 05:48 AM nintendo said this in Post #447 (https://vborg.vbsupport.ru/showthread.php?postid=366507#post366507)
Yes, it works allright!! The 'Unknown Location' members are the Googlebot eating the 'HTML version' of the site thanks to this script.
http://www.vgchat.com/html_version/
http://www.world-of-nintendo.com/nintendo/googlebot_html_version.gif

How do you know it's a google bot?

Craig

[Alien]

I have this exact same problem as this guy, which seemed to go without an answer. I'd love to get this up and running, help! :)

I have everything installed that I need to have, and in my error log I receive the following:

[Sun Mar 16 23:05:18 2003] [alert] [client *.*.*.*] /pathtopublic_html/forums/archive/.htaccess: ForceType not allowed here

One difference I have is we have a subdomain for forums, so I use http://forums.aliensoup.com/archive

Will this stop this from working properly? Thank you to anyone who can help provide answers!

-Jason

01-20-03 at 08:17 PM Geographic said this in Post #237 (https://vborg.vbsupport.ru/showthread.php?postid=344210#post344210)
HELP

So i got .htaccess issues.

What do I do about it?

I should go to the httpd conf file and change what exactly?

mod_mime and mod_rewrite are installed
with these commands:

LoadModule rewrite_module modules/mod_rewrite.so
AddModule mod_rewrite.c

duplicate for mod_mime


mod_mime_magic is disabled... Don't know could that be a problem?

#AddModule mod_mime_magic.c

I have my own server but I'm not totally familiar with it.
Can anybody step by step me through that?

Red Hat 6.1 running Apache.

Can someone show exactly what they changed to get it to work when they had .htaccess problems?


would leaving links like this:

forums.com/forum/archive/forum.php/7.html

hurt my chances with the googlebot?


Would someone uploading a working conf file for comparison be
against apache's rules?

[Alien]

Any ideas guys? :D

[nintendo]

:::How do you know it's a google bot?

The IP address.

freshBot: 64.68.82.* bah, listed for only a few days, short dinner date then it dumps you.
deepcrawler: 216.239.46.* Good bot, it likes you, listed until death do you depart. So don't make him mad or it will divorce your site. You can divorce the Googlebot by using your robot.txt file. It's much cheaper and faster than going to the courts.

It almost crashed my server once, taking the server load up to 24.56, 10.49, 5.14 slowing the server down for about five minutes!!

http://www.world-of-nintendo.com/nintendo/server_load.gif

[cklaszlo]

Anyone got a solution for editing online.php so it shows up correctly rather than unknown location.

I've added a few locations using Chens Custom Online Locations hack, but there are thousands of posts in the archive.

Thanks
Craig

I'd be very interested in this too. Anyone have a good solution?

[Alien]

Hmm everything seems correct, I'm not sure what could be causing this but would really like to get going before the next dance... Help! Make work! Fix! :D

[ph1`]

google is all over my archives now with 50+ bots, wonder if i'll get listed this time, last dance i didnt.

[Alien]

You still around, SkuZZy? :D

[Alien]

I guess not. :P

[Alien]

I'm gonna miss the dance ain't I? First the prom, now this! :)

[SkuZZy]

03-26-03 at 01:23 AM Alien said this in Post #458 (https://vborg.vbsupport.ru/showthread.php?postid=373345#post373345)
I'm gonna miss the dance ain't I? First the prom, now this! :)


The dance is coming soon, and the deep spider won't be for a week atleast.

[Alien]

SkuZZy: Any idea what the problem is in this post?

https://vborg.vbsupport.ru/showthread.php?postid=367916#post367916

I've tried everything. :(

[cklaszlo]

We've been getting crawled by:

inktomisearch.com

We've been getting indexed with them for 2 months now. But I haven't seen an increase in pages listed in their DB. Does any one have any info about inktomi and their indexing practices.

Also FYI, Google is doing another major crawl of our archive!!!

[tpearl5]

I need help QUICK

I just changed servers and aparently this machine handels subdomains differently.

Before everything work work find at http://board.midibuddy.net/archive2/

Now it doesn't. The archive does load at http://midibuddy.net/board/archive2/ however.

I NEED the URL to be the first one since this is the one that google indexes. Please tell me there is a way to change this???

[cklaszlo]

??? Howdy.

Seams that the discussionhere has dwindled.

Anyway our pages have been removed from google.

http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=site%3Awww.thrillnetwork.com+skuzzy&btnG=Google+Search

Where as this search used to bring up all the Archive pages now only produces one link.

Anyone else experience this?

[Elmer]

* Elmer click install and ask -'could you please add my archive to the demo list?'-
http://www.2spoemas.com/foros/archive/

:)

[Xenon]

/me agrees and added it to the list

[Xenon]

* Xenon agrees and added it to the list

:)

[esfron]

Thx

[jw791]

I installed it - easy! Works great - thank you!

http://www.talkpets.com/forums/archive/

Question tho'... why cant I see these html archive files in ws-ftp?

[Xenon]

Today at 23:19 jw791 said this in Post #468 (https://vborg.vbsupport.ru/showthread.php?postid=383906#post383906)
Question tho'... why cant I see these html archive files in ws-ftp?


because those html files don't exists really, they are redirections :)

[jw791]

Today at 02:29 PM Xenon said this in Post #469 (https://vborg.vbsupport.ru/showthread.php?postid=383912#post383912)
because those html files don't exists really, they are redirections :)

OK, sorry for my ignorance, but when I run index.php where are they? Are they created on the fly?

[Xenon]

if you call showthread/1-1.html .htacces redirects it to showthread. with parameters threadid1 and page 1.
now the page is generated dynamically :)

[Littlebit]

Hi :) I'm interested in your hack, but am curious about the .htaccess file...is it no longer neccessary with your hack or is it still... and if so, is it secure?
Thanks :)

[Xenon]

it is neccessary, and why should it be insecure?

just look into the code to see there is nothing harmful in it.

[Littlebit]

Hi Xenon :),
I just thought that having a .htaccess file in user directories can be like extending some of your Webmaster privileges to anyone who can edit those files.
I could look at the source code, but don't know much of anything about the strings so I wanted to ask you...
Hey, I downloaded your hack but the zip did not even contain a .htaccess file.
:surprised:

[Xenon]

look at the attached screenshot, the first file in the archive is clearly named as .htaccess

if you don't see that file you either haven't downloaded the zip correctly, or you are using a crappy linux OS which doesn't show you that file

[Alien]

I just wanted to extend a thank you for this hack.

I got hammered hard in the last week and a half by google, and inktomi both (and luckily seperately lol).. Real hard heh. As many as 55 bots on at once (and that happened several times), thanks so very much. :D

-Jason

[Littlebit]

Xenon,
I see it now...I hadn't realized I had an option to turn on/off view of hidden files. .htaccess was a hidden file. Clicking install :) Thank you

[Xenon]

:)

enjoy it :)

[Aurore]

Very goog indeed. But I've renamed forum=>forum.php - topic=>topic.php to make it work since my host does not uses mod_Rewrite.

http://forums.aurore-france.net/archive/

[Guidster]

Man! What is up with Google this month? I got the notification of the dance and I think my jaw hit the floor when I saw a whopping 28 listings in the www2 index. I hope that things correct themselves or it is going to be a SLOW month!

[cklaszlo]

ouch. Our listings dropped too. From like 1900 to 568. I hope this gets better too. Does any one know why this happens?

[FTDave]

Hi.
This looks like a very good hack. I edited my config.php file. I uploaded all files in Ascii. When I try to access the archive directory I get the famous 500 internal server error. Where do I start to see what could be wrong? Could it be that .htaccess is not right on the server? Thank you.

[FTDave]

update.
I was curious so I removed the .htaccess file. MY 500 error went away.
It will show the very first page but when I click on someting it says page not found.

I guess I need the .htaccess file. :)
Anyone have any idea why I'm getting this error.
I ahve access to anything on the server if you have any suggestions for me.
Thanks.

[FTDave]

hmm?
I checked my server error_log and this is what it says.

[Sun May 11 08:33:49 2003] [alert] [client 205.203.167.5] /usr/local/apache/htdocs/forums/archive/.htaccess: ForceType not allowed here

What does this mean?

[Xenon]

it seems you are not allowed to use .htacces on your server..

[FTDave]

Hi. It is my server. Do I need to do something in hte httpd.conf file to fix it?
Thanks.
Dave.

[FTDave]

You're absolutely right. I checked my httpd.conf file and changed it to allow this then rebooted apache.

Thanks.

[Xenon]

:)
welcome :)

[version2]

Im getting a 404 error.

http://www.mysticwicks.com/archive/forum/2-1.html

And in the error log...

[Fri May 16 16:10:39 2003] [error] [client 205.240.74.14] File does not exist: /var/www/html/mw/archive/forum/2-1.html, referer: http://www.mysticwicks.com/archive/

:) What did I do wrong?

[version2]

Oh...and mod_mime is installed. :)

[version2]

Egads...is anyone still using this hack? Or am I the only one who has any problems left? :D

[version2]

[i]01-22-03 at 05:46 PM NTLDR said this

As for Apache2 Error 404s I don't know and am no longer bothered about ;) [/B]

Did you get rid of Apache2? That is what Im using.

[version2]

Well, THANKS for your help! :D

Just kidding. I fixed it and it now works with Apache2. You have to add this to your http.conf:

AcceptPathInfo On

Add this where you put all your AllowOverride stuff or in your VirtualHost paramaters. Enjoy folks.

http://www.mysticwicks.com/archive/

Now I just have to pretty it up.

CLicks INSTALL.

Please add me to the list, Xenon!

[Xenon]

added

[julius]

Before I installed this hack I had 200 pages in google.
After I installed the hack I had for 2 months 14.000 pages.
Now I have onlly 280 pages!
Why? :(

[Guidster]

If you look back a little bit, I posted that Google is flaky this month. Almost everyone that is listed is in an uproar. I do not have a single liting from my archive in the listing this month. I went from 2000 pages to 28. Gonna be a slow month I am afraid. I am ready for the next deep crawl to see what happens.

[cnczone]

Me to what is going on?

[cnczone]

How do you know when your being spidered?

[Xenon]

you can see a lot of googlebots in your who's online ^^

[Guidster]

There is also an application that is available from the Google Dance/SEO Chat folks called Google Tracker that can be installed to trace the path of the bot through your site...

There is always the logs to parse through as well.