vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)
-   -   vB Easy Archive FINAL - Search Engine Spiderable Hack! Get your posts listed @ google (https://vborg.vbsupport.ru/showthread.php?t=47087)

NTLDR 01-21-2003 09:41 PM

Has anyone got this going under apache2? I've got mod_mime loaded and the index loads but I get a 404 error for the rest of the pages. Apache2 doesn't seem to like stuff like this, I can't get it to protect directories either :(

SkuZZy 01-21-2003 11:18 PM

Quote:

Originally posted by NTLDR
Has anyone got this going under apache2? I've got mod_mime loaded and the index loads but I get a 404 error for the rest of the pages. Apache2 doesn't seem to like stuff like this, I can't get it to protect directories either :(
As far as I know, old versions of apache don't seem to work with this. I'm not sure why, it would seem they would. I'll look into it more.

Ember 01-22-2003 02:05 PM

You can change http://a51gaming.madsims.net/forum/archive

to

www.a51forums.com/archive

I am just uploading it now :) Thought I should notify you of the change!

Ember 01-22-2003 02:16 PM

Change of plan, it wont work for some reason.

Can I get me a copy of the old one?

NTLDR 01-22-2003 03:48 PM

Quote:

Originally posted by SkuZZy
As far as I know, old versions of apache don't seem to work with this. I'm not sure why, it would seem they would. I'll look into it more.
I had this running on Apache/1.3.26 fine before, I've now switched to a dedicated server with Apache2, the archive wouldn't work at all, I allways got Error 404 for forums and topics, and I couldn't seem to protect directories either.

So I've removed Apache2 and installed Apache/1.3.27 and now I get No forum/thread specified. Again mod_mime is installed and everything else seems to work well.

Altarion 01-22-2003 06:33 PM

Hey scuzzy.... Great hack! If you do a update tho, I suggest (a) changing to this
chdir("../");
require("global.php");

And (b)modifying it so it will display categories in multi-teir layouts, like mine and a few others here (DON"T YELL AT ME, I KNOW HOW I CAN DO IT ONE WAY, but i have co-admins who don't always pay that much attention)

Thanks :D

NTLDR 01-22-2003 08:46 PM

Quote:

Originally posted by NTLDR
So I've removed Apache2 and installed Apache/1.3.27 and now I get No forum/thread specified. Again mod_mime is installed and everything else seems to work well.
Worked that one out ;) This doesn't work with register_globals = off.

In showthread and forumdisplay change:

PHP Code:

$REQUEST_URI 

to:

PHP Code:

$_SERVER['REQUEST_URI'

Then it will work :D

As for Apache2 Error 404s I don't know and am no longer bothered about ;)

Guidster 01-23-2003 03:53 AM

Can you experts out there clue me in as to why I would be listed in Google for the first time about 10 days ago and now I am no longer in the index at all?

The URL that was referenced was the one that I had originally submitted--not the archive one that Google gorged on at the beginning of the month. I just find it odd that I would be there and then suddenly absent. I hope that the archive didn't PO them!

SkuZZy 01-23-2003 03:56 AM

Quote:

Originally posted by Guidster
Can you experts out there clue me in as to why I would be listed in Google for the first time about 10 days ago and now I am no longer in the index at all?

The URL that was referenced was the one that I had originally submitted--not the archive one that Google gorged on at the beginning of the month. I just find it odd that I would be there and then suddenly absent. I hope that the archive didn't PO them!

What you saw was everflux. Your site was never actually in good, they just spidered you and you showed up in their search for a week or so, then disappeared. It happens to all new sites being added. Wait until the end of the month (about one week or less) and all your pages should be added when the "google dance" happens. That is when good updates all their links and adds new pages they indexed throughout the month. It definately was not the archive that did this ;)

Guidster 01-23-2003 04:03 AM

Quote:

Originally posted by SkuZZy


What you saw was everflux. Your site was never actually in good, they just spidered you and you showed up in their search for a week or so, then disappeared. It happens to all new sites being added. Wait until the end of the month (about one week or less) and all your pages should be added when the "google dance" happens. That is when good updates all their links and adds new pages they indexed throughout the month. It definately was not the archive that did this ;)

Whew! Thanks, Skuzzy! I didn't think that it was, but I am new AND a little paranoid I guess!

Rand M 01-23-2003 08:43 PM

Thanks for a great hack SkuZZy... installed and running :)

http://www.toyotaimportsforum.co.uk/forum/archive/

One more for your list!

FlyBoy73 01-25-2003 12:19 AM

SkuZZy,
Thank you so much for a great hack and all your personal help installing it. I can't wait for the dance..
Please feel free to add me to the list also if you would like.
Thanks,
David

SkuZZy 01-25-2003 12:24 AM

Quote:

Originally posted by Ember
You can change http://a51gaming.madsims.net/forum/archive

to

www.a51forums.com/archive

I am just uploading it now :) Thought I should notify you of the change!

Congrats on the domain name!!! I updated your URL. I also added 3 other new URL's to the list ;) If anyone has an archive not on the list and wants to be added, PM me. Never can have too many demos ;)

Guidster 01-25-2003 04:01 AM

Quote:

Originally posted by SkuZZy


Congrats on the domain name!!! I updated your URL. I also added 3 other new URL's to the list ;) If anyone has an archive not on the list and wants to be added, PM me. Never can have too many demos ;)

You can add mine if you like:

www.allthingsmoto.com/archive

SkuZZy 01-25-2003 04:09 AM

Quote:

Originally posted by Guidster


You can add mine if you like:

www.allthingsmoto.com/archive

Added sir :D

smestas 01-25-2003 08:57 AM

SkuZZy,

I was having problems with v2 and a blank page but I just installed the new script v2.3 and it works great.

Thanks!

SkuZZy 01-26-2003 08:25 AM

Google is dancing! For those of you who installed this hack in early to mid-janary, check to see if google spidered your pages. There is a simple way to check:

1. Go to http://www2.google.com (notice the www2).

2. Type in: site:www.yoursite.com yoursite. (So for instance, I would type in "site:www.battleforums.com battleforums")

3. Check to see how many pages are listed. I notice some sites have alot of pages listed. Thrillnetwork has over 8000 pages indexed for example (not all of them from the archive).

BE SURE TO POST YOU RESULTS! :D

SkuZZy 01-26-2003 08:39 AM

Some interesting statistcs, for those who doubt the power of this hack. Over 29,000 archive pages have been spidered in the first month alone! Search for Archive SkuZZy Xenon in google and see the results!

http://www3.google.com/search?hl=en&...e+SkuZZy+Xenon

floridaideal 01-26-2003 01:50 PM

Hi Skuzzy, I just added my link to the index on the site and also submitted it to Google.

Do you think I will get a chance of spidering? or is it too late?

By the way do you think my link is ok on my index page http://www.top-forums.com (its right at the bottom of the page)

Also the link to the http://www.top-forums.com/forum/archives/

Does that look ok to you guys? Thanks all this is a great hack, I could not get vbspider to spider on google so I hope this one works better for me.

Thanks

Stuart

Guidster 01-26-2003 02:49 PM

Quote:

Originally posted by SkuZZy
The dance is usually at the end of the month, often between the 25th and 30th and the "major crawl" happens 5 days after the dance BEGINS (approximately). So if the dance started on January 28th, the major crawl would probaly begin around the 3rd.
I saw the first deep crawl last evening into my archive. It would appear that they have been dancing all week and have begun the deep spidering.

Guidster 01-26-2003 03:19 PM

Quote:

Originally posted by SkuZZy
Google is dancing! For those of you who installed this hack in early to mid-janary, check to see if google spidered your pages. There is a simple way to check:

1. Go to http://www2.google.com (notice the www2).

2. Type in: site:www.yoursite.com yoursite. (So for instance, I would type in "site:www.battleforums.com battleforums")

3. Check to see how many pages are listed. I notice some sites have alot of pages listed. Thrillnetwork has over 8000 pages indexed for example (not all of them from the archive).

BE SURE TO POST YOU RESULTS! :D

My site was only a couple of weeks old when I added this hack and got in before the last dance. Still, this hack has allowed Google to index over 300 pages of archive content and is currently deep crawling it again this month. This is great and is really going to help get my fledgling board off to the races! Thanks, Skuzzy!

Hey, Skuzzy--What is the sequence for the data at www2 to be added to the www index? Cudos for figuring all of this stuff out! I have also heard of a tool that can display the PR of a page. Is this a worthwhile thing to have and if so, where would one get it?

kuska 01-26-2003 04:09 PM

Does anyone know the dates of google dances? I had one google spider visit my forum yesterday but it did not pick up the link to my archive : \
I have the link to my archive visible but in my footer. Inktomi Slurp picked up on the link and was all over my archive forums but not Google. Any ideas?

Guidster 01-26-2003 05:51 PM

Quote:

Originally posted by kuska
Does anyone know the dates of google dances? I had one google spider visit my forum yesterday but it did not pick up the link to my archive : \
I have the link to my archive visible but in my footer. Inktomi Slurp picked up on the link and was all over my archive forums but not Google. Any ideas?

This is a good sign. I saw the exact same thing earlier this month. At first they just nibble at it, then they really get after it. Watch for lots of activity in the coming days.

NTLDR 01-26-2003 06:23 PM

Seems to have worked pretty well, I think it has indexed every viewable thread by guests up till the time it spidered which is pretty good indeed :D

Quote:

Results 1 - 10 of about 979. Search took 0.23 seconds.

subduck 01-26-2003 06:34 PM

This is the coolest hack ever!!!!!!!!

John 01-26-2003 06:44 PM

"The Google Dance" has indeed started - the 7 servers are currently out of sync.

Check progress here: http://www.google-dance.com/

subduck 01-26-2003 06:46 PM

lol! It looks like you guys don't need to worry about people backward links to your site! you have hundreds now!

NTLDR 01-26-2003 09:33 PM

I've just taken a look at my server stats and I've had 85 visitors come in via Google in the last 6.5 hours, thats about 14 an hour and the weekend is pretty inactive for me, this hack seems to have done a great job :D

SkuZZy 01-26-2003 10:00 PM

wow, i'm glad to hear all the success that everyone has had. The www2 server is what google uses to test out their new index, before they have it go "live". The test lasts 5 days and then the www server is fully updated. That is when the dance is considered "finished". Then the "big spidering" begins and it can last anywhere from 5 days to 3 weeks.

Smoothie 01-27-2003 02:13 AM

Quote:

Originally posted by SkuZZy
Google is dancing! For those of you who installed this hack in early to mid-janary, check to see if google spidered your pages. There is a simple way to check:

1. Go to http://www2.google.com (notice the www2).

2. Type in: site:www.yoursite.com yoursite. (So for instance, I would type in "site:www.battleforums.com battleforums")

3. Check to see how many pages are listed. I notice some sites have alot of pages listed. Thrillnetwork has over 8000 pages indexed for example (not all of them from the archive).

BE SURE TO POST YOU RESULTS! :D

Results from Macfora.com;
Results 1 - 10 of about 6,490. Search took 0.22 seconds

Smoothie 01-27-2003 02:16 AM

Is it wise to leave links in your header/navbar in your archive page? The reason I ask is because I understand that google starts at the first link it sees and works down from there. If it crawls those links, will it stop? Does anyone know?

@SkuZZy-

Have we found a way to eleiminate that blank page we get when clicking the category link?

SkuZZy 01-27-2003 02:25 AM

Quote:

Originally posted by Smoothie
Is it wise to leave links in your header/navbar in your archive page? The reason I ask is because I understand that google starts at the first link it sees and works down from there. If it crawls those links, will it stop? Does anyone know?

@SkuZZy-

Have we found a way to eleiminate that blank page we get when clicking the category link?

The blank page when clicking a category would probably be fixable, but I haven't bothered to fix it because it doesn't really matter whether or not it works. The bottom line is, this archive works. It's by all means not a "pretty archive" but it gets the job done and it gets pages spidered. Google doesn't start at the first link it sees and goes from there... google will spider all links. Regardless of what is at the top. Don't worry about the first link being a dead one, google really doesn't care.

SkuZZy

SkuZZy 01-27-2003 02:27 AM

Quote:

Originally posted by Smoothie
Results from Macfora.com;
Results 1 - 10 of about 6,490. Search took 0.22 seconds

Holy mother. Congrats, that is a great amount of pages! You need to remember, google only allows so many pages to be indexed per cycle. So even though you have more thn 6000 threads, that's all google is gonna give you this time around ;) The next month, another 6000, ect. This is to prevent googleboe from being overloaded, as spidering 50000 page sites isn't exactly easy to do. Please let me know if your hits increase. Right now google is still dancing, but watch in the next 3 days and see after that if your hits increase, they are bound to!

Smoothie 01-27-2003 02:42 AM

I noticed that past few days, google has been very active on my site.

And about the dead links in the navbar, yes, that was a big concern. Someone told me that if google tries to spide these first, there could be a problem, but judging from my results, I see that is not the case.

And yes, we get some forum members who actually go the archive to read it, thats why I asked about the dead category links.

BTW, here's my seach link;
Macfora.com

SkuZZy 01-27-2003 03:18 AM

muhahaha, smoothie. You are going to get tons of hits once google is done dancing. That is some spider you got! Congrats.

And now on a sad note, i've released the final version of this hack. I suggest everyone download it, because it's the final version... and it's stable. Hopefully this hack shall live on forever. Sorry to everyone for not getting around to adding cool new stuff, like the category thing mentioned above. Just no time lately and i've got to stop doing some stuff and with vb3 coming up, there realy is no need to continue this archive. All the die hards who stay with vb 2.2.9, this hack will be for you ;)

SkuZZy

Smoothie 01-27-2003 03:21 AM

Is there any need to upgrade if the version we use works? What is new about this final version?

SkuZZy 01-27-2003 03:37 AM

Quote:

Originally posted by Smoothie
Is there any need to upgrade if the version we use works? What is new about this final version?
It won't hurt to upgrade. Just upload all files EXCEPT config.php and nothing will change at all, atleast not that google will know of
. Nothing much changed, just cleaned up the code a bit and changed the copyright around. Everyone should upgrade though, just because... final versions rock :D

Smoothie 01-27-2003 03:59 AM

@SkuZZy-

many thanks for this hack!

subduck 01-27-2003 06:54 AM

When we upgrade to vbulletin3, will our forums history in google still redirect to the proper pages? or will they all be dead links?

SkuZZy 01-27-2003 07:07 AM

Quote:

Originally posted by subduck
When we upgrade to vbulletin3, will our forums history in google still redirect to the proper pages? or will they all be dead links?
Good question. I'm assuming the old links will work. The URL hasn't changed in vb3... it still looks the same.


All times are GMT. The time now is 07:40 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02573 seconds
  • Memory Usage 1,848KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_php_printable
  • (17)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete