hi,
something similar exists in vb 3.5 extensions, but does supports 3.x as well, can you compare features of both these hacks ?

I'm afraid I can't since I can't get this hack working on my forum right now, if it's running here I'll post my experience and such for you.

I really really like this idea, but is the below a concern? I do not want to do negative thing for my forums.

http://www.google.com/webmasters/guidelines.html

301 Redirect is Google preferred way. Google penalize web site with duplicated contents. So if you have two URLs showing the same contents, Google prefers you redirect from one URL to the other. Have both will attract penlty.

What we do here is not sneaky. We have the actual contents, we just want Google to show one version of it. We do not want Google to give us higher page rank than the pages actually worth, we just want Google to index the actually contents, instead of looping through the endless internal links.

I moved my forums to a new domain a few weeks ago (just before I released this hack). There are so far 150,000 pages indexed by Google already. Without sitemap Yahoo has only indexed just over 20,000 pages.

As the error suggests. The permission is denied in the '/home/ftpusers/otfans/html/forums/' directory. That is the base vB directory. Where I described very clearly as "the same directory where you find showthread.php etc.".

People keep asking the directory name to change. I do not know, because that can be anything. In your case it is 'forums', others may be 'public_html'...

hm, with vb 3.5 rc3, I get the following errors when accessing the forums_sitemap.php:

Any ideas? :confused:

No idea, I have upgraded to RC3. The hack works without any further modification.

If you have changed the php file, make sure your upgrade does not copy over them. I did not copy new files to 'archive directory'.

Hey lierduh,

Thanks for your hacks over the years, always fine jobs :)

I'm another one wanting instructions for the archive/index.php and archive/global.php

I've applied the rest of the instructions, and all appears to be working fine. And I do get the concept of removing the PDA crud, and redirecting humans out of the archive... but I'm a little lost with the diff output you've supplied... a more primitive ><+/- lines would've confused me less ;)

If you do ever find time to update the instructions, it will be very appreciated by quite a few of us. And I realise how much of a pain that is, as I'm supposed to be porting my own hacks and really don't like the idea much.

Cheers

DavidK

Hello DavidK,

The reason it has confused you is my diff were based on the RC2 and modified RC2 files. I presume now you have got RC3 files.

I have attached the diff between RC3 and modified RC2 files (this will confuse everyone else, but not you:)). Otherwise, if you still have the RC2 files, then the coloured diff will make a lot of sense.:)

Much better :D Thanks :)

And yes... it will confuse everyone now ;)

i can't seem to get the files to be created :(

from the instructions, which where just about to much info for me..
i upload the forums_sitemap.php to arcives then chmom the arcives folder 775,
then make the sceduled task, run it, and the files should be made..
what part am i missing ? thankyou..

at rc3 now.

I'm also still having that errors, weird =/ lierduh, could it be that you've tested it with PHP4 rather than PHP5?

I do not have php5 to test.

Instead of calling the script directly, have you tried using the Schedule Task's "Run Now" button?

thank you very much! I just started trying to do this my self but found this :D yay!

Yes, that results into for like thousand times.

Weird thing is that I'm 100% sure that I chmodded everything correctly. (It's on a debian host, if that is relevant)

That means the script can't write to archive directory. What is the persmission for this directory?

Right now it's 0777 and it's working, I thought I already changed it some days ago, it was 0775 (which should be enough as far as I know)

Thanks for your support ;)

OK I have this installed and it seems to be working as described. When viewing who's online I can see the search engines looking at threads with url's similar to the following.

/archive/index.php/t-6044.html

When clicked by a human user they are redirected to

/showthread.php?t=6044

So I can only assume this works since all the spiders on the board are seeing the archived version, and when users click they are taken to the full version.

I do have a couple of questions however, since I am not too familiar with Google Sitemaps. Does the script automatically upload the sitemap to Google without any further action aside from making the Scheduled Task? I have made the task in the manager and run it (every day at 1AM), and it has created the files ( [xml] in forum root and specific [gz] forums in archive folder).

[upon further thinking, would I be correct in saying I need to let google know about the xml file in the root of the site?]

What is the affect on other search engines? I see yahoo, msn, ask, and others viewing similar archives, so I assume the affect is similar to what is happening with google, but they are not getting a map.

Last question, does this basically mean that other SEO hacks are not required, since the spiders will never see the rewritten urls anyhow?

Allot of assumptions up there. :ermm:

Oh and one last thing, I do use mod rewrite on my server for many sites, and have had no issues, but the command you say to enter to resolve the index.php issue seems to bog the server, making any urls that point directly to it, as in /index.php, not load. I suppose this could be a conflict within my htaccess file, but not too certain where to start looking. (however, I did try it with only the codes you provided (and RewriteEngine on) and have the same problem.

Thanks for your time and the hack.

Each time the script is run.

1) It re-generates all the sitemaps. Makes sense because you have more threads/posts now.

2) It notifies Google about new sitemaps being available. You will notice Google fetches these files soon afterwards.

If you have the scheduled task logged. The end of the log is the response sent by Google. It should say:

======================
Sitemap Notification Received

Your Sitemap has been successfully added to our list of Sitemaps to crawl. If this is the first time you are notifying Google about this Sitemap, please add it via http://www.google.com/webmasters/sitemaps so you can track its status. Please note that we do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about when or if they will appear.
======================

One thing to remember is under your Google sitemap account. The 'last submitted' does not reflect the auto ping/submit. It only logs the manual submit you do by push the button at Google sitemap account.

Other search engine do not accept sitemaps as far as I know, at least not using Google's sitemap format. The redirects however works for all the major search engine which I believe benefits the indexing.

I do not recommend using SEO at least for existing sites. The chances are Google has already indexed part of your forums using links like /showthread.php?t=12345. Now if you rewrite all the URLs, Google will have two copies of the same contents for that thread. (one with the traditional URL, one from your new rewrite URL). This will lead Google panalizing your site ranking. Some smarter SEO scheme redirect your old URL to the new one does not suffer this, but it becomes a very complicated add-on. It may break every time a mojor vB version is released. I elect not to use such scheme. For the record, I used URL rewrite SEO back in vB2 era. In my .htaccess, I still need to redirect my old rewritten vB2 URLs in fear of Google penalizing my site. Basically the vB archive is very static, it was designed for SEO in the first place anyway. Think about how many clickable links a normal showthread brings to you, it becomes a mess for search engines no matter how smart your SEO is.

For index.php redirect, my working version is:

RewriteEngine on

#...

RewriteCond %{QUERY_STRING} ^$
RewriteRule ^index.php$ / [R=301,L]

If it does not work for you, I would check the http logs. Failing that, log your rewrite! (you need to do this in your http.conf, consult apache manual for log level etc.:))

Thanks for the reply, that clears up alot... I had to verify site ownership via google, the logs for the cron showed exactly that.

One thing I notice however, while looking now, is that the google spider is viewing a few regular threads, while the google adsense spider is viewing the archive, also viewing the archive is msnbot yahoslurp and askjeeves.

Wonder why google is seeing a regular thread now.

Going to look into the htaccess in a bit.

vB detects search engine by the following string:
google|msnbot|yahoo! slurp (in the init.php file)

Any user agent matchines that will be detected as search engine.
So check your http access log and see what user agent does 'Google' use. It should say "Googlebot".

Looking at the logs I have 2 versions of Google coming up...

Googlebot/2.1; +http://www.google.com/bot.html)"
Google/2.1"

The first one looks legit, but why is the second not sending an origen? They both have the same IP.

/me installed

Thank you!

I'm not very clear about how to complete step 3. It says:

Where do I find the instructions to make the changes? :)

Does this work with the latest version? Anyone?

Thanks

Steps 1 and 2 worked great for me :D . I don't know how to do step 3 (optional)

Step 3 is actually pretty simple, but not with the instructions in the download, look at the attachments in this post here.

The lines preceeded by << mean remove, or comment out, and replace with the following lines preceeded with >> or add after.

will this hack bring spiders to my forum?

Ok I will give this a try. We do't have too mutch pages listed (642) and most of them are from pages with no real content (memberlist and such)

So I will install this and let you know what the results are in about 3-4 months.

Ty.

this is a very confusing hack :confused:
it's not working I guess, I ran the task and it did create the sitemap files from 36 to 81
but after that it gives me loads of errors, saying it couldn't create g_sitemap.xml and something like that
so I CHMODED the /html/ folder and tried again, it gave me this time an error donno what it is, so I refresh

is it working or no?

this is the error:

....

Hmz not easy....

I understand chmod 755 (it was actually already set to 777), but what do you mean with: '#chown apache.MYUSER_GROUP archive' and 'MYUSER_GROUP is the user group my login belongs to. #ls -l will show that' ? That's chinese to me !

PLease explain, thank you

The problem is no one wants to know/learn about the very basics of directory/file permission.

If you have 777, it means everyone can write to it. You won't need to bother with 'chown'. The example I provided was for someone who knows a bit more and wanting to have the more secure way. I will rewrite the instruction at next release and have one set of simple instruction for newbies. In the meantime, if you are still interested about directory permissions, please read:

https://vborg.vbsupport.ru/showpost....6&postcount=27

what about my error?

It means your web server could not connect to Google at port 80. It is possible Google was down at the time, or your web server's firewall prevents the script doing so. I just tried manually:

http://www.google.com/webmasters/sit...Fg_sitemap.xml

It returned:
=========
Sitemap Notification Received

Your Sitemap has been successfully added to our list of Sitemaps to crawl. If this is the first time you are notifying Google about this Sitemap, please add it via http://www.google.com/webmasters/sitemaps so you can track its status. Please note that we do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about when or if they will appear.
===============

You can always comment out those lines in the script and manually submit the sitemaps.

To those unsure about this hack, I would say persevere.

My number of Google spiders hasn't increased dramatically, but they're being far more efficient.

A month ago the number of pages I had in Google was only 66,000, now I have over 846,000 pages indexed: http://www.google.co.uk/search?q=site%3Awww.bowlie.com

It really is worth it, although the code edits in archive/index.php can be a bugger to get your head around at first.

I do. That's why I replied:
The more I can lurn , the better :)

thank you, I will look into it and try to setup the hack

BTW I created a google sitemap account after the task. Is this needed, or do these need to be joined or something? And how does the vb task send the sitemaps if the google account wasn't even setup?

Also:
What weblog is there in vbulletin, if you mean currently active users: i don't see it there.


Alot of newbie questions again :D

Ok I know chmod, and set my archive/internal files to 777 .. I still get

I also did my root as 750 ..

my direction setup is url/forum/index.php for forums, I changed your robots text to disallow: /forum/files/ etc

and I still get error above, I also setup my .htaccess etc,

I'm running vbulletin v3.5.0 gold.

any sugguestions?

i'm getting the same error!