Member Bots 1.0 - Allow bots to act as members

--------- Member Bots 1.0 ---------
Written by Trilljester - http://www.trilliummud.com

This small mod allows you to treat search engine robots (i.e. GoogleBot, Yahoo Slurp) as registered users instead of guests, as they are treated by default in VBulletin.

This is handy if your site allows limited access to guests, but full access to registered users. By default, a search engine robot would only be allowed to index what a guest could see, but with this mod, it would allow them to fully index your site as a registered user.

This mod uses the spiders_vbulletin.xml file for determining if a visitor is a bot or not, so make sure you have that file up to date. The one that comes with VB is pretty sparse, but it works. If you're only interested in GoogleBot, then the default file works great.

WARNING! This mod requires a very small edit to a VBulletin core file. This means whenever you upgrade to a new version of VBulletin, you'll need to re-do this edit, if you overwrite this file. If the idea of editing a VBulletin core file bothers you, don't install this mod! I won't be held responsible for your forum being fouled up. This mod is very safe however, so don't fret.

Install Instructions:

1. Unzip the product-memberbot.xml file on your system.

2. Go to your VBulletin AdminCP and click on Manage Products under the Plugin Manager section.

3. Click on Add/Import Product.

4. Click on Browse on the first line (Upload the XML from your computer) and locate the product-memberbot.xml file. Click on Import to upload it to the system.

5. Click on Plugin Manager, and ensure that the new plugin named Bot Checker, which is found under the init_startup hook, is enabled.

6. Now the fun part, you'll need to edit the core VBulletin file init.php which is located in the includes directory under your forum home.

For example, if your forum home is located in /home/blah/forum, then init.php is in /home/blah/forum/includes

If you can edit the file on the server without having to download it, more power to you.

In init.php, find this line (should be near the very bottom of the file, it was line 403 in mine):

if (!empty($db->explain))

Add ABOVE this line:

if ($is_bot == 1) {
        $vbulletin->userinfo['usergroupid'] = 2;
}

NOTE: My registered users group is ID #2. Yours may be different, you may change this to any group ID you like.

7. Save init.php and upload back to your server if you had to download the file to edit it.

If you're editing on the server, save it, and that's it!

================================================== ========
That's it! Search Engine bots should now "see" your forum as a regular user, not a guest.

Questions, comments, improvements are more than welcome. Please use this thread as a cental support site. It makes it easier for me to help out. Also, you may visit my site and fire me off a PM for help, or just send a quick thanks or shout out.

Enjoy!
// trilljester

[GamerJunk.net]

Amazing! Any plans to have them show as users on the memberlist like GoogleBot and YahooSlurp?

[AshokForums.com]

thanks for this excellent hack.. love it

[kall]

Originally Posted by Smiry Kin's

hmm i read some one doing this was agaisnt the bots agreements etc?

Highly unlikely.

If this is cloaking, having a Guest usergroup is in and of itself, cloaking.

If a registered user happens to come across the Google Cache of one of your pages, and goes to visit it, they see something different as well.

[trilljester]

Here's what Google wrote back to me:

Thank you for your note. We recognize your concern. Please be advised that
we don't personally review individual sites, nor do we comment on
webmaster techniques or the details of our search technology beyond what
appears on our site.

We've dedicated an entire section of our site to answering the most common
questions from those who maintain and/or promote websites. You'll find all
of our publicly available information information posted at
http://www.google.com/webmasters/index.html

I did some more research on their site, and nowhere in Google's definitions does this qualify as a "bad" practice. However, it could be loosely, and I mean, very loosely defined as cloaking, in the sense that 2 sets of content are being presented. However, you are not hiding content from anyone.

There are a lot of websites out on the net that require you to register or even pay to see content. Yet, they're indexed and served in Google. If this was a rampant violation of their terms of service, then they'd clearly state it. Again, it's your choic to use this mod or not. If you are extremely worried, then don't use it. Simple as that!

Onwards:

Amazing! Any plans to have them show as users on the memberlist like GoogleBot and YahooSlurp?

Well, on my site, when Googlebot shows up, it displays them as their true identity on the Who's Online display. It's probably not a good idea to make them actual members, they're just borrowing the members' group.

[Zia]

hmm..i dont its would be better to install it or not...
some whre in a famous webmaster forum i read "cloaking' spider/bot maight be a reason banned the site from the search engine...

...looks for xpert's comment....

[Corriewf]

This is a great mod!

Think about the logic you guys are preaching. With the same logic, we could get banned for an indexed thread that has been moved to a private area..... There is no black and white here but nothing in life is. If the bot has indexed the material available in your site then your fine by Google's TOS. Cloaking is making material that does not actually exist on your site.

Install and have fun!

[Cap'n Steve]

If you're worried about the engines getting annoyed with you, the Search Engine-Friendly Archive is much shadier than this.

[Snake]

Thank you very much, just installed the hack and it works great!

[kofoid]

YAY YAY YAY! I have been wanting this for a while - THANK YOU

[bashy]

Hi

I still have bots getting the No Permissions when trying to view a forum that users are allowed to view?