KX - Spider Permissions

[Ziki]

This modification was brought to you by
KXDesign
http://www.kxdesign.com/


~Modification name
Spider Permissions

~Modification description
This modification allows you to set view permissions for the spiders/bots on your website or you can place them in your desired usergroup.


~Modification options

Enable?
Place into usergroup?
Usergroup ID
Can view forums?
Can view threads?
Can view who's online?
Can view profiles?
Can view albums?


~Modification info
File uploads: 0
File edits: 0
Templates: 0
Template edits: 0
Plugins: 1
SQL Queries: 0
Phrases: 17
Settings: 8
Hooks: 0

Install time: 3 sec
Install level: Easy

~Modification installation

Step 1: Import Product


~Modification changelog


1.0.0 - First Release



~Modification copyright
This may not be distributed,released or claimed as your work without author's permission.

[Ziki]

~Reserved~

[7lanet]

Is it work with Version 3.7

[Ziki]

Is it work with Version 3.7

https://vborg.vbsupport.ru/showthread.php?t=196418

That is the 3.7 version but you can install this one as well,because they are almost identical

[7lanet]

thanks

this modification great
^_^

[7lanet]

google web masters considered it a violation of laws
The google disable archive site which uses this method
Is this true ؟

[ssslippy]

Sites using this will get removed from google.

[7lanet]

Sites using this will get removed from google.

Are u sure ?

[joh]

That is BS, I have a similar mod on one of my forums where the spiders act as a member and they are allowed to few all my rooms. https://vborg.vbsupport.ru/showthread.php?t=108070

I have not been band and my traffic has increase by 5x. I will be installing this mod on my 3.8 if it works as good as the other one, and if the old one that I have can not work on 3.8

How does it compeer to the link above and does it do the same thing but just for 3.8?

[forumsonsuz]

thanks

[Lizard King]

Who ever installs this board is putting their ite in great danger. If someone reports their site to google their site will be removed from SE index forever.

Joh , if you want to proove it is bullshit or not , you can share your link with me and i'll be gladly report your site to google.

[joh]

Lizard King, it is people like you that may cause a broad to get ban, stop hating on the guy mod and make one that is better.

This is not the format for you, it is to talk about if the mod works as he said it would.
A lot of people may speak out against a lot of the mods on this site, but yet they use them, At one point it was said that using vbseo was going to get you band because it ping Google, place links from all your forums in your footer on your front page so that the bots can index them all, which means that you could get band from Google too.

some of you made a stink about the mod that let you read all pms on your site.
the list can go on and on and on. Stop the hate, stop telling people that things will happen simply because they install a mod.

My question above still stand.
Lizard King the most people like you can hope for is that people do not click install and that the coder stop coding.

[Ziki]

That is BS, I have a similar mod on one of my forums where the spiders act as a member and they are allowed to few all my rooms. https://vborg.vbsupport.ru/showthread.php?t=108070

I have not been band and my traffic has increase by 5x. I will be installing this mod on my 3.8 if it works as good as the other one, and if the old one that I have can not work on 3.8

How does it compeer to the link above and does it do the same thing but just for 3.8?

Well,this actually works :),it allows you to have specific permissions and place them into a usergroup you want :)

Are u sure ?

Partly I guess.It shows the content that registered users can see (which are humans) but guests might not (depends on your options).So it's not a complete violation.But yes,it can happen.

Who ever installs this board is putting their ite in great danger. If someone reports their site to google their site will be removed from SE index forever.

Joh , if you want to proove it is bullshit or not , you can share your link with me and i'll be gladly report your site to google.

(View above)

[dirtyfeast]

I am not sure if this works. All my spiders still get the "Viewing 'No Permission' Message & Viewing Error Message". I have installed, set permissions, and it is active. Did I miss something. Are there other edits, or permissions that must be made. Is there a special spiders mod that must be installed. Let me know. Thanks for your time.

[Ziki]

No this should be all.Can someone also report this as not working?

[Zia]

it can be done via robotx.txt .........

[Ziki]

it can be done via robotx.txt .........

No it can't :)

[Lizard King]

It can be done via robot.txt Ziki.

[Lizard King]

Lizard King, it is people like you that may cause a broad to get ban, stop hating on the guy mod and make one that is better.

This is not the format for you, it is to talk about if the mod works as he said it would.
A lot of people may speak out against a lot of the mods on this site, but yet they use them, At one point it was said that using vbseo was going to get you band because it ping Google, place links from all your forums in your footer on your front page so that the bots can index them all, which means that you could get band from Google too.

some of you made a stink about the mod that let you read all pms on your site.
the list can go on and on and on. Stop the hate, stop telling people that things will happen simply because they install a mod.

My question above still stand.
Lizard King the most people like you can hope for is that people do not click install and that the coder stop coding.
vBSEO is using all white hat seo tactics and there is no feature that will get you banned on google. I donot have hate at all. I just warn people with the truth.

If they use this mod and they choose this mod to display bots different content than guest. It will get their site dropped out of SE 's index.

[Ziki]

It can be done via robot.txt Ziki.

You can block them from accessing files and folders but not forum features.

vBSEO is using all white hat seo tactics and there is no feature that will get you banned on google. I donot have hate at all. I just warn people with the truth.

If they use this mod and they choose this mod to display bots different content than guest. It will get their site dropped out of SE 's index.

Like I said people can access the content that the SE can as well (like threads and forums).

[dirtyfeast]

So is there an answer to my question. The problem I am having.

[Hostboard]

I am trying to understand what this does and how adding:


Disallow: /member.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Etc...


in a robots.txt file is any different outside of assigning a specific user group within VB to do the same?

[Ziki]

Try allowing them to access usercp :)

[Hostboard]

Try allowing them to access usercp :)

Then just use:


Disallow: /usercp.php


Your reply was less then helpful. I still do not understand.

In fact a robotos.txt file for VB should encompass:


User-agent: *
Disallow: /ajax.php
Disallow: /attachment.php
Disallow: /calendar.php
Disallow: /cron.php
Disallow: /editpost.php
Disallow: /global.php
Disallow: /image.php
Disallow: /inlinemod.php
Disallow: /joinrequests.php
Disallow: /login.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /moderator.php
Disallow: /newattachment.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /poll.php
Disallow: /postings.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /register.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showgroups.php
Disallow: /subscription.php
Disallow: /threadrate.php
Disallow: /usercp.php
Disallow: /usernote.php


No being a hater, I just do not understand what differences your plugin accomplishes that a robots.txt file can't do.

[Ziki]

I said allow bots to access usercp not to disallow ;)

[Hostboard]

Devils advocate...

I thought unless you dis-allow that robots will auto spider your site. Also why allow usercp.php? I would think this would be duplicate information. Duplicate information is not seo friendly plus a bandwidth waster?

[Ziki]

Well I don't care what people do with this modification,I released this because of a request.

[eJM]

Lizard King, it is people like you that may cause a broad to get ban, stop hating on the guy mod and make one that is better.
Don't bash people who are trying to help us understand the ramifications of using a particular modification. I certainly don't want to break the Google rules and if there is a chance that could happen and the mod author forgot to tell us about that issue, then I appreciate someone like Lizard King letting me know. It sounds like if you knew a law or rule was being broken that might harm someone's website, you wouldn't say a word. Someone like that would be far more problematic than Lizard King.

It can be done via robot.txt Ziki.
Not always. I am checking this mod out for a specific purpose that can't be handled by robots.txt.

vBSEO is using all white hat seo tactics and there is no feature that will get you banned on google. I donot have hate at all. I just warn people with the truth.

If they use this mod and they choose this mod to display bots different content than guest. It will get their site dropped out of SE 's index.
Keep doing what you're doing, Lizard King. I appreciate your efforts. However, this mod may just be the thing some of us might need to allow the Adsense spiders to crawl private forums that also have ads.

Well I don't care what people do with this modification,I released this because of a request.
I do appreciate your hard work on this, but I always appreciate these kinds of efforts more when the author of a mod also cares about how the modification is used. If it might cause a problem with a search engine's rules about accessing content that an unregistered visitor couldn't also access, you might put a disclaimer in the modification's description. Give the user the information they require to make a wise decision.

What I am interested in doing, and I think there might be a wide use for this, is to allow the Mediapartners-Google* spiders to crawl private forums that also contain Adsense ads. I run a forum for flooring professionals and consumers. There are a number of forums on my site that only pros can view and participate in. Consumers and, of course, unregistered or unlogged in members can't view the content there. I need a way to allow the Adsense only spiders to access the private forums to help them place content relevant advertisements.

This is NOT against the Google rules. There is a place you can set up authentication for the Adsense spiders (https://www.google.com/adsense/edit-auth), but it doesn't appear to work with vBulletin private forums. If this modification works for that particular bot, then you truly do have a very valuable tool to offer vBulletin webmasters.

My hope is that the modification has the ability to select which bots you want to give access - and that the Mediapartners-Google* bot is one of them. Maybe you could edit your description to let us know this and post some screen shots of the options panel. As it is, I have to download and install to know the answers to these questions.

Anyway, thanks for offering a solution - if it is a solution to what I need.

Best R'gards,

Jim McClain

[eJM]

Well, apparently this mod does not give you very many choices. There are no choices as to which spiders can have access and which ones cannot. Although spiders can be added to specific usergroups, there doesn't seem to be a way a spider could log in or utilize a password. If there is some way that any ol' spider can have access to private areas of my forum, then what is there to prevent a malicious user from impersonating a search spider to gain access to secure areas of my forum?

Sorry, this doesn't seem to be a secure modification with enough options to make it useful to me.

Jim

[Ziki]

This modification places the spiders defined in your spiders.xml into the usergroup specified.So you cannot fool it ;)

[ShawneyJ]

wow this looks sweet, its like what IP.Boards have right? put the spiders in there own group.
and alow only the spider that are in spider display in admin panel. as for vbulletin, the spiders allowed would be vbulletin spider list.xml right? i think thats what your saying above ;)
this would be good to get members to sign up to view the thread they found on google that the bots can only see. nice job ;)

[eJM]

It is a violation of Google's rules. They may consider this "cloaking." One of the definitions of cloaking is:
Serving different content to search engines than to users.
It's likely that putting a spider in a special position to see content that is not available to unregistered users would be considered cloaking ind it will get your website removed from the Google indexed pages.

Jim

[Ziki]

Well but users can see the content that the spiders can as well.Let's say you disabled guest viewing of your threads,but registered still can see.So spiders can actually see,what users can as well.

[eJM]

That's not the same. If you have to devise special code, whether that be template edits or a plug-in, you are giving the spiders an advantage website visitors do not have. If the content is blocked from view until a user meets certain requirements, like registering, agreeing to terms and being added to a special usergroup, then you are causing the spiders to circumvent these requirements, which the normal user cannot do.

If the possibility exists for a user of this modification to be banned or penalized by Google or another popular search engine, you should state it in the modification description. If you have proof that search engines will not punish users of this mod, then there isn't a problem. It's my opinion that you have an ethical responsibility to your customers (a customer isn't always someone who pays a fee) to keep them informed of any possible consequences from using this software.

Jim

[Ziki]

Ok Sir.

[ShawneyJ]

That's not the same. If you have to devise special code, whether that be template edits or a plug-in, you are giving the spiders an advantage website visitors do not have. If the content is blocked from view until a user meets certain requirements, like registering, agreeing to terms and being added to a special usergroup, then you are causing the spiders to circumvent these requirements, which the normal user cannot do.

If the possibility exists for a user of this modification to be banned or penalized by Google or another popular search engine, you should state it in the modification description. If you have proof that search engines will not punish users of this mod, then there isn't a problem. It's my opinion that you have an ethical responsibility to your customers (a customer isn't always someone who pays a fee) to keep them informed of any possible consequences from using this software.

Jim



well if IP.Boards are allowed to do it the last few years, then so can we yeah? it comes standard with IP.Boards, its bloody built in m8...get banned from google. you mean black listed? i'd doubt that to. its all about the guests, what users can see bots can see, also i see no IP.Boards black listed. copy and paste where its says on google about "guests" or send me a link. ""Guests"". no offense but send some proof on this page, as what you told Ziki to add is going to scare everyone off from using this addon completely.
yeah agreed, be safe, but bring on the proof first.
also hows the bots going to no what guests can see or cant see?
sorry but this is just the same as what comes standard with IP.Boards, if they can use it so can we true? other wise this to me makes no damn sense.

[eJM]

well if IP.Boards are allowed to do it the last few years, then so can we yeah?
Congratulations. You have caused Ziki to edit the description back to the way it was, without the disclaimer. I don't use IPBoards, so I wouldn't know if what you say is true or not. I do use Google and the quote I provided above comes from here: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=66355 No, it doesn't use the exact term "Guests", but then it doesn't use many other terms that might imply the same thing. It uses "users." Why don't you provide the same sort of "proof?"

... what you told Ziki to add is going to scare everyone off from using this addon completely.
yeah agreed, be safe, but bring on the proof first.
Exactly. Where's your proof? Look, this debate could go on and on. If you want to have a debate about what Google and other search engines like and don't like, start a new topic. I merely agreed with at least one other very experienced webmaster that the possibility exists that this mod is in violation of Google terms. The debate rages on and unless we get a definitive answer from Google itself, regarding the practice of allowing it to index content that its users will click the link to and get an error message instead of the content, then neither you nor I will have the "proof."

As I said before, that you apparently didn't read even though you quoted it, it's my opinion that Ziki has an ethical responsibility to his customers to warn them of the possibilities. I've got nothing to gain or lose by keeping people from using this mod. It's only my desire to help my fellow webmasters who may be looking for legitimate ways to bring relevant content, including advertising, to my users. This product does not appear to be that and it may be detrimental to my ranking and indexing in Google and other search engines.

I'm taking a wild guess here that, based on the content of your avatar and signature, you're prob'ly around a teenager. I don't know how much experience you have as a webmaster, but mine goes back to about the time you were born (in other words, in the teens). I'm old and I hope I am wise in some matters. My personal rulZZZ: don't argue with children. I'm done with this conversation.

[Ziki]

Congratulations. You have caused Ziki to edit the description back to the way it was, without the disclaimer. I don't use IPBoards, so I wouldn't know if what you say is true or not. I do use Google and the quote I provided above comes from here: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=66355 No, it doesn't use the exact term "Guests", but then it doesn't use many other terms that might imply the same thing. It uses "users." Why don't you provide the same sort of "proof?"


Exactly. Where's your proof? Look, this debate could go on and on. If you want to have a debate about what Google and other search engines like and don't like, start a new topic. I merely agreed with at least one other very experienced webmaster that the possibility exists that this mod is in violation of Google terms. The debate rages on and unless we get a definitive answer from Google itself, regarding the practice of allowing it to index content that its users will click the link to and get an error message instead of the content, then neither you nor I will have the "proof."

As I said before, that you apparently didn't read even though you quoted it, it's my opinion that Ziki has an ethical responsibility to his customers to warn them of the possibilities. I've got nothing to gain or lose by keeping people from using this mod. It's only my desire to help my fellow webmasters who may be looking for legitimate ways to bring relevant content, including advertising, to my users. This product does not appear to be that and it may be detrimental to my ranking and indexing in Google and other search engines.

I'm taking a wild guess here that, based on the content of your avatar and signature, you're prob'ly around a teenager. I don't know how much experience you have as a webmaster, but mine goes back to about the time you were born (in other words, in the teens). I'm old and I hope I am wise in some matters. My personal rulZZZ: don't argue with children. I'm done with this conversation.

I guess it's time I enter into this conversation.My ethical responsibility to warn the users would be if I was sure this modification would negatively affect their website.I am sure that this does not conflict with Google TOS,as spiders can see the same content as the board's users can.

As another point that's off this topic,is that you shouldn't judge someone according to their age.For first it's discrimination and as a second point,you would be surprised how many adults behave childish.

[eJM]

I guess it's time I enter into this conversation.My ethical responsibility to warn the users would be if I was sure this modification would negatively affect their website.I am sure that this does not conflict with Google TOS,as spiders can see the same content as the board's users can.
You have the right to choose your own ethical standards. I believe you are wrong about the board's users being able to see the content Google spiders index. If Google indexes a private forum and it appears as a search result for a given term, when the searcher clicks the link, they will receive a "No Permissions" message. You have given Google a view of content that no one else can see unless they meet certain requirements. That could be defined as cloaking, but it is almost certainly subterfuge.

As another point that's off this topic,is that you shouldn't judge someone according to their age.For first it's discrimination and as a second point,you would be surprised how many adults behave childish.
I am prejudiced against children. I don't like them. So sue me.

Jim

[Ziki]

You have the right to choose your own ethical standards. I believe you are wrong about the board's users being able to see the content Google spiders index. If Google indexes a private forum and it appears as a search result for a given term, when the searcher clicks the link, they will receive a "No Permissions" message. You have given Google a view of content that no one else can see unless they meet certain requirements. That could be defined as cloaking, but it is almost certainly subterfuge.

Guests get that message even if you disabled forum view and has your forum in index.

I am prejudiced against children. I don't like them. So sue me.

Jim

I think there is a big difference between children,teenagers and adults,although there is a thin line between children and adults.I would like to end this discussion as it's becoming hatred.

[unp]

webmasterworld.com used this technique for years, results and description would appear in google but clicking the links would take you to a login page,
They were never penalized,

[ShawneyJ]

Why don't you provide the same sort of "proof?"

IP.Boards AdminCP
Treat spider/bot as part of which group?
This is useful as you can specify group settings for a 'bot' group which can restrict access to certain areas, etc. just like Ziki's addon. Please note: you can make a bot group and give bots there own permissions.
https://vborg.vbsupport.ru/external/2008/12/83.jpg

Force spider/bot to use skin
This is useful as you can create a 'lite' optimized skin for your spiders to use to reduce bandwidth and increase the depth of the bot crawling.
https://vborg.vbsupport.ru/external/2008/12/84.jpg

Spider Bot User-Agent
Use this section to list which search engine user-agents you'd like to be treated as a search engine visit along with a 'human' name for the online list in a bot=Human Name format.(one per line)
Example: googlebot=Google.com
https://vborg.vbsupport.ru/external/2008/12/85.jpg

based on the content of your avatar and signature, you're prob'ly around a teenager. I don't know how much experience you have as a webmaster, but mine goes back to about the time you were born (in other words, in the teens). I'm old and I hope I am wise in some matters. My personal rulZZZ: don't argue with children. I'm done with this conversation.

well you didnt learn much in that time lol.
did you no that Avril is an Adult which mite make you think im one to lol.
actually im not going to flam anyone like you are, if im wrong i dont cry like
a little girl. no offense but your just plain wrong.
you started something you couldnt finish.


also, sorry Ziki, your thread has been attacked, its a sweet hack and keep up the good work m8.
this will be my last post unless i need support for your addon.
but at least now everyone can see that IP.Boards/invision comes with
this built in and can see how useful your addon/hack is ;)
cheers.

[eJM]

IP.Boards AdminCP
Treat spider/bot as part of which group?
This is useful as you can specify group settings for a 'bot' group which can restrict access to certain areas, etc. just like Ziki's addon. Please note: you can make a bot group and give bots there own permissions.
Notice that is says "restrict access" not gain access.
Force spider/bot to use skin
Spider Bot User-Agent
Neither of these are a violation of any rule.

well you didnt... ssssniiiip
As you kids like to say, whateverrrrr.

also, sorry Ziki, your thread has been attacked...
BS! An issue has been brought to light. You can delude yourself all you want, but it is still an issue. Your proof is not. Ask IPBoard support whether that feature is designed to give access to Google when it blocks others. It doesn't. You can use it to gain access to restricted areas if you want, but it was NOT designed to violate any Google or other search engine terms of service. I am familiar with the product. I've been managing message boards of all types since 1992.

If you want to continue this discussion further, then honor the mod author by complying with his request. Start a new thread somewhere else about it.

Jim

[Ziki]

Ok that does it.I had to report both of you.If you can't discuss in a mature way about the topic (Is this modification SE friendly?) and just insult yourself,please do it in a private discussion.This is not a place for such conversation.

[Marco van Herwaarden]

I am sure that this does not conflict with Google TOS,as spiders can see the same content as the board's users can.

Sorry to burst the bubble, but those warnings are correct.

Serving different content (layout is not an issue) to spiders and to visitors (and not registered members) is considered cloaking and can get you blacklisted on SE's.

[Leo Brazil]

It is a violation of Google's rules. They may consider this "cloaking." One of the definitions of cloaking is:

It's likely that putting a spider in a special position to see content that is not available to unregistered users would be considered cloaking ind it will get your website removed from the Google indexed pages.


In fact this is kind of violation of Google's rules. But it takes time to Google punish you but trust me they'll.
And don't forget you're playing like "My Game, my rules", I mean Google can simply ban you from their lists and you'll have to do whatever they want. The problem is it takes a lot of time to get back to their lists if you were banned.

I can tell you that because I've seen this before, a friend of mine had an issue with Google about cloaking. His board was on the web about two years with a kind of cloaking, he always tried to convince us that wasn't against rules and something. One day he was notified by Google and he took almost one year to get back to their lists.
The problem is if you are listed due cloaking there's no argue with them at all, you're guilt and that's it and have to do whatever they want.

I'm not telling this because I think this is a BS, waste of time and nothing like this. In fact I think it's got potential. This this is for who really wants what they want and the risks for.

[Orakk]

it can be done via robotx.txt .........
That text file is just a polite request which many, if not most bots ignore. One only need check their log files to see this is true and is common knowledge via google search the thing is close on useless.

Fair enough, if you don't want to risk up setting google then so be it but there are many of us who see being indexed as a waste of time and invasion of our privacy.

Thanks again for this plugin, home is much calmer now.. :up:

[hscorp]

ziki i really appreciate this mod and i hope it work

but it doesn't look to work with me

see the wol pic and setting pic

https://vborg.vbsupport.ru/

https://vborg.vbsupport.ru/

[Ziki]

Try placing them into a usergroup.

[CableSux]

I am not sure if this works. All my spiders still get the "Viewing 'No Permission' Message & Viewing Error Message". I have installed, set permissions, and it is active. Did I miss something. Are there other edits, or permissions that must be made. Is there a special spiders mod that must be installed. Let me know. Thanks for your time.

No this should be all.Can someone also report this as not working?

Try placing them into a usergroup.

I'm having the same problem with the 3.7 version.
https://vborg.vbsupport.ru/showpost.php?p=1742504&postcount=33
I'm getting ready to upgrade to 3.8. Has anything changed in 3.8 to make it work?
.

[intell_iut]

not work for me too
maybe better choice is that put bot in usergroup base on it's IP , googlebot and yahoo slurp have constant IP

[Welshy2008]

So Gentlemen,

What happens when your board is open to the public (Full view or limited views before registering)? I mean, SE's rank it and it's contents.

Should you decide later on that you wish to Stop those views for a little while, What happens then? Is that against SE terms etc?

Genuine Question.

[Welshy2008]

Until I know for certain, I will not be installing it.

But - nethertheless - A great mod, Congratulations. And Good Luck to all who use it. :)

[drupy]

Installed. Thank you!

[mpoorrajab]

ty very much Ziki https://vborg.vbsupport.ru/external/2009/10/7.gif

https://vborg.vbsupport.ru/external/2009/10/5.gif

[Orakk]

ziki i really appreciate this mod and i hope it work

but it doesn't look to work with me

see the wol pic and setting pic

http://img156.imageshack.us/img156/7494/01ti9.jpg

http://img156.imageshack.us/img156/2946/02bt4.jpg
I'm guessing you'll be going to get a google.bot added to the spiders_vbulletin.xml (http://spiderlist.codeforgers.com/) so you'll need block them via .htaccess but for those bots on the list we use a separate spidieDungeon.group (id:101) with zero permissions.

[TheSupportForum]

i am currently using tubenow link checker, and downloaded this mod that is needed for it to work

if anyone has got this and tubenow link checker works please advise me, i followed everthing on www.tubenow.net (http://www.tubenow.net) website and it dont work

here what happens
http://tubenow.net/link-checker/?url=www.tutorials4you.co.uk/showthread.php?t=8

http://www.tutorials4you.co.uk/images/Untitled.jpg

i get the following message
Error: Unable to find download links.[ Try again (http://tubenow.net/link-checker/?url=tutorials4you.co.uk/showthread.php?t=8#) ]

[Manoel J?nior]

Me too!

:(

[final kaoss]

<a href="http://tubenow.net/link-checker/?url=tutorials4you.co.uk/showthread.php?t=8#" target="_blank">http://tubenow.net/link-checker/?url...hread.php?t=8#</a>

Try removing the pound sign

[TheChaosFactor]

For all those who read this thread, as I did, and were still unsure of what to think, I bring you:

Cutts explained that when Google's crawlers visited a BMW page, it saw blocks of text with repeated key search words such as "neuwagen," which means "new car" in German. However, when a user visited the listed page they would be automatically redirected to another page with less text and more pictures, which was more attractive than the page the crawler saw, but would have scored lower in Google's PageRank system.

"This is a violation of our Webmaster quality guidelines, specifically the principle of 'Don't deceive your users or present different content to search engines than you display to users,'" Cutts' blog said.

Full article: (http://news.cnet.com/Google-blacklists-BMW.de/2100-1024_3-6035412.html)

It took BMW about a week's time, and who knows how much to their team of lawyers, to get re-listed. Don't think you'll get nearly the same treatment without being able to blame an SEO company, and have some lawyers do some fancy talk for you. Google is srs bsns.

And I'm not hating. I couldn't have written a mod like this myself if my life depended on it. :cool:

[Marek58]

Polish version

[Meola]

I know, phpBB has an ability to define different permission for the Bots and guests, and I've never heard any phpBB forum has been banned by Google yet.

[mxracer95]

Cutts explained that when Google's crawlers visited a BMW page, it saw blocks of text with repeated key search words such as "neuwagen," which means "new car" in German. However, when a user visited the listed page they would be automatically redirected to another page with less text and more pictures, which was more attractive than the page the crawler saw, but would have scored lower in Google's PageRank system.

"This is a violation of our Webmaster quality guidelines, specifically the principle of 'Don't deceive your users or present different content to search engines than you display to users,'" Cutts' blog said.

That supports the interpretation that this mod is OK.

It seems obvious to me that Google's intent is to ensure people don't mislead search engines with content that doesn't exist, such as when you click on a google result you get taken to a malicious website, not the website and content you wanted to see.

Locking content behind a username/password is not changed, malicious or misleading content.

As was mentioned earlier in this thread, newspapers and television stations regularly do this. A news story will be visible in a search result, but when you click the link to the website, you can't see it unless you register and log in.



That said, I have a question about this mod.

My website requires users to log in. This mod allows you to assign search bots to a usergroup, but does it bypass the login requirement? In otherwords, does it show bots private forums without them logging in?

[mohammadxxx]

Hi

I installed this product and change all you say .

#
ACP -> vBulletin Options -> KX - Spider -> Select yes on view threads.
#
ACP -> Style Manager -> Your Template -> BB Code Layout -> bbcode_code
and
# accomplish this change too :
edit "/includes/xml/spiders_vbulletin.xml"


in topic, above the links show HyperLink Check Download Links , but when i click on Check Download Links in new page (on your website) it don't check any of my links in my topic .
why ?
please help

kind regards
mohammadxxx

[Alfa1]

Google specifically asks webmasters to give Googlebot access to restricted pages. I have seen this at various Google webmaster blogs and webmaster help pages. However, you must show the whole access page/thread to the visitors. Any other pages may be restricted from the visitors.

See here:
http://googlewebmastercentral.blogspot.com/2008/10/first-click-free-for-web-search.html

Webmasters wishing to implement First Click Free should follow these guidelines:
All users who click a Google search result to arrive at your site should be allowed to see the full text of the content they're trying to access.
The page displayed to all users who visit from Google must be identical to the content that is shown to Googlebot.
If a user clicks to a multi-page article, the user must be able to view the entire article. To allow this, you could display all of the content on a single page—you would need to do this for both Googlebot and for users. Alternately, you could use cookies to make sure that a user can visit each page of a multi-page article before being asked for registration or payment.First Click Free is also what webmasterworld.com is using.

I hope this clears it up.

Ziki would you consider to add First Click Free functionality?

[Ricsca]

for vbulletin 4.0.x version?

[ibanez32]

Need VB4 verssion, thx in advance

[MadK]

Likewise, looking for vB4 version! ;)

[hawa.ly]

thank you

[Rick7C2]

Need this for vb 4.0.x

[ins3]

Also need this for vB 4.0.x

[Brandon Sheley]

allowing spiders access to content guest can't see = cloaking = google ban
Cloaking, sneaky Javascript redirects, and doorway pages (http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=66355)

If your site contains elements that aren't crawlable by search engines (such as rich media files other than Flash, JavaScript, or images), you shouldn't provide cloaked content to search engines. Rather, you should consider visitors to your site who are unable to view these elements as well. For instance:

stay away from this mod ;)

[Greight]

allowing spiders access to content guest can't see = cloaking = google ban
Cloaking, sneaky Javascript redirects, and doorway pages (http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=66355)



stay away from this mod ;)
Does it talk about illegality of asking people to register on your page for free, then showing them the page? I think we should put a stop to this cloaking idea r lets say nonsense. Spiders separate permissions is something standard and working in phpBB and as said above no one ever never has heard of a page of phpBB based forums to be blocked by any search engine.

BTW waiting for vB 4 version impatiently.

[BirdOPrey5]

It seems very clear to me there is valid use for this mod and potential for abuse. If you use this mod to restrict what googlebot can see you're 100% in the clear. If you use this mod to give googlebot access to private forums then you are playing with fire.

That said I don't care about googlebot, I'm wondering does anyone know how I could use this mod to give only the google adsense bot access to "private" forums? Basically I want ad-sense to act like a logged in registered user, this is not against any TOS, in fact it is encouraged. Google's own authentication/log in system doesn't seem to work with vBulletin.

[josantos]

I?m waiting for vB 4 version impatiently too :up::up:

[Super-six]

If you want to have a debate about what Google and other search engines like and don't like, start a new topic.


You should follow your own advice and take the depression-fueled
moral & ethical crusade with you. This wasnt your thread.


I'm taking a wild guess here that, based on the content of your avatar and signature, you're prob'ly around a teenager. I don't know how much experience you have as a webmaster, but mine goes back to about the time you were born (in other words, in the teens). I'm old and I hope I am wise in some matters. My personal rulZZZ: don't argue with children. I'm done with this conversation.



I am prejudiced against children. I don't like them. So sue me.
Jim


Wow. Who's the "child" here? :erm:
Its you that actually sounds like a toddler on a rampage.

[onealien]

i am trying to install the linkchecker add-on...in the instruction text doc is says to first add this mod....wat do they have to do with each other?

is there any other link checker add-ons that someone can recommend where you do not have to mess with all of this?

thanks,

[ouch]

is there a VB4 version of this

[Ricsca]

Foru vbulletin version 4 ?

[madshark]

It works fine with VB4 from what I know.

[Eslob]

Well ..

the most profitable thing to do is to cut the first post and keep little content for search engines to index & adsense.

this can be seen by visitors & bots, while members can see the full post.

which i think its the safest way to be away from Google violations.

[no-limits]

canot see any spiders in my usergroup.
my spider usergoup is stil empty.

admin cp
Enable this modification = yes
Do you want to place the spiders into a special usergroup?If no you can apply the permissions below. = yes
Enter here the ID of the usergroup where the spiders should be placed. = 42 (my spider group)

rest all = no

any ideas ??