The Arcive of Official vBulletin Modifications Site.

tgillespie · #1 06-20-2006, 12:34 AM

Just curious if anyone has ever looked into making a search engine spider conditional? I have certain aspects on my web site, such as text links and buttons that I would rather not let spiders visit. With vBulletin's engine, would it be possible to create a conditional that sorted spiders by hostmask or IP? For example:

<if conditional="host8.google.com"><else />My Friends Site</if>

Not looking for someone to make it, just checking if its possible. Most logical condition would be based on hostmask or ip range.

If done, I think it could greatly help the ability to better SEO your forums.

amykhar · #2 06-20-2006, 12:09 PM

I'll do it today if I have time. I can use code from my spider's hack to set it up.

Amy

xStylezx · #3 06-20-2006, 12:36 PM

Id be interested in knowing how to do this in vbulletin myself

amykhar · #4 06-20-2006, 12:45 PM

A word of warning - I'm releasing this for 3.6 because that's what I'm running and that's what I can test on. I suspect, however, that the 3.6 version will work on 3.5 as well.

tgreer · #5 06-20-2006, 02:25 PM

No plugin or special coding is required. All of this can be accomplished via a standard "robots.txt" file. A simple web search should reveal all you need to know about this standard technique. Also, I believe there is a thread already on vbulletin.com which lists some of the standard pages within vbulletin that spiders should not try to access.

amykhar · #6 06-20-2006, 02:27 PM

Tgreer, you are correct. I misunderstood his post to read that he had certain portions of his individial pages that he wanted to exclude from spiders.

tgreer · #7 06-20-2006, 02:31 PM

Just out of curiosity, then... wouldn't excluding certain portions of pages from spiders be considered "spoofing", and negatively affect SERPS?

amykhar · #8 06-20-2006, 02:40 PM

Probably. But, it would have been a fun academic exercise to code

tgreer · #9 06-20-2006, 03:01 PM

No doubt, but the original poster was wanting it for SEO purposes. I don't know for certain, but I'm pretty sure that hiding certain sections of pages from a search engine would have a negative impact on SEO.

But then again, since the conditional would be implemented in PHP, I'm not sure how a search engine would KNOW...

tgillespie · #10 06-20-2006, 05:14 PM

Quote:

Originally Posted by tgreer

No doubt, but the original poster was wanting it for SEO purposes. I don't know for certain, but I'm pretty sure that hiding certain sections of pages from a search engine would have a negative impact on SEO.

But then again, since the conditional would be implemented in PHP, I'm not sure how a search engine would KNOW...

I'm pretty sure it wouldn't hurt SERPS because the HTML is just never rendered and the spider wouldn't know. The same technique is used in vBSEO's signature hide hack which disables signatures for guests which also disables it for spiders. Used properly, a conditional like this would greatly benifit boards/sites.

amykhar, I would really appreciate it. 3.6 is fine as I am sure it will most likely be backwards compatible, if not, I'm sure I could manage.

Well seeing as it might be possible, I guess I'll start a mods request thread.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.04426 seconds Memory Usage 2,250KB Queries Executed 11 (?)
More Information
Template Usage: (1)SHOWTHREAD (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)ad_showthread_beforeqr (1)ad_showthread_firstpost (1)ad_showthread_firstpost_sig (1)ad_showthread_firstpost_start (1)bbcode_quote (1)footer (1)forumjump (1)forumrules (1)gobutton (1)header (1)headinclude (1)navbar (3)navbar_link (120)option (1)pagenav (1)pagenav_curpage (1)pagenav_pagelink (10)post_thanks_box (10)post_thanks_button (1)post_thanks_javascript (1)post_thanks_navbar_search (10)post_thanks_postbit_info (10)postbit (10)postbit_onlinestatus (10)postbit_wrapper (1)spacer_close (1)spacer_open (1)tagbit_wrapper Phrase Groups Available: global inlinemod postbit posting reputationlevel showthread	Included Files: ./showthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/functions_bigthree.php ./includes/class_postbit.php ./includes/class_bbcode.php ./includes/functions_reputation.php ./includes/functions_post_thanks.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete showthread_start showthread_getinfo forumjump showthread_post_start showthread_query_postids showthread_query bbcode_fetch_tags bbcode_create showthread_postbit_create postbit_factory postbit_display_start post_thanks_function_post_thanks_off_start post_thanks_function_post_thanks_off_end post_thanks_function_fetch_thanks_start post_thanks_function_fetch_thanks_end post_thanks_function_thanked_already_start post_thanks_function_thanked_already_end fetch_musername postbit_imicons bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete postbit_display_complete post_thanks_function_can_thank_this_post_start pagenav_page pagenav_complete tag_fetchbit_complete forumrules navbits navbits_complete showthread_complete
Messages:

The Arcive of Official vBulletin Modifications Site.

It is not a VB3 engine, just a parsed copy!