Go Back   vb.org Archive > vBulletin Modifications > Archive > vB.org Archives > vBulletin 2.x > vBulletin 2.x Beta Releases
Reload this Page Have Googlebot Crawl EVERYTHING (theoretically)
FAQ Community Calendar Today's Posts Search

Reply
Page 1 of 7 1 23 Last »
 
Thread Tools
Have Googlebot Crawl EVERYTHING (theoretically) Details »»
Have Googlebot Crawl EVERYTHING (theoretically)
Version: 1.00, by filburt1 filburt1 is offline
Developer Last Online: May 2007 Show Printable Version Email this Page
Version: 2.3.x Rating:
Released: 03-24-2003 Last Update: Never Installs: 107
Is in Beta Stage  
No support by the author.

Googlebot supposedly won't crawl links that have sessionhashes (32 character hex strings) in the URLs. So this hack detects when Googlebot is browsing your forum, and if it is, deletes the sessionhash. So, theoretically, Googlebot will be able to crawl every single guest-accessible link on your board. All of them. This means every thread, every forum, every post, even useless stuff like the memberlist and the FAQ.

Simple to install (1 step). This won't get instant gratification because Googlebot will only reindex your site on its next crawl.

Show Your Support

  • This modification may not be copied, reproduced or published elsewhere without author's permission.

Comments
  #2  
Old 03-25-2003, 05:00 PM
Intex Intex is offline
 
Join Date: Apr 2002
Posts: 576
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Another great hack filburt1. Not sure I need it crawling my internal site, but nevertheless very cool.
Reply With Quote
  #3  
Old 03-25-2003, 05:22 PM
colicab-d's Avatar
colicab-d colicab-d is offline
 
Join Date: Dec 2002
Location: Glasgow
Posts: 382
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
lol so thats why u wanted the sessions code
Reply With Quote
  #4  
Old 03-25-2003, 05:38 PM
filburt1 filburt1 is offline
 
Join Date: Feb 2002
Location: Maryland, US
Posts: 6,144
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
No it's not...
Reply With Quote
  #5  
Old 03-25-2003, 05:54 PM
VampireMan VampireMan is offline
 
Join Date: Jan 2003
Location: Leicestershire uk
Posts: 102
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
simple but very effective hack , thanks :-)
Reply With Quote
  #6  
Old 03-25-2003, 06:02 PM
Xanthine Xanthine is offline
 
Join Date: Mar 2002
Posts: 232
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Sounds good thanks Filburt
Reply With Quote
  #7  
Old 03-25-2003, 06:35 PM
Overgrow's Avatar
Overgrow Overgrow is offline
 
Join Date: Nov 2001
Posts: 320
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Excellent


One note: This may not allow Google to see all of your posts (theoretically). If you have old ones that would not naturally show up on forumdisplay, Google will not use the select box to see all of the old posts.

Elegantly simple though and makes perfect sense.


[high]* Overgrow clicks install[/high]
Reply With Quote
  #8  
Old 03-25-2003, 06:42 PM
Davey Davey is offline
 
Join Date: Nov 2002
Location: England
Posts: 383
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Filburt how did you find that special part of code through all that 'mess'?
.
Thanks.
*Installs*.

Dave.
Reply With Quote
  #9  
Old 03-25-2003, 08:31 PM
Webdork Webdork is offline
 
Join Date: Dec 2001
Location: Melbourne, Australia
Posts: 37
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
On my site I require registration for users to view threads. Is there a way to allow Google to become a "registered" user so it can crawl all threads still?
Reply With Quote
  #10  
Old 03-25-2003, 08:32 PM
filburt1 filburt1 is offline
 
Join Date: Feb 2002
Location: Maryland, US
Posts: 6,144
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Yes but smart users can get around it...

edit: actually I thought of a way to fix that, too. Expect that hack eventually.
note to self: check first few IP bits and HTTP_USER_AGENT, update $bbuserinfo.
Reply With Quote
Reply
Page 1 of 7 1 23 Last »

« Previous Thread | Next Thread »

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT. The time now is 04:08 AM.

Contact Us - Archive - Top

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.04180 seconds
  • Memory Usage 2,287KB
  • Queries Executed 23 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_post
  • (1)navbar
  • (6)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (2)pagenav_pagelink
  • (10)post_thanks_box
  • (10)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (10)post_thanks_postbit_info
  • (9)postbit
  • (10)postbit_onlinestatus
  • (10)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 
Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 
Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete