Go Back   vb.org Archive > vBulletin Modifications > Archive > vB.org Archives > vBulletin 2.x > vBulletin 2.x Full Releases
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools
Deferred Threadviews v1.00 (to reduce server load) Details »»
Deferred Threadviews v1.00 (to reduce server load)
Version: 1.00, by Jujubee Jujubee is offline
Developer Last Online: May 2009 Show Printable Version Email this Page

Version: 2.2.x Rating:
Released: 06-21-2002 Last Update: Never Installs: 34
 
No support by the author.

Hack name: Deferred Threadviews
Version: 1.00
Purpose:
To reduce server load by not updating the view count on every page view but instead keeping the it in memory and writing to the db only occasionally.
Functions:
? Logs threadviews to a temporary table, and periodically commits data to the thread table.
? Frequency of writes controlled by server load and a random number.
? Includes optional email notification.
vB version tested on: 2.2.6
PHP version tested on: 4.2.0
Acknowledgements: Thanks to MattR for the "store threadviews as list of non-unique ids and use count(*)" suggestion that removed 1 query per pageview.



The updating of threadviews has been identified as a main cause of high server loads. This hack basically writes threadviews to a HEAP table (i.e. temporary table residing in memory) and will only update the thread count when the load is below a specified value and a random # value is hit.

(The impetus for this hack was an email from my host: They threatened to boot me off my server due to CPU usage "abuse", so as part of a larger effort, I decided to write this hack. *crossing fingers*)

SUMMARY of changes:
1) Set 1 Admin CP option
2) Create MySQL table
3) Insert code into SHOWTHREAD.PHP

The code will optionally send you an email every time the threadviews are committed to the thread table.

More details are in the text file.


This is what the email you'll get looks like:
Quote:
2395 Total Views (original number of write queries)
170 Threads (new number of write queries)

93% of Writes Avoided [2225 avoided]
14:1 Original Writes vs. Consolidated Writes

Version History:

0.9 - Initial Release
0.91 - removed need for reading of temp table before writing to it (thanks to MattR). Note that if you've been using 0.9, you must re-create the my_threadviews table as it's structure has changed.
0.92 - added reporting of the ratio of Original Writes vs. Consolidated Writes. Use this to help in tweaking the frequency of updates. If the ratio is low, (e.g. 2:1, 5:1, etc) it probably means updates are occuring too often. (The ratio will also be low if you have many separate threads that are viewed a small number of times.)
1.00 - No code changes. Changed version to 1.00 since it's been stable for the past 2 months.

Show Your Support

  • This modification may not be copied, reproduced or published elsewhere without author's permission.

Comments
  #32  
Old 06-26-2002, 02:50 PM
The Piper's Avatar
The Piper The Piper is offline
 
Join Date: Apr 2002
Location: Brazil
Posts: 63
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

This seems really fantastic, and I have installed it already, but so far I've gotten no e-mails whatsoever. Since my whole site is built around vBulletin 1000 or so threadviews should happen pretty quick. I get around 10,000-12,000 page views every day! Any clue of why I didn't get any e-mails with the stats? Thanks a lot!
Reply With Quote
  #33  
Old 06-26-2002, 02:58 PM
Jujubee Jujubee is offline
 
Join Date: Nov 2001
Posts: 39
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally posted by The Piper
This seems really fantastic, and I have installed it already, but so far I've gotten no e-mails whatsoever. Since my whole site is built around vBulletin 1000 or so threadviews should happen pretty quick. I get around 10,000-12,000 page views every day! Any clue of why I didn't get any e-mails with the stats? Thanks a lot!
The only reason you wouldn't get the emails is if your server load is consistently above 2.0. Login to the Admin CP and check your current load.


Also, check the my_threadviews table via PHPmyadmin to make sure it's being written to.
Reply With Quote
  #34  
Old 06-27-2002, 02:07 AM
The Piper's Avatar
The Piper The Piper is offline
 
Join Date: Apr 2002
Location: Brazil
Posts: 63
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally posted by Jujubee


The only reason you wouldn't get the emails is if your server load is consistently above 2.0. Login to the Admin CP and check your current load.


Also, check the my_threadviews table via PHPmyadmin to make sure it's being written to.
Ok, I just got several e-mails today, so this is definately working. Been getting anywhere from 40-80% avoided queries. Most of them around the 70% mark. Definately an AWESOME hack, this should be in vB 3.x!

Thanks a lot for such a great hack and for your help!
Reply With Quote
  #35  
Old 06-27-2002, 02:19 AM
Jujubee Jujubee is offline
 
Join Date: Nov 2001
Posts: 39
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Glad to be of help.


BTW, the server my machine is on has been running at >5 load avg all day, and the temp table's got 14000 rows in it.... that's gonna be one crazy update. Will post when I get the email. aranoid:
Reply With Quote
  #36  
Old 06-27-2002, 05:25 AM
Regs Regs is offline
 
Join Date: Oct 2001
Location: Vancouver BC
Posts: 322
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Jujubee,

What has been the response at VO? Are they taking notice of the extraordinary steps you have taken to lessen the load?

Cheers,

~Regs.
Reply With Quote
  #37  
Old 06-27-2002, 06:28 AM
Jujubee Jujubee is offline
 
Join Date: Nov 2001
Posts: 39
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally posted by Regs
Jujubee,

What has been the response at VO? Are they taking notice of the extraordinary steps you have taken to lessen the load?

Cheers,

~Regs.
PMed you -- don't want to throw off this thread.
Reply With Quote
  #38  
Old 06-27-2002, 02:45 PM
Jujubee Jujubee is offline
 
Join Date: Nov 2001
Posts: 39
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Latest email from 24-hour period:

Quote:
16924 Total Views (original number of write queries)
434 Threads (new number of write queries)

16490 Writes Avoided (97% avoided)
For the statisticians out there, this is a 39:1 ratio of original queries versus consolidated queries. Which in english means there were originally 39 times the number of queries the were actually written to the disk.
Reply With Quote
  #39  
Old 06-28-2002, 08:06 AM
Mark Hewitt Mark Hewitt is offline
 
Join Date: Oct 2001
Posts: 174
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

This sounds like a good hack but is there a way to decouple it from sever load and make it just write periodically?

For whatever reason the server load monitoring doesn't work on my server. However I would like to be able to reduce the load.
Reply With Quote
  #40  
Old 06-28-2002, 02:00 PM
Jujubee Jujubee is offline
 
Join Date: Nov 2001
Posts: 39
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally posted by Mark Hewitt
This sounds like a good hack but is there a way to decouple it from sever load and make it just write periodically?

For whatever reason the server load monitoring doesn't work on my server. However I would like to be able to reduce the load.
Easy:

Find
PHP Code:
if ($loadavg[0]<&& mt_rand(1,500)=='44'){ 
and remove " $loadavg[0]<2 && "
Reply With Quote
  #41  
Old 07-02-2002, 06:20 PM
Raz's Avatar
Raz Raz is offline
 
Join Date: Oct 2001
Posts: 84
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Jujubee, a great hack! Devs, should really consider making something similar as part of VB3.

But if your load is so high, why not just disable views? They're not all that important if you're having server load problems.
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 11:26 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02344 seconds
  • Memory Usage 2,312KB
  • Queries Executed 27 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)bbcode_php
  • (6)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_post
  • (1)navbar
  • (6)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (11)post_thanks_box
  • (11)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (11)post_thanks_postbit_info
  • (10)postbit
  • (11)postbit_onlinestatus
  • (11)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete