Deferred Threadviews v1.00 (to reduce server load)

Hack name: Deferred Threadviews
Version: 1.00
Purpose:
To reduce server load by not updating the view count on every page view but instead keeping the it in memory and writing to the db only occasionally.
Functions:
? Logs threadviews to a temporary table, and periodically commits data to the thread table.
? Frequency of writes controlled by server load and a random number.
? Includes optional email notification.
vB version tested on: 2.2.6
PHP version tested on: 4.2.0
Acknowledgements: Thanks to MattR for the "store threadviews as list of non-unique ids and use count(*)" suggestion that removed 1 query per pageview.



The updating of threadviews has been identified as a main cause of high server loads. This hack basically writes threadviews to a HEAP table (i.e. temporary table residing in memory) and will only update the thread count when the load is below a specified value and a random # value is hit.

(The impetus for this hack was an email from my host: They threatened to boot me off my server due to CPU usage "abuse", so as part of a larger effort, I decided to write this hack. *crossing fingers*)

SUMMARY of changes:
1) Set 1 Admin CP option
2) Create MySQL table
3) Insert code into SHOWTHREAD.PHP

The code will optionally send you an email every time the threadviews are committed to the thread table.

More details are in the text file.


This is what the email you'll get looks like:

2395 Total Views (original number of write queries)
170 Threads (new number of write queries)

93% of Writes Avoided [2225 avoided]
14:1 Original Writes vs. Consolidated Writes

Version History:

0.9 - Initial Release
0.91 - removed need for reading of temp table before writing to it (thanks to MattR). Note that if you've been using 0.9, you must re-create the my_threadviews table as it's structure has changed.
0.92 - added reporting of the ratio of Original Writes vs. Consolidated Writes. Use this to help in tweaking the frequency of updates. If the ratio is low, (e.g. 2:1, 5:1, etc) it probably means updates are occuring too often. (The ratio will also be low if you have many separate threads that are viewed a small number of times.)
1.00 - No code changes. Changed version to 1.00 since it's been stable for the past 2 months.

[eva2000]

Originally posted by PPN
Here is a file to let you use a cron job to do it instead of in showthread, its more efficent in my opinion.

Open up the file and change the path to your config.php, upload now to your server, and then create a cron job.

Cpanel users can do this in the Advanced section

just type

php /home/path/to/deffered_views.php

and set it to run every 30mins

interesting idea... anything else needed to be done (edit files etc ? )

or just this 2 steps before this cronjob ?

[1) ADMIN CP CHANGES:

	 Admin CP -> vB Options -> *NIX Load Limit -> set to any "non-zero number"

NOTES: I use 10 as the "non-zero number", but you can set it to anything you want,
You can view your current load on the main page of the Admin CP.
Remember that your board will be inaccessable when the server load goes
above this number, so don't set it too low! The "correct" number is a matter of debate.



2) CREATE MYSQL TABLE

PHP Code:

        CREATE TABLE my_threadviews (
          threadid mediumint(9) NOT NULL default '0'
        ) TYPE=HEAP; 


NOTES: Via PHPmyAdmin: paste the above into the box labeled "Run SQL query/queries on database ...."

[Martin64]

Sounds like a great hack for big size boards, I would install it if I hadn't moved my forums to a seperate server already (had some troubles earlier).

Good job!

[Kaelon]

I've just installed this hack on v2.2.9, and I'll report on any improvements. My forums regularly have between 60 and 80 users and I've been looking to pin-point and reduce weird server spikes that raise my load (which is ordinarily between 1 and 2) up to 3 and sometimes 4 or 5.

Kaelon

[Kaelon]

Some preliminary results on my board:

273 Total Views (original number of write queries)
131 Threads (new number of write queries)

52% of Writes Avoided [142 avoided]
2:1 Original Writes vs. Consolidated Writes

This was in a 17 minute period. So, good deal thus far.

Kaelon

[Mijae]

Last time I tried to use this hack it reset my Who's Online record to 0, but I will install it again now that I have upgraded

[Mijae]

I get safe mode errors when I set *NIX Load Limit to anything but 0. Damn.

[mini2]

Installed first time no problem, thanks!

Paul

[Overgrow]

Excellent! I just stumbled on this while searching for something else. Should be mandatory for big boards sticking with 2.x for a while.

[Overgrow]

I'm rarely getting more than 50% writes avoided.. which may be expected when you have many different threads that users are opening. I've turned the delay setting up to 1000 since I don't want to wait too long for the updates even if it would mean a higher hit ratio.

Also for those of you running separate DB servers, the load checker will do nothing as the built in load check is only on your web server. fastforward released a remote db-server load checker that works fine here, I'll find it if anyone needs.

[scotty]

hm - I installed the hack and everything worked fine...

...except the "mark read" feature of thread, that the icon of the thread is changed.

anyone else reporting this problem?