vb.org Archive - Deferred Threadviews v1.00 (to reduce server load)

Page 2 of 3

Show 40 post(s) from this thread on one page

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)

- - Deferred Threadviews v1.00 (to reduce server load) (https://vborg.vbsupport.ru/showthread.php?t=40137)

Jujubee

07-03-2002 04:28 AM

Quote:

Originally posted by Raz
But if your load is so high, why not just disable views? They're not all that important if you're having server load problems.

I consider views important to spur participation. Zeroes all over makes the board seem dead. :)

BTW, latest email after about 3 days without an update:

Quote:

19063 Total Views (original number of write queries)
400 Threads (new number of write queries)

18663 Writes Avoided (98% avoided)
48:1 Original Writes:New Writes

Erwin

07-04-2002 03:10 AM

Cool. I've installed this. Will let you know how it goes for me.

Erwin

07-04-2002 08:19 PM

Okay, my first email:

Quote:

4018 Total Views (original number of write queries)
535 Threads (new number of write queries)

3483 Writes Avoided (87% avoided)

Not bad at all... :)

Thanks for a great hack!

Jujubee

07-04-2002 10:12 PM

Ok, I've updated the hack a little. I've added reporting of the ratio of Original Writes vs. Consolidated Writes.

Email will look like:

Quote:

2395 Total Views (original number of write queries)
170 Threads (new number of write queries)

93% of Writes Avoided [2225 avoided]
14:1 Original Writes vs. Consolidated Writes

I use the ratio to tweak the frequency of the updates. A low ratio means updates are occuring too often. There is no "correct" ratio as it depends on the number of threads being updated -- you don't want too many threads being updated at the same time.

IMHO, ratios of less than 5 probably means that updates are too frequent.

(for those using a previous version, only the code has changed -- the db table remains unchanged)

Erwin

07-08-2002 01:21 AM

Possible bug:

The thread views are not being recorded properly. I have a few threads with many replies where the thread views are persistently at 0 and not updated.

Do these threadviews get updated when the db write occurs?

Jujubee

07-08-2002 02:30 AM

Quote:

Originally posted by Erwin
Possible bug:

The thread views are not being recorded properly. I have a few threads with many replies where the thread views are persistently at 0 and not updated.

Do these threadviews get updated when the db write occurs?

The threadview count is updated only when the db write occurs. So depending on the frequency of your updates, the count will remain at 0 for new threads for a while.

Is the db being written to daily? (i.e. how often do you get an email?)

Martin CX

07-08-2002 01:11 PM

This looks like a marvelous hack!

I would install it, but since vB is the auxillary forum on my site (waiting for vB3) I don't have the need for it just yet. I hope, though, vB3 has a similar feature (or perhaps you'll be able to port it real quick? ;)) since I most often have around at 2-300 online users/guests simultaneously during the day hours.

Erwin

07-08-2002 11:22 PM

Quote:

Originally posted by Jujubee

The threadview count is updated only when the db write occurs. So depending on the frequency of your updates, the count will remain at 0 for new threads for a while.

Is the db being written to daily? (i.e. how often do you get an email?)

I figured it out. :) Thanks!

Once a day or once every 2 days. It's working fine, and there is a noticeable difference.

stilger

08-04-2002 12:25 PM

Hello.. Im thinking about installing this hack on a site that I support that averages about 400 users logged in at a time. Sometimes peaking to around 2000 (well vbulletin reports the most ever as being 2410 I don't know how much you can trust this though). We currently spread our load over 2 servers using Round Robin DNS and have a separate server for all database functions. Will this hack work on multiple server site? I am thinking because of how this hack works there should be no problem but I wanted to make sure. It sounds like this hack would help with the occasional To Many Connections messages we get..... Thanks for the info.

Tim Wheatley

08-07-2002 10:36 PM

Installing tonight: 12:40am UK time 8-AUG.

Will post my first e-mail. Looks like it'll really help me. We get a lot of guests visiting and not posting (just viewing) as well as 200-300 logged in users.

Tim Wheatley

08-09-2002 09:24 PM

I've decided to keep this hack installed. But have not decided to keep the original settings.

I have removed the line which tells it to work under a certain server usage. And have just got it running so it moves them to the database periodically.

I've attached a text file which is a downloaded post from my admin forum on my forums, this contains over 50 e-mails sent by this hack.

I have now commented out the e-mail lines because I'm happy with it's performance at these settings.

FFMania

08-11-2002 02:42 AM

Have some problem......when I find this code....

PHP Code:


		
			
    if ($noshutdownfunc) {

      $DB_site->query("UPDATE thread SET views=views+1 WHERE threadid='$threadid'");

    } else {

      $shutdownqueries[]="UPDATE LOW_PRIORITY thread SET views=views+1 WHERE threadid='$threadid'";

    }

I can't fiind it since I have lesane Store hack installed..so the code becomes...

PHP Code:


		
			
if ($noshutdownfunc) {

  $DB_site->query("UPDATE thread SET views=views+1 WHERE threadid='$threadid'");

  

  // Store hack by Lesane



  $storeview=$DB_site->query_first("select views, postuserid from thread WHERE threadid='$threadid'");

  $views=$storeview[views];

  $postuserid=$storeview[postuserid];



  $storeadmin = $DB_site->query_first("SELECT * FROM storeadmin");

  $view1=$storeadmin[view1];

  $view2=$storeadmin[view2];

  $view3=$storeadmin[view3];

  $view4=$storeadmin[view4];

  $view5=$storeadmin[view5];

  $viewpoint1=$storeadmin[vpoint1];

  $viewpoint2=$storeadmin[vpoint2];

  $viewpoint3=$storeadmin[vpoint3];

  $viewpoint4=$storeadmin[vpoint4];

  $viewpoint5=$storeadmin[vpoint5];



  if ($views == $view1) 

  {

    $DB_site->query("UPDATE user SET storep=storep+$viewpoint1 where userid='$postuserid'");

  }

  elseif ($views == $view2) 

  {

    $DB_site->query("UPDATE user SET storep=storep+$viewpoint2 where userid='$postuserid'");

  }

  elseif ($views == $view3) 

  {

    $DB_site->query("UPDATE user SET storep=storep+$viewpoint3 where userid='$postuserid'");

  }

  elseif ($views == $view4) 

  {

    $DB_site->query("UPDATE user SET storep=storep+$viewpoint4 where userid='$postuserid'");

  }

  elseif ($views == $view5) 

  {

    $DB_site->query("UPDATE user SET storep=storep+$viewpoint5 where userid='$postuserid'");

  }

  

} else {

  $shutdownqueries[]="UPDATE LOW_PRIORITY thread SET views=views+1 WHERE threadid='$threadid'";

Any idea what I should do?

Courage

08-11-2002 12:42 PM

I'm the only one with a problem.. :)

After I installed this hack I got 200 Email errors in ~ 5 min :o

Quote:

Database error in vBulletin :

Link-ID == false, connect failed
mysql error: Too many connections

mysql error number: 1040

I'm using Vb 2.2.2 witht A LOT of hacks :o

Tim Wheatley

08-11-2002 01:35 PM

Update! This is going to be uninstalled from my boards. Last nights busy period was very busy and all went well. Then it got to about 1am when the visitors start to go down and BOOM - the board was killed for 5mins. I kept trying to enter and when I eventually did the server stats said that usage had been:

10.40 7.00

This caused massive problems and also (I believe) all the view counts were lost.

It seems that when it comes to updating the stats it's such a massive undertaking that it kills the server pc.

Tim Wheatley

08-23-2002 05:42 PM

I have reinstalled this.

Now have it functioning correctly.

My 'normal users online' is around 100-150 at once. Peaking maybe at 250 at times. A lot of guests visit and a lot of people read threads and don't post, let me know if anyone wants to see my settings for it...

Capt PPRuNe

08-24-2002 03:08 PM

Errr... can someone shed some light on the reason every time I try to download this hack i get v0.91 instead of v0.92?

having installed it I am getting the emails but without the ratio bit added to v0.92.

Looks like it will be a great hack as i have over 50,000 registered users and at least twice that many guests with an average of 250-350 users on at any one time peaking to about 550 every day.

Dedicated server with average load at around 1.5-2.5 but can peak to about 6 or 7 and ocassionally even more.

Jujubee

08-25-2002 04:24 PM

Quote:

Originally posted by stilger
Hello.. Im thinking about installing this hack on a site that I support that averages about 400 users logged in at a time. Sometimes peaking to around 2000 (well vbulletin reports the most ever as being 2410 I don't know how much you can trust this though). We currently spread our load over 2 servers using Round Robin DNS and have a separate server for all database functions. Will this hack work on multiple server site? I am thinking because of how this hack works there should be no problem but I wanted to make sure. It sounds like this hack would help with the occasional To Many Connections messages we get..... Thanks for the info.

Doh! I stopped getting notification emails... sorry for the long delay. :rolleyes:

I don't see why this wouldn't work with multiple web servers, but I can't be 100% without intimate knowledge of your setup. (that's me covering my ass :p )

The hack makes cumulative updates to the db, so while the two webservers would update the db at different times, it should still work. You'll probably have to tweak the update frequency to get it to your liking, but otherwise, I don't forsee a problem.

Jujubee

08-25-2002 04:31 PM

Quote:

I'm not familiar with the Lesane hack, but it looks way more complicated than this hack. Maybe you could ask them...

stilger

08-27-2002 12:29 AM

I just installed this. Tweaking now.

I would like to see your settings....

Quote:

Originally posted by Tim Wheatley
I have reinstalled this.

Now have it functioning correctly.

My 'normal users online' is around 100-150 at once. Peaking maybe at 250 at times. A lot of guests visit and a lot of people read threads and don't post, let me know if anyone wants to see my settings for it...

Darth Cow

08-27-2002 04:56 AM

Hmmm... thank you. Just installed and should do wonders for our forums :cool:.

Tim Wheatley

08-31-2002 02:01 AM

Code:

if ($loadavg[0]<1 && mt_rand(1,5)=='44'){

Darth Cow

09-03-2002 03:41 AM

Quote:

Originally posted by Raz
Jujubee, a great hack! Devs, should really consider making something similar as part of VB3.

I believe they already have :). I read somewhere on the alpha forums that thread views are now only updated every hour :cool:.

Ritsui

09-03-2002 11:53 PM

How about scaling dynamically with load? Something like:

PHP Code:


		
			
$loadMult = (($loadavg[0] * 100) +5);    //change these values to tweak frequency of writes

if ( mt_rand(1,$loadMult)=='1' ) {

...snip...

This would seem to work better at least with smaller forums that have occasional very heavy peaks where you still want views to be updated frequently, but less based proportionally on load.

Jujubee

09-06-2002 01:09 AM

Quote:

Originally posted by Tim Wheatley

Code:

if ($loadavg[0]<1 && mt_rand(1,5)=='44'){

Tim, have you been getting any emails with the above settings? PHP will produce a random number between 1 and 5. This won't ever equal 44. :)

If you want more frequent updates, change the 44 to 1, then change the 5 to something higher. A limit of 5 would mean the views would be update once every 5 pageviews -- not much of a savings IMHO. :)

Jujubee

09-06-2002 01:13 AM

Quote:

Originally posted by Ritsui
How about scaling dynamically with load? Something like:

PHP Code:

$loadMult = (($loadavg[0] * 100) +5); //change these values to tweak frequency of writes if ( mt_rand(1,$loadMult)=='1' ) { ...snip...

This would seem to work better at least with smaller forums that have occasional very heavy peaks where you still want views to be updated frequently, but less based proportionally on load.

Sure, this would work. But personally, I don't want it doing any updates when the load is high. I don't mind losing views if the server crashes.

With your code, for example, the load could be 50 and it would still update (if the correct random number was produced). Hundreds of consecutive db writes when the load is at 50+ won't be a pretty sight. :)

Mijae

12-04-2002 10:14 PM

Quote:

Originally posted by FFMania
Have some problem......when I find this code....

PHP Code:

if ($noshutdownfunc) { $DB_site->query("UPDATE thread SET views=views+1 WHERE threadid='$threadid'"); } else { $shutdownqueries[]="UPDATE LOW_PRIORITY thread SET views=views+1 WHERE threadid='$threadid'"; }

I can't fiind it since I have lesane Store hack installed..so the code becomes...

PHP Code:

if ($noshutdownfunc) { $DB_site->query("UPDATE thread SET views=views+1 WHERE threadid='$threadid'"); // Store hack by Lesane $storeview=$DB_site->query_first("select views, postuserid from thread WHERE threadid='$threadid'"); $views=$storeview[views]; $postuserid=$storeview[postuserid]; $storeadmin = $DB_site->query_first("SELECT * FROM storeadmin"); $view1=$storeadmin[view1]; $view2=$storeadmin[view2]; $view3=$storeadmin[view3]; $view4=$storeadmin[view4]; $view5=$storeadmin[view5]; $viewpoint1=$storeadmin[vpoint1]; $viewpoint2=$storeadmin[vpoint2]; $viewpoint3=$storeadmin[vpoint3]; $viewpoint4=$storeadmin[vpoint4]; $viewpoint5=$storeadmin[vpoint5]; if ($views == $view1) { $DB_site->query("UPDATE user SET storep=storep+$viewpoint1 where userid='$postuserid'"); } elseif ($views == $view2) { $DB_site->query("UPDATE user SET storep=storep+$viewpoint2 where userid='$postuserid'"); } elseif ($views == $view3) { $DB_site->query("UPDATE user SET storep=storep+$viewpoint3 where userid='$postuserid'"); } elseif ($views == $view4) { $DB_site->query("UPDATE user SET storep=storep+$viewpoint4 where userid='$postuserid'"); } elseif ($views == $view5) { $DB_site->query("UPDATE user SET storep=storep+$viewpoint5 where userid='$postuserid'"); } } else { $shutdownqueries[]="UPDATE LOW_PRIORITY thread SET views=views+1 WHERE threadid='$threadid'";

Any idea what I should do?

I have the same problem, I tried adding your hack anyways deleting some lines from Store Hack but I started getting php errors, so I have no idea what to do.

Erwin

12-05-2002 01:18 AM

The good news is that vB3 will have this built-in. :)

eoc_Jason

12-31-2002 06:52 AM

I like this hack but it has one quirk... late at night when not many people are on it doesn't update that often and people are asking why...

Would it be possible to code in a timer so that it would write every 5min or so? But also have the random number generation so that it will write properly during normal usage?

ladyfyre

01-04-2003 05:20 PM

How can i set this so that it updates every 500 threadviews, without any conditionals? The server load on the mysql server is not a factor...it is the php server load that is an issue and something in apc is not playing nicely with this hack. I would like to keep it, but would prefer being able to set a fixed # at which it will dump.

I have been trying to use this:

PHP Code:


		
			
if (mt_rand(1,500)=='44'){//change these values to tweak frequency of writes

But it is not updating at 500 views.

Scott MacVicar

01-05-2003 06:20 PM

Here is a file to let you use a cron job to do it instead of in showthread, its more efficent in my opinion.

Open up the file and change the path to your config.php, upload now to your server, and then create a cron job.

Cpanel users can do this in the Advanced section

just type

php /home/path/to/deffered_views.php

and set it to run every 30mins

eva2000

01-08-2003 09:21 PM

Quote:

Originally posted by PPN
Here is a file to let you use a cron job to do it instead of in showthread, its more efficent in my opinion.

Open up the file and change the path to your config.php, upload now to your server, and then create a cron job.

Cpanel users can do this in the Advanced section

just type

php /home/path/to/deffered_views.php

and set it to run every 30mins

interesting idea... anything else needed to be done (edit files etc ? )

or just this 2 steps before this cronjob ?

Quote:

[1) ADMIN CP CHANGES:

Code:

Admin CP -> vB Options -> *NIX Load Limit -> set to any "non-zero number"

NOTES: I use 10 as the "non-zero number", but you can set it to anything you want,
You can view your current load on the main page of the Admin CP.
Remember that your board will be inaccessable when the server load goes
above this number, so don't set it too low! The "correct" number is a matter of debate. :)

2) CREATE MYSQL TABLE

PHP Code:


		
			
        CREATE TABLE my_threadviews (

          threadid mediumint(9) NOT NULL default '0'

        ) TYPE=HEAP;

NOTES: Via PHPmyAdmin: paste the above into the box labeled "Run SQL query/queries on database ...."

Martin64

01-08-2003 11:56 PM

Sounds like a great hack for big size boards, I would install it if I hadn't moved my forums to a seperate server already (had some troubles earlier).

Good job! :)

Kaelon

02-08-2003 02:03 AM

I've just installed this hack on v2.2.9, and I'll report on any improvements. My forums regularly have between 60 and 80 users and I've been looking to pin-point and reduce weird server spikes that raise my load (which is ordinarily between 1 and 2) up to 3 and sometimes 4 or 5.

Kaelon

Kaelon

02-08-2003 03:23 AM

Some preliminary results on my board:

Quote:

273 Total Views (original number of write queries)
131 Threads (new number of write queries)

52% of Writes Avoided [142 avoided]
2:1 Original Writes vs. Consolidated Writes

This was in a 17 minute period. So, good deal thus far.

Kaelon

Mijae

02-25-2003 03:37 AM

Last time I tried to use this hack it reset my Who's Online record to 0, but I will install it again now that I have upgraded :)

Mijae

02-25-2003 03:54 AM

I get safe mode errors when I set *NIX Load Limit to anything but 0. Damn.

mini2

02-25-2003 08:04 PM

Installed first time no problem, thanks!

Paul

Overgrow

03-04-2003 11:55 AM

Excellent! I just stumbled on this while searching for something else. Should be mandatory for big boards sticking with 2.x for a while.

Overgrow

03-04-2003 02:12 PM

I'm rarely getting more than 50% writes avoided.. which may be expected when you have many different threads that users are opening. I've turned the delay setting up to 1000 since I don't want to wait too long for the updates even if it would mean a higher hit ratio.

Also for those of you running separate DB servers, the load checker will do nothing as the built in load check is only on your web server. fastforward released a remote db-server load checker that works fine here, I'll find it if anyone needs.

scotty

05-06-2003 09:35 AM

hm - I installed the hack and everything worked fine...

...except the "mark read" feature of thread, that the icon of the thread is changed.

anyone else reporting this problem?

All times are GMT. The time now is 07:06 AM.

Page 2 of 3

Show 40 post(s) from this thread on one page

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.01775 seconds Memory Usage 1,897KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (3)bbcode_code_printable (9)bbcode_php_printable (17)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)pagenav (1)pagenav_curpage (2)pagenav_pagelink (1)post_thanks_navbar_search (1)printthread (40)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start pagenav_page pagenav_complete bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: