Go Back   vb.org Archive > vBulletin Modifications > Archive > vB.org Archives > General > Big Board Discussions
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools
Big Board - Datastore Setup has large amount of memory. Details »»
Big Board - Datastore Setup has large amount of memory.
Version: , by Hastin Hastin is offline
Developer Last Online: Feb 2015 Show Printable Version Email this Page

Version: Unknown Rating:
Released: 09-16-2007 Last Update: Never Installs: 0
 
No support by the author.

Hello All,

Seem to be having an issue with my board (18k members, 2.1 million posts). My datastore memory usage seems high - about 7M. We just switch to a new server (Dual Opteron, 4GB of RAM), and during our popular time (about 1k members), we are still eating up though it. Load on the CPU is fine, but memory usage goes through the roof.

I've got xcache going for both PHP, and the datastore. As well, I've got sphinx search for the searching. I've disabled all my hacks, and then checked the usage, and it's about the same. Any other tips?

Thanks!

Quote:
Datastore Setup
Time Before: 0.00286 seconds
Time After: 0.04625 seconds
Time Taken: 0.04339 seconds

Memory Before: 321.609 KB
Memory After: 7,379.039 KB
Memory Used: 7,057.430 KB

Show Your Support

  • This modification may not be copied, reproduced or published elsewhere without author's permission.

Comments
  #2  
Old 09-16-2007, 04:40 PM
Zachery's Avatar
Zachery Zachery is offline
 
Join Date: Jul 2002
Location: Ontario, Canada
Posts: 11,440
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

How many usergroups and forums do you have?
Reply With Quote
  #3  
Old 09-16-2007, 04:42 PM
Hastin Hastin is offline
 
Join Date: Sep 2004
Posts: 14
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

We've got a club thing going on, so I'd say there's about 40 usergroups, and 60ish forums.

Is that too many?
Reply With Quote
  #4  
Old 09-16-2007, 04:45 PM
Zachery's Avatar
Zachery Zachery is offline
 
Join Date: Jul 2002
Location: Ontario, Canada
Posts: 11,440
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Its not too many but its why you're going to see such high memory values. Its got to load permissions for every forum and every usergroup on the forumhome.
Reply With Quote
  #5  
Old 09-16-2007, 04:55 PM
Hastin Hastin is offline
 
Join Date: Sep 2004
Posts: 14
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Okay. We were thinking of breaking out to a new system, and this is the kicker.
Reply With Quote
  #6  
Old 09-16-2007, 05:19 PM
Zachery's Avatar
Zachery Zachery is offline
 
Join Date: Jul 2002
Location: Ontario, Canada
Posts: 11,440
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Reducing your usergroups would be a good first step, or at least trying to consoldate if possible.
Reply With Quote
  #7  
Old 09-17-2007, 07:00 PM
Marco van Herwaarden Marco van Herwaarden is offline
 
Join Date: Jul 2004
Posts: 25,415
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

I do the technical administration of a board that has an similar server. Concurrent online users vary from ~600 up to peaks of ~2.400. The board has over 600 forums with around 20 usergroups, a very bad style (from performance POV) and a few modifications that also add up to degrading the performance. It however runs smooth most of the time. Only at the real extreme peaks (we sometimes go from 1.000 to over 2.200 in just a few minutes) combined with the mail server sending out an mail listing, we do have a short degradation of the responsetimes.

What harddisk do you use?

Also did you already try posting in the Server Configuration forum at vBulletin.com?
Reply With Quote
  #8  
Old 09-27-2007, 12:33 PM
Dismounted's Avatar
Dismounted Dismounted is offline
 
Join Date: Jun 2005
Location: Melbourne, Australia
Posts: 15,047
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Marco van Herwaarden View Post
I do the technical administration of a board that has an similar server. Concurrent online users vary from ~600 up to peaks of ~2.400. The board has over 600 forums with around 20 usergroups, a very bad style (from performance POV) and a few modifications that also add up to degrading the performance. It however runs smooth most of the time. Only at the real extreme peaks (we sometimes go from 1.000 to over 2.200 in just a few minutes) combined with the mail server sending out an mail listing, we do have a short degradation of the responsetimes.
Now that's we call a BIG board .
Reply With Quote
  #9  
Old 09-27-2007, 01:25 PM
edgecutioner edgecutioner is offline
 
Join Date: Jul 2007
Location: Qatar
Posts: 120
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by Dismounted View Post
Now that's we call a BIG board .
aren't I a fool to think that my board was big... anyways, i'm satisfied with it and isn't that the goal of every forums? to make every member happy?
Reply With Quote
  #10  
Old 09-27-2007, 02:16 PM
Marco van Herwaarden Marco van Herwaarden is offline
 
Join Date: Jul 2004
Posts: 25,415
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Actually i do not even consider that as a big board, medium sized maybe, or high-medium based on the extensive use of sub-forums and other resource intensive stuff. A real big board would need to run on a good multi-server setup.
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 01:10 AM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.05780 seconds
  • Memory Usage 2,289KB
  • Queries Executed 25 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (3)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_post
  • (1)navbar
  • (6)navbar_link
  • (120)option
  • (10)post_thanks_box
  • (10)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (10)post_thanks_postbit_info
  • (9)postbit
  • (10)postbit_onlinestatus
  • (10)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete