Go Back   vb.org Archive > vBulletin Article Depository > Read An Article > vBulletin 4 Articles
Reload this Page FractalizeR: VBulletin 4 extensive optimization guide (Part I)
FAQ Community Calendar Today's Posts Search

Reply
Page 2 of 3 1 2 3
 
Thread Tools
FractalizeR: VBulletin 4 extensive optimization guide (Part I)
FractalizeR's Avatar
FractalizeR
Join Date: Oct 2005
Posts: 368

 

Russia, Moscow
Show Printable Version Email this Page Subscription
FractalizeR FractalizeR is offline 05-02-2010, 10:00 PM
Preface

Actually, I never bothered to optimize my server for VBulletin, because the server we purchased for forum is really powerful. But once upon a time logwatch told me, that smartd told her, that he found several relocated sectors on one of server HDDs. And then instantly I realized, that many time ago when I assembled that server, I configured HDDs to be in RAID-0 (striped mode)? Oh?. When you will make some stupid thing on your server, please keep in mind, that there are some people that can do things even more stupid?. Nothing critical happened, because I, actually, have all database backups. Database snapshots are made on server using this script of mine every hour or so (BTW, database backup on my server of 1.12Gb database (reported at admincp) takes around a minute to complete together with compression).

So, we had to visit datacenter and remove/replace HDDs. To allow our forum users to continue using forum until we fix that, we moved our forum to a new server. After forum move, I upgraded it to v4 to benefit from friendly URLs. And there it started. Server load almost immediately reached 4.0 (server had two dualcore CPUs, so this is almost 100% of standard load) and those were even not peak hours. Of course, that was not because of V4 (I turned off both that huge CMS issuing ~200 queries on home page and blogs). This was because that new server where the forum was moved to was less powerful. Of course, 4.0 is not a heavy load for server (once one of my servers reached around 100.0 and still I was able to login by SSH ).
But nevertheless, that was the moment, where optimization hysteria first visited my head.

I used my Linux knowledge combined with Google power to provide you with this optimization guide. It is concentrated around things, that can be easily and simply applied to your server without too much work. Most of the time it explains basic things, that seem apparent. But I am sure, 90% of beginner administrators never use them altogether. This guide does not pretend to be complete one. To follow most of it, you have to have some basic Linux knowledge and root access to your server. However, many of the described optimizations can be done just with Control Panel administrator and FTP accesses. Please feel free to criticize it and add things to it. If you are going to republish it, I will be most pleased, but please keep my copyright on it and a link to my website where I will try to maintain up-to-date version of it with all improvements and corrections you might send me.
This guide mostly provide things, that are safe to your server or could be easily rolled back. However, please keep in mind, that you apply them on your own peril.
If
  • you have RedHat or CentOS operating system on your server
  • your forum have around 200.000 ? 400.000 posts or more
  • your forum have around 100 users online or more at regular daytime
  • you have root access to your forum machine and are willing to provide me with root password
  • you are willing to report optimization results after some time on vbulletin.org
you can contact me to help you to apply some or all optimizations I described here. Please note, that giving someone root password to your machine can be dangerous. Root access allows administrator to do EVERYTHING on the machine. Think about it twice before asking someone to help you with forum using root password. In all cases you need to change root password immediately after all tasks you requested are performed. This can be easily done by logging to your machine as root and issuing passwd command.
I cannot promise to do all instantly because of the amount of daily job I have, but will do my best.

Generic all-good server optimizations

This section will list optimizations, that are, actually, good for any forum and any server.

Optimize all *.png images in forum folder

VBulletin forum images and, especially, addon and style images can be optimized reducing their size up to 50% from original. Install OptiPNG tool (for example, from RPMForge repository. If you are on RedHat or CentOS you can install RPMForge repository quickly this way and install OptiPNG from there using yum install optipng after.) and use it to optimize forum images. On Linux you can just enter forum directory and execute there the following command, after you have optipng rpm installed:
find -name "*.png" | xargs optipng This will run optipng command on every *.png image in the directory you are currently in (check that with pwd, it should be forum directory) and below. OptiPNG will use it?s own default settings this way. They are ok 99% of cases. I leave studying it?s options to you.
If you can?t run optipng on server, just download all images and optimize them on your home PC (there is a Windows version of OptiPNG too). But remember to do that again after forum update, because update package may overwrite your optimized images.
OptiPNG does not affect the quality of them. At least it is not supposed to, so? you have a backup anyway, right?

Prevent browser from resize images

If you are developing a skin or just make new forum status icons, keep in mind, that you should not make browser resize your images (for example, if your icon on disk is 128?128, png, but you specified in your style, that it is only <img width=?48″ height=?48″ src=???/>. This will make an impression on user, that page is loading too slow (browser will be busy resizing all your images and that takes CPU time).

Use external YUI

It is faster to use YUI from Yahoo or Google servers. You can set which one to use YUI from in VBulletin Options -> Server optimizations -> Use remote YUI. I prefer Google

Move your attachments into filesystem

Move your attachments into filesystem. Go to your AdminCP, expand the Attachments menu on the left side and click Attachments Storage Type.

Move Avatars into filesystem

The same as with attachments, you can move avatars into file system. Go to your AdminCP, expand the Avatars menu on the left side and click Avatars Storage Type.

Prevent search engine spiders from visiting areas of your forums that they do not need to spider

Create a robots.txt file with the following and upload it to your public_html directory:
User-agent: *
Disallow: /forums/admincp/
Disallow: /forums/clientscript/
Disallow: /forums/cpstyles/
Disallow: /forums/customavatars/
Disallow: /forums/customprofilepics/
Disallow: /forums/images/
Disallow: /forums/modcp/
Disallow: /forums/ajax.php
Disallow: /forums/attachment.php
Disallow: /forums/calendar.php
Disallow: /forums/cron.php
Disallow: /forums/editpost.php
Disallow: /forums/global.php
Disallow: /forums/image.php
Disallow: /forums/inlinemod.php
Disallow: /forums/joinrequests.php
Disallow: /forums/login.php
Disallow: /forums/member.php
Disallow: /forums/memberlist.php
Disallow: /forums/misc.php
Disallow: /forums/moderator.php
Disallow: /forums/newattachment.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/poll.php
Disallow: /forums/postings.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/profile.php
Disallow: /forums/register.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/search.php
Disallow: /forums/sendmessage.php
Disallow: /forums/showgroups.php
Disallow: /forums/subscription.php
Disallow: /forums/threadrate.php
Disallow: /forums/usercp.php
Disallow: /forums/usernote.php If you use something other than /forums/ for your forums directory, you will want to adjust the paths accordingly (just like /usernote.php if your forum is in www root)

Disable File Access Time updates in your filesystem

If your forum is running on VDS/VPS or dedicated server, you need to disable updating access times for your filesystem to improve performance. You can do that by adding noatime attribute into your /etc/fstab so that the partition you have forum files on have this option enabled like this for example:

Code:
/dev/md2                 /                       ext3    defaults,noatime        1 1
For changes to take effect, you need to reboot your server with reboot or shutdown -r now command.
Please note, if you make a mistake in editing fstab, you may end up with non-booting system and you will need to boot Linux into single user mode or startup from emergency disk to repair this file. This is especially dangerous if you have no physical access to your server.

Install SPRI

SPRI is a small bash script, that runs regularly and adjusts process priorities to distribute system load. You can get it here. Sometimes benefits of installing SPRI to your server are as much as 20% lowered server load.
You will need to wget it, untar it, and then run /install.sh from unpacked folder.

Continue to Part II
Reply With Quote
  #12  
Old 06-01-2010, 12:47 PM
ChopSuey ChopSuey is offline
 
Join Date: Jun 2009
Location: Alaska
Posts: 2,140
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Okay! I installed OptiPNG on my my server dont know if it loads great because i have high speed internet lol.....can somebody tell me?
Reply With Quote
  #13  
Old 07-23-2010, 06:45 PM
Bluefin221 Bluefin221 is offline
 
Join Date: Jul 2008
Location: Scotland
Posts: 290
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
I've installed RPMForge and now when I try to install optipng or even memcached I get the following result?

yum install optipng
Loaded plugins: rhnplugin, security
Excluding Packages in global exclude list
Finished
Setting up Install Process
No package optipng available.
Nothing to do

This happens for both optipng and memcache.

--------------- Added [DATE]1279914623[/DATE] at [TIME]1279914623[/TIME] ---------------

When I try to install the file I'm also getting this:

./rpmforge_inst.sh
FractalizeR's RPM Forge Repository Installation Script for CentOS v0.91

Detected architecture is x86_64
Detected CentOS Release: Enterprise, Major=Enterprise, Minor=
Repository file /etc/yum.repos.d/CentOS-Base.repo is not readable or writtable!

When I go to that folder CentOS-Base.repo does not even exist only file there is rhel-debuginfo.repo
Reply With Quote
  #14  
Old 07-24-2010, 06:56 AM
FractalizeR's Avatar
FractalizeR FractalizeR is offline
 
Join Date: Oct 2005
Location: Russia, Moscow
Posts: 368
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
You seem to run not CentOS Linux. Please post here the result of uname -a command.
Reply With Quote
  #15  
Old 07-24-2010, 02:43 PM
Bluefin221 Bluefin221 is offline
 
Join Date: Jul 2008
Location: Scotland
Posts: 290
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Quote:
Originally Posted by FractalizeR View Post
You seem to run not CentOS Linux. Please post here the result of uname -a command.
I managed to get it installed through another route, I ran the png script in my forum route and also installed memcache. However I do not know if I have it working properly. When I edited the config.php file and uncommented out the */ /* at the top and bottom it returned an error on the site pointing towards the class_datastore.php file held within the includes folder.
Reply With Quote
  #16  
Old 07-25-2010, 07:37 AM
FractalizeR's Avatar
FractalizeR FractalizeR is offline
 
Join Date: Oct 2005
Location: Russia, Moscow
Posts: 368
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Please provide complete error message here.
Reply With Quote
  #17  
Old 07-28-2010, 02:47 AM
Bluefin221 Bluefin221 is offline
 
Join Date: Jul 2008
Location: Scotland
Posts: 290
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Quote:
Originally Posted by FractalizeR View Post
Please provide complete error message here.
Sorry I've not got back to you I didn't want to bring my board down while it had a lot of users.

Fatal error: Memcache is not installed in [path]/includes/class_datastore.php on line 183
Reply With Quote
  #18  
Old 07-28-2010, 06:35 AM
FractalizeR's Avatar
FractalizeR FractalizeR is offline
 
Join Date: Oct 2005
Location: Russia, Moscow
Posts: 368
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
So, it looks like your PHP does not have Memcache extension.
Reply With Quote
  #19  
Old 07-29-2010, 02:14 AM
Bluefin221 Bluefin221 is offline
 
Join Date: Jul 2008
Location: Scotland
Posts: 290
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Quote:
Originally Posted by FractalizeR View Post
So, it looks like your PHP does not have Memcache extension.
I installed the memcache PHP extension and still errors persisted, so I decided to start from the beginning and removed rpm through ssh interface, I noticed though when I installed I was presented with errors upon installation.

Code:
Configuration file /etc/yum/pluginconf.d/rhnplugin.conf not found
Unable to find configuration file for plugin rhnplugin
Configuration file /etc/yum/pluginconf.d/security.conf not found
Unable to find configuration file for plugin security
After the install I tried to run memcached again on install I'm presented with numerous errors.

Code:
memcached-1.4.5-1.el5.rf.x86_64 from rpmforge has depsolving problems
--> Missing Dependency: perl(AnyEvent::Socket) is needed by package memcached-1.4.5 -1.el5.rf.x86_64 (rpmforge)
memcached-1.4.5-1.el5.rf.x86_64 from rpmforge has depsolving problems
--> Missing Dependency: perl(AnyEvent) is needed by package memcached-1.4.5-1.el5.r f.x86_64 (rpmforge)
memcached-1.4.5-1.el5.rf.x86_64 from rpmforge has depsolving problems
--> Missing Dependency: perl(AnyEvent::Handle) is needed by package memcached-1.4.5 -1.el5.rf.x86_64 (rpmforge)
memcached-1.4.5-1.el5.rf.x86_64 from rpmforge has depsolving problems
--> Missing Dependency: perl(YAML) is needed by package memcached-1.4.5-1.el5.rf.x8 6_64 (rpmforge)
memcached-1.4.5-1.el5.rf.x86_64 from rpmforge has depsolving problems
--> Missing Dependency: perl(Term::ReadKey) is needed by package memcached-1.4.5-1. el5.rf.x86_64 (rpmforge)
Error: Missing Dependency: perl(AnyEvent::Socket) is needed by package memcached-1.4. 5-1.el5.rf.x86_64 (rpmforge)
Error: Missing Dependency: perl(AnyEvent) is needed by package memcached-1.4.5-1.el5. rf.x86_64 (rpmforge)
Error: Missing Dependency: perl(AnyEvent::Handle) is needed by package memcached-1.4. 5-1.el5.rf.x86_64 (rpmforge)
Error: Missing Dependency: perl(YAML) is needed by package memcached-1.4.5-1.el5.rf.x 86_64 (rpmforge)
Error: Missing Dependency: perl(Term::ReadKey) is needed by package memcached-1.4.5-1 .el5.rf.x86_64 (rpmforge)
This is proving to be a bit of a nuisance for something which looks fairly easy to install.
Reply With Quote
  #20  
Old 07-29-2010, 06:36 AM
FractalizeR's Avatar
FractalizeR FractalizeR is offline
 
Join Date: Oct 2005
Location: Russia, Moscow
Posts: 368
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Your OS installation seem to be broken. If even it is not, anyway, it doesn't look like regular CentOS installation to me. I suggest you ask someone to help you with it because I think, it is not something you can easily solve on forum.
Reply With Quote
  #21  
Old 08-04-2010, 04:03 AM
Bluefin221 Bluefin221 is offline
 
Join Date: Jul 2008
Location: Scotland
Posts: 290
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default
Quote:
Originally Posted by FractalizeR View Post
Your OS installation seem to be broken. If even it is not, anyway, it doesn't look like regular CentOS installation to me. I suggest you ask someone to help you with it because I think, it is not something you can easily solve on forum.
rpmforge is broken with missing files and folders so anything I try to install returns errors, I have tried again and again to uninstall rpmforge, reinstall it, update it but still the files/folders are missing.
Reply With Quote
Reply
Page 2 of 3 1 2 3

« Previous Thread | Next Thread »

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT. The time now is 09:59 AM.

Contact Us - Archive - Top

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.08742 seconds
  • Memory Usage 2,345KB
  • Queries Executed 26 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (3)bbcode_code
  • (4)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_article
  • (1)navbar
  • (4)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (2)pagenav_pagelink
  • (11)post_thanks_box
  • (1)post_thanks_box_bit
  • (11)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (1)post_thanks_postbit
  • (11)post_thanks_postbit_info
  • (10)postbit
  • (11)postbit_onlinestatus
  • (11)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 
Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 
Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • fetch_musername
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • post_thanks_function_fetch_thanks_bit_start
  • post_thanks_function_show_thanks_date_start
  • post_thanks_function_show_thanks_date_end
  • post_thanks_function_fetch_thanks_bit_end
  • post_thanks_function_fetch_post_thanks_template_start
  • post_thanks_function_fetch_post_thanks_template_end
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete