Optimize tips & tricks

[pr0team.net]

Hello,

Anyone have big boards tips & tricks for vbulletin big board (ajo.pl) ??
Before post this theard, i search in this forum without results.

[Kentaurus]

Hello,

Anyone have big boards tips & tricks for vbulletin big board (ajo.pl) ??
Before post this theard, i search in this forum without results.

I had to do a lot to have my forum running smoothly. Vbulletin is a great product, but add traffic and a lot of posts... and you'd better have a good server and do some optimization. I run CemZoo.com, an Anime/Gaming community for Spanish speakers, currently with somehow ~4M posts and ~50k members.

First... If you haven't already, posting a thread in the server optimization forum at vbulletin.com is a good idea. Some fine tuning of the mysql and apache parameters can help a little. And a lot more if you have never given mysql some tuning.

A good server helps... it's often not enough. I found that with an IDE/SATA drive the hard disk was constantly a bottleneck. A fast disk helps, a lot. But if you need to decide between adding more RAM and a fast disk, I'll always go with more RAM. Anything less than 2Gb of RAM is not enough for running a large forum.

Apache seems to come short for larger server loads. I recently tried litespeed, and other lite web server solutions. They helped. They can reduce the load of the server a 30 o 40%, that's really nice. Apache is a good server, but somehow litespeed runs faster specially when processing a lot of php.

Do not run php without a cache or accelerator, eaccelerator, mturck cache, x-cache... you can do some homework and check which one is better, or which one you like most. They do the work of compiling php. The cache alone reduced the load of my server by 50%. I use eaccelerator.

Install an image server... I serve dynamic content with one web server, and the images with another web server. Of course you need two servers or at least two IP addresses for doing this. By doing this you can turn keepalive off on your main server (that servers dynamic content), and have a server dedicated to static content that can also be cached for a long time (like images).

If you can afford it, separate the database from the web/content server. Have apache/litespeed run in one server, this server needs a lot of processing power. Have another server run the mysql database. This server needs a lot of RAM and a fast disk. You can even load-balance the sites this way.

I do archive posts. I only keep 3 months or less. That's like 600,000 posts. I had to do that when I had millions of posts that were never read, and were keeping mysql busy especially with searches. I keep two forums, one for the main, updated, recently read content, and another forum with the archived content. I use a custom-solution for my archiving needs. And I use two vbulletin licenses for this strategy.

If you are fan of hacks and mods, keep in mind performance. Never, ever, install a hack that adds a query to every page. Any hack that queries or modifies the post table should be a warning, it must be well written or it will make your forums very slow. Hacks for showthread are often more intensive that hacks for say, newthread.

Make an habit of keeping an eye on your server load and memory usage. Memorize those numbers, or at least be familiar with them, with the "usual" load of the server. After installing a hack, keep an eye on the load, if something strange happens, most likely it's because of the hack you just installed. The tiniest hack, that only adds a couple plugins can have a negative effect on performance if, for example, makes some bad regexps. Sometimes a hack can run beautifully with 10 users online, and take the forum down with 100 users.

Because of that, never install more than one modification at once, check your server, install the mod, monitor your server(s). If anything weird comes up, be ready to disable the mod.... check your server again. A test environment helps, but you'll need to test it with a heavy load sometime. Installing several hacks in a unattended way is dangerous.


Of course, this is only my experience, and some "quick tips" that I can give. Results may vary from person to person, and from forum to forum as each forum and each environment is unique :) Feel free to share your comments.

[teedizz]

wow...VERY helpful info there Ken. Thanks.

[dbembibre]

I have a forum (Spanish BMW owners), with more that 2 million of posts and 30.000 members.
My setup for this is a Dual dual xeon 2.8, 2 gb of ram and ultra scsi disks with Fedora core 4.

I use php 4.4 compile by hand and eAccelerator (php cache), mysql 4.0.21 compiled with the intel c++ compiler (20% faster that with gcc), and many tweakings in tcp/ip stack, apache and php.

I can handle 1000 constants users within a load more higher off 1.4

If you need more that this, put a lighthttpd as image server, one server for database or memcached and another server for apache

Excuse my broken English

[psico]

Hola Kentaurus!! La sigo en ingles para que nos entiendan todos ;)

I'm the owner of www.psicofxp.com a spanish forum with 200.000 users, 2.4 MM posts, 2800 users online on peak hours and 100.000+ visits per day (unique ips).
I have 2 DB servers (1 Master and 1 Slave) and 4 Webservers in a load balance solution provided by the hosting, the load balance solution is caching the static content so I don?t need to implement an image server. The slave DB server had 2 more roles: memcached and nfs server (all the php files resides in this server).

I?m very interested in your custom solution to archive old posts, can you explain me how you did that? Or at least help me somehow?

Tnx in advance!

Ismael Briasco

[Kentaurus]

Hola Kentaurus!! La sigo en ingles para que nos entiendan todos ;)

I'm the owner of www.psicofxp.com (http://www.psicofxp.com) a spanish forum with 200.000 users, 2.4 MM posts, 2800 users online on peak hours and 100.000+ visits per day (unique ips).
I have 2 DB servers (1 Master and 1 Slave) and 4 Webservers in a load balance solution provided by the hosting, the load balance solution is caching the static content so I don?t need to implement an image server. The slave DB server had 2 more roles: memcached and nfs server (all the php files resides in this server).

I?m very interested in your custom solution to archive old posts, can you explain me how you did that? Or at least help me somehow?

Tnx in advance!

Ismael Briasco

It's was custom programmed, I run two separate forums with two separate licenses. Each week I archive all the posts that are older than 3 months to the "backup" or "archive" forum. That means copying data from the thread and post tables, and from the dependent tables as well (attachment, editpost, deletelog, etc). Sorry, it was a custom solution and I don't have it available for sharing since it was customized for my site (and making it generic for any forum is not really a possibility).

With this, I can run two different mysql servers, one for the archive and one for the main forum, in different servers. And I can load balance with a little more flexibility. I can implement rules for deactivating the archive without deactivating the main forums... I just can manage them separately.

Also, the mysql load went down by a lot, leaving 4M posts in a single mysql table makes joins a little painful. it was faster to just leave the 200-300,000 posts that people want to read and check everyday, and the rest is in an archive, still accesible, but you cannot longer reply, and if the archive loads 5 times slower... it doesn't matter to me because that's not the active users.

[pr0team.net]

Yes, actually i use :
Dual xeon 2.6, 2 GB ram
lighttpd with php 4.4.4 (xcache)
and load above 50 :( - 900 online


last pid: 2805; load averages: 23.33, 31.08, 31.91 up 0+01:28:18 12:14:36
180 processes: 24 running, 155 sleeping, 1 stopped
CPU states: 92.1% user, 0.0% nice, 5.3% system, 2.6% interrupt, 0.0% idle
Mem: 809M Active, 186M Inact, 178M Wired, 244K Cache, 112M Buf, 829M Free
Swap: 4069M Total, 4069M Free

PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
2334 www 1 107 0 24248K 18504K RUN 0:21 1.27% php-cgi
2338 www 1 4 0 28460K 22788K accept 0:21 1.27% php-cgi
2353 www 1 107 0 25792K 19360K RUN 0:21 1.22% php-cgi
2359 www 1 4 0 28628K 22952K accept 0:21 1.22% php-cgi
2355 www 1 106 0 27488K 21812K RUN 0:21 1.22% php-cgi
2346 www 1 107 0 27776K 22100K RUN 0:21 1.17% php-cgi
2361 www 1 106 0 26272K 20600K RUN 0:21 1.12% php-cgi
2313 www 1 4 0 24552K 18824K accept 0:21 1.12% php-cgi
2326 www 1 105 0 27848K 22192K RUN 0:21 1.07% php-cgi
2336 www 1 104 0 25336K 19592K RUN 0:20 1.07% php-cgi
2322 www 1 4 0 28552K 22896K accept 0:19 1.07% php-cgi
2354 www 1 4 0 27792K 22120K accept 0:21 1.03% php-cgi
2335 www 1 107 0 24588K 18844K RUN 0:21 1.03% php-cgi
2342 www 1 107 0 24516K 18772K RUN 0:20 1.03% php-cgi
2369 www 1 106 0 28760K 23108K RUN 0:20 1.03% php-cgi
2363 www 1 4 0 27708K 22032K accept 0:22 0.93% php-cgi
2364 www 1 4 0 25516K 19784K accept 0:22 0.93% php-cgi

[NameClan.com]

thank you very much, it is very useful for me!

[Keyser Soze]

Install an image server... I serve dynamic content with one web server, and the images with another web server. Of course you need two servers or at least two IP addresses for doing this. By doing this you can turn keepalive off on your main server (that servers dynamic content), and have a server dedicated to static content that can also be cached for a long time (like images).How exactly did you do that? I am running two different webservers and separated php-files and images on almost every of my websites to these two different servers, but not in my vbulletin-forum so far, because I don't know how this is possible.

[Kentaurus]

How exactly did you do that? I am running two different webservers and separated php-files and images on almost every of my websites to these two different servers, but not in my vbulletin-forum so far, because I don't know how this is possible.

For the style, in the style options, you can configure the image paths. Just point them to any server that you are using as an image server :)

[Keyser Soze]

Thanks a lot :)
Did you also move your attachments and avatars to another server or just the style-images?

[Kentaurus]

Thanks a lot :)
Did you also move your attachments and avatars to another server or just the style-images?

Also the attachments and avatars. You need to specify a filepath location for the attachments & avatars, and have the image server point there. Also modify the url to the image's server.

If they are two physically distinct servers you'll need something like nfs to keep the avatar & attachments & profile pics synchronized.

[Keyser Soze]

Yes I could change the avatar-path in the AdminCP to an external server and rsync the avatar-directory... But what about uploading? If someone wants to upload a new avatar, vBulletin will most probably try to write to the directory which is an external URL in this case. Is vBulletin able to write images to an external server without modifying the vB-Core-Files?

[kontrabass]

Also the attachments and avatars. You need to specify a filepath location for the attachments & avatars, and have the image server point there. Also modify the url to the image's server.

If they are two physically distinct servers you'll need something like nfs to keep the avatar & attachments & profile pics synchronized.

Have you experience with running 2 http servers on the same box, one for images? Just curious to see what if any performance that might bring about to have one caching and the other dynamic on the same box...

[Zia]

well ..
if there 2 physically different server..one for vb itself
and 2nd for only avtar/profile pic/sigs/attachment

how to do it?