Quad Opterons + FreeBSD + Lighttpd + FastCGI/PHP + APC = Fast vB :)
 

Yup, I migrated my forums to new servers (dual xeon to quad opterons) using completely new software - FreeBSD, Lighttpd instead of Apache. Lighttpd is really amazing, and I recommend that everyone switch from Apache to Lighttpd. APC works fine on the setup too. Just sharing.

Good to hear everything is going well. ;)

FreeBSD is quite a robust OS with a very good upgrade system. I highly recommend it as a good basis for a server.

Well, all benchmarks I saw suggest otherwise - FreeBSD scales worse than Linux in multiprocessor setup. YMMV, of course, but I also can't see how the FreeBSD ports system is superiour to the yum/apt repositories. And if you're into recompiling everything to tune up for your system, Gentoo's ebuilds are just for that.

edit: I see that you didn't make it to SoftLayer...

Sounds good, Erwin. Thanks for sharing your experience with this setup.
I'm wondering if I could get away with something like this. My new Adaptec SAS RAID controller card doesn't have the widest range of drivers available yet, thus I'm using CentOS 4.1 rather than the latest version.
I'm always down for going faster ;)

how is converting apache over to lighhtpd

Erwin, did you notice less load on the servers vs apache? Any optimizations? I converted to lighttpd but didn't see a noticable drop in load but the pages are generated slightly faster.

hmm i guess im gonna switch from apache =]

but i like gentoo as my OS =]

Only if you compile the wrong kernel. :) FreeBSD 6.0 using AMD64 SMP kernel on a Quad Opteron just plain flies.

Ports are better than yum/rpm because ports automatically fixes all broken dependencies and installs everything automatically using 1 command: portinstall apache or portinstall mysql5

Can't get any easier than that. ;)

The only downfall of FreeBSD is its lack of driver support - I had to change my RAID card and go with a RAID 10 rather than a 0+1 setup. But it's all good.

On FreeBSD, it's this: portinstall lighttpd.

;)

Maybe because your webserver wasn't badly overloaded? I can tell you that I was able to consolidate my multiple webservers into a single more powerful one which makes maintaining files and the datastore cache much easier.

Gentoo's portage system does all of that too, and i personally would have gone with Gentoo over FreeBSD.

Yeah but Gentoo is still Linux and it basically copied the BSD ports system. BSD is the oldest *nix around AFAIK. In any case, there is no one OS that suits everyone. :) They are all good in their own way. I just wanted to try something different. :)

I'm glad its working for you, what is your current setup now, and would you mind sharing who you went with?

For the database server, it's a dual processor/dual core Opteron 270 with 8Mb RAM, 8 x 15k SCSI PCI-X Raid 10 array, FreeBSD 6, MySQL 5 - runs like the wind. Loads never hit 0.5. :) The main webserver is the Quad Opteron with similar specs. We also have image server, irc, attachment, dns, and a slave search server.

We went with webnx in the end - best prices in town - look for Dan Paultz and tell him Erwin sent you - he'll look after you well. :)

Well, but you haven't compared it to Linux, have you?
I already posted some links to FreeBSD/Linux comparisons in MySQL performance, here're some more:
FreeBSD 5.x: http://unix.derkeiler.com/Mailing-Li...5-06/0331.html
FreeBSD 6.x:
http://www.archivesat.com/FreeBSD_Pe...read412787.htm
http://www.archivesat.com/FreeBSD_Pe...read420280.htm

I see some confusion here: yum does resolve the missing dependencies! I'm not sure what do you mean by "broken" dependencies, probably 3rd party rpm packages? But the same goes for the ports - if you install some 3rd party program manually, outside the ports system (doing usual "./configure; make; make install"), you have to resolve its dependencies manually.

There's one fundamental difference between the ports and rpm-based package managers - rpm's are all binary, and you build your ports from the source when installing them. It could be considered as an advantage as you're supposedly tuning the package for your system when building it (with the right parameters), but some time ago I saw the tests of Gentoo (which is touting the same approach as an advantage) vs some binary-based distribution didn't reveal any substantial performance advantages.

And finally, you can easily rebuild RPM package from the source rpm with options tailored for your architecture, too (that's what I did for our install, rebuilt php rpms with Opteron optimizations). It's as easy as doing "rpmbuild --target=athlon --rebuild yourpackage.src.rpm"

Thanks for the links, it's very specific stuff there. Interesting.

I know about yum - I ran RHEL servers for 4 years. :)

I'm not here to start an OS-debate. :) I'm just posting my experiences which mirrors the experience of many others. :)

I would have went with a little more ram. :p

My swap is still zero. :) Loads are still less than 1. I think it's okay for now considering all it has is the one database as a dedicated database server, MINUS any search or IP searches - we have another dedicated slave database Dual Xeon with 4 MB or RAM to do just search or IP searches. :)

Hopefully you mean GB rather than MB. ;)

LOL!

Yeah, Gb. The last time I had 4 Mb RAM was on a 386... My 286 had 1 Mb RAM and my XT had 64kb.

Those cpu loads are very nice :D

I recommend giving Litespeed Webserver a try as well...I much prefer it over Lighttpd.

Theres that whole cost thing.
Enterprise Edition (4 CPU) $1199 ;)

The standard version is free and has no limits as long as it keeps to one CPU (or a dual core) and 300 concurrent users. But the benchmarks speak for themselves when you're under high load.

I do love both however. :)

Lighttpd works very well if the config is tweaked. I'm using the Quake.net config example - works great.

Where would a person find this "Quake.net config" ?
:)

Someone on vB.com posted a link to it in the large forums thread. :) Should still be there.

gotta back up Zachery here. Gentoo is plain awesome. upgrading just doesn't get easier than Gentoo has it - execute "emerge -u world" in screen and forget about it. it's very stable - I have a single-CPU Gentoo box that has 260 days uptime and handles ~15k uniques a day. and needless to say, Gentoo logo is the best. Pacman forever! :D

Ah, thanks ;)

http://nopaste.php-q.net/206712

Mind you, my lighttpd is causing server errors at the moment, and I'm trying to troubleshoot it. It's possible APC doesn't want to work it. I can't get eA to work with it.

You are trying to run both at the same time erwin? eA and APC accomplish the same thing.

I've got lighttpd running on a new server just for serving images, and large downloads (E3 is a killer on the site in question). Has made a huge differnece offloading just the images to another server. Loads on the three servers are below 1 and its running great.

No, not at the same time - one or the other.

I had to remove the quake.net tweaks in the end. :) I was getting server errors with them for some reason...

By the way, my servers are with www.webnx.com and they are by far the cheapest providers of top end hardware I've found. We're talking Quad Opterons or Dual/Dual Opterons with 15k SCSI hardware RAID arrays for a few hundred bucks - and their service has been great so far - unmanaged servers but they help out wherever they can. I highly recommend them. :) You can contact Dan Paultz, who's a great guy, if you're interested in some high-end boxes to lease. The provisioning of the boxes take a while but once they're up they're good.

Erwin, which method did you decide to go for when doing virtial hosts? If you have any?

Virtual hosts are flexible in lighttpd - I don't do them at the moment but if I did I would be IP based.

I was refering to the simple version or the conditional based version.

What kind of problems are you having with lighttpd?

We're using lighttpd + phpfcgi + 64bit CentOS and lately experiencing segfaults from the php processes. I see a few bug listings regarding segfaults on APC and have a hunch it might be that.

BTW, eA does not work on a 64bit OS.