thread with 100,000+ replies very slow

[tomotron]

I have a thread on my forum that currently has 110,884 replies. When trying to open a page in that thread or even going to the last page which might only have a few posts, it's VERY slow compared to the rest of the site.

I've read a few docs that claim the length of a thread shouldn't effect the load time, but I can tell you there is a very clear difference.

Is there anything I can do to speed things up, aside from closing the thread? It's sort of a long running tradition on the forum and I'd hate to see it go.

Any suggestions? Thanks!

[snakes1100]

I would first suggest checking mysql during the query process of that thread, you might be able to optimize mysql via my.ini, could be curable with a few variable changes and restarting mysql.

You can also look into spliting the thread, check vbulletin.com on splitting thread topics, plenty of ideas from the others that have had the same issue.

[nexialys]

100 000+ replies to a single thread ?!

don't you think it would be better to split the thread and close the first part so people just answer the follow up ?!

next time, just close and re-open a new thread to follow up the discussion after some K... or you just have one thread on your forum ?!

[Lynne]

We have a thread with a bit over half that many replies and so far I have decided not to split it. It takes about 1.2 seconds to load compared to .1 seconds on other pages, so there is definitely a difference for load times of big threads. I've heard some people just like to split them (and close them) at some point. I think there is a mod to automatically do this, but if I recall there are problems with it. If it's just a single thread, it should be no problem for you to just deal with it on your own.

[tomotron]

Yup, 100k replies in one thread. We have quite a few threads with replies in the thousands, and have never had a problem until recently. Most everything I've read says thread size should not have a significant effect on server performance or load times.

It's a decent sized forum,
Threads: 25,905, Posts: 1,279,387, Members: 55,056

but I visit other larger forums with longer threads that never seem to have similar problems.

snakes1100, thanks for the reply. Our mysql configuration is pretty well optimized, or so I think at least. I'm going to post in the server optimization forum over at vbulletin.com and see if I missed something though.

--------------- Added 1224091682 at 1224091682 ---------------

We have a thread with a bit over half that many replies and so far I have decided not to split it. It takes about 1.2 seconds to load compared to .1 seconds on other pages, so there is definitely a difference for load times of big threads. I've heard some people just like to split them (and close them) at some point. I think there is a mod to automatically do this, but if I recall there are problems with it. If it's just a single thread, it should be no problem for you to just deal with it on your own.

I did end up closing it and started a new monthly version of the thread. It's an "off-topic" thread that's been running for probably over 5 years now, so It's a tradition that I wanted to keep alive and a monthly off-topic thread should keep my users happy.

Like I said though, most everything I'v read said thread length shouldn't effect load times (even though I too noticed a clear difference), and I visit other forums that are more active and have longer threads. I was hoping to find a workaround or tips on optimizing my server/database to handle these types of situations.

[Lynne]

Like I said though, most everything I'v read said thread length shouldn't effect load times (even though I too noticed a clear difference), and I visit other forums that are more active and have longer threads. I was hoping to find a workaround or tips on optimizing my server/database to handle these types of situations.
But do you know what those other sites are using for their server setup? That will make a difference in how quick the load times are. You didn't say what your server set up is, but ours is a two server setup. It would make a difference too on how the servers are configured - RAID0, RAID1, RAID10?

[tomotron]

yeah that's why I'm going to post up over at the server optimization forum at vbulletin.com...

we're currently running on a load balanced cluster with 3 servers,

Database Server
Dell PE1850 Dual Xeon (2 x 3.2 Ghz)
2GB Ram
2 x 146 GB SCSI (RAID 1)

2 Front End Nodes
Dell PE1425 Dual Xeon (2 x 2.8 Ghz)
1GB Ram
80 GB SATA-II

[snakes1100]

With 1.2 million post, 2gb of ram is fine, but the 100k reply post, that will be the drain on the remaining ram in the system, i would go thru the DB server and disable any programs running that arent needed, i would check the ram usage at the point of load when that thread is opened to view it on the last page as well as using mytop to view the process live.

You might want to try and play with these 3 a little, but with 2gb, your pressing your luck.
join_buffer_size
read_buffer_size
sort_buffer_size

[tomotron]

thanks snakes, hopefully the solution is something as simple as adding ram to my DB server. i've posted up in the server optimization forum at vbulletin.com, but there might not be much i can do besides beefing up hardware to deal with huge threads.

[azn_romeo_4u]

You're seriously on a cluster with a forum with 1.2mill posts? How much you paying? Cost you are blowing your money big time.

I got a forum with a 50k thread, with some other threads in the 10k running pretty speedily. I have about 1.7mill post though. It's on a shared server running with some other heavy hitters. Some sites doing like 50gb's a day off of hundreds of requests per sec and my load is under 1.

Are you running apache?

[tomotron]

You're seriously on a cluster with a forum with 1.2mill posts? How much you paying? Cost you are blowing your money big time.

I got a forum with a 50k thread, with some other threads in the 10k running pretty speedily. I have about 1.7mill post though. It's on a shared server running with some other heavy hitters. Some sites doing like 50gb's a day off of hundreds of requests per sec and my load is under 1.

Are you running apache?

the site is much more than just a forum, and serves close to 10 million page views to just under a million unique visitors a month. if we could get away with sitting on a shared server we would, but i think we'll just have to keep blowing our money big time on our cluster instead :rolleyes:

yes, we're running apache.

[Alfa1]

Definitely increase the RAM. Big time.

[azn_romeo_4u]

I'm not trying to tease you or anything. Just saying. IMO it is a waste of money, unless you're like some video site. Uploading videos and converting it or using one of those other boxes to do the conversion whatever etc.

I'm at about close to your range, with about 750k uniques a month on one server. I'm doing like 90gb's of transfer per day. I'm using instead of apache,

www.litespeedtech.com

If you're using cpanel or plesk, it supports that. Works way better than apache at serving php, and static content. I'm running a version of it that cost 45 dollars a month. I did the switch a few months ago. Works wonder. There's a more expensive version, 100 bucks a month that lets you run 8 core cpu's. I'm running a 2 core version.

I think you should look into it. It's 30 day money back, with a 15 day trial on the 2 core version, so you can basically run this thing for 45 days for free.

Easy to install, works with a lot of stuff like memcache, apc, xcache, eacellera, mod_security, sohusian etc. They also got a load balancer solution.

There's other options like http://nginx.net/ if you don't want to spend money. I know a forum that has this installed, that runs with 20,000 concurrent connections. Suppose to handle 100,000 connections find apparently O_O.

[djxcee]

You should be fine with your current setup but I do highly recommend you to try LiteSpeed if you want to further upgrade your server.

I'm using it on my VPS and it's rocks :D

[Dismounted]

I'll step in for TECK and recommend nginx. :)

"nginx - the raise of a new giant (http://www.vbulletin.com/forum/showthread.php?t=253741)" - TECK

[Mosh]

I'll step in for TECK and recommend nginx. :)

"nginx - the raise of a new giant (http://www.vbulletin.com/forum/showthread.php?t=25374)" - TECK
It is actually here - "nginx - the raise of a new giant (http://www.vbulletin.com/forum/showthread.php?t=253741)" - TECK :)

[Dismounted]

Fixed my link - somehow left the one off the end :p.

[punchbowl]

should be 'rise' or am I missing some in techy joke?

As in Terminator 3 : Rise of the Machines

[Dismounted]

It should be rise. Don't know of any tech jokes about it...

[TECK]

Raise: http://www.answers.com/topic/raise
To move to a higher position; elevate: raised the loads with a crane. See synonyms at lift.
To set in an upright or erect position: raise a flagpole.
To erect or build: raise a new building.
To cause to arise, appear, or exist: The slap raised a welt.
To increase in size, quantity, or worth: raise an employee's salary.
To increase in intensity, degree, strength, or pitch: raised his voice.

[tmc]

Those are all verbs, TECK.. :) Still should be rise. "______ of a new giant" requires a noun, and the only common use of "raise" as a noun is "pay increase". But I really don't care, just thought it was odd you came in this thread just to post the definition of raise. :rolleyes:

@ the OP:

But I definitely wanted to give my +1 to nginx. Litespeed is simply too expensive, and Lighttpd is unstable (so I hear), slower (usually), and not as fully featured as nginx. And of course if you're talking about the free version of Litespeed, nginx still wins, IMO. Converting all your .htaccess code isn't too hard either. I would say I prefer nginx in that regard anyway. Doesn't bog your server down by looking for a .htaccess file in every single directory back to the web root..

Nevertheless, that's really not going to help you with your large thread problems. You discussed a simple enough solution, and that's auto-closing big threads.

And this bit simply isn't true, sadly: Like I said though, most everything I'v read said thread length shouldn't effect load times (even though I too noticed a clear difference), and I visit other forums that are more active and have longer threads. I was hoping to find a workaround or tips on optimizing my server/database to handle these types of situations.Every "large" thread I've ever come across has 1+ second load times, no matter what the server configuration is.