Painfully slow forum, virtual server, database issue?

[GONUMBER6]

About 2 weeks ago my forum has started to progressively become painfully slow. It is always slow in the mornings.

Godaddy blames vbulletin and mySQL database.

Here is some of my chat with them:


Thank you for contacting Live Chat support for Virtual and Dedicated Servers. This is David. May I start with your name and the issue which you are currently experiencing?
bulldognews: Hi David, It's me Lisa again! I have done troubleshooting for 24 hours now and my site is still painfully slow. :(
David C -Server Concierge: What is your site?
bulldognews: http://www.englishbulldognews.com
David C -Server Concierge: Do I have your permission to access the server?
bulldognews: Yes please.
David C -Server Concierge: It could take up to 5 minutes for me to look at this. I appreciate your patience.
bulldognews: No problem
David C -Server Concierge: It looks like the issue is specifically with your database queries, did you try disabling any plugins on the site or enable slow MySQL query logging?
bulldognews: I disabled everything right now, but no idea about the MySQL
David C -Server Concierge: Well your site is a vBulletin site so it has to connect to the database to pull the information out to load the site, this is what's taking a long time. To find out what specifically is doing that you'd have to enable the slow query logging feature in MySQL, there's a tutorial on how to do this here: http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html
bulldognews: let me have a look, thanks
David C -Server Concierge: You're welcome, can I help you with anything else?
bulldognews: Oh no that is all greek to me
bulldognews: I am planning on moving to an assisted dedicated server plan in a few weeks, but until then I have to try to figure this out on my own.... :(
David C -Server Concierge: With an assisted server our assisted team can turn on the MySQL slow query logging for you, but they can't troubleshoot your actual site code or database structure.
David C -Server Concierge: Did you make the site or did someone else build it?
bulldognews: I did everything on my own, and I have been fighting an early morning slowness issue for almost a year, I am guessing. I turned off a modification about 2 weeks ago that uses alot of resources (SEO rewrite software) and it was lightning fast for a few days and then progressively slower again each day.
David C -Server Concierge: Okay, well I don't know your level of expertise in web development or server administration, but I'll give you the basic rundown to be on the safe side. vBulletin is built in PHP and writes to the MySQL database to store content. To display any of the pages, it has to connect to the database, search for the content it needs, and then display it. If you have a lot of plugins installed, those also run each time the page is accessed. For example, if you have a statistics plugin that caches where all your visitors come from, what page they visit, when they visited it, etc, that has to run as well on each page load.
David C -Server Concierge: Databases get larger and larger over time with plugins like that, so it takes longer to load each table since there's more data in it than there was before.
David C -Server Concierge: That will eventually slow down the site.
David C -Server Concierge: That's what the slow query logging does - it shows you what queries to the database are taking the longest to finish.
David C -Server Concierge: It's more than likely a plug-in doing it, the only way to really test that would be to enable the query logging to see which query it is, or just try disabling all the plugins and see if the site comes back up to speed.
David C -Server Concierge: You could also go into your MySQL database and optimize it to reduce the overhead, that would flush out any old data that's no longer needed and bring the size down a bit as well.
David C -Server Concierge: I haven't used vBulletin personally, but I've seen a number of people with WordPress issues, and a lot of the time the problem was due to plug-ins like YARPP, StatPress, and so forth that aren't part of the base WordPress install.
bulldognews: Yes I did turn off all modifications (no help at all), and this issue started just a couple of weeks ago.
bulldognews: Also my other forum Taste tested recipes is slow to load so how can that be?
David C -Server Concierge: What's the URL for that?
bulldognews: http://tastetestedrecipes.com
bulldognews: It's not so slow that it is painful but it should be super fast since it is small and almost out of the box.
David C -Server Concierge: That one is loading fairly quick for me.
bulldognews: Hmmmm.... okay. Not for me. Unless I visit a cached page.
David C -Server Concierge: All those pages are loading in under a second for me, so that one seems fine.
David C -Server Concierge: The englishbulldognews.com is definitely really slow for me though.
David C -Server Concierge: If you've already tried turning off all the plugins, the only other thing I can think of is if it's taking along time to load the external content on that site.
David C -Server Concierge: Did you try disabling the ads as well?
David C -Server Concierge: I can get to a test page almost instantly, so it has to be either the database or some of the external content.
David C -Server Concierge: http://www.englishbulldognews.com/phpinfo.php loads right away.
bulldognews: Yes it sure does.

[Lynne]

What are your server loads like when everything is running slow? Are you on shared hosting? If you are, do you realize your sites could be slow due to someone else's site on the server hogging the resources?

[nhawk]

I have to agree with Lynne.. after accessing your site it seems to be an overall server load issue, not a site load isssue.

[GONUMBER6]

I would agree normally, but my other forum http://tastetestedrecipes.com is working fine on the same server.

I am on a virtual dedicated server, and sometimes the site slowness is due to exactly what you are describing. But in this particular case, it is not. I am not technically advanced enough at this point to be able to fix whatever is going on with it now.

[Lynne]

So do you have access to tweak your server? If so, you may want to post in the Server Configuration forum on vb.com and get George's advice.

More advice - get firebug and see what is going on (look at the Net tab and then reload the page and you will see). For instance, on your taste tested site, it sits there spinning and spinning long after the page is rendered. It's because it is trying to get this image - http://www.englishbulldognews.com/forums/images/English_Bulldog_Square.png You can also just see that everything on your englishbulldognews takes forever to load. Honestly, it's ridiculous how long it is taking to load the images. I wish the host support could look at that. There is definitely something going on.

[GONUMBER6]

Lynne that makes perfect sense because that image is on the other forum that is slow.

I wish I knew what else you were talking about on the other stuff. I do have access, but my knowledge is limited. I will check out firebug, but I don't know if I can even begin to relate to it but it is worth a shot.

I did submit a ticket on vb.com last night.

--------------- Added 1321730757 at 1321730757 ---------------

Yeah, I got the firebug but it does not do me any good, since I don't know how to use it.

--------------- Added 1321735288 at 1321735288 ---------------

Okay it is saying that it is taking 40seconds of waiting time to load a page, while the receiving part is only in milliseconds. Do you know what the "waiting" is for firebug Lynne?

--------------- Added 1321743165 at 1321743165 ---------------

Hired a professional to fix, this was the issue:

This problem was due to your resolves on the server being down, I have switched them to google dns and remote lookups are no longer taking a long time. The page has went from taking 11 seconds to load to loading in around 1 second.

---

Here is the output of the important part of the trace that shows the problem.

3519 0.000054 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 71
3519 0.000055 connect(71, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("208.109.188.2")}, 28) = 0
3519 0.000068 fcntl64(71, F_GETFL) = 0x2 (flags O_RDWR)
3519 0.000037 fcntl64(71, F_SETFL, O_RDWR|O_NONBLOCK) = 0
3519 0.000037 gettimeofday({1321741059, 624622}, NULL) = 0
--
3519 0.003837 sendfile64(69, 70, [15674], 114769) = 13032
3519 0.000162 poll([{fd=69, events=POLLOUT}], 1, 120000) = 1 ([{fd=69, revents=POLLOUT}])
3519 0.160484 sendfile64(69, 70, [28706], 101737) = 17376
3519 0.000139 poll([{fd=69, events=POLLOUT}], 1, 120000
3499 0.135138 ) = 0 (Timeout)
3499 0.000058 waitpid(-1, 0xbfcd27d8, WNOHANG|WSTOPPED) = 0
3499 0.000044 select(0, NULL, NULL, NULL, {1, 0}
3519 0.025202 ) = 1 ([{fd=69, revents=POLLOUT}])
3519 0.000044 sendfile64(69, 70, [46082], 84361) = 24616
3519 0.000106 poll([{fd=69, events=POLLOUT}], 1, 120000) = 1 ([{fd=69, revents=POLLOUT}])
--
3519 0.000046 close(69) = 0
3519 0.000055 read(8, 0xbfcd2773, 1) = -1 EAGAIN (Resource temporarily unavailable)
3519 0.000047 close(70) = 0
3519 0.000056 semop(4325385, 0xb7cdae98, 1
3518 0.078649 ) = 0 (Timeout)
3518 0.000054 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 71
3518 0.000054 connect(71, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("208.109.188.2")}, 28) = 0
3518 0.000072 fcntl64(71, F_GETFL) = 0x2 (flags O_RDWR)
3518 0.000038 fcntl64(71, F_SETFL, O_RDWR|O_NONBLOCK) = 0
3518 0.000037 gettimeofday({1321741060, 723580}, NULL) = 0

[Lynne]

That you for posting what the issue was. It sounds like this is something your host should have been able to figure out. But, I'm just glad you have it fixed.

(Sorry I didn't reply - I was gone for the weekend.)