vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   Forum and Server Management (https://vborg.vbsupport.ru/forumdisplay.php?f=232)
-   -   What needs to handle 2500 users online at once (https://vborg.vbsupport.ru/showthread.php?t=294308)

AndrewSimm 01-23-2013 11:36 PM
What needs to handle 2500 users online at once
 
On 01/05 we had over 1,200 online at once and I expect around 2,500 on 02/06. There will be several times that day when post will spike to about 800 in about a 10 second span. Currently I have

Intel Xeon E3-1230 V2 3.3GHz
100 mbps Uplink
16 GB DDR3 Memory
1 TB RAID-1 Drives
10 TB Bandwidth

My current host is hostgator and we are getting around 230,000 pageviews a day. I expect over 500,000 on NSD.

When we had 1,200 online things got really slow and almost crashed. What are some suggestions and does anyone have any experience with vbulletin and amazon aws. I got a vanilla install up but had issues importing my database due to the mysql memory size on amazon RDS.

For those with experience is amazon the way to go or multiple servers? Also what about rackspace and how do they compare to amazon?

Max Taxable 01-23-2013 11:43 PM
Are all of these actual human users, or mostly "Guests?" Eliminate most of the automated bandwidth leeches by blocking Baidu and other bad user agents. Use this Mod for that:

Ban Spiders by User Agent

Link to the 4.x.x version and some nice lists of bad actors to block, in the thread. Cut your leech traffic almost totally out.

AndrewSimm 01-23-2013 11:46 PM
No they are human. We do a pretty good job with the spiders.

Max Taxable 01-23-2013 11:47 PM
Quote:
Originally Posted by AndrewSimm (Post 2399051)
No they are human. We do a pretty good job with the spiders.
1,200 to 2,000 human, logged in users online?

I gotta see this great board, link please?

snakes1100 01-24-2013 12:55 AM
You've never seen a site like that MT?

Ill PM you a couple lol

You would need to give us a bit more detail about the servers state when it loads up, cpu usage, ram usage, disk i/o stats etc.

AndrewSimm 01-24-2013 01:04 AM
<a href="http://canesinsight.com" target="_blank">http://canesinsight.com</a>

Isn't that busy right now. Currently There are currently 808 users online. 664 members and 144 guests

Max Taxable 01-24-2013 01:15 AM
Quote:
Originally Posted by snakes1100 (Post 2399059)
You've never seen a site like that MT?.
Actually I have, just like to see more of them.
Quote:
Originally Posted by AndrewwSimm
http://canesinsight.com

Isn't that busy right now. Currently There are currently 808 users online. 664 members and 144 guests
Thanks! It looks GREAT!

AndrewSimm 01-24-2013 01:26 AM
Quote:
Originally Posted by Max Taxable (Post 2399062)
Actually I have, just like to see more of them.Thanks! It looks GREAT!
thanks, I am just worried about the crashing.

snakes1100 01-24-2013 01:31 AM
Quote:
Originally Posted by snakes1100 (Post 2399059)
You would need to give us a bit more detail about the servers state when it loads up, cpu usage, ram usage, disk i/o stats etc.
Any info?

Max Taxable 01-24-2013 01:31 AM
Quote:
Originally Posted by AndrewSimm (Post 2399064)
thanks, I am just worried about the crashing.
I would be too.

AndrewSimm 01-24-2013 02:15 AM
Currently

2.95 4.03 5.29 | 1,043 Users Online (830 members and 213 guests)

It spikes during announcements. Basically I get post bombed.

--------------- Added [DATE]1358997646[/DATE] at [TIME]1358997646[/TIME] ---------------

Running apache, nginx, and DP Sphinx, Tables are still MyISAM

snakes1100 01-24-2013 02:27 AM
Rather high load for that amount of users on, but we dont know much about your setup without details.

nginx & apache may not be tuned/optimized.

hacks installed make a difference.

no cache installed?

the disk setup you have isnt very good for a db server.

AndrewSimm 01-24-2013 03:12 AM
I am not running a lot of hacks. vBDynamics and vbGallery but those don't get a lot of traffic. Datastore in the file system. No memcache or xcache yet.

snakes1100 01-24-2013 03:18 AM
File based caching is not as fast as memory.

Without know nginx & apache conf, not much more can really be said, not to mention is if sql is optimized as well.

With slow 1tb sata drives, thats not going to be good performance wise, for sql either.

final kaoss 01-24-2013 02:04 PM
It shows that alot of your resources aren't being cached.

I suggest you do the following.

*Install cloudflare and disable it's rocketloader feature. It can help serve content as a cdn as well as help block attackers/bad bots.

*Install the htaccess found in my guide to help with caching, gzip & more.

*Try out vb4 supercharged.

*Optimize your images

*Where possible, add css sprites.

And last but not least, make a post here to help fine tune your server hardware/software.
https://www.vbulletin.com/forum/foru...-Configuration
If you need help with any of the above, simply look in my signature for the link to the popular guide.

nhawk 01-24-2013 03:59 PM
I don't know why you're running Apache and Nginx, but it should be one or the other. Not both.

Normally when Nginx is installed apache is disabled.

snakes1100 01-24-2013 04:54 PM
Quote:
Originally Posted by final kaoss (Post 2399177)
It shows that alot of your resources aren't being cached.

I suggest you do the following.

*Install cloudflare and disable it's rocketloader feature. It can help serve content as a cdn as well as help block attackers/bad bots.

Cloudfare is a bad idea on a large heavy traffic site like his.

*Install the htaccess found in my guide to help with caching, gzip & more.

He dont need to, thats already being done by nginx if its setup correctly

*Try out vb4 supercharged.

That hack hasnt been updated in over a year & dont work correctly on later versions of vb.

*Optimize your images

*Where possible, add css sprites.

And last but not least, make a post here to help fine tune your server hardware/software.
https://www.vbulletin.com/forum/foru...-Configuration
If you need help with any of the above, simply look in my signature for the link to the popular guide.
Quote:
Originally Posted by nhawk (Post 2399198)
I don't know why you're running Apache and Nginx, but it should be one or the other. Not both.

Normally when Nginx is installed apache is disabled.
Nhawk, nginx is in proxy mode, then passes the php process off to apache, nginx is caching images & gzipping content to name a few, as well as handling a wide range of attacks that apache could never handle that are built into nginx by default.

final kaoss 01-24-2013 04:59 PM
Sorry snake but the points you've highlighted in red are null & void as per the results of a webpagetest & yslow report.

Also, cloudflare is a good choice for any site, as they help to serve content & thus reduce the amount of static requests that each page requires to load. It does not in anyway make site performance worse.

http://www.webpagetest.org/pagespeed...run=1&cached=0

As you can see, the points you tried to argue against are not being done by his current setup, hence why I posted what I posted.

Also, vb4 supercharged is working fine for me regardless of how old it is, I still use it to this day and recommend it.
Quote:
Originally Posted by snakes1100 (Post 2399204)
Nhawk, nginx is in proxy mode, then passes the php process off to apache, nginx is caching images & gzipping content to name a few, as well as handling a wide range of attacks that apache could never handle that are built into nginx by default.

snakes1100 01-24-2013 05:03 PM
Quote:
Originally Posted by final kaoss (Post 2399207)
Sorry snake but the points you've highlighted in red are null & void as per the results of a webpagetest & yslow report.

Also, cloudflare is a good choice for any site, as they help to serve content & thus reduce the amount of static requests that each page requires to load. It does not in anyway make site performance worse.

http://www.webpagetest.org/pagespeed...run=1&cached=0

As you can see, the points you tried to argue against are not being done by his current setup, hence why I posted what I posted.
Apparently, you dont read either, you should try to read what i said again, the points are valid & you gave very bad advice on a few points.

1. You tell him to use a hack that dont even work & is full of bugs.

2. I stated if his nginx was setup right, he dont need to, so apparently its not setup right.

final kaoss 01-24-2013 05:07 PM
1. With the correct settings vb4 supercharged works just fine for combining js & css, that is all that you really need it for since there are other ways to cache it & compress it without relying on the plugin to do it for you. Also what bugs are there with it? With the advice from my article you can run it pretty much with zero worries.

Quote:
For this step, we need to install vb supercharged. WARNING! Some people have reported conflicts when using this mod with other mod's. With Google Closure Compiler OFF, I have not encountered any conflicts!

My advice is to enable every option except for these:
Html Optimiser
Javascript Optimiser
Attachment Optimiser
2. I realize that you was stating this as well and I pointed it out with a report.

Quote:
Originally Posted by snakes1100 (Post 2399210)
Apparently, you dont read either, you should try to read what i said again, the points are valid & you gave very bad advice on a few points.

1. You tell him to use a hack that dont even work & is full of bugs.

2. I stated if his nginx was setup right, he dont need to, so apparently its not setup right.

snakes1100 01-24-2013 05:18 PM
Quote:
Originally Posted by final kaoss (Post 2399211)
1. With the correct settings vb4 supercharged works just fine for combining js & css, that is all that you really need it for since there are other ways to cache it & compress it without relying on the plugin to do it for you. Also what bugs are there with it? With the advice from my article you can run it pretty much with zero worries.

My advice is to enable every option except for these:
Html Optimiser
Javascript Optimiser
Attachment Optimiser
Thats a lot of options your disabling for a hack that works right.

final kaoss 01-24-2013 05:22 PM
Yep but I've been able to throw just about any mod I want onto it with no worries with it setup like that, including chats & arcades to test with.

AndrewSimm 01-25-2013 01:10 AM
I tried the super charged mod about a year ago and had some issues with it. I am already using gzip

my htaccess is below if you would like to look at it.

I noticed an improvement when I set up ngnix. Maybe it could be done better but it seems to be working in some capacity.

Does anyone have a link to some independent benchmarks done with cloudflare?


Code:
RewriteOptions inherit
####Charset
AddDefaultCharset Off

## Expires
<ifModule mod_expires.c>
        ExpiresActive On
        ExpiresDefault "access plus 1 seconds"
        ExpiresByType text/html "access plus 1 seconds"
        ExpiresByType image/gif "access plus 3456000 seconds"
        ExpiresByType image/jpeg "access plus 3456000 seconds"
        ExpiresByType image/png "access plus 3456000 seconds"
        ExpiresByType text/css "access plus 3456000 seconds"
        ExpiresByType text/javascript "access plus 3456000 seconds"
        ExpiresByType application/javascript "access plus 3456000 seconds"
        ExpiresByType application/x-javascript "access plus 3456000 seconds"
</ifModule>

## Compression
<ifmodule mod_headers.c>
        <ifmodule mod_deflate.c>
                AddOutputFilterByType DEFLATE text/html text/css text/xml application/x-javascript
                BrowserMatch ^Mozilla/4 gzip-only-text/html
                BrowserMatch ^Mozilla/4\.0[678] no-gzip
                BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
        </ifmodule>
</ifmodule>

## Rewrites
<ifmodule mod_rewrite.c>
        RewriteEngine on
        DirectoryIndex index.php

        ReWriteRule ^media/m(\d+).*/tags$ media.php?do=tags_edit&mid=$1 [QSA]
        ReWriteRule ^media/m(\d+).*/edit$ media.php?do=details_edit&mid=$1 [QSA]
        ReWriteRule ^media/m(\d+).*/report$ media.php?do=report&mid=$1 [QSA]
        ReWriteRule ^media/m(\d+).*/c(\d+)$ media.php?do=comment_edit&cmt=$2 [QSA]
        ReWriteRule ^media/m(\d+).*/p(\d+).*$ media.php?do=details&mid=$1&pid=$2 [QSA]
        ReWriteRule ^media/m(\d+).* media.php?do=details&mid=$1 [QSA]
        ReWriteRule ^media/p(\d+).*/edit$ media.php?do=playlist_edit&pid=$1 [QSA]
        ReWriteRule ^media/p(\d+).* media.php?do=playlist&pid=$1 [QSA]
        ReWriteRule ^media/c(\d+).* media.php?do=category&cid=$1 [QSA]
        ReWriteRule ^media/u(\d+).* media.php?do=user&uid=$1 [QSA]
        ReWriteRule ^media/tag/(.*) media.php?do=tag&tid=$1 [QSA]
        ReWriteRule ^media/results/(.*) media.php?do=results&query=$1 [QSA]
        ReWriteRule ^media/advresults/(.*) media.php?do=advresults&query=$1 [QSA]
        ReWriteRule ^media/letter/(.*) media.php?do=letter&query=$1 [QSA]
        ReWriteRule ^media/browse.* media.php?do=browse [QSA]
        ReWriteRule ^media/search.* media.php?do=search [QSA]
        ReWriteRule ^media/random.* media.php?do=random [QSA]
        ReWriteRule ^media/submit.* media.php?do=submit [QSA]
        ReWriteRule ^media/tagcloud.* media.php?do=tagcloud [QSA]
        ReWriteRule ^media/playlists.* media.php?do=playlists [QSA]
        ReWriteRule ^media/pcreate.* media.php?do=playlists_create [QSA]
        ReWriteRule ^media/pmine.* media.php?do=playlists_mine [QSA]
        ReWriteRule ^media/favorites.* media.php?do=favorites [QSA]
        ReWriteRule ^media/subscriptions.* media.php?do=subscriptions [QSA]
        ReWriteRule ^media/admin/edit/s(\d+).* media.php?do=admin_host_edit&sid=$1 [QSA]
        ReWriteRule ^media/admin/delete/s(\d+).* media.php?do=admin_host_delete&sid=$1 [QSA]
        ReWriteRule ^media/admin/export/s(\d+).* media.php?do=admin_host_export&sid=$1 [QSA]
        ReWriteRule ^media/admin/(\w+).* media.php?do=admin_$1 [QSA]

        # Media Library from Video Directory
        RewriteCond %{QUERY_STRING} do=viewdetails&videoid=(\d+)
        RewriteRule ^video\.php$ media.php?do=details&mid=%1
        RewriteCond %{QUERY_STRING} viewcategory&categoryid=(\d+)
        RewriteRule ^video\.php$ media.php?do=category&cid=%1
        RewriteCond %{QUERY_STRING} viewuser&userid=(\d+)
        RewriteRule ^video\.php$ media.php?do=user&uid=%1
        RewriteCond %{QUERY_STRING} viewtag&tag=(.*)
        RewriteRule ^video\.php$ media.php?do=tag&tid=%1
        ReWriteRule ^video\.php$ media.php
 
 # Send hardcoded ipa scores to arcade instead
  RewriteCond %{QUERY_STRING} act=Arcade [OR]
  RewriteCond %{QUERY_STRING} autocom=arcade
  RewriteRule ^index.php arcade.php [L,QSA]

</ifmodule>

<Files 403.shtml>
order allow,deny
allow from all
</Files>

Options -Indexes
deny from 71.76.197.71
deny from 193.200.150.125
deny from 173.63.38.218
deny from 67.159.60.83
deny from 89.145.95.2

RewriteCond %{HTTP_HOST} ^www.canesinsight.com$ [NC]
RewriteRule ^(.*)$ http://canesinsight.com/$1 [R=301,L]

deny from 72.228.132.15
RewriteCond %{HTTP_HOST} ^canesinsight\.co$ [OR]
RewriteCond %{HTTP_HOST} ^www\.canesinsight\.co$
RewriteRule ^/?$ "http\:\/\/canesinsight\.com" [R=301,L]
--------------- Added [DATE]1359080196[/DATE] at [TIME]1359080196[/TIME] ---------------

Quote:
Originally Posted by nhawk (Post 2399198)
I don't know why you're running Apache and Nginx, but it should be one or the other. Not both.

Normally when Nginx is installed apache is disabled.
I installed it as a cpanel plugin. I believe it comes set for nginx to serve the static content and apache the dynamic. I didn't see a easy way to just turn apache off and use nginx and didn't want to screw things up in my two busiest weeks of the year.

final kaoss 01-25-2013 02:29 AM
Here is the modified htaccess that I recommend for you (includes all of your old rewrite rules yet improves upon the caching/gzip.)

Although if you use cloudflare you would want to load only www urls and no urls that do not have www.
https://discussion.dreamhost.com/thr...html#pid158213
Quote:
Originally Posted by https://discussion.dreamhost.com/thread-136317-post-158213.html#pid158213

RE: Cloudflare without "www"
Hi,

Using CloudFlare through the panel option requires the www entry (DNS RFC compliance only allows us to provision via CNAME).

Note: This does not apply if you are signing up directly for CloudFlare.
PHP Code:
# Author: Final Kaoss (aka: Extreme-Gaming)
# Version 2.7
RewriteEngine on

# This file is only needed if you have set the Forum Component URL in your admincp and you are 
# using the mod_rewrite option for friendly urls.  If this is the case, copy this file
# to your forum component stub directory.

## Rewrites
<ifmodule mod_rewrite.c>
    RewriteEngine on
        DirectoryIndex index.php

        ReWriteRule ^media/m(\d+).*/tags$ media.php?do=tags_edit&mid=$1 [QSA]
    ReWriteRule ^media/m(\d+).*/edit$ media.php?do=details_edit&mid=$1 [QSA]
    ReWriteRule ^media/m(\d+).*/report$ media.php?do=report&mid=$1 [QSA]
    ReWriteRule ^media/m(\d+).*/c(\d+)$ media.php?do=comment_edit&cmt=$2 [QSA]
    ReWriteRule ^media/m(\d+).*/p(\d+).*$ media.php?do=details&mid=$1&pid=$2 [QSA]
    ReWriteRule ^media/m(\d+).* media.php?do=details&mid=$1 [QSA]
    ReWriteRule ^media/p(\d+).*/edit$ media.php?do=playlist_edit&pid=$1 [QSA]
    ReWriteRule ^media/p(\d+).* media.php?do=playlist&pid=$1 [QSA]
    ReWriteRule ^media/c(\d+).* media.php?do=category&cid=$1 [QSA]
    ReWriteRule ^media/u(\d+).* media.php?do=user&uid=$1 [QSA]
    ReWriteRule ^media/tag/(.*) media.php?do=tag&tid=$1 [QSA]
    ReWriteRule ^media/results/(.*) media.php?do=results&query=$1 [QSA]
    ReWriteRule ^media/advresults/(.*) media.php?do=advresults&query=$1 [QSA]
    ReWriteRule ^media/letter/(.*) media.php?do=letter&query=$1 [QSA]
    ReWriteRule ^media/browse.* media.php?do=browse [QSA]
    ReWriteRule ^media/search.* media.php?do=search [QSA]
    ReWriteRule ^media/random.* media.php?do=random [QSA]
    ReWriteRule ^media/submit.* media.php?do=submit [QSA]
    ReWriteRule ^media/tagcloud.* media.php?do=tagcloud [QSA]
    ReWriteRule ^media/playlists.* media.php?do=playlists [QSA]
    ReWriteRule ^media/pcreate.* media.php?do=playlists_create [QSA]
    ReWriteRule ^media/pmine.* media.php?do=playlists_mine [QSA]
    ReWriteRule ^media/favorites.* media.php?do=favorites [QSA]
    ReWriteRule ^media/subscriptions.* media.php?do=subscriptions [QSA]
    ReWriteRule ^media/admin/edit/s(\d+).* media.php?do=admin_host_edit&sid=$1 [QSA]
    ReWriteRule ^media/admin/delete/s(\d+).* media.php?do=admin_host_delete&sid=$1 [QSA]
    ReWriteRule ^media/admin/export/s(\d+).* media.php?do=admin_host_export&sid=$1 [QSA]
    ReWriteRule ^media/admin/(\w+).* media.php?do=admin_$1 [QSA]

    # Media Library from Video Directory
    RewriteCond %{QUERY_STRING} do=viewdetails&videoid=(\d+)
    RewriteRule ^video\.php$ media.php?do=details&mid=%1
    RewriteCond %{QUERY_STRING} viewcategory&categoryid=(\d+)
    RewriteRule ^video\.php$ media.php?do=category&cid=%1
    RewriteCond %{QUERY_STRING} viewuser&userid=(\d+)
    RewriteRule ^video\.php$ media.php?do=user&uid=%1
    RewriteCond %{QUERY_STRING} viewtag&tag=(.*)
    RewriteRule ^video\.php$ media.php?do=tag&tid=%1
    ReWriteRule ^video\.php$ media.php
 
 # Send hardcoded ipa scores to arcade instead
  RewriteCond %{QUERY_STRING} act=Arcade [OR]
  RewriteCond %{QUERY_STRING} autocom=arcade
  RewriteRule ^index.php arcade.php [L,QSA]

</ifmodule>

<Files 403.shtml>
order allow,deny
allow from all
</Files>

Options -Indexes
deny from 71.76.197.71
deny from 193.200.150.125
deny from 173.63.38.218
deny from 67.159.60.83
deny from 89.145.95.2

RewriteCond %{HTTP_HOST} ^www.canesinsight.com$ [NC]
RewriteRule ^(.*)$ http://canesinsight.com/$1 [R=301,L]

deny from 72.228.132.15
RewriteCond %{HTTP_HOST} ^canesinsight\.co$ [OR]
RewriteCond %{HTTP_HOST} ^www\.canesinsight\.co$
RewriteRule ^/?$ "http\:\/\/canesinsight\.com" [R=301,L]


#Deny attempts to view the Htaccess file.
<Files .htaccess>
Order allow,deny
Deny from all
</Files>
# BEGIN Browser Caching/Headers
<IfModule mod_mime.c>
    AddType text/css .css
    AddType text/richtext .rtf .rtx
    AddType image/svg+xml .svg .svgz
    AddType text/plain .txt
    AddType text/xsd .xsd
    AddType text/xsl .xsl
    AddType video/asf .asf .asx .wax .wmv .wmx
    AddType video/avi .avi
    AddType image/bmp .bmp
    AddType application/java .class
    AddType video/divx .divx
    AddType application/msword .doc .docx
    AddType application/x-msdownload .exe
    AddType image/gif .gif
    AddType application/x-gzip .gz .gzip
    AddType image/x-icon .ico
    AddType image/jpeg .jpg .jpeg .jpe
    AddType application/vnd.ms-access .mdb
    AddType audio/midi .mid .midi
    AddType video/quicktime .mov .qt
    AddType audio/mpeg .mp3 .m4a
    AddType video/mp4 .mp4 .m4v
    AddType video/mpeg .mpeg .mpg .mpe
    AddType application/vnd.ms-project .mpp
    AddType application/vnd.oasis.opendocument.database .odb
    AddType application/vnd.oasis.opendocument.chart .odc
    AddType application/vnd.oasis.opendocument.formula .odf
    AddType application/vnd.oasis.opendocument.graphics .odg
    AddType application/vnd.oasis.opendocument.presentation .odp
    AddType application/vnd.oasis.opendocument.spreadsheet .ods
    AddType application/vnd.oasis.opendocument.text .odt
    AddType audio/ogg .ogg
    AddType application/pdf .pdf
    AddType image/png .png
    AddType application/vnd.ms-powerpoint .pot .pps .ppt .pptx
    AddType audio/x-realaudio .ra .ram
    AddType application/x-shockwave-flash .swf
    AddType application/x-tar .tar
    AddType image/tiff .tif .tiff
    AddType audio/wav .wav
    AddType audio/wma .wma
    AddType application/vnd.ms-write .wri
    AddType application/vnd.ms-excel .xla .xls .xlsx .xlt .xlw
    AddType application/zip .zip
</IfModule>
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType text/css A2628000
    ExpiresByType text/richtext A3600
    ExpiresByType image/svg+xml A3600
    ExpiresByType text/plain A3600
    ExpiresByType text/xsd A3600
    ExpiresByType text/xsl A3600
    ExpiresByType video/asf A2628000
    ExpiresByType video/avi A2628000
    ExpiresByType image/bmp A2628000
    ExpiresByType application/java A2628000
    ExpiresByType video/divx A2628000
    ExpiresByType application/msword A2628000
    ExpiresByType application/x-msdownload A2628000
    ExpiresByType image/gif A2628000
    ExpiresByType application/x-gzip A2628000
    ExpiresByType image/x-icon A2628000
    ExpiresByType image/jpeg A2628000
    ExpiresByType application/vnd.ms-access A2628000
    ExpiresByType audio/midi A2628000
    ExpiresByType video/quicktime A2628000
    ExpiresByType audio/mpeg A2628000
    ExpiresByType video/mp4 A2628000
    ExpiresByType video/mpeg A2628000
    ExpiresByType application/vnd.ms-project A2628000
    ExpiresByType application/vnd.oasis.opendocument.database A2628000
    ExpiresByType application/vnd.oasis.opendocument.chart A2628000
    ExpiresByType application/vnd.oasis.opendocument.formula A2628000
    ExpiresByType application/vnd.oasis.opendocument.graphics A2628000
    ExpiresByType application/vnd.oasis.opendocument.presentation A2628000
    ExpiresByType application/vnd.oasis.opendocument.spreadsheet A2628000
    ExpiresByType application/vnd.oasis.opendocument.text A2628000
    ExpiresByType audio/ogg A2628000
    ExpiresByType application/pdf A2628000
    ExpiresByType image/png A2628000
    ExpiresByType application/vnd.ms-powerpoint A2628000
    ExpiresByType audio/x-realaudio A2628000
    ExpiresByType application/x-shockwave-flash A2628000
    ExpiresByType application/x-tar A2628000
    ExpiresByType image/tiff A2628000
    ExpiresByType audio/wav A2628000
    ExpiresByType audio/wma A2628000
    ExpiresByType application/vnd.ms-write A2628000
    ExpiresByType application/vnd.ms-excel A2628000
    ExpiresByType application/zip A2628000
</IfModule>
<IfModule mod_deflate.c>
    <IfModule mod_setenvif.c>
        BrowserMatch ^Mozilla/4 gzip-only-text/html
        BrowserMatch ^Mozilla/4\.0[678] no-gzip
        BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
        BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
    </IfModule>
    <IfModule mod_headers.c>
        Header append Vary User-Agent env=!dont-vary
    </IfModule>
    <IfModule mod_filter.c>
        AddOutputFilterByType DEFLATE text/css application/x-javascript text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon
    </IfModule>
</IfModule>
<FilesMatch "\.(css|CSS|)$">
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public, must-revalidate, proxy-revalidate"
    </IfModule>
    FileETag MTime Size
    <IfModule mod_headers.c>
         Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(rtf|rtx|svg|svgz|txt|xsd|xsl|RTF|RTX|SVG|SVGZ|TXT|XSD|XSL|)$">
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public, must-revalidate, proxy-revalidate"
    </IfModule>
    FileETag MTime Size
    <IfModule mod_headers.c>
         Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|swf|tar|tif|tiff|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip|ASF|ASX|WAX|WMV|WMX|AVI|BMP|CLASS|DIVX|DOC|DOCX|EXE|GIF|GZ|GZIP|ICO|JPG|JPEG|JPE|MDB|MID|MIDI|MOV|QT|MP3|M4A|MP4|M4V|MPEG|MPG|MPE|MPP|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|PNG|POT|PPS|PPT|PPTX|RA|RAM|SWF|TAR|TIF|TIFF|WAV|WMA|WRI|XLA|XLS|XLSX|XLT|XLW|ZIP)$">
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public, must-revalidate, proxy-revalidate"
    </IfModule>
    FileETag MTime Size
    <IfModule mod_headers.c>
         Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
    </IfModule>
</FilesMatch>
# End Browser Caching/Headers 
Unfortunately I don't have any benchmark comparisons for before & after cloudflare but here is my latest gtmetrix score on my site that does have cloudflare enabled.
http://gtmetrix.com/reports/www.vide...t.com/lcSDSldq

Also I know nothing about nginx rewrite rules atm so I can't help you with that, maybe I'll try to learn it within the coming month's.

cloferba 01-25-2013 03:09 AM
what does ReWriteRule+^media/p(d+).*/edit$+media.php?do=playlist_edit&pid=$1+[QSA] exactly do?

final kaoss 01-25-2013 03:13 AM
That I believe is for some kindof media addon that used to be popular but is now abandoned, brb.

This is the addon that requires all of the "media" rewrites..
https://vborg.vbsupport.ru/showthread.php?t=240677

AndrewSimm 01-25-2013 03:35 AM
Quote:
Originally Posted by final kaoss (Post 2399313)
Here is the modified htaccess that I recommend for you (includes all of your old rewrite rules yet improves upon the caching/gzip.)

Although if you use cloudflare you would want to load only www urls and no urls that do not have www.
https://discussion.dreamhost.com/thr...html#pid158213


PHP Code:
# Author: Final Kaoss (aka: Extreme-Gaming)
# Version 2.7
RewriteEngine on

# This file is only needed if you have set the Forum Component URL in your admincp and you are 
# using the mod_rewrite option for friendly urls.  If this is the case, copy this file
# to your forum component stub directory.

## Rewrites
<ifmodule mod_rewrite.c>
    RewriteEngine on
        DirectoryIndex index.php

        ReWriteRule ^media/m(\d+).*/tags$ media.php?do=tags_edit&mid=$1 [QSA]
    ReWriteRule ^media/m(\d+).*/edit$ media.php?do=details_edit&mid=$1 [QSA]
    ReWriteRule ^media/m(\d+).*/report$ media.php?do=report&mid=$1 [QSA]
    ReWriteRule ^media/m(\d+).*/c(\d+)$ media.php?do=comment_edit&cmt=$2 [QSA]
    ReWriteRule ^media/m(\d+).*/p(\d+).*$ media.php?do=details&mid=$1&pid=$2 [QSA]
    ReWriteRule ^media/m(\d+).* media.php?do=details&mid=$1 [QSA]
    ReWriteRule ^media/p(\d+).*/edit$ media.php?do=playlist_edit&pid=$1 [QSA]
    ReWriteRule ^media/p(\d+).* media.php?do=playlist&pid=$1 [QSA]
    ReWriteRule ^media/c(\d+).* media.php?do=category&cid=$1 [QSA]
    ReWriteRule ^media/u(\d+).* media.php?do=user&uid=$1 [QSA]
    ReWriteRule ^media/tag/(.*) media.php?do=tag&tid=$1 [QSA]
    ReWriteRule ^media/results/(.*) media.php?do=results&query=$1 [QSA]
    ReWriteRule ^media/advresults/(.*) media.php?do=advresults&query=$1 [QSA]
    ReWriteRule ^media/letter/(.*) media.php?do=letter&query=$1 [QSA]
    ReWriteRule ^media/browse.* media.php?do=browse [QSA]
    ReWriteRule ^media/search.* media.php?do=search [QSA]
    ReWriteRule ^media/random.* media.php?do=random [QSA]
    ReWriteRule ^media/submit.* media.php?do=submit [QSA]
    ReWriteRule ^media/tagcloud.* media.php?do=tagcloud [QSA]
    ReWriteRule ^media/playlists.* media.php?do=playlists [QSA]
    ReWriteRule ^media/pcreate.* media.php?do=playlists_create [QSA]
    ReWriteRule ^media/pmine.* media.php?do=playlists_mine [QSA]
    ReWriteRule ^media/favorites.* media.php?do=favorites [QSA]
    ReWriteRule ^media/subscriptions.* media.php?do=subscriptions [QSA]
    ReWriteRule ^media/admin/edit/s(\d+).* media.php?do=admin_host_edit&sid=$1 [QSA]
    ReWriteRule ^media/admin/delete/s(\d+).* media.php?do=admin_host_delete&sid=$1 [QSA]
    ReWriteRule ^media/admin/export/s(\d+).* media.php?do=admin_host_export&sid=$1 [QSA]
    ReWriteRule ^media/admin/(\w+).* media.php?do=admin_$1 [QSA]

    # Media Library from Video Directory
    RewriteCond %{QUERY_STRING} do=viewdetails&videoid=(\d+)
    RewriteRule ^video\.php$ media.php?do=details&mid=%1
    RewriteCond %{QUERY_STRING} viewcategory&categoryid=(\d+)
    RewriteRule ^video\.php$ media.php?do=category&cid=%1
    RewriteCond %{QUERY_STRING} viewuser&userid=(\d+)
    RewriteRule ^video\.php$ media.php?do=user&uid=%1
    RewriteCond %{QUERY_STRING} viewtag&tag=(.*)
    RewriteRule ^video\.php$ media.php?do=tag&tid=%1
    ReWriteRule ^video\.php$ media.php
 
 # Send hardcoded ipa scores to arcade instead
  RewriteCond %{QUERY_STRING} act=Arcade [OR]
  RewriteCond %{QUERY_STRING} autocom=arcade
  RewriteRule ^index.php arcade.php [L,QSA]

</ifmodule>

<Files 403.shtml>
order allow,deny
allow from all
</Files>

Options -Indexes
deny from 71.76.197.71
deny from 193.200.150.125
deny from 173.63.38.218
deny from 67.159.60.83
deny from 89.145.95.2

RewriteCond %{HTTP_HOST} ^www.canesinsight.com$ [NC]
RewriteRule ^(.*)$ http://canesinsight.com/$1 [R=301,L]

deny from 72.228.132.15
RewriteCond %{HTTP_HOST} ^canesinsight\.co$ [OR]
RewriteCond %{HTTP_HOST} ^www\.canesinsight\.co$
RewriteRule ^/?$ "http\:\/\/canesinsight\.com" [R=301,L]


#Deny attempts to view the Htaccess file.
<Files .htaccess>
Order allow,deny
Deny from all
</Files>
# BEGIN Browser Caching/Headers
<IfModule mod_mime.c>
    AddType text/css .css
    AddType text/richtext .rtf .rtx
    AddType image/svg+xml .svg .svgz
    AddType text/plain .txt
    AddType text/xsd .xsd
    AddType text/xsl .xsl
    AddType video/asf .asf .asx .wax .wmv .wmx
    AddType video/avi .avi
    AddType image/bmp .bmp
    AddType application/java .class
    AddType video/divx .divx
    AddType application/msword .doc .docx
    AddType application/x-msdownload .exe
    AddType image/gif .gif
    AddType application/x-gzip .gz .gzip
    AddType image/x-icon .ico
    AddType image/jpeg .jpg .jpeg .jpe
    AddType application/vnd.ms-access .mdb
    AddType audio/midi .mid .midi
    AddType video/quicktime .mov .qt
    AddType audio/mpeg .mp3 .m4a
    AddType video/mp4 .mp4 .m4v
    AddType video/mpeg .mpeg .mpg .mpe
    AddType application/vnd.ms-project .mpp
    AddType application/vnd.oasis.opendocument.database .odb
    AddType application/vnd.oasis.opendocument.chart .odc
    AddType application/vnd.oasis.opendocument.formula .odf
    AddType application/vnd.oasis.opendocument.graphics .odg
    AddType application/vnd.oasis.opendocument.presentation .odp
    AddType application/vnd.oasis.opendocument.spreadsheet .ods
    AddType application/vnd.oasis.opendocument.text .odt
    AddType audio/ogg .ogg
    AddType application/pdf .pdf
    AddType image/png .png
    AddType application/vnd.ms-powerpoint .pot .pps .ppt .pptx
    AddType audio/x-realaudio .ra .ram
    AddType application/x-shockwave-flash .swf
    AddType application/x-tar .tar
    AddType image/tiff .tif .tiff
    AddType audio/wav .wav
    AddType audio/wma .wma
    AddType application/vnd.ms-write .wri
    AddType application/vnd.ms-excel .xla .xls .xlsx .xlt .xlw
    AddType application/zip .zip
</IfModule>
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType text/css A2628000
    ExpiresByType text/richtext A3600
    ExpiresByType image/svg+xml A3600
    ExpiresByType text/plain A3600
    ExpiresByType text/xsd A3600
    ExpiresByType text/xsl A3600
    ExpiresByType video/asf A2628000
    ExpiresByType video/avi A2628000
    ExpiresByType image/bmp A2628000
    ExpiresByType application/java A2628000
    ExpiresByType video/divx A2628000
    ExpiresByType application/msword A2628000
    ExpiresByType application/x-msdownload A2628000
    ExpiresByType image/gif A2628000
    ExpiresByType application/x-gzip A2628000
    ExpiresByType image/x-icon A2628000
    ExpiresByType image/jpeg A2628000
    ExpiresByType application/vnd.ms-access A2628000
    ExpiresByType audio/midi A2628000
    ExpiresByType video/quicktime A2628000
    ExpiresByType audio/mpeg A2628000
    ExpiresByType video/mp4 A2628000
    ExpiresByType video/mpeg A2628000
    ExpiresByType application/vnd.ms-project A2628000
    ExpiresByType application/vnd.oasis.opendocument.database A2628000
    ExpiresByType application/vnd.oasis.opendocument.chart A2628000
    ExpiresByType application/vnd.oasis.opendocument.formula A2628000
    ExpiresByType application/vnd.oasis.opendocument.graphics A2628000
    ExpiresByType application/vnd.oasis.opendocument.presentation A2628000
    ExpiresByType application/vnd.oasis.opendocument.spreadsheet A2628000
    ExpiresByType application/vnd.oasis.opendocument.text A2628000
    ExpiresByType audio/ogg A2628000
    ExpiresByType application/pdf A2628000
    ExpiresByType image/png A2628000
    ExpiresByType application/vnd.ms-powerpoint A2628000
    ExpiresByType audio/x-realaudio A2628000
    ExpiresByType application/x-shockwave-flash A2628000
    ExpiresByType application/x-tar A2628000
    ExpiresByType image/tiff A2628000
    ExpiresByType audio/wav A2628000
    ExpiresByType audio/wma A2628000
    ExpiresByType application/vnd.ms-write A2628000
    ExpiresByType application/vnd.ms-excel A2628000
    ExpiresByType application/zip A2628000
</IfModule>
<IfModule mod_deflate.c>
    <IfModule mod_setenvif.c>
        BrowserMatch ^Mozilla/4 gzip-only-text/html
        BrowserMatch ^Mozilla/4\.0[678] no-gzip
        BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
        BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
    </IfModule>
    <IfModule mod_headers.c>
        Header append Vary User-Agent env=!dont-vary
    </IfModule>
    <IfModule mod_filter.c>
        AddOutputFilterByType DEFLATE text/css application/x-javascript text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon
    </IfModule>
</IfModule>
<FilesMatch "\.(css|CSS|)$">
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public, must-revalidate, proxy-revalidate"
    </IfModule>
    FileETag MTime Size
    <IfModule mod_headers.c>
         Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(rtf|rtx|svg|svgz|txt|xsd|xsl|RTF|RTX|SVG|SVGZ|TXT|XSD|XSL|)$">
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public, must-revalidate, proxy-revalidate"
    </IfModule>
    FileETag MTime Size
    <IfModule mod_headers.c>
         Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
    </IfModule>
</FilesMatch>
<FilesMatch "\.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|swf|tar|tif|tiff|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip|ASF|ASX|WAX|WMV|WMX|AVI|BMP|CLASS|DIVX|DOC|DOCX|EXE|GIF|GZ|GZIP|ICO|JPG|JPEG|JPE|MDB|MID|MIDI|MOV|QT|MP3|M4A|MP4|M4V|MPEG|MPG|MPE|MPP|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|PNG|POT|PPS|PPT|PPTX|RA|RAM|SWF|TAR|TIF|TIFF|WAV|WMA|WRI|XLA|XLS|XLSX|XLT|XLW|ZIP)$">
    <IfModule mod_headers.c>
        Header set Pragma "public"
        Header append Cache-Control "public, must-revalidate, proxy-revalidate"
    </IfModule>
    FileETag MTime Size
    <IfModule mod_headers.c>
         Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
    </IfModule>
</FilesMatch>
# End Browser Caching/Headers 
Unfortunately I don't have any benchmark comparisons for before & after cloudflare but here is my latest gtmetrix score on my site that does have cloudflare enabled.
http://gtmetrix.com/reports/www.vide...t.com/lcSDSldq

Also I know nothing about nginx rewrite rules atm so I can't help you with that, maybe I'll try to learn it within the coming month's.
I changed my htaccess to yours. What exactly does it do differently than mine?

final kaoss 01-25-2013 03:43 AM
What it has done is improved your score in a few areas.
http://gtmetrix.com/compare/x3r3SsIp/6uFU7ZYI

Your webpagetest score is also much better than it was previously as well now.
http://www.webpagetest.org/result/130125_G1_3JM/

Although webpagetest archive results are a bit harder to find..

If you still have that custom htaccess, something is still not allowing you to properly cache images it looks like and it could be nginx is my guess. The only stuff that should be failing is external resources.

http://www.webpagetest.org/pagespeed...run=1&cached=0

AndrewSimm 01-25-2013 03:50 AM
give me a min and I will turn nginx off

--------------- Added [DATE]1359089623[/DATE] at [TIME]1359089623[/TIME] ---------------

I ran

root@can [/etc/init.d]# httpd stop
Stopping nginx daemon: nginx.

--------------- Added [DATE]1359089969[/DATE] at [TIME]1359089969[/TIME] ---------------

Hm I am not sure I have a way to just stop nginx

final kaoss 01-25-2013 04:09 AM
Ok, I do know that cloudflare does support caching but I haven't had a chance to try it out without the modified htaccess. That could help you in the meantime. Remember to disable rocketloader.

AndrewSimm 01-25-2013 04:30 AM
Ok I think I fixed nginx

final kaoss 01-25-2013 04:32 AM
Yes that does seem to be a good deal better for you!

One last major thing that I strongly suggest you do, if you haven't already, is make a optimization request here:
https://www.vbulletin.com/forum/foru...-Configuration

While I kindof know my way around apache, the guy who answers threads there is a guru at what he does! It should definitely hellp you with memory settings and all of the other necessary hardware/mysql settings stuff.

Also, check and see if there are login issues after applying the htaccess. In my guide I noticed there were for me and I posted my fix on post 3 of the guide.

AndrewSimm 01-25-2013 04:45 AM
Quote:
Originally Posted by final kaoss (Post 2399332)
Yes that does seem to be a good deal better for you!

One last major thing that I strongly suggest you do, if you haven't already, is make a optimization request here:
https://www.vbulletin.com/forum/foru...-Configuration

While I kindof know my way around apache, the guy who answers threads there is a guru at what he does! It should definitely hellp you with memory settings and all of the other necessary hardware/mysql settings stuff.

Also, check and see if there are login issues after applying the htaccess. In my guide I noticed there were for me and I posted my fix on post 3 of the guide.
I can log in fine. I submitted a request. Thank for all your help. Let me know if there is anything else you can think of.

--------------- Added [DATE]1359093176[/DATE] at [TIME]1359093176[/TIME] ---------------

hmm yslow does agree with my expiree headers

final kaoss 01-25-2013 04:54 AM
The final thing I can suggest to you now is to try to losslessly compress your images and use css sprites. Sometimes the lossless compression can make a huge difference.



This covers the following PageSpeed Rules:

Optimize Images

If you have access to Visual Studio 2010 (grab the trial edition here which should be compatible with the addon), you can complete this step in just a few clicks using this addon
http://visualstudiogallery.msdn.micr...77c3?SRC=VSIDE

You can also use pnggauntlet which will compress images as well.

This is great to use on new skins/themes and also non animated gifs for Smileys, Signatures, Photo Album & Group Images and Avatars that your forum stores on your server!

Please note that both of these methods produce lossless images, meaning that even though it uses various methods to decrease file size there is absolutely no visual loss in quality of the image.

For non-windows users, I suggest Imageoptim & Trimage



Step 4: Add CSS Sprites.
Not compatible with VB3

This covers the following PageSpeed Rules:

Combine images using CSS sprites

Code:
Install the addon United-Forum CSS Sprites and follow all of the instructions.  Can be found in the sprite_0.6.3a.zip file.
2 more things from the top of my head to consider (which I haven't had a chance to try yet) are mod_spdy & mod_pagespeed from google.

Carnage 01-26-2013 09:42 AM
Looking at it, this is one of those instances where you may just have to throw hardware at the issue. A second CPU, more ram or faster drives could help you out depending on where your bottleneck is. (From the limited info you've given us, I suspect harddrives will give the biggest win)

Before doing that, I would highly recomend looking at mariaDB as an alternative to myisam and switching fully over to nginx + php fcgi will help save some resources as well.

final kaoss 01-26-2013 07:06 PM
Well, not quite. I do agree that you may need some more hardware. Try putting the static content (js/css/html/images etc... pretty much all of your files) onto one server.

Grab a second server and use that one to only host mysql. I think your 16GB of ram should be more than enough.

Also, you may want to check out google page speed service. I've been receiving much better optimization reports recently while using it!
Page Speed 1.12 Score: 92/100 Compared to 82 or so while using cloudflare instead..

vbresults 01-26-2013 11:20 PM
You don't need better hardware. The server needs to be tuned by a professional.


All times are GMT. The time now is 02:22 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.03273 seconds
  • Memory Usage 2,211KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_code_printable
  • (2)bbcode_php_printable
  • (17)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (37)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 
Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 
Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete