vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Board Optimization - vB Accelerator (https://vborg.vbsupport.ru/showthread.php?t=244391)

Vitaly 06-10-2010 10:00 PM

vB Accelerator
 
1 Attachment(s)
ported & significantly improved vb 3.8 branch.

This mod is suited for medium and big boards. It reduces server load, caused by thumnails and big attachments. Includes significant recommendations for server tuning, to speedup pages loading.

what's the problem with?
  1. In original vB all thumbnails a downloaded via php. So, if you page contains 10-20 attached images, then each page php request will be followed by 10-20 thumbnails php requests. That's very bad.
  2. When attachments body transfered via php, that's much more waisteful, than direct transfer of static file. Especially for big files.
  3. Browser does lots of additional requests to static file, to check if modified
  4. JS/CSS not compressed
what this mod does:
  1. All thumbnails will have DIRECT links (served as static files). Much lower reply latency and server load.
  2. If you have nginx webserver, you php-fcgi will not participate in file transfer any more. It will reply with X-Accel-Redirect header. Then nginx will proceed attachment as static file.
    • No more problems with multiple downloads of 10-mb files.
    • No needs to restrict attachments and full-size images for guests and search bots.
  3. Static files are properly cached/compressed after tuning
how to install:
  1. Make sure, attachments are stored on disk, NOT in database.
  2. Make sure, that thumbnails are in web-accessible area. If not - reconfigure your web-server first.
  3. Check manually, that you can download any .thumb file via browser.
  4. !!! Tune cache/compression settings for static files (see example from next post)
  5. Import product XML & tune settings.
  6. Enjoy
This mod is developped here: http://github.com/rcdesign/vb-accelerator . Patches are welcome.

history:

0.9
- fixed path for duplicated attachments

0.8
- fixed back timestamp parameter & substitutions for XML output
- added conversion for assets.php thumbnails (in attachments manager)

0.7
- ported to vb4
- added basic CDN support for thumbnails

Vitaly 06-11-2010 06:51 AM


Nginx config sample


Code:

# Modified gzip settings: added CSS/JS,
# but completely disabled for broken browsers.
gzip_types      text/plain text/css application/x-javascript text/xml text/javascript;
gzip_disable    "MSIE [1-6].(?!.*SV1)";


# globally disable external access to attachments, but enable for X-Accel-Redirect
location /uploads {
    internal;
}
# enable thumbnails direct access, but nothing else!
location ~* /uploads/(.*)\.thumb$ {
    expires 24h;
}


# Expires for static files.
location /images/ {
    expires 24h;
}
location /clientscript/ {
    expires 24h;
}
location /custom { # All at once: profile pics, avatars, group images
    expires 24h;
}




Apache config example


TBD :) . Please, help to fill.

funmasti 06-11-2010 09:41 AM

Thanks... anyone tried it?

gwerzal 06-11-2010 10:11 AM

This looks good. thanks for sharing.

Vitaly 06-12-2010 12:35 PM

Updated mod to catch XML output & timestamp issues.

TeknoSounds 06-12-2010 05:46 PM

Will this work along side vbOptimize and vB4: Supercharged? Are there conflicts? Duplicated features?

ChopSuey 06-12-2010 05:54 PM

Quote:

Originally Posted by TeknoSounds (Post 2052600)
Will this work along side vbOptimize and vB4: Supercharged? Are there conflicts? Duplicated features?

vBOptimize is more of caching, vB4 supercharges is like "Store CSS As files" option but a little better. I don't see any conflicts.

Darkimmortal 06-12-2010 10:25 PM

Oh cool, this looks like an improved version of my 'Attachment Optimiser' in vB4 SuperCharged - installed :)

Dr.osamA 06-12-2010 10:59 PM

Thanks
installed

but i am not sour if it is working

anyone tried it?

waiting for tests

Vitaly 06-13-2010 12:31 AM

Quote:

Originally Posted by Darkimmortal (Post 2052690)
Oh cool, this looks like an improved version of my 'Attachment Optimiser' in vB4 SuperCharged - installed :)

In fact, on the second page load, other feature doesn't make sence, if you set long expire for static content :) . You can check with firebug NET bookmark.

Optimizing the first page load is potentially interesting, but price in your case is very high - big server load and lots of possible conflicts.

kawe 06-13-2010 12:36 AM

This is .htaccess on Nginx ?

PHP Code:

# Modified gzip settings: added CSS/JS,
# but completely disabled for broken browsers.
gzip_types       text/plain text/css application/x-javascript text/xml text/javascript;
gzip_disable     "MSIE [1-6].(?!.*SV1)";


# globally disable external access to attachments, but enable for X-Accel-Redirect
location /uploads {
    
internal;
}
# enable thumbnails direct access, but nothing else!
location ~* /uploads/(.*)\.thumb$ {
    
expires 24h;
}


# Expires for static files.
location /images/ {
    
expires 24h;
}
location /clientscript/ {
    
expires 24h;
}
location /custom # All at once: profile pics, avatars, group images
    
expires 24h;


And about " path to attachments "
is full path like : /home/www/html/sites.com/attachment
or
just /attachment/

Vitaly 06-13-2010 12:46 AM

Quote:

Originally Posted by kawe (Post 2052715)
This is .htaccess on Nginx ?

Nginx doesn't support .htaccess. That's pаrt of config file. Please, read nginx documentation for details.

kawe 06-13-2010 12:58 AM

Quote:

Originally Posted by Vitaly (Post 2052720)
Nginx doesn't support .htaccess. That's pаrt of config file. Please, read nginx documentation for details.

yeah i add this code to /etc/nginx/sites-enable/mydomain.com :D
under my vBseo config :) its right ?

Vitaly 06-13-2010 01:01 AM

Quote:

Originally Posted by TeknoSounds (Post 2052600)
Will this work along side vbOptimize and vB4: Supercharged? Are there conflicts? Duplicated features?

vbOptimize do different things. It can make sence, if you have DB on separate server. Or if you setup many dirty-written mods :) . No conflicts. If you have local DB, then built-in query cache do most things for you.

Supercharged have some interesting ideas, but not balanced, IMHO. The same result can be acheived with much less efforts & server load :)

vb Accelerator - if you dont use attachments, skip it :) . But set "not expire" headers for static content. It really worth to do.

Vitaly 06-13-2010 01:06 AM

Quote:

Originally Posted by kawe (Post 2052723)
yeah i add this code to /etc/nginx/sites-enable/mydomain.com :D
under my vBseo config :) its right ?

Gzip options can be in other place. Look there they are in your config, compare difference and fix.

Other rewrites should be BEFORE vbseo.

Fenriz 06-14-2010 06:24 PM

Can I use Mod Version: 0.8 with vbulletin 3.8.5?

Vitaly 06-15-2010 05:39 AM

NO. You should use version from this topic: https://vborg.vbsupport.ru/showthread.php?t=207566

tupique 06-22-2010 09:30 AM

Installed, but downloaded from other forum by accident. However, I don't see that the URL is changing. I still see attachment.php as part of the URL. Am I missing something here?

Vitaly 06-22-2010 10:54 AM

You should check thumbnails path, but not full-size attachments links.

If you have problems, please provide more detailed information. For example - incorrect thumbnails URLs

tupique 06-22-2010 11:45 AM

ahh I see.. so this mod is used to change the thumbnails URL only, not the full image? I did simple checking (disabling and enabling this mod), and I saw what you meant. Thanks a lot!

Another question, the URL in my forum is like this:

http://<domain hidden>/img/1/1.thumb?d=1276962745

Could you remove the query string parameter? It ruins the purpose of image caching, since the parameter value always changing while the image stays the same.

Vitaly 06-22-2010 11:59 AM

Quote:

Originally Posted by tupique (Post 2057413)
so this mod is used to change the thumbnails URL only, not the full image?

The goal of this mod is to reduce server load. Direct thumb links affect load. Full attach links - not.

There can be 20 thumbs on thread page. But not 20 fullsize images, downloaded at once.

Quote:

Originally Posted by tupique (Post 2057413)
http://<domain hidden>/img/1/1.thumb?d=1276962745

Could you remove the query string parameter? It ruins the purpose of image caching, since the parameter value always changing while the image stays the same.

This parameter doesn't affect cacheing. It's fixed for each thumbnail.

tupique 06-22-2010 12:08 PM

great! thanks!

tupique 06-22-2010 02:10 PM

Could you maybe do the same for image.php (to display user's avatar)? It's hard to cache image served by PHP script.

Vitaly 06-22-2010 02:24 PM

It's already done in vbulletin settings. Please, read vb manual. This mod is NOT buggy, and works properly. Really.

ssslippy 07-03-2010 06:45 PM

Do attachments themselves need to be in a public accessible area or just the thumbnails?

Vitaly 07-03-2010 06:52 PM

Just the thumbnails

Xencored 07-03-2010 07:00 PM

Fact my thumbs seem tons faster to load thanks!!

now Vitaly could you explain this part please

"CDN list
Enter up to 4 mirror domains, to load local images faster. For example:"

Thanks!

Ps. how do i know if i have a NGINX websever lol sorry :x

Vitaly 07-03-2010 07:30 PM

You'd better to read yahoo website optimization manuals.

In 2 words:

1. Browser allows only 2 parallel connections to each domain, and 8 total.
2. If you do mirrors for your images, it can download up to 8 images simultaniously
3. It's better to keep mirrors on separate domain, without cookies, to speed up requests

That can make some sence, ONLY if you have tons of thumbnails on each single page.

I did this feature for fun, and didn't meashured effect yet. Anyway, it's not as significant, as (direct links + proper cache headers for static files).

Quote:

Ps. how do i know if i have a NGINX websever lol sorry :x
If you ask such question, then 99% you don't have nginx :)

Xencored 07-03-2010 08:16 PM

Quote:

Originally Posted by Vitaly (Post 2063979)
You'd better to read yahoo website optimization manuals.

In 2 words:

1. Browser allows only 2 parallel connections to each domain
2. If you do mirrors for you images, it can download up to 8 images simultaniously
3. It's better to keep mirrors no separate domain, without cookies, to speed up requests

That can make some sence, ONLY if you have tons of thumbnails on each single page.

I did this feature for fun, and didn't meashured effect yet. Anyway, it's not as significant, as (direct links + proper cache headers for static files).

Thanks for that i dont use any mirrors so i can leave that blank

Quote:

Originally Posted by Vitaly (Post 2063979)
If you ask such question, then 99% you don't have nginx :)

Guess not than :)

Thanks mate the speed on pages like >>This<< is tons better

Nomination!

Vitaly 07-04-2010 07:01 AM

Quote:

Originally Posted by Animemike (Post 2064015)
Thanks mate the speed on pages like >>This<< is tons better

You also have to tune apache config. Remove ETAG headers, and add Expires (in next 7 days) headers instead for all static files.

Xencored 07-04-2010 12:35 PM

Quote:

Originally Posted by Vitaly (Post 2064222)
You also have to tune apache config. Remove ETAG headers, and add Expires (in next 7 days) headers instead for all static files.

Oh but it seems to have speeded it up with out that
so if i work out how to do above it will be even faster?:D

doraj 07-04-2010 03:11 PM

Hello Vitaly,

in the AdminCP Options, at "Path to attachments", need to insert the patch of the thumb right?

But it is the same of the attachments? If not, How can I find the correct path? I remember to have setup the attachments stored on disk, but I don't rememeber a specific path where are store my thumb.

All my attachments are in : "forum/attachments"

Thank you

Vitaly 07-04-2010 03:31 PM

Try /forum/attachments or /attachments

It's easy to check:
- open page source
- check generated thumbnal paths
- if not correct - play with setting to fix difference.

Of cause, you should check, that thumbnails can be downloaded via www (that's mentioned in installetion steps).

Vitaly 07-04-2010 03:42 PM

Quote:

Originally Posted by Animemike (Post 2064316)
Oh but it seems to have speeded it up with out that
so if i work out how to do above it will be even faster?:D

Sorry to disappoint, but I have no apache. So, I can say WHAT should be done, but not HOW. Try to ask your admin or google this question :) . Don't forget to share result, if you have some.

ImmortalForums 07-25-2010 03:58 AM

Hmm have an odd issue. I have attachments on a subdomain along with all the url rewrites working. The attachment thumbnails show however when you click on the attachment image they do not load in the lightbox. Any advice?

I do run imagemagick of that has any effect.

Vitaly 07-25-2010 12:44 PM

Do those work:

- if you switch off lightbox
- if you switch off this mod

?

The other way to check is to install firebug addon for firefox, and check what happens in network tab when image fails to load.

ImmortalForums 07-25-2010 09:23 PM

Turns out I am getting 404 errors on the lightbox and if I load the attachment directly. If I disable this mod the attachments work fine and the thumbs still work.

So it seems something from this mod is breaking the attachments themselves.

Vitaly 07-26-2010 04:12 AM

Can you check with firebug, which URL failed for lightbox?

ImmortalForums 07-26-2010 05:01 PM

Played with it some more it turns out its the X-Accel-Redirect. Updated my NGINX just to make sure it wasnt a bug.


Failing URL is
http://immortal-guild.net/attachment...1&d=1271597314

Vitaly 07-26-2010 05:26 PM

This mod doesn't affect fullsize attachments, if you turn off "nginx acceleration" option.

If problem caused by this option, then absolute disk path can be wrong, or some rare nginx limitation for X-Accel-Redirect (see docs).


All times are GMT. The time now is 10:24 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01265 seconds
  • Memory Usage 1,838KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_code_printable
  • (1)bbcode_php_printable
  • (14)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (1)pagenav_pagelink
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete