vB Accelerator

[Vitaly]

vb 4.0 version here (https://vborg.vbsupport.ru/showthread.php?t=244391)


This mod is suited for medium and big boards. It reduces server load, caused by thumnails and big attachments.

what's the problem with?
In original vB all thumbnails a downloaded via php. So, if you page contains 10-20 attached images, then each page php request will be followed by 10-20 thumbnails php requests. That's very bad.
When attachments body transfered via php, that's much more waisteful, than direct transfer of static file. Especially for big files.what this mod does:
All thumbnails will have DIRECT links (served as static files). Much lower reply latency and server load.
If you have nginx (http://nginx.net/) webserver, you php-fcgi will not participate in file transfer any more. It will reply with X-Accel-Redirect header. Then nginx will proceed attachment as static file.
No more problems with multiple downloads of 10-mb files.
No needs to restrict attachments and full-size images for guests and search bots.
That works for forum posts, blogs, albums. Everywhere!how to install:
Make sure, this files are stored on disk, NOT in database:
post attachments & thumbnails
blog attachments & thumbnails
album images & thumbnails
Make sure, that thumbnails are in web-accessible area. If not - reconfigure your web-server first. If you own big board - then you know how to do that :) .
Check manually, that you can download any .thumb file via browser.
Import this product, and set options for you configuration.
Be happy to reduce server count in your cluster :)PS. It's tested and works fine at my production site for 1 week. But I'll be happy to know server load statistics from big boards owners.

This mod is developped here: http://github.com/rcdesign/vb-accelerator . Patches are welcome.

History

0.4
- fixed bug with PHP wagnings in server logs.

0.3
- first public release

[Vitaly]

Faq

Q1. I don't have nginx (http://nginx.net/). Will this mod work for me?

Yes, partially. Enable direct thumbnail links, but disable nginx acceleration. You'll still have serious benefit, if your forum is plenty of attachments.

Q2. What about lignttpd, apache and so on? They have X-Sendfile, alternative to nginx X-Accel-Redirect.

I know, but:
I don't have such configurations to test, and I don't have time to implement that. You can do it yourself or hire someone (NOT ME) for this work.
X-Sendfile don't support partial content download :( . Nginx (http://nginx.net/)'s X-Accel-Redirect it the best you can use.Q3. Is it safe to give direct access for thumbnails?

Why not? You can give direct access for thumbnails, but disable direct access to original files. Just configure your server properly. See examples below.

Q4. Thumbnails looks like files with strange extention and mime-type. Will those be ok in browser?

Yes. HTML-page address those files via imagetag. So, browser show those properly, even with wrong mime and strange extention.

Q5. Why don't you fix that for sure?

Because that requires vB sources modification, and i hate such things. And if everyting works fine, who cares about mimes :) ?

Q6. My forum has 20 users online, and I don't see difference at my 32-core server with 128Gb ram and 8 SSD raid array.

You a happy man, really.

Q7. Does it work with VBSEO ?

If attachment rewrite is disabled - yes, it works. And check, that attachments path is separated in vbseo rewrite rules in webserver config. I use vbseo in this mode with no problems. Sorry, no plans now to make better integration with vbseo. But code is reuseable, or you can send your patches :)



Server configuration examples

let's say, whe have the following folders structure:

/uploads
/uploads/blogs/* <- blog attachments & thumbnails
/uploads/posts/* <- forum attachments & thumbnails
/uploads/albums/* <- album images
/uploads/al_thumbs/* <- album previews

So, we have to setup direct access for thumbtails, but protect original files.

Nginx

# globally disable external access, but enable for X-Accel-Redirect
location /uploads {
internal;
}
# enable direct access to blogs & posts thumbnails, but nothing else!
location ~* /uploads/(.*)\.thumb$ {
}
# enable direct access to album thumbnails
location /uploads/al_thumbs/ {
}

Apache/Lighttpd

TBD. You are welcome to share your one :)

[lonasdigital]

Gracias Voy a probarlo ahora mismo .

[Frondy]

Well, it works! Thank you!

[rob01]

is this a good option for small forums?

[JamesAB]

Q4. Thumbnails looks like files with strange extention and mime-type. Will those be ok in browser?

Yes. HTML-page address those files via imagetag. So, browser show those properly, even with wrong mime and strange extention.
I'm not using this mod, but I recently looked into serving static thumbnails on vBulletin.

In apache httpd.conf, I know you can use:
AddType image/jpeg .thumb

[lazydesis]

what is this nginx ?i have centos OS and apache .. will this increase performance on that?

[Vitaly]

is this a good option for small forums?

Everything is relative in this life :) . That depends on attachments count and CPU/IO load. If you have no attachments at all, you'll have nothing with this mod.

But nobody prohibit you to try and compare if server load changes :) . At least, direct linked thumbnails will be loaded faster for your users.

I'm not using this mod, but I recently looked into serving static thumbnails on vBulletin.

In apache httpd.conf, I know you can use:
AddType image/jpeg .thumb

Yes. Exactly. But in fact that's not required. My rule is simple: the less customisations - the less possible mistakes in the future :) . You can always add it later, if something goes wrong.

[Vitaly]

what is this nginx ?i have centos OS and apache .. will this increase performance on that?

Yes. Nginx/lighttpd/lightsphere/zeus + php-fcgi will give you serious benefits over apache + mod_php on burst loads.

In fact, that should be done before any futher vb optimizations. Maybe I'm a bit categoric, but in my opinion apache+mod_php system is absolutely inpredictable at big loads.

Server optimisation should be done prior to any scripts optimization. That's essential.

[Bounce]

I don't have an uploads folder so I don't know the paths

Path to etc

How do I find this out?

[Vitaly]

ACP -> Attachments -> Storage type
ACP -> User Albums -> Storage type

You'd better look to vB documentation about details. That's VERY CPU/Disk intencive operation, and you have to backup your board.

[Bounce]

ACP -> Attachments -> Storage type
ACP -> User Albums -> Storage type

thanks

You'd better look to vB documentation about details. That's VERY CPU/Disk intencive operation, and you have to backup your board.yip aware o that,ty :up:

[mindhunter77]

Anyone have any code to make the thumbnails accessible in LITGHTPD ?

[relaxiha]

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 43

ERROR

[Vitaly]

relaxiha,

1. Which php version?

2. Fatal error or warning message in webserver log?

3. At which page?

http://forum.rcdesign.ru/f14/

Seems to work. Go to any topic with attachments and see direct links for preview sources.

[chesterano]

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30


I have Apache please more detalis how setup direct access for thumbtails, because stop show ;(

[Vitaly]

Try attached file, that should help with warning mistake.

[chesterano]

Working :) You are my Idol Thank You, everything is ok, only under IE and Opera thumbtails, not show. Under Fiefox and Chrome everything is ok... little bit strange? but I try checking config browse
Thank You

[Vitaly]

Hm... that's strange... Find any direct link in FF, and check manually, that it's accessible from IE/Opera via web. Don't hesitate to provide screenshots of broken pages. I have very weak telepathy skills, to understand the problem from description "it doesn't work"

Nobody of my regular 6000 users reported problem with IE. Try this advice: https://vborg.vbsupport.ru/showpost.php?p=1762207&postcount=9

May be, apache work not as nginx with unknown extention types, no idea.

[chesterano]

I install plugin again and now is ok, under IE, Opera, Chrome, Firefox :)

Thank You, Thank You, Thank You...Vitaly

Regards
chesterano

[Vitaly]

Since no more bugs reported - version 0.4 released. If fixes php warnings in webserver logs. Nothing special.

[mindhunter77]

preeeeeettttyyyyy please make this work with lighttpd lol...

[relaxiha]

Thank You, Thank You, Thank You...Vitaly

[xxalexkimxx]

Thank you very much.

[Xencored]

Thanks installed

[newmomsforum]

Hi

I have VBSEO running on one of my forums and the attachment URL's are re-written to include the name of the post and forum.

When I Set direct links for thumbnail images to Yes, this breaks all of my images (I assume there is a comflict between VBSEO and your great mod).

Should this work with VBSEO or hasn't it been tested.

Thanks in advance :)

[Vitaly]

I have vbseo installed, but URLs conversion for attachments is turned off in my configuration.

Sorry, no plans to integrate with vbseo attachments urls conversion.

[mindhunter77]

Request for lighttpd integration =)

[thcf]

Hi there does with work with vbseo and vboptimise?

[mindhunter77]

How do I make the thumbs web accessible? They are located in a web accessible DIR as well as 777 chmod but I still can not bring them up in a browser.

[vktechnology]

How do I make the thumbs web accessible? They are located in a web accessible DIR as well as 777 chmod but I still can not bring them up in a browser.


I have same problem
my path is "/home/palungjit/board/attachments"
When i put this path it doesnt show thumbnail

after i t put "attachments" then it work

[vktechnology]

My board just in stall this plugin

Posts: 1,773,284
Members: 254,310

Attachments : 255.61 GB
Attachments : 495,833 reccords

I thinks this plugin will help to reduce server load.

[mindhunter77]

how did you make the .thumb web accessible?

[mindhunter77]

is there a way to verify this is working, I think I finally got it all figured out but wanted to verify.

[mindhunter77]

Finally got this going I think, only one drawback from my site, it seems that this conflicts with this mod,

WATERMARK ATTACHMENTS
https://vborg.vbsupport.ru/showthread.php?t=201972

If I turn off Enable large downloads acceleration, the watermarking works again.

[MmS1]

Not working
the attachments still /attachment.php?attachmentid=2403
plz help me am having big problem with attachments

[mindhunter77]

Yea, mine look this

http://www.xxxxxxxxx.com/uploadedimages/1/20016.thumb?d=1222135419

So I guess it's not working

[MmS1]

It's oky working fine with me
But i can't download files pic's oky i can but files i can't

even the attachments convert them into direct link or just thumb only

[mindhunter77]

could someone who has this working post their nginx.conf so I can see what I'm doing wrong, this still isn't working.

[Vitaly]

For all, who think, that mod is not working:

1. Check that thumbnails are accessible via www (type URL to any thumbnail in browres path). Browser should offer to download file or to show thumbnail (if youcared about mime type)

2. Look into page source, and check "direct" path to any thumbnail. If pattern is different - fix appropriate mod settings.

3. Split your job in 2 parts:
- try to do direct thumbnails working first
- try to make work nginx accelerated download. DONT turn it ON with another web-server.

4. If you have VBSEO - don't enаble attachments rewrite.


Sorry, but can't consult everyone about each particular configuration :( . I hope, that general recommendations will help.

[mindhunter77]

1. Check that thumbnails are accessible via www (type URL to any thumbnail in browres path). Browser should offer to download file or to show thumbnail (if youcared about mime type)

This is part of my issue, I got nginx running, that was pretty easy, but for some reason I can not access thumbs via the browser, I disabled vbseo attachments, chmod 777, but still no go.

[Vitaly]

I think, this page contains whe most actual resources, if you have troubles with nginx configuration:

http://nginx.net/

There are some links, where you can ask any questions about nginx in english.

[mindhunter77]

Everything seems to be working on my site now, it's just when I check the headers when I go to vist an image I get this

http://www.xxxxxxxx.com/forum/uploadedimages/1/38251.thumb?d=1236711928


Notice the ?d=1236711928

Does that mean it's not working and something is wrong?

[Vitaly]

Does that mean it's not working and something is wrong?

Thhat's ok. Don't care.

?d=XXX - timestamp, used to refresh browser cache, if you update image. It's generated by vB and left intact for compatibility reasons.

[mindhunter77]

ok great then, looks like I got this working...

I will try and help anyone else if I can, so if you have a question ask..

[mindhunter77]

Oh, by the way, vitaly I was able to still use vbseo on attachments as well =)

[Vitaly]

You mean, "attachments rewrite" vbseo options? I din't checked it at all, because don't use it (that option is off in my vbseo config).

Anyway, check twice, that your nginx config has separate "location" for thumbnails, and that this location stays before vbseo location. For example, you can temporary add deny rule for thumbnails location, to check that it's really catched.

vbseo php code should NOT be called for direct thumbnail links. Or you'll get no benefits from this mod.

Mod idea is:
- minimise php calls (no calls for thumbnails at all)
- minimise useless data transfer (can make sense for big attachments)

[mindhunter77]

It strange, I have rewrites enabled for attachments, yet everything works. My headers are showing that they are coming through as direct links like

http://www.xxxxxxxx.com/forum/uploadedimages/1/38251.thumb?d=1236711928

I must be doing something wrong then lol,

[mindhunter77]

vitaly,

can you think of a good way to tell if this is working? I disabled vbseo attachment rewrites, and it still appears to be working, thumbnails are showing up, but is there a way to test if it's actually doing what it's superposed to?

[Vitaly]

If you turned on "nginx acceleration option", then it's ok.

If you wish to check thumbnails - add "deny all" to thumbnails location from nginx config (see sample from first post). If thumbnails disappears - then all is ok, roll back your nginx config. If thumbnails still shown - then they goes via vbseo rewrite rules and you have to fix nginx config. See my previous recommendation.

The last way to check - turn mod off when server is loaded and check load difference. ~20% in my case. Of cause, if you have few attachments, there will be no difference at all.

VBSEO itself doesn't affect bebefits of this mod, if you've managed to get direct links to thumbnails and proper nginx config. But I've not checked combination with "attachments rewrite" option, so, can't comment if it conflicts with this mod or not.

[MmS1]

Thanks i have two problem
1 when i active this mod i can't download any files "attachments"

2 It's only for thumbs i mean is there any option to make it with other attachments i have a big Problem with attachments server load i have more then 60 user download attachments each One 20 mb and up to 40
Thats will help me alot

[Vitaly]

MmS1,

Do you use nginx webserver? I have no problems with full attachments (not thumbs), but this option work ONLY with nginx. With other webserver "nginx acceleration" must be disabled, see FAQ. Don't enable all mod options "just for fun", if you don't have nginx webserver.

If you use nginx, but full attaches can't be downloaded, check twice absolute path to directory (see config samples in FAQ)

Try to re-read FAQ and my explanations on previous page.

[mindhunter77]

This is the config I used when I had your mod working AND vBSEO attachment rewrites going. Now from the site, everything worked. In fact, load even when down. Yet I'm not sure if it's correctly working or not.

Notice I did not add your config options as this seemed to work.

server {
listen 80 default;
server_name www.xxxx.com xxxx.com;
server_name_in_redirect on;


location / {
root /www/vhosts/xxxx.com/htdocs;
index index.php;
autoindex on;
access_log off;

}

location ~ \.php$ {
root /www/vhosts/xxxx.com/htdocs;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_intercept_errors on;
}


location /forum/ {

if (!-e $request_filename) {
rewrite ^/forum/(.*)$ /forum/vbseo.php last;
}

}
if ($request_filename ~ "\.php$" ) {
rewrite ^(.*)$ /forum/vbseo.php last;
}



location ~* \.(swf|pdf|ico|css|js|gif|jp?g|attach|thumb|png)(\ ?[0-9]+)?$ {
expires max;
root /www/vhosts/xxx.com/htdocs;
}



location /status {
stub_status on;
access_log off;
}

}

[Vitaly]

At first glance, that can work :)

IMHO, there are no reasons to add "expires max". Very cheap economy, but potential problems with browser cache, if you upload new CSS for some mod and so on.

[mindhunter77]

ok, set at 30d; ??

[Vitaly]

No. Remove it at all.

[mindhunter77]

ok,

For those who want to test this mod with vBSEO still handling your attachments this is my working conf file nginx. This one is updated from the last one I posted and has some fixes. Keep in mind a few things.

1) In my conf where it says /thumbnails is not attachment thumbnails, I embed flv video on my site and this is where the flv thumbs go. You will not need this.

2) I placed this in the mod config area, and have both options set to YES.

/forum/uploadedimages

3) My server was running between 2.00 and 3.00 at all times before this mod, it now runs at 1.0 or less. I could probably get it even lower then that if I disabled the handling of attachments through vBSEO all together, but I get a lot of google image traffic and don't want to risk loss of ranks.

4) My forum gets about 50,000 unique per day, 90% of all threads contain pictures and video with picture thumbnails at 400px.



server {
listen 80 default;
server_name www.xxxx.com xxxx.com;
server_name_in_redirect on;



location / {
root /www/vhosts/xxxx.com/htdocs;
index index.php;
autoindex on;
access_log off;

}

location ~ \.php$ {
root /www/vhosts/xxxx.com/htdocs;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_TRANSLATED $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_intercept_errors on;
}



location /forum/ {

if (!-e $request_filename) {
rewrite ^/forum/(.*)$ /forum/vbseo.php last;
}

}
if ($request_filename ~ "\.php$" ) {
rewrite ^(.*)$ /forum/vbseo.php last;
}


location /forum/uploadedimages {
root /www/vhosts/xxxx.com/htdocs;
expires 24h;
}



location /forum/thumbnails {
root /www/vhosts/xxxx.com/htdocs;
expires 24h;
}


location ~* ^.+\.(gif|png|ico|css|zip|tgz|gz|rar|pdf|bmp|js|sw f) {
access_log off;
expires 24h;
root /www/vhosts/xxxx.com/htdocs;

}


location /status {
stub_status on;
access_log off;
}

}

[MmS1]

MmS1,

Do you use nginx webserver? I have no problems with full attachments (not thumbs), but this option work ONLY with nginx. With other webserver "nginx acceleration" must be disabled, see FAQ. Don't enable all mod options "just for fun", if you don't have nginx webserver.

If you use nginx, but full attaches can't be downloaded, check twice absolute path to directory (see config samples in FAQ)

Try to re-read FAQ and my explanations on previous page.

Thanks alot but iam not running with nginx
I have hostshare is that oky i mean they can install nginx 4 me or not

My attachments path out forum path how can mange this

mydomain.com/vb this is my forum path
mydomain.com/vbatttatn this is my attachments path

[mindhunter77]

nginx is a webserver, for this to work 100% you need to be using nginx as your webserver, your host may not just switch that for you, you will have to ask them

[MmS1]

thanks i'll ask them
and about the path

My attachments path out side forum path how can mange this

mydomain.com/vb this is my forum path
mydomain.com/vbatttatn this is my attachments path

[lebmedia]

I Have A Question Before Installation
The Attachment And Albums And Blogs Stored In The FTP
But About The Thumbs ? From Where Can I Move Them To The FTP ?

[mindhunter77]

You need to have everything stored in the file system.. NOT in the database.

FTP has nothing to do with it.

[MmS1]

thanks i'll ask them
and about the path

My attachments path out side forum path how can mange this

mydomain.com/vb this is my forum path
mydomain.com/vbatttatn this is my attachments path
Edit/Delete Message Reply With Quote

[lebmedia]

You need to have everything stored in the file system.. NOT in the database.

FTP has nothing to do with it.

Yeah I Know That Man , I Mean By The FTP THe File System
but about the thumbs ?

[mindhunter77]

once you move the attachments to the file system, the thumbs will be placed there as well. You don't need to do anything separate for them, just make sure your attachments are served from the file system.

[lebmedia]

once you move the attachments to the file system, the thumbs will be placed there as well. You don't need to do anything separate for them, just make sure your attachments are served from the file system.

thanks so much mate for your help

[lebmedia]

Why The Attached Pictures Look Like This ??

Wen Uploading A Small Image To The Attachments t look Like This

https://vborg.vbsupport.ru/attachment.php?attachmentid=99563&d=1243209261

And Wen Upload A Big Pictures It Look Like This

https://vborg.vbsupport.ru/attachment.php?attachmentid=99564&d=1243209261

[R-D]

Works great for me.

I just switched from inline attachments to thumbnails, so this seemed like a logical step too. Lets just say anything that uses less server resources is a huge plus in my book.

Added this to my .htaccess to prevent .attach viewing on an Apache server. Seems to work fine.
RewriteCond %{REQUEST_FILENAME} \.attach [NC]
RewriteRule ^.* - [F,L]


Thanks for the modification, Vitaly!

[Fenriz]

Do I need to "Enable large downloads acceleration" if I have nginx as a frontend and apache as a backend?

[Vitaly]

Probably, that will work. Try to enable. I haven't tested such combination.

[zyadalseef]

thank u
I will rty it

[R-D]

The nginx part of this mod also works perfectly if you run nginx as a reverse proxy, with an Apache backend, even if Apache handles your PHP files.

After some basic nginx configuration (listed in post #2), it's very good to be able to let nginx handle the attachments instead of Apache. With attachments and all my static files now handled by nginx, a very basic VPS now goes a lot faster.

[love.]

thanks

[MmS1]

Again can this hack work with cpanel
Is there any way this hack work with files lika thumb

[PossumX]

Any chance of this mod being updated for vB 4.0 when the time comes?

[Vitaly]

Any chance of this mod being updated for vB 4.0 when the time comes?

All my mods will certently be updated for 4.0. At least, because I have to migrate myself :) . I think, that will take 3-4 months, because 4.0 must become stable, and i have a lot of code to convert for 4.0.

[Rafa-el]

Does this plugin work for vB 3.8.5?

[Vitaly]

I did't upgraded my 3.8.4 yet. But don't expect problems with 3.8.5

[magmf]

do you have anything for vbulletin 4 ?

[mindhunter77]

Wouldn't this be useful for other areas? I know it covers thumbs, but what about any folder that has strictly static files, wouldn't it be best to have nginx serving those as well? I mean my style folder has hundreds of images that I would assume could benefit from something like this as well?