vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.8 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=235)
-   -   Board Optimization - vB Accelerator (https://vborg.vbsupport.ru/showthread.php?t=207566)

Vitaly 03-06-2009 10:00 PM

vB Accelerator
 
1 Attachment(s)
vb 4.0 version here

This mod is suited for medium and big boards. It reduces server load, caused by thumnails and big attachments.

what's the problem with?
  1. In original vB all thumbnails a downloaded via php. So, if you page contains 10-20 attached images, then each page php request will be followed by 10-20 thumbnails php requests. That's very bad.
  2. When attachments body transfered via php, that's much more waisteful, than direct transfer of static file. Especially for big files.
what this mod does:
  1. All thumbnails will have DIRECT links (served as static files). Much lower reply latency and server load.
  2. If you have nginx webserver, you php-fcgi will not participate in file transfer any more. It will reply with X-Accel-Redirect header. Then nginx will proceed attachment as static file.
    • No more problems with multiple downloads of 10-mb files.
    • No needs to restrict attachments and full-size images for guests and search bots.
  3. That works for forum posts, blogs, albums. Everywhere!
how to install:
  1. Make sure, this files are stored on disk, NOT in database:
    • post attachments & thumbnails
    • blog attachments & thumbnails
    • album images & thumbnails
  2. Make sure, that thumbnails are in web-accessible area. If not - reconfigure your web-server first. If you own big board - then you know how to do that :) .
  3. Check manually, that you can download any .thumb file via browser.
  4. Import this product, and set options for you configuration.
  5. Be happy to reduce server count in your cluster :)
PS. It's tested and works fine at my production site for 1 week. But I'll be happy to know server load statistics from big boards owners.

This mod is developped here: http://github.com/rcdesign/vb-accelerator . Patches are welcome.

History

0.4
- fixed bug with PHP wagnings in server logs.

0.3
- first public release

Vitaly 03-06-2009 11:54 PM

Faq

Q1. I don't have nginx. Will this mod work for me?

Yes, partially. Enable direct thumbnail links, but disable nginx acceleration. You'll still have serious benefit, if your forum is plenty of attachments.

Q2. What about lignttpd, apache and so on? They have X-Sendfile, alternative to nginx X-Accel-Redirect.

I know, but:
  • I don't have such configurations to test, and I don't have time to implement that. You can do it yourself or hire someone (NOT ME) for this work.
  • X-Sendfile don't support partial content download :( . Nginx's X-Accel-Redirect it the best you can use.
Q3. Is it safe to give direct access for thumbnails?

Why not? You can give direct access for thumbnails, but disable direct access to original files. Just configure your server properly. See examples below.

Q4. Thumbnails looks like files with strange extention and mime-type. Will those be ok in browser?

Yes. HTML-page address those files via imagetag. So, browser show those properly, even with wrong mime and strange extention.

Q5. Why don't you fix that for sure?

Because that requires vB sources modification, and i hate such things. And if everyting works fine, who cares about mimes :) ?

Q6. My forum has 20 users online, and I don't see difference at my 32-core server with 128Gb ram and 8 SSD raid array.

You a happy man, really.

Q7. Does it work with VBSEO ?

If attachment rewrite is disabled - yes, it works. And check, that attachments path is separated in vbseo rewrite rules in webserver config. I use vbseo in this mode with no problems. Sorry, no plans now to make better integration with vbseo. But code is reuseable, or you can send your patches :)



Server configuration examples

let's say, whe have the following folders structure:

/uploads
/uploads/blogs/* <- blog attachments & thumbnails
/uploads/posts/* <- forum attachments & thumbnails
/uploads/albums/* <- album images
/uploads/al_thumbs/* <- album previews

So, we have to setup direct access for thumbtails, but protect original files.

Nginx

# globally disable external access, but enable for X-Accel-Redirect
location /uploads {
internal;
}
# enable direct access to blogs & posts thumbnails, but nothing else!
location ~* /uploads/(.*)\.thumb$ {
}
# enable direct access to album thumbnails
location /uploads/al_thumbs/ {
}

Apache/Lighttpd

TBD. You are welcome to share your one :)

lonasdigital 03-07-2009 12:05 AM

Gracias Voy a probarlo ahora mismo .

Frondy 03-07-2009 12:25 AM

Well, it works! Thank you!

rob01 03-07-2009 02:03 AM

is this a good option for small forums?

JamesAB 03-07-2009 02:25 AM

Quote:

Q4. Thumbnails looks like files with strange extention and mime-type. Will those be ok in browser?

Yes. HTML-page address those files via imagetag. So, browser show those properly, even with wrong mime and strange extention.
I'm not using this mod, but I recently looked into serving static thumbnails on vBulletin.

In apache httpd.conf, I know you can use:
Code:

AddType image/jpeg .thumb

lazydesis 03-07-2009 02:33 AM

what is this nginx ?i have centos OS and apache .. will this increase performance on that?

Vitaly 03-07-2009 02:38 AM

Quote:

Originally Posted by rob01 (Post 1762197)
is this a good option for small forums?

Everything is relative in this life :) . That depends on attachments count and CPU/IO load. If you have no attachments at all, you'll have nothing with this mod.

But nobody prohibit you to try and compare if server load changes :) . At least, direct linked thumbnails will be loaded faster for your users.

Quote:

Originally Posted by JamesAB (Post 1762207)
I'm not using this mod, but I recently looked into serving static thumbnails on vBulletin.

In apache httpd.conf, I know you can use:
Code:

AddType image/jpeg .thumb

Yes. Exactly. But in fact that's not required. My rule is simple: the less customisations - the less possible mistakes in the future :) . You can always add it later, if something goes wrong.

Vitaly 03-07-2009 02:46 AM

Quote:

Originally Posted by lazyindian (Post 1762212)
what is this nginx ?i have centos OS and apache .. will this increase performance on that?

Yes. Nginx/lighttpd/lightsphere/zeus + php-fcgi will give you serious benefits over apache + mod_php on burst loads.

In fact, that should be done before any futher vb optimizations. Maybe I'm a bit categoric, but in my opinion apache+mod_php system is absolutely inpredictable at big loads.

Server optimisation should be done prior to any scripts optimization. That's essential.

Bounce 03-07-2009 06:59 AM

I don't have an uploads folder so I don't know the paths

Path to etc

How do I find this out?

Vitaly 03-07-2009 07:06 AM

ACP -> Attachments -> Storage type
ACP -> User Albums -> Storage type

You'd better look to vB documentation about details. That's VERY CPU/Disk intencive operation, and you have to backup your board.

Bounce 03-07-2009 07:42 AM

Quote:

Originally Posted by Vitaly (Post 1762297)
ACP -> Attachments -> Storage type
ACP -> User Albums -> Storage type

thanks
Quote:

Originally Posted by Vitaly (Post 1762297)
You'd better look to vB documentation about details. That's VERY CPU/Disk intencive operation, and you have to backup your board.

yip aware o that,ty :up:

mindhunter77 03-08-2009 10:27 PM

Anyone have any code to make the thumbnails accessible in LITGHTPD ?

relaxiha 03-08-2009 11:10 PM

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 43

ERROR

Vitaly 03-09-2009 06:13 AM

relaxiha,

1. Which php version?

2. Fatal error or warning message in webserver log?

3. At which page?

http://forum.rcdesign.ru/f14/

Seems to work. Go to any topic with attachments and see direct links for preview sources.

chesterano 03-09-2009 02:01 PM

PHP Code:

WarningInvalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'
d code on line 30

Warning
Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'
d code on line 30

Warning
Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'
d code on line 30

Warning
Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30

Warning: Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'
d code on line 30

Warning
Invalid argument supplied for foreach() in [path]/includes/class_postbit.php(265) : eval()'d code on line 30 


I have Apache please more detalis how setup direct access for thumbtails, because stop show ;(

Vitaly 03-09-2009 03:12 PM

1 Attachment(s)
Try attached file, that should help with warning mistake.

chesterano 03-09-2009 04:11 PM

Working :) You are my Idol Thank You, everything is ok, only under IE and Opera thumbtails, not show. Under Fiefox and Chrome everything is ok... little bit strange? but I try checking config browse
Thank You

Vitaly 03-09-2009 04:29 PM

Hm... that's strange... Find any direct link in FF, and check manually, that it's accessible from IE/Opera via web. Don't hesitate to provide screenshots of broken pages. I have very weak telepathy skills, to understand the problem from description "it doesn't work"

Nobody of my regular 6000 users reported problem with IE. Try this advice: https://vborg.vbsupport.ru/showpost....07&postcount=9

May be, apache work not as nginx with unknown extention types, no idea.

chesterano 03-09-2009 04:42 PM

I install plugin again and now is ok, under IE, Opera, Chrome, Firefox :)

Thank You, Thank You, Thank You...Vitaly

Regards
chesterano

Vitaly 03-10-2009 04:25 PM

Since no more bugs reported - version 0.4 released. If fixes php warnings in webserver logs. Nothing special.

mindhunter77 03-10-2009 06:48 PM

preeeeeettttyyyyy please make this work with lighttpd lol...

relaxiha 03-11-2009 03:16 AM

Thank You, Thank You, Thank You...Vitaly

xxalexkimxx 03-11-2009 05:02 AM

Thank you very much.

Xencored 03-19-2009 03:32 PM

Thanks installed

newmomsforum 04-20-2009 09:09 AM

Hi

I have VBSEO running on one of my forums and the attachment URL's are re-written to include the name of the post and forum.

When I Set direct links for thumbnail images to Yes, this breaks all of my images (I assume there is a comflict between VBSEO and your great mod).

Should this work with VBSEO or hasn't it been tested.

Thanks in advance :)

Vitaly 04-20-2009 01:19 PM

I have vbseo installed, but URLs conversion for attachments is turned off in my configuration.

Sorry, no plans to integrate with vbseo attachments urls conversion.

mindhunter77 05-07-2009 07:29 AM

Request for lighttpd integration =)

thcf 05-07-2009 02:30 PM

Hi there does with work with vbseo and vboptimise?

mindhunter77 05-07-2009 06:07 PM

How do I make the thumbs web accessible? They are located in a web accessible DIR as well as 777 chmod but I still can not bring them up in a browser.

vktechnology 05-10-2009 08:00 PM

Quote:

Originally Posted by mindhunter77 (Post 1806519)
How do I make the thumbs web accessible? They are located in a web accessible DIR as well as 777 chmod but I still can not bring them up in a browser.


I have same problem
my path is "/home/palungjit/board/attachments"
When i put this path it doesnt show thumbnail

after i t put "attachments" then it work

vktechnology 05-10-2009 08:11 PM

My board just in stall this plugin

Posts: 1,773,284
Members: 254,310

Attachments : 255.61 GB
Attachments : 495,833 reccords

I thinks this plugin will help to reduce server load.

mindhunter77 05-13-2009 10:06 PM

how did you make the .thumb web accessible?

mindhunter77 05-16-2009 09:34 PM

is there a way to verify this is working, I think I finally got it all figured out but wanted to verify.

mindhunter77 05-17-2009 08:48 PM

Finally got this going I think, only one drawback from my site, it seems that this conflicts with this mod,

WATERMARK ATTACHMENTS
https://vborg.vbsupport.ru/showthread.php?t=201972

If I turn off Enable large downloads acceleration, the watermarking works again.

MmS1 05-17-2009 10:17 PM

Not working
the attachments still /attachment.php?attachmentid=2403
plz help me am having big problem with attachments

mindhunter77 05-18-2009 02:24 PM

Yea, mine look this
Code:

http://www.xxxxxxxxx.com/uploadedimages/1/20016.thumb?d=1222135419
So I guess it's not working

MmS1 05-18-2009 08:40 PM

It's oky working fine with me
But i can't download files pic's oky i can but files i can't

even the attachments convert them into direct link or just thumb only

mindhunter77 05-18-2009 09:18 PM

could someone who has this working post their nginx.conf so I can see what I'm doing wrong, this still isn't working.

Vitaly 05-20-2009 03:47 PM

For all, who think, that mod is not working:

1. Check that thumbnails are accessible via www (type URL to any thumbnail in browres path). Browser should offer to download file or to show thumbnail (if youcared about mime type)

2. Look into page source, and check "direct" path to any thumbnail. If pattern is different - fix appropriate mod settings.

3. Split your job in 2 parts:
- try to do direct thumbnails working first
- try to make work nginx accelerated download. DONT turn it ON with another web-server.

4. If you have VBSEO - don't enаble attachments rewrite.


Sorry, but can't consult everyone about each particular configuration :( . I hope, that general recommendations will help.


All times are GMT. The time now is 02:10 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01602 seconds
  • Memory Usage 1,837KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (3)bbcode_code_printable
  • (1)bbcode_php_printable
  • (7)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (1)pagenav_pagelink
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete