[How to] Cut down on memory usage

[Brad]

This is in the manual but I've yet to see it mentioned here.

The plugin system works by storing all plugin code for all scripts in memory, so you can quickly find your plugins using large amounts of memory if they contain a lot of code.

A simple way to avoid this problem is to use the plugin code simply to call an external script, which contains all the complex code. In this way the code is only loaded when it is actually required.

For example, a plugin could contain this:
$tmp_uid =& $vbulletin->userinfo['userid'];

$db->query_write("
INSERT INTO " . TABLE_PREFIX . "profilelog
(userid, dateline)
VALUES
($tmp_uid, " . TIMENOW . ")
");

or alternatively, that code could be placed into a file called (for example) plugins/my_script.php, and the plugin itself would contain this:
include('./plugins/my_script.php');

Naturally, the second option will use up far less memory than the first, and this saving will become more and more beneficial as the amount of code to be run increases.

You should also take a look at this project: https://vborg.vbsupport.ru/showthread.php?t=107315

[Paul M]

Would this slow execution if lots of plugins were calling lots of files ?

[The Geek]

Would this slow execution if lots of plugins were calling lots of files ?

A lot less then housing all the PHP for all files in memory at all time.

The bulk of my processing is done via external files.

[Milad]

Would this slow execution if lots of plugins were calling lots of files ?

I agree

I think files modifications would be better for this purpose

[Marco van Herwaarden]

I have been thinking about creating an extension that will allow 'include-files' for plugins. They would be loaded as regular plugins (not file uploads), and can be loaded into memory whenever needed (for example: require_pluginlibrary_once('myfunctionlibraryplugi n'); ).

Like this you might get the best of both worlds:
- No file uploads
- Only loaded into memory when needed

[PennylessZ28]

I like that concept, I think I am going to play with this some more.

[99SIVTEC]

i'm ditching the plugin system on most of my larger sites. I'm just hardcoding the hacks. The plugin system is great in principal, but it eats up memory, increases queries, and slows down execution.

[PennylessZ28]

i'm ditching the plugin system on most of my larger sites. I'm just hardcoding the hacks. The plugin system is great in principal, but it eats up memory, increases queries, and slows down execution.

Oh so true. If only there was a way to use this concept of the xml plugin to also upload a file like the above. So its not stored in memory.

[Marco van Herwaarden]

i'm ditching the plugin system on most of my larger sites. I'm just hardcoding the hacks. The plugin system is great in principal, but it eats up memory, increases queries, and slows down execution.
If coded for saving memory(ie. 99% of executable code in include files), a plugin don't need to use much memory, nor does it need to use more queries or slow down significantly.

[amykhar]

This is in the manual but I've yet to see it mentioned here.

I feel better now. Somebody griped when I wrote my Amazon plugin this way :D

[99SIVTEC]

Just having the plugin system turned on uses extra queries. A good programmer ca hack vbulletin in such a way as to add few if any extra queries, but the system itself is what adds the extra queries.

If coded for saving memory(ie. 99% of executable code in include files), a plugin don't need to use much memory, nor does it need to use more queries or slow down significantly.

[Marco van Herwaarden]

Just having the plugin system turned on uses extra queries. A good programmer ca hack vbulletin in such a way as to add few if any extra queries, but the system itself is what adds the extra queries.

Not sure on the exact amount of added queries when the plugin system is turned on, but it is not much, and they are good optimiyed and won't effect your board much from a performance point of view.

Remember that the number of queries is almost irrelevant for performance, it is the execution time that counts.

[PennylessZ28]

Seems to me that just putting the plugin code in a php file and replacing it with

include('./plugins/my_script.php');

Doesn't work 100%

[Marco van Herwaarden]

Nobody ever said that things would work without any modification.

[Andreas]

1. Turning the Plugin System On or Off does not have any impact on the amout of queries.
2. Memory Footprint is not a such a big issue. In most cases, forumcache (which is loaded on every page) for example will be a lot bigger then the plugins.

Having to eval() the code all the time, that is an issue.

Therefore, complex plugin code (especially in high-traffic places like global.php, postbit, showthread, forumdisplay, etc.) should be moved out to include files.
(Could even be done automatically. I wrote a proof-of-concept hack that writes include-files if the plugin-code on one hook is > X bytes long.)

[Andreas]

Seems to me that just putting the plugin code in a php file and replacing it with

include('./plugins/my_script.php');

Doesn't work 100%

Hmm ... what problems did you have?

If you use

include(DIR . '/plugins/my_script.php');


it should work 100%.

[PennylessZ28]

Hmm ... what problems did you have?

If you use

include(DIR . '/plugins/my_script.php');


it should work 100%.

I converted two plugins to test this. VBSHOUT for one, which failed 100%.

And then estakis referral stats. That one works, half way.

It's depositing all the code in raw format on the page.

Crazy I tell ya.

[Andreas]

Erm ... are you sure you put


<?php
// Plugin Code here
?>


in the files?

[PennylessZ28]

Erm ... are you sure you put


<?php
// Plugin Code here
?>


in the files?

Yes I'm stupid. I forgot that. LMAO

[Trigunflame]

Solution to all your problems :)

https://vborg.vbsupport.ru/showthread.php?t=107315

[pmfp]

I have a problem with include's and phrases. It has been recreated in the files test.php and includes/test_include.php.

The problem is that the code is executed and the template fetched, but the phrase is not being printed. The nature of the real script means that I cannot use another plugin to insert the code, so that is out of the question.
Any ideas on how to do this properly?

test.php:
<?php

error_reporting(E_ALL & ~E_NOTICE);

define('THIS_SCRIPT', 'db_update');
$phrasegroups = array('test_phrases');

include('includes/test_include.php');

require_once('./global.php');

did_it();
?>
includes/test_include.php:

<?php

function did_it() {
$varvar = "set";

eval('print_output("' . fetch_template('test_splitter') . '");');
}
?>
test_splitter template:
<tr>
<td width="310" valign="top" class="thead">
<if condition="$varvar==set">$vbphrase[test_hello_world]</if>
</td>
<td valign="top" class="thead"></td>
</tr>

[thincom2000]

I just added about 10 of my larger plugins to files, and I have noticed a considerable lag on the forum. Page generation times have increased from .10 seconds to 2 seconds or more, and server load has gone up from about 1.00 to anywhere from 8 - 40. 40!!!!!

EDIT: Looks like it was just a really bad night for server load. My forum seems to be running at the normal load today.

[Whissi]

Do you still believe that outsourcing hugh plugins to includes will speed up your forum?
I don't think so. It sounds like it should improve the performance, but in real scenarios, it doesn't matter.

At least, have a look in the official addons like the blog software. They put their code in the plugins and not in files they include. They did it with high traffic hooks like fetch_userinfo... Haven't they thought about performance or doesn't that matter...