Cache the datastore in the filesystem

Ok, the purpose of this hack is to move the huge parts of the datastore,that are called each time you access a page, into a file unserialized and passed through var_export in order to save the time used to load the data from the database and to unserialize it. If you're using a PHP optimizer, like eAccelerator for example, this may save even more time I think. The result is, that huge arrays, like the forumcache are now loaded simply by executing

PHP Code:

require_once(DATASTORE_FILE_DIR.'datastore_cache.php'); 


in your init.php.

Additionally this saves 4 calls to the function unserialize, which is broken in PHP 5.0.3 and 4.3.10, so this should be a good work around until PHP 5.0.4 and/or PHP 4.3.11 has been released.

It tries to finish, what has been started in the code and AFAICS you can even find the beginnings of that in the init.php of vB3.0.6,although I'm still running vB 3.0.3, so I can't tell, that this works without problems in newer versions.

Some Notes:

• I recommend to place rebuild_dscache.php into a protected directory and to alter the first chdir(...); in the script, so that no one can abuse this, by flooding your forum with calls to that script.
• Additionally you should remove the backup-files named datastore_cache_TIMESTAMP.php in your includes dir (or wherever the datastore-cache is stored) from time to from to save disc-space.
• Use this at your own risk and make sure to backup first before installing this hack.
• This hack is not designed for a multi-webserver setup. So unless the data on the filesystem is being updated on the fly its not recommended to install this.

Changelog:
0.8.1 Alpha:

• removed a relic of an old hack from dscache.txt
• Added config_dscache.php
• backups can be turned off now
• you can change the directory, where the datastore_cache.php is stored now
• Fixed a bug, that may cause unserialize-errors in the CP out of the blue
• rebuild_dscache.php is verbosive now

0.8.2 Alpha:

• Added CVS-Ids to some files
• Added a note to rebuild_dscache.php
• Added a note about PHP < 4.2.0 to dscache.txt

0.8.3 Alpha:

• Removed some relics of the old backup functionality
• some changes to make it possible to run it at the commandline (see below)
• Allow to specify a key in the config-file, so this script will only run if you specify the correct key (-k KEY on the commandline or ?key=KEY in the URL)
• You can make the script silent by adding -s respectively ?silent=1

1.0 (Final):

• No changes at all. All files are the same as in Version 0.8.3 Alpha.

[mark99]

If somebody could help with the above post then I'd be most grateful, it's a good solution for easing DB load.

[kall]

Originally Posted by Erwin

I got it working with my version of ushop - you basically need to add the ushop variables to datastore templates too.

Could you please share how you managed it?

[Stadler]

Originally Posted by mark99

1) When do you actually need to run the 'rebuild' process? Judging by the data - only when you change an option in the admin CP or add/edit one of your forums/category names & options. Does it need to be rebuilt after every new post or is it just doing the more static data?

2) Can I remove the admincp/rebuild....php script from the server when not needing to be used or does something require it to be there?

3) If I need to take my forums offline to change the options, then (as per above) will this corrupt my database? I'm confused about how this can be prevented as admins always need to update options/add forums etc. now and then. Do you take it offline, run the rebuild, update the options, run the rebuild, put it online and then run the rebuild again? Am a little confused

Do you have a single or multiple webservers, where your forum is running on? If your forum only runs on a single werserver, theres nothing to worry about.

PS: Sorry for the late reply. I've been busy with other projects recently.

[Stadler]

I have updated some files. I've only updated some of the documentation and no functionality so far, so you don't need to update anything. You'll only waste your time

For people with PHP < 4.2.0 I've added a note, what they could do to get the hack running.

Erwin: Scott pointed me to the following: How about triggering rebuild_dscache.php on all your server with something, like
fopen('http://www01.example.com/path/to/rebuild_dscache.php?silent=1&key=SECRET');
?

PS: PHP 4.3.11RC1 and PHP 5.0.4RC1 are out: http://news.php.net/php.general/210866

[Erwin]

Originally Posted by kall

Could you please share how you managed it?

editing init.php and adding the ushop datastore into the cache too.

SUGG: drop the config_dstore.php file and add the $dscache_dir var in the config.php

PHP Code:

define('CACHE_DATASTORE_FILE', true);
define('DATASTORE_FILE_DIR', './includes/'); 


$dscache_dobackup and BACKUP_DATASTORE_FILE are useless for now... would need to set it somewhere or delete it!

also, for security, isn't it suggested to NOT chmod the includes dir ?!

[Stadler]

Originally Posted by nexialys

$dscache_dobackup and BACKUP_DATASTORE_FILE are useless for now... would need to set it somewhere or delete it!

D'Oh, right. I removed the backups from the script but forgot to mention that, sry. I'll remove the redundant code.

However: I'm working on enabling rebuild_dscache.php for running as a shell script. You will then be able to specify a key, so the script will only run, if you call it with rebuild_dscache.php?k=sdiztgejklt for example.
Or with php rebuild_dscache.php -k sdiztgejklt on your favorite shell.

Another parameter would be ?silent=1 or -s to make the script silent, except error-messages.

[Stadler]

Ok, can anyone test the recent version?
It works fine on my test-system (commandline and through URI)

Changes:
0.8.3 Alpha:

• Removed some relics of the old backup functionality
• some changes to make it possible to run it at the commandline (see below)
• Allow to specify a key in the config-file, so this script will only run if you specify the correct key (-k KEY on the commandline or ?key=KEY in the URL)
• You can make the script silent by adding -s respectively ?silent=1

[Acido]

OK, now i have this hack installed.
But i cant see any datastore_cache_TIMESTAMP.php file in my includes dir (i'm usign defaults dirs of the hack).

How i can to know if this is working?

[Stadler]

Originally Posted by Acido

OK, now i have this hack installed.
But i cant see any datastore_cache_TIMESTAMP.php file in my includes dir (i'm usign defaults dirs of the hack).

How i can to know if this is working?

These are relics of the old backup functionality and won't be created anymore. I've left the note about them for users who still have old backup files in their includes-dir (or whereever they are storing their datastore_cache.php.

You should take a look at datastore_cache.php. It should contain the expanded arrays of the usergroupcache or the forumcache for example.