Realtime Template Compressor: Reduces page sizes, loads pages faster, saves bandwidth

This is a port of Logican's Realtime Template Compressor hack from vBulletin 2.2.x to vBulletin 3.5.0 Beta 1. I wrote this by rewriting a version of the original hack, that I wrote for my forums when they ran 3.0.x, so that it would function as a plugin for 3.5.0.

I had originally sent this to Logican for him to release it but he replied granting me permission to release it; hence why I'm releasing this.

Installation is very simple. Just import the plugin via vBulletin's "Download / Upload Plugins" page in the Plugin System in the Admin Control Panel and you're done.

I'll try to correct any bugs that you might find. Let me know if you like this plugin and don't forget to click the install link if you install this.

Update History

Version 1.1.0: The expression was replaced with a more efficient one that increases the compression level from approximately 1-2% to approximately 4-8%.
Version 1.0.0: Initial Release

Originally Posted by ryuji

i meant in your templates... you can remove all the spaces but on my templates it removed code it wasnt supposed to when i did that, best way is to download the whole style and then use a find and replace to add a space to all of them

So are you saying that if I have:

<!--This is my comment-->

That I need to change them to:

<!-- This is my comment -->

Like that?

[ryuji]

thats all i had to do.. and it solved issues with the scripts

[Hellcat]

I played around with this a bit
And after talking to my co-admin (who has quite some knownledge of regular expressions) he told me to exclude <script> and other tag blocks is not possible with one RegEx, it would need additional code.

Well, so I started making some code

This I what I currently have, it gave me a "compression" ratio vom ~10% to ~15% in my tests, while leaving user input, [code] posts and JS fully intact (so far )

I added some comments, so you can see what is done where

PHP Code:

global $ascript, $atext, $apre;

// definition of callback functions
function cb_script($matches)
{
  global $ascript;

  $i = count($ascript) + 1;
  $h = md5($matches[0]);
  $ascript[$i][0] = $matches[0];
  $ascript[$i][1] = $h;

  return $h;
}

function cb_text($matches)
{
  global $atext;

  $i = count($atext) + 1;
  $h = md5($matches[0]);
  $atext[$i][0] = $matches[0];
  $atext[$i][1] = $h;

  return $h;
}

function cb_pre($matches)
{
  global $apre;

  $i = count($apre) + 1;
  $h = md5($matches[0]);
  $apre[$i][0] = $matches[0];
  $apre[$i][1] = $h;

  return $h;
}


// init some variables/arrays
$page_byte=strlen($output);
$ascript = array();
$atext   = array();
$apre    = array();

// cut from output, and store away for later re-inclusion, everything between <textarea> and <pre> tags
// (we do not want to touch those blocks as they may contain wanted whitespace)
$output=preg_replace_callback('/<textarea.*?[^\btextarea>\b]*?textarea>/si', "cb_text", $output);
$output=preg_replace_callback('/<pre.*?[^\bpre>\b]*?pre>/si', "cb_pre", $output);

// strip all whitespace at the beginning of every new line (but leave the linebreak intact)
$output=preg_replace('/\r\n(\s*)/', "\n", $output);

// cut from output, and store away for later re-inclusion, everthing between <script> tags
// (we don't want to strip the linebreaks from script code since that would render the scripts non working)
$output=preg_replace_callback('/<script.*?[^\bscript>\b]*?script>/si', "cb_script", $output);

// strip all linebreaks and HTML-Comments from the output
// (since we first strip all linebreaks, even multi-line comments will get striped this way! :-))
$output = str_replace( "\n" , "" , $output );
$output=preg_replace('/<!-- .*? -->/', "", $output);

// re-insert everything we saved before
foreach ( $ascript as $block)
{
  $output = str_replace( $block[1] , $block[0] , $output );
}
foreach ( $atext as $block)
{
  $output = str_replace( $block[1] , $block[0] , $output );
}
foreach ( $apre as $block)
{
  $output = str_replace( $block[1] , $block[0] , $output );
}

// calculate some nice looking numbers :-)
$pagenew_byte=strlen($output);
$page_kilobyte=number_format(($page_byte/1024),2);
$pagenew_kilobyte=number_format(($pagenew_byte/1024),2);
$pagesaved_byte=$page_byte-$pagenew_byte;
$pagesaved_kilobyte=number_format((($pagesaved_byte)/1024),2);
$pagesaved_perc=number_format(((100*$pagesaved_byte)/$page_byte),2);

// and add the compression stats at the very bottom of the page
$output = str_replace('</body>', '<center><span class="smallfont">[<em>page compression: </em>' . $pagenew_kilobyte . 'k/' . $page_kilobyte . 'k (<strong>' . $pagesaved_perc . '%</strong>)]</span></center></body>', $output); 


That fixed the advanced mode bug, Hellcat, but hosed tons of other stuff on my forums. For one, the edit/quote buttons at the bottom right of posts were almost overlapped, and the breadcrumbs had no spacing at all.

A step in the right direction though, to be sure.

Screenshot attached of what I'm experiencing. I'd LOVE to get this sorted out - your code, albeit buggy on my particular theme, quadrupled my compression savings.

For reference, a screeny of my forums using my existing code, which only strips comments.

[Hellcat]

Hmm.... I think I know what that might cause....
If that's it it would reduce the ratio, but increase compatibility....
[high]* Hellcat starts tweaking
[/high]

[EDIT]
OK, try this:

In my above posted code, find theese two lines:

$output = str_replace( "\n" , "" , $output ); 
$output=preg_replace('/<!-- .*? -->/', "", $output);

and replace them by those:

$output = str_replace( "\n" , " " , $output );
$output = preg_replace('/<!--[^\{]*?-->/', "", $output);

Like I already said, this drops the ratio a bit again, but should raise compatibility.
At least I could confirm this in my last tests....

[high]* Hellcat waits for feedback now [/high]

[The Chief]

does this work with vB Drupal?

[Hellcat]

Originally Posted by gamebgs

does this work with vB Drupal?

Hmm, can't test that since I don't have that installed.
There's only one way to find out

But it should, yes....

[The Chief]

Originally Posted by Hellcat

Hmm, can't test that since I don't have that installed.
There's only one way to find out

But it should, yes....

I'll try that right now

is it supposed to take all the lines away from the source?

[The Chief]

and the result is: Nope it doesn't work with vb drupal...

works fine for my forums though so installed...

/me clicks install