PDA

View Full Version : substr is BAD! fetch_trimmed_title is GOOD!


Makc666
01-20-2007, 12:37 AM
A little note for all coders.

I was inspecting MOD:
Cyb - Advanced Forum Statistics
https://vborg.vbsupport.ru/showthread.php?t=122986

And saw that php's substr(); function is used there.
For example:
$getstats_starter = substr($getstats_starter[username], 0, $trimusername) . '...';

substr(); is a bad one for cutting words up to set limit.

You have to use fetch_trimmed_title(); function which is built-in vbulletin.

You asked why?
1-st, VB's uses this function to cut thread titles.
2-nd, most forum uses UTF8.

For example Russian letter in UTF-8 will look like: &_236; (you have to replace _ with #)
So when you type some word in Russian which consists of 5 letter it will look like:
&_236;&_236;&_236;&_236;&_246; (you have to replace _ with #)
And in this way it is storied in databse.
So it it 30 symbols if you count.

When you use substr(); function you cut by symbols [U]and not by letters!

So if you set to with substr(); function to cut after 27 letter and you will cut the example above you will get:
&_236;&_236;&_236;&_236;&_2

As you see the last letter in UTF-8 format was cuted.

And when this one will be displayed on the webpage you will see a bug.

That is why you have to use fetch_trimmed_title(); function.
As it cuts only whole words.
fetch_trimmed_title() relies on spaces when cutting.

If me return to
Cyb - Advanced Forum Statistics
https://vborg.vbsupport.ru/showthread.php?t=122986

in all code I replaced lines like:
$getstats_starter[username] = substr($getstats_starter[username], 0, $trimusername) . '...';

with a good one:

$getstats_starter[username] = fetch_trimmed_title($getstats_starter[username], $trimusername);

Here how this function looks like in:
functions.php

// ################################################## ###########################
/**
* Trims a string to the specified length while keeping whole words
*
* @param string String to be trimmed
* @param integer Number of characters to aim for in the trimmed string
*
* @return string
*/

Hubbitus
07-07-2008, 03:07 PM
For example Russian letter in UTF-8 will look like: &_236;
So when you type some word in Russian which consists of 5 letter it will look like:
&_236;&_236;&_236;&_236;&_246;
Excuse me, where it look like this?
In UTF8 (http://en.wikipedia.org/wiki/UTF8) any character in the Unicode standard, yet the initial encoding of byte codes and character assignments for UTF-8 is backwards compatible with ASCII. So, dependency of character it may be represent from 1 (first 127 characters ASCII) to 4 bytes. Russian Cyrillic characters in UTF-8 represents by 2 byte per character.

Your example is strange, and I can't represent it on my system. I'm guess what you meant HTML mnemonic entities (http://www.w3schools.com/tags/ref_entities.asp), but in this case mnemonic must look like
ì
not like
&_236;

And in this way it is storied in databse.
If my guess about html-entities is true, this behaviour you get only with mismatching used character encodings. For example, on vbulletin board you use UTF8 character encoding, but stores information in database (or tables, or may by only row) with another, single-byte charset, for example windows-1251 (CP1251, microsoft-1251). In this case, this transformation made by browser, to allow store characters, which are not be acceptable by form (if charsets correctly provided in HTML-code).

On my board on UTF-8 with database collation utf8_general_ci I'm haven't this troubles - all Russian and many more characters saved as is, without similar transformation.

When you use substr(); function you cut by symbols and not by letters!
So, about HTML-entities I'm wrote above. But we still has problem on not ASCII characters, which are be represent more then 1 byte per character. PHP function substr() do not handle characters (or symbols if you wish), it is truncate string by amount of bytes (keep in mind it is same in ASCII)!
So, if you wish cut by characters, you just may use multibyte functions like mb_substr() (http://php.net/mb_substr) (which comes from mbstring extension) or iconv_substr() (http://php.net/iconv_substr) (this is from "more standard" and widely distributed iconv).

So, if none of this extension available, in multibyte strings, you may safely emulate substr by regular expression on this manner:
$getstats_starter['username'] = preg_replace('#(.{0,' . (int)$trimusername . '}.*?)[^\pL].*#u', '\\1', $getstats_starter['username']);

That's all!! You don't need bulky functions!

Or, for clarity, you may wrap it in function, if wish:
function pcre_trim ($title, $chars = 70){
return preg_replace('#(.{0,' . (int)$chars . '}.*?)[^\pL].*#u', '\\1', $title);
}

//Anywhere after, use it:
$getstats_starter['username'] = pcre_trim($getstats_starter['username'], $trimusername);

Makc666
07-08-2008, 04:57 AM
Excuse me, where it look like this?
In UTF8 (http://en.wikipedia.org/wiki/UTF8) any character in the Unicode standard, yet the initial encoding of byte codes and character assignments for UTF-8 is backwards compatible with ASCII. So, dependency of character it may be represent from 1 (first 127 characters ASCII) to 4 bytes. Russian Cyrillic characters in UTF-8 represents by 2 byte per character.

Your example is strange, and I can't represent it on my system. I'm guess what you meant HTML mnemonic entities (http://www.w3schools.com/tags/ref_entities.asp), but in this case mnemonic must look like
ì
not like
&_236;

If you try to quote your post or my post here with
ì
you will get a letter instead of code...

Try to post something like:
&_236;&_236;&_236;&_236;&_236;&_236;
(you have to replace _ with #)

You will get this one:
ìììììì

kotlt99
08-29-2008, 06:39 AM
With VBB 3.7 , Has it effect ?

Dismounted
08-29-2008, 07:39 AM
This post will apply regardless of vBulletin version - it will apply as long the the substr() function behaves like it currently does.